Arweave aims to make the Internet Archive's data accessible forever

The developers behind decentralized data storage platform Arweave have teamed up with the Internet Archive on a project to make access to the popular archive's data permanently "fault-tolerant."

The Internet Archive is a non-profit library of digital content, ranging from websites to music and even software. But this "amazing resource" is also "fundamentally centralized," says Sam Williams, Arweave's founder. 

So, Arweave's developers are adding torrents — files that can be distributed and accessible via a peer-to-peer network  — to Arweave's "next-generation archive." These torrents consist of three things: "essential" metadata about the data in the Internet Archive, "data integrity hashes" that can be used to cryptographically verify that the data has not been altered since it was uploaded to the internet, and data about where on the network the information can be found.

The goal is to build a fault-tolerant record of what has been added to the archive and when — "and make sure that access to the data is available in a distributed way rather than through centralized server," Williams told The Block in an interview.

Based on a blockchain-like data structure called "the blockweave," Arweave features a unique pay-once pricing model that the platform's creators say makes it possible to store data in a "permanent" way. The "permaweb" that arises from the blockweave is home to around 250 decentralized applications, according to Williams. 

This particular application reflects what Arweave is at its core, said Williams. The platform is set up as "an archive for all of the data put inside it," he said. "Some of that can just be metadata and access information for other pieces of data, as in the case here."

Williams said the team has already archived 700,000 torrent files about information found in the Internet Archive. For now, Arweave is funding the storage. "But we are also interested in a potential community project — depending on how the community feels about it — to fund the storage of the full dataset potentially."

Arweave has drawn investments from Andreessen Horowitz, Union Square Ventures and Coinbase Ventures.