Decentralised Storage with Arweave

Vic Genin
5 min readAug 26, 2022

--

Arweave introduces an entirely new economic model to the market, one that was never possible before the advent of permissionless crypto networks: permanent storage.

It addresses this by requiring miners to refer to data that was previously stored, proving they still have it. Much like in Bitcoin where miners must expend CPU cycles, in Arweave, miners must put up storage, or they will not be able to participate.

With permanent storage, users pay a one-time, up-front fee to store the data forever. The protocol accomplishes this by leveraging crypto-economic game theory and creating an endowment to compensate miners for ensuring data availability, reliability and permanence. It uses economics to incentivize people to store the data for long periods of time for the first time ever. This combination makes either public or private data permanent.

The incentivization is thus part of the core layer, baked into the protocol. Data simply can’t go missing, and the network is being paid to keep it (and continuously proving they still are). Which data to recall is decided upon by pseudo-random non-predictable algorithm, so for each block, the entirety of the network must include a hash of the underlying data, fetching it from other peers, should they not have it locally, and as such, the data is organically replicated across the network.

Consensus Mechanism

Proof of access is a novel consensus mechanism that produces a positive externality of data storage. Instead of competing to burn as much electricity as possible (as in Bitcoin), miners compete to provide as many replications of the data held in the system as they can. Further, as the blockweave expands in size, the amount of electricity expended in the mining process decreases.

The nodes are periodically asked for a random piece of data from the entire Arweave network. If the node can verify they are storing that piece of data they get awarded extra AR tokens. This means that every single node on the network benefits from holding a copy of the entire data.

Wildfire is the Arweave’s self-organising network topology system. It ensures that miners are selfishly incentivised to store and share data as quickly as possible with other miners in the network, in order to build a positive reputation. While more complex under the hood, Wildfire can be summarised as: ‘if you share with me, I will share with you’.

As nodes in blockweave networks require fast access to data in order to mine efficiently, they are selfishly-incentivised to give data to other members of the network promptly and continuously, autonomously improving the sharing to lightning-fast speeds.

Tokenomics & Sustainability

The formula used for the fee calculation in essence incentivises the nodes to hold the data for well over 200 years. The fee takes into calculation the rising price of AR over time alongside the decreasing price of physical storage. And on top of that the Arweave team deliberately underestimated the decreasing cost of storage over the years in order to make it even more safe.

$AR token seems like a pure utility token so the value is not tied to the success of the project.

However, what if the endowment (prepaid funds for some file) is worth $1000 one day and $50 a year later? Can we guarantee forever storage? The logical answer would probably be no.

A potential improvement would be to extend Arweave protocol to add expiry functionality and reduce the costs for the users. After all, many users don’t really need their data to be stored for 200 years. Potential improvement can be to let users decide for how long they want to store it. Each can have individual pricing models and miners who agree to those terms. This will also improve the sustainability of the model.

Scalability

The Arweave blockchain is capable of handling over 5,000 transactions per second.

As the nodes are incentivised to hold a copy of the entire data, this creates a mechanism of redundancy and sharding — ensuring its scalability.

In order to support a network that allows for long-term on-chain data storage, the Arweave needs a system that supports unlimited sized blocks. The Arweave achieves this by using a system that decouples transaction distribution from that of block distribution in the network.

This allows only a ‘shadow’ of the block to be moved around the system (the instructions necessary to rebuild the block from its constituent transactions), rather than full block itself. This means that the information required to process large blocks can be distributed across the network in just a few kilobytes.

Gateways

Gateways are the portal to the permaweb. Every application that reads data, sends transactions, and queries Arweave needs one. They ensure that your file data is sufficiently seeded to the underlying mining nodes of the network, cache and serve your data from the moment your transactions reach the network, and provide a searchable index for the transaction tag data that powers some of the permaweb’s most essential dApps.

The main gateway most applications use is arweave.net — operated by the Arweave team. But it stands at risk of being overloaded, as it is the major access point in an otherwise highly decentralized system.

The problem is that gateways are generally difficult and costly to run and upkeep. There’s no upside for the gateway operator, either. It’s a pure expense. In the case of arweave.net, the Arweave team is incentivized to keep it alive, but unlike being a miner there’s no way for operators to earn back enough to cover costs or make profits.

Security and Privacy

When data is first written, it is signed with the uploader’s private key and is therefore guaranteed to be intact, from a security, and integrity standpoint. If a malicious miner were to misrepresent recalled data at a later date, the rest of the network would disregard this untruth easily and immediately.

Arweave is a permanent data storage, so that means it can’t be edited or deleted. However, you can upload updated versions, so you might construct a system to facilitate the appearance of mutable data, but with a permanent history of its edits.

Overall

Arweave is very interesting protocol and for someone who works in the data and the blockchain sphere, it’s the one to bookmark for sure.

--

--

Vic Genin
Vic Genin

Written by Vic Genin

🚀 Building & scaling Web3, AI & DeFi solutions | Led innovations with Binance ,GameStop & Polkadot |Tap the follow and get notifications for my new articles!

No responses yet