What is the Bitcoin Mempool?
The term “Mempool” is certainly not one of the more well known buzzwords in Bitcoin, however it’s an important technical aspect that you need to understand if you want to know how Bitcoin transactions work.
While some have leveled criticisms at the current design for slowing down the network and sometimes being subject to spam, it helps maintain the integrity of the entire system.
The memory pool (Mempool) is a collection of unconfirmed transactions that await confirmation by all Bitcoin nodes. Think of it as a waiting area for unconfirmed transactions, and if the network is ever overloaded, the list can be quite long. Once it is confirmed, it is cemented into the blockchain and will remain there as long as the network is alive.
The change of transparent (yet optional) memory pools came with the BIP 35 update, which extends the “get data” function to include querying transactions from the memory pool. Since anyone running a full node on their computer or server can query the Mempool, statistics about unconfirmed transactions are made public.
When a user sends a transaction from their wallet to another address, transactions are queued in a list to be confirmed by nodes. After the first confirmation, it then goes into the Mempool to await confirmation by miners.
A Bitcoin node will have a series of checks before letting a transaction into the Mempool, which may include:
Orphaned transactions are those missing their parental funding source, and the network rules indicate that they must not be propagated to the network. The node may run the steps above until the parent is found, or until the node decides to reject them.
If the transaction meets the pool’s criteria, it will add it to the memory pool and broadcast it to the network. Some criteria can be changed at the whim of a node’s owner, and there may be custom functions involved.
It is best to visualize a Mempool as a prioritized task list for miners to pick up transactions as they move onto each block. Since each block has a limited size, an unusually high amount of transactions will cause the list to get larger, and will likely make confirmations longer.
A nifty feature about Bitcoin is that a user can choose their transaction fee, and the higher it is, the more likely miners will prioritize their transaction. Tacking a high transaction fee compared to others within the Mempool will likely make yours go through over others. Even after being moved into the next block, transactions will continue to be verified to maintain network integrity.
It should also be noted that it’s completely optional for a node operator to enable the Mempool, but transactions will not be made transparent until they’ve been confirmed. There is no consistency of transactions between nodes, and most Mempools will likely look completely different from one another.
Bitcoin nodes may hold their own Mempool, and their sizes will differ by location. These nodes also prioritize transactions and set minimal transaction fees as a means to regulate an overloaded Mempool.
Once miners pick up the transactions and the node receives a new block, the node will wipe out transactions added to the previous one, along with any conflicting transactions. This is why memory pool graphs commonly have sharp drops and rises.
By default, unconfirmed transactions will drop off a nodes Mempool after two weeks if no miner picks it up to be confirmed. This is really up to the node operator, and there are indeed nodes with unlimited time limits that will hypothetically store a transaction forever.
If you find yourself with a stuck transaction for an exceedingly long time, you can pay an accelerator service to push the transaction through for you (usually mining pools offer this service). But the better thing to do is flag all transactions you make as RBF or “replace by fee’. If your transaction gets stuck, you can just just replace it with a different transaction with a higher fee set, killing the original but using the same inputs.
While it won’t matter much to the average Bitcoin user, miners and mining pools are especially important. If a node receives more large transactions, then miners will have more opportunity to obtain rewards by confirming them.
Since Bitcoin is a decentralized, Proof-of-Work system, the Mempool is an important buffer to the integrity of transactions. Transactions need to be verified by miners before added into the next block in the blockchain.
Since the fee structure is based on supply (block space) and demand (transactions), the transparent Mempool is important to help nodes, wallets, and exchanges set transaction fees to assure reasonable delivery times.
Those willing to contribute more fees towards miners will be rewarded with speed while helping to maximize the usage of blockspace.
For example, imagine the pallate below is a block and each box sitting on it is a btc transaction. By using the highest fees as the standard to get your box on the pallete, you ensure that the block space is used most efficiently. This is actually very similar to how shipping prices are determined in the real world.
Unlike hosting a mining pool or Lightning Node, there is no direct financial incentive for hosting a full node with a memory pool. This is why the Bitcoin network depends on enthusiasts to volunteer their network and computing power to create more nodes.
In a free-market fashion, fees adjust based on supply & demand. If the memory pool is overburdened and there is a lack of miners in the network, the fees can get absurdly high.
Miners are incentivized to confirm transactions since they get a share of those transaction fees after every block mined.
While a user can set their custom fee amount, Bitcoin exchanges and online wallets tend to give users recommended fees so their transaction is completed in a reasonable amount of time.
It is entirely possible to send off a zero-fee transaction and have it confirmed, but it’s also possible to have it sitting in the queue for ages. Much like traffic on a highway, how easily you can use it depends on how much other people want to use it. And lile toll roads, you can sometimes pay to go around it.
To get an idea of how congested a transaction is within the queue, one will have to look at the current Mempool size, which varies drastically by day. For example, a Mempool size of 3 megabytes will take 3 blocks to confirm.
Bitcoin Cash and other forks of Bitcoin have experimented with the idea of increased block sizes to alleviate congestion in the Mempool and keeping fees quite low. While it may give temporary decreases to fees, it also increases the resources needed to operate full nodes, which goes against the decentralized ideology of Bitcoin.
The Lightning Network is a relatively new concept to help users bypass high fees and overburdened Mempools. The sender opens a payment channel with someone elses node and they create a series of IOU’s.
It is sort of similar to a tab you run up at the bar. Instead of paying for each drink as you drink them, the bar just keeps track of everything you’ve ordered and you pay it all at once at the end. Lightning channels work the same way.
The memory pull is full when it reaches the maximum size set by the node operator, which can be from thousands to millions.
If there is no limit, it can be considered “full” when the transactions exceed the current network’s capacity to process them.
While it is possible to set any limit or none at all, the limitations in hardware and network infrastructure could contribute to a perceived limitation.
Usually, the Mempool is mainly stored in RAM, and a dedicated server with extremely high ram is desirable, yet expensive. An overloaded server could result in crashed software and a wiped-out memory pool, acting as a speed bump for the entire network.
To mitigate crashing during high-traffic sessions, a minimum fee threshold is set up to prioritize transactions. For those with zero-fee or very low fee transactions, it can be seemingly impossible to send a transaction in an overloaded network. By default, the transaction fees scale back down to the “minrelayfee” value set by the node.
There are outside services, known as Bitcoin Transaction Accelerators, that use dedicated mining power to confirm a single transaction. This is usually hosted by large mining pools, and they work by selectively including the transaction in the upcoming block. Of course, The fee is rather expensive compared to paying a normal transaction fee.
It’s possible for someone with deep pockets to spam the network with what’s known as a flood attack. This is when someone sends lots of low-value transactions, causing nodes to fill up and leave a huge queue in the Mempool. This is why some users still campaign for increased block sizes to mitigate such attacks. However, demand for block space is endless when it is free, so increasing block size would not actually decrease the spam.