The Dynamics of Private Mempools and Their Implications for Bitcoin

In this article I'll be looking at the dynamics of private mempools, and the implications that has for the utility of the public mempool, mining incentives, and the health of the Bitcoin network overall.

In the last Mempool article, I went through the dynamics of transaction propagation when different nodes on the network are running different mempool relay policies. In this piece I’ll be looking at the dynamics of private mempools, and the implications that has for the utility of the public mempool, mining incentives, and the health of the Bitcoin network overall.

At the heart of the purpose of the mempool is facilitating the aligned incentives of two different parties, miners and transacting users. Users want to transact, and are willing to pay miners’ transaction fees in order to do so. Miners want to make money, and transaction fees are an additional source of revenue in addition to the new coin subsidy in each block, as well as a necessary primary revenue source to cultivate in the long term as the subsidy dwindles.

Bitcoin is a system secured by incentives. This core dynamic is what drives the security of the system, you have a customer(s) and a provider, and the two of them attempting to fulfill their wants and needs is what ensures the blockchain continues ticking forward with a sufficient amount of thermodynamic security.

Attempts to introduce friction into this facilitation mechanism does not ultimately do anything at all to change the incentives of these two parties. A user who wants to make a certain kind of transaction is still going to want to make that transaction, and pay for it. A miner who is willing to accept those kinds of transactions is still going to want to accept them, and collect the fee by including them in a block.

If the transaction is valid, then these two parties are still going to have their unmet wants and needs, and are still going to be strongly motivated to meet them in some form or fashion.

Miner APIIndividual end users are not necessarily capitalized enough or competent enough in order to route around friction artificially introduced between both ends of a coincidence of wants, but miners most definitely are. As the old adage goes, “if you build it, they will come.”

The preferential situation for miners is obviously to acquire fee paying transactions in-band through the public mempool. It requires the lowest overhead possible for them, simply running a standard Bitcoin client out of the box, it is a very resilient propagation mechanism that ensures a very high degree of reliability in getting miners the highest fee paying transactions, and they don’t have to do anything. Just download the client and run it.

However, in a very hostile environment such as a network wide effort to filter consensus valid transactions during their propagation across the network, that traditional assumption can be drawn into question.

In such a scenario miners have every incentive to set up out-of-band mechanisms for accepting transactions that are not properly being relayed across the network. Marathon’s Slipstream API for non-standard transactions is not the only example of this. There is in fact a long standing precedent from almost ten years ago that was widely implemented by many mining pools, and still exists to this day. Transaction accelerators.

We now live in a world of Full-RBF, where any transaction, regardless of using the historical “opt-in” flag, can be fee-bumped. Any node who has upgraded to Full-RBF will relay any transaction that is spending an unconfirmed output already pending in the mempool as long as it is paying a higher fee. This has not always been the case. Historically only transactions that were originally made with a flag to opt-in to RBF use could be replaced and expected to propagate across the network.

Transaction accelerators were created by miners in order to facilitate this behavior for transactions that did not opt-in to RBF use.

Third Party APIsWhile the overhead is not exorbitantly high for a miner or pool to create their own transaction submission API, it isn’t free. It still does require at least one developer and time to go through the design and release cycle of any piece of software. The curve isn’t particularly exaggerated, but still does favor larger miners over smaller ones in terms of how much resources they will have to devote to such an endeavor.

Mempool.space has proven that it is a viable endevour for a third party unrelated to miners to create such an API, allowing miners to simply connect to their service rather than expend the effort to create one themselves from scratch. This does have its issues though, such a third party is not going to build and operate such a service for free. They will want their cut.

There are two ways that this dynamic can go, either these services wind up requiring a higher cost in order to allow both the miners and service providers to earn revenue, or miners will have to share a smaller cut of the revenue in order for such services to remain competitive with directly miner operated ones. This means miners using a third party submission API rather than their own will earn less revenue than the miners operating their own API.

Private Order FlowEither of the above possibilities introduces serious problems when it comes to the overall system incentives, reliability of end-user software, and potentially even the security model of second