Can someone please explain FIBRE to me like I'm 5 and why is it useful?

Question

I read the website but they imply you already know why this is valuable. Can someone please illuminate me on this issue?

Answer 1

First the "why it matters":

Fibre (Fast Internet Bitcoin Relay Engine) is a protocol which attempts to deliver Bitcoin blocks around the world with delays as close to the physical limits as possible.

This is very important because Bitcoin mining is intended to be a fair lottery: If you have x% of the hashrate you should find x% of the blocks, on average. But when it takes time to communicate blocks mining acts more like a race than a lottery: the bigger miner gets an unjust share, creating a pressure to centralize.

This is because when a miner makes a block other miners can't work on extending it until they've heard of it, during that time they might make their own blocks. If you are a small miner and make a block this works against you because no one attempts to mine the successor to your block until later. If a larger miner mines a block it also works against you, because you've fallen behind.

This race behavior is the same that enables selfish mining.

The race effect can also benefit attackers trying to overpower the network, because they don't need to compete with hashrate lost to stale blocks.

Improving propagation speeds reduces the race and makes mining more fair, lowers pressure to centralize, and improves network security.

Now, "why is it hard":

The world is a big place and a round trip from San Francisco to Australia is well over 200ms typically. These delays are substantially due to speed of light in fiber and while better routing can improve them somewhat, they aren't going away without through-earth-neutrino communications or other stuff from science fiction.

International internet links frequently have packet loss. If you send a message on a 200ms RTT path you can't hear about any missed packets for 200ms, and they can't get any replacements for another 100ms. Traditional transmission protocols like TCP do a poor job of achieving link capacity when delays are high especially if there is any loss.

In theory the problem is aided somewhat because 99.9% of the block data is already known because the block mostly contains transactions that have already been broadcast, but unfortunately a node sending a block doesn't know which transactions you'll be missing by the time their block shows up and you telling them will take too long just due to the above mentioned round trip times.

What Fibre does do to solve it:

Fiber lets you effectively build a distributed node that internally moves a block around the world very close to the lowest one way communications delay between the computers comprising it.

The author of fibre used to run one such node on public systems which were well maintained and anyone could connect to. The normal Bitcoin P2P protocol, with BIP152, was used for the last mile-- since that is what BIP152 is optimized for.

Similar to BIP152 (which it was co-developed with) Fiber first sends a short block sketch: A list of short hashes and lengths that allow the far end to use its mempool to lay out transactions into a block, with holes in positions where the transactions were missing or the hash was ambiguous.

Then Fibre breaks the blocks into packets and sends error correction data. The error correction lets the receiver recover the block as soon as it has received as many packets as missing chunks, even though the sender doesn't know whats missing. Error correcting coding is also used on the initial sketch. This is similar to how Raid or PAR encoding works, but the implementation is made much more complicated by needing to be very fast.

All of this communication happens over UDP with pre-programmed transmission rates, so it doesn't need to ramp up slowly or wait for lost packets.

The sender also sends different chunks to each of its Fibre peers and those peers themselves relay the chunks among each other as soon as they get them, in order to make use of all the interface bandwidth available. When a Fibre peer gets enough to decode the block they do, and start emitting novel chunks.

This process avoids any duplicated data being sent and is highly robust to packet loss as well as link and node failures and achieves very consistent delay.

At every step Fibre trades off bandwidth for latency and can send several times the size of the block data in order to achieve the lowest delay across the long-haul links that it rides over.

There is a lot more to the details of it, probably enough for two dozen questions.

Where did Fibre come from?

Fibre was built by Matt Corallo and was the product of several years of experience running earlier fast relay networks as several years of research into fast block propagation techniques. Matt took many ideas that were far too computationally slow to be practical, solved the remaining engineering challenges, optimized the heck out of it and then put the result up on a well curated public system. Good network management and sysadmin work are almost as important to getting low delays as the protocol itself.

Answer 2

FIBRE is valuable as it allows for miners to extremely quickly propagate their blocks to other miners and to the rest of the network. The goal of FIBRE is to reduce latency in block transmission. Miners will thus waste less time on trying to mine a block that has already been found so they will reduce their stale block rates and be working on the latest block almost as soon as it is found and broadcast to the network.