Lightning Node High Availability/High Throughput

Question

Are there any resources/best practises on running a highly available/high throughput lightning node? If my lightning service where to take off, should I be running multiple lightning nodes or a single lightning node? Would it be possible to scale horizontally? Maybe each channel could be managed on a different machine?

Answer 1

So there are a couple things to consider here:

1. The database, it needs to be remotely available if you are going to failover your Lightning node. We talk about adding support to eclair for a remote database (postgres) in our blog post here
2. Safely accessing the database. If you have two nodes writing to the database at the same time, it is possible that they write conflicting versions which could result in a penalty transaction on the blockchain. This means you need some sort of locking mechanism to access the database. We have implemented this on eclair as well, you can find the blog post about it here and the actual PR on eclair here.
3. The final thing that needs to be considered is how to make the key material available in a highly available, yet secure way. This is still something we are exploring at Suredbits. The "dumb" thing to do is just replicate the keys across every node you have.

Now you should be able to spin up an arbitrary amount of nodes. There should be 1 master node that is actively updating the database, and n shadow nodes that are waiting for the master node to fail. For implementing this, you can use something like Amazon's ELB and health check's to determine when the current master node fails, and you should promote one of the "shadow" nodes.

You can see a demo of this here on youtube

Answer 2

Are there any resources/best practises on running a highly available/high throughput lightning node?

In Lightning Network, any disturbances (electricity outage, server crash) while running the application can result in permanent loss of funds. It is thus recommended that you run your server in a managed facility like cloud or colo if you are not able to provide high reliability of the servers yourself. This should especially be true in your case since you will be running multiple nodes that will route payments. Any server malfunction can have direct monetary losses.

For attaining high throughput (higher lightning routing) you can use autopilot feature to determine which nodes to connect to and what amount of liquidity you must have in each channel. The things that you might want to consider include increasing the centrality of your node, increase probability to have well funded routes, and improve your presence within your own geography.

If my lightning service where to take off, should I be running multiple lightning nodes or a single lightning node?

Depends on the service that you want to provide. If you want to provide a custodial solution, then running one lightning node may be considered. However, increasing the number of nodes will increase the probability of your routing capabilities, prevent bottlenecks and hence provide a better customer service.

If you want the users to be in charge of their funds, you have no option but to use multiple lightning nodes, one for each user.

Maybe each channel could be managed on a different machine?

No need to run different nodes on a different machine. You can create separate directories within the same machine and run the lightning node.