FAQ: Speeding up IBD (a full node's first sync)

Why is the first sync slow?

When you fire up the BTG Core Wallet for the first time, you are starting a Full Node on the BTG network. Instead of trusting anyone else for the state or history of the BTG blockchain, it can validate everything itself, going back to the first Bitcoin blocks from 2009.

(This is technically called IBD, for Initial Block Download.)

How? It connects to the network of BTG full nodes and asks them for the historical blocks, and then validates every transaction and every block right up until today. But this can take a long time!

On a slow computer with a slow hard drive, it can take a week, or even weeks. Network bandwidth is rarely a problem, and while a slow networks connection might also hamper you, the slowdown is usually in the validation work. Your node will connect to many other nodes and download some blocks from each, which is generally faster than your computer can process it all.

The number one reason for slow initial sync is the constant need to look up prior transaction outputs from the chainstate database - each new transaction uses prior outputs as its own inputs, so a full node must keep track of all of the valid outputs in history in a database it can check. This database is referred to as the chainstate. As I write, the chainstate folder takes up about 2.52 GB.

How can we speed it up?

Caching! If we can give the node more RAM in which it can cache a copy of that chainstate data, it won’t have to go looking it up in the database on your drive.

Providing more cache can dramatically increase your validation speed, even if your databases are already on a fast SSD because RAM is still much faster, but if your database are on a hard drive, the boost is even bigger. People often go from needing 7 days to sync on a modern desktop computer to doing so in < 12 hours.

Here’s all it takes

Add a line like this to your bitcoingold.conf file:

dbcache=2000

The 2000 means to lock up 2000 MB (about 2 GB) of the RAM for the cache. If you don’t tell it to do this, the default is only 450 MB… which is plenty for use after you are all synced up, and leaves memory free for you computer - why waste RAM? But increasing it temporarily while doing the initial sync will make it much faster.

You can set dbcache even higher if you have lots of RAM… but the bigger you go, the less additional performance gain you get. Most of the gain is there once you get to around the 2 GB level.

Important: if your computer doesn’t have much RAM, don’t lock most of it up for the cache! If you only have 4GB and you lock up 3GB for this cache, the remaining RAM may not be enough for the rest of the full node and the rest of the computer’s system processes, never mind anything else you want to run. You can slow down the computer so much that your sync actually takes longer.

And remember when sync is complete to remove that line from your config so that you’re not wasting all that RAM any more!

Examples, using the Bitcoin Gold GUI

Go into Setting, Options:

image

Then pick Open Configuration File:
image

Hit OK, and you’ll see your config file in a text editor.

If this is a brand new node, it will probably be empty. Just add this text:

image

Save the file and close the text editor. Then exit the node (File, Exit), wait for it to cleanly shut down, and then start it back up. When the node starts up, it will read the bitcoingold.conf file and act accordingly.

If you already have other stuff in your config (maybe you’re duplicating what you set up elsewhere), you can add the dbcache command in the top section:

image

or somewhere inside the [main] section:

image

Whichever you do, just make sure to go back and undo this after the node has completed syncing up!

To undo this, simply:

  1. Edit the file to delete the dbcache line and save it,
  2. Restart your node so that it re-reads the file.

If running your node from the command line

Simply add -dbcache=2000 to your command line switches. This should work whether you’re running the GUI or just the bgoldd daemon.

1 Like