Compact channel.db before lnd-rescue backup?

[rootzoll]

I can see on some testdevices the channel.db getting very big. That slows down the backup process thats recommended during update, etc so far that you get the feeling the system is on hold.

@openoms Is there an easy way to compact (reduze in size) the channel.db before we make a lnd-rescue file?

[openoms]

Surely doable and built-in to lnd, needs running lnd with the autocompact flag like:

sudo -u bitcoin /usr/local/bin/lnd --configfile=/home/bitcoin/.lnd/${netprefix}lnd.conf --db.bolt.auto-compact

and wait the message when the compacting is finished.

Running it the first time will take long as well, but the resulting file should be less vulnerable.

Should probably be a separate script which could be called from REPAIR also.

[rootzoll]

OK did a test run on my testing node ... the results are great .. the resulting file is over 78x smaller.

This was the result message (maybe to check for success):

2021-12-07 09:35:24.471 [INF] CHDB: DB compaction of /home/bitcoin/.lnd/data/graph/mainnet/channel.db successful, 9894187008 -> 125960192 bytes (gain=78.55x)

BUT that command is really hard to put into a RaspiBlitz script, because it keeps runnig after that with wLND waiting to get wallet unlocked and is not finishing:

2021-12-07 09:36:06.851 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access

So it would need a more complex monitoring script - running compacting in the background with writing logs into file and monitoring for success or error and then shutting down the lnd again.

@openoms Basically I get the feeling - this is not how the compacting feature is meant to be used. Maybe it makes more sense to run the lnd in general with the config --db.bolt.auto-compact-min-age= .. so that it get compacted on every restart after a certain time limit? So that way it would also happen if people simply restart their RaspiBlitz or even when they falsh a new sd card and recover. What was your concern of not switching this feature on by default on RaspiBlitz?

[openoms]

My concern with using it by default is the lack of control. It might take a half an hour when one only needs a quick restart (especially that there are plenty of unnecessary restarts still).

Did you look at the PR #2761 yet? Can test from the repair menu.
I think not having an error can be considered success in this case. If the compaction is not successful the database is not copied over and if compacting was done within the default 168hours it doesn't run either (and there is no benefit to run it more of often I think).

[rootzoll]

Ah OK ... havent checked the RP #2761 yet (insert tomato eyes smiley here) .. so perfect. Lets offer it as a repair option with the v1.7.2

[rootzoll]

Merged PR to dev and changed the interaction to a whiptail dialog so that it fits better into update process dialogs.
TODO: Test Update-Process

[rootzoll]

On my test when using the "UPDATE > RELEASE" option from menu (and then saying DOWNLOAD BACKUP) it compacts but then is exiting the dialog - seems to send a terninate signal somehow.

@openoms that killall command seems to have side effects - is it possible somehow to traget it more to the lnd process? https://github.com/rootzoll/raspiblitz/blob/95f979c7342e556939e7ebe1dd821af7724fa2c1/home.admin/config.scripts/lnd.compact.sh#L81

[openoms]

tested ok now.
Also now both rescue files are offered if LND and CL are active (not just the default).

[rootzoll]

Check if the pre-compact backups are not part of mnt/hdd/lnd/ directory anymore after compacting .. so that they dont make the lnd-rescue file bigger. If they are still there after compact, move them to a directory out side lnd data dirs (e.g. under data-storage that will not be part of migration to free space)

[rootzoll]

Ah OK ... I dont see any backup filese in the data dir. Putting back to final testing.

[rootzoll]

Tested this multiple times - looks good. Closing issue for release.