Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plugin idea: YA Bitcoin backend plugin #112

Closed
darosior opened this issue May 20, 2020 · 3 comments · Fixed by #248
Closed

plugin idea: YA Bitcoin backend plugin #112

darosior opened this issue May 20, 2020 · 3 comments · Fixed by #248
Labels

Comments

@darosior
Copy link
Member

We could have a plugin that completely reproduce bcli's behaviour (hit bitcoind) but which would in addition fall back to other block sources if bitcoind can't serve that block (for example if it's been pruned already). Here is an example use case of nix-bitcoin.

We could fall back to a randomly-picked explorer from a list, cross-check its hash with bitcoind and serve it to lightningd.

There could also be a fallback to the Blockstream satellite :).

@cdecker cdecker added the ideas label Aug 28, 2020
@jsarenik
Copy link
Contributor

There is no cross-reference, so adding it: https://github.com/clightning4j/btcli4j

@cdecker
Copy link
Contributor

cdecker commented Apr 13, 2021

I had something like this for a while, as a --bitcoin-cli argument, that would run against the pruned bitcoind, except for blocks which it'd fetch from a blockexplorer:

#!/usr/bin/env python3
from urllib import request
import subprocess
import sys
import json

cmd = sys.argv
cmd[0] = 'bitcoin-cli'
args = [c for c in cmd[1:] if c[:1] != '-']

if len(args) == 2 and args[0] == 'getblock':
    # Download from blockstream.info instead
    url = 'https://blockstream.info/api/block/{hash}/txids'.format(hash=args[1])
    r = request.Request(url, headers={'Accept': 'application/json'})
    content = json.loads(request.urlopen(r).read().decode('ASCII'))
    json.dump({'tx': content}, sys.stdout, indent=2)

elif len(args) == 3 and args[0] == 'getblock' and args[2] == 'false':
    try:
        out = subprocess.check_output(cmd).decode('ASCII')
        sys.stdout.write(out)
        sys.exit(0)
    except subprocess.CalledProcessError as exc:
        #sys.stderr.write("Could not retrieve block from local bitcoind, falling back to online explorers")
        pass
    url = 'https://blockexplorer.com/api/rawblock/{hash}'.format(hash=args[1])
    r = request.Request(url, headers={
        'Accept': 'application/json',
        'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36'
    })
    content = json.loads(request.urlopen(r).read().decode('ASCII'))
    sys.stdout.write(content['rawblock'])
    sys.stdout.write('\n')

else:
    try:
        out = subprocess.check_output(cmd).decode('ASCII')
        sys.stdout.write(out)
    except subprocess.CalledProcessError as exc:
        sys.exit(exc.returncode)

Can't remember how well this was working, and today I'd probably use blockstream.info as backend, but thought I'd share it here

@jsarenik
Copy link
Contributor

jsarenik commented Apr 13, 2021

@cdecker Amazing! Thank you! An eye-opener for me. So here is my take in POSIX-shell compatible with busybox:

#!/bin/sh

exec </dev/null
ADD="/$(echo $* | grep -o "testnet\|signet")"
URL="https://mempool.space$ADD"

bcli() {
  test "$TEST_GETBLOCK" = 1 && return 1
  bitcoin-cli $*
}

getesplora() {
  END=$1; shift
  #wget -q -O- --header='Accept: application/json' "$URL/api/block/$2/$END"
  curl -s -o- -H='Accept: application/json' "$URL/api/block/$2/$END"
}

ARGS=$(echo $* | sed 's/-[^ ]\+ //')
OPTS=$(echo $* | tr ' ' '\n' | grep '^-')
shift $(echo $OPTS | wc -w)
test "$1" = "getblock" && {
  ARGN=$(echo $ARGS | wc -w)
  if test "$ARGN" = 2
  then
    bcli $OPTS $ARGS 2>/dev/null \
      || { printf '{"tx":'; getesplora txids $ARGS; printf '}'; }
  elif test "$ARGN" = 3
  then
    bcli $OPTS $ARGS 2>/dev/null \
      || { getesplora raw $ARGS | od -v -A n -t x1 \
             | tr -d ' \n'; echo; }
  fi
} || bitcoin-cli $OPTS $ARGS

Works also with -testnet and -signet (though tested only on the former so far).

Recap: all one needs to do is to add --bitcoin-cli PATH/TO/bcli-esplora.sh. Simply export TEST_GETBLOCK=1 before running lightningd --bitcoin-cli ... in order to test esplora-only-getblock-mode (still needs local pruned bitcoind for hello world etc.).

NOTE: The txids part was not tested yet (manually yes, but how to trigger lightningd to ask for it? @cdecker ).

vincenzopalazzo added a commit to vincenzopalazzo/plugins that referenced this issue Jun 5, 2021
The bitcoin backend plugin is inspired by lightningd#112 lightningd#214

Signed-off-by: Vincenzo Palazzo <[email protected]>
vincenzopalazzo added a commit to vincenzopalazzo/plugins that referenced this issue Jun 5, 2021
The bitcoin backend plugin is inspired by lightningd#112 lightningd#214

Signed-off-by: Vincenzo Palazzo <[email protected]>
vincenzopalazzo added a commit to vincenzopalazzo/plugins that referenced this issue Jun 9, 2021
The bitcoin backend plugin is inspired by lightningd#112 lightningd#214

Signed-off-by: Vincenzo Palazzo <[email protected]>
cdecker pushed a commit that referenced this issue Jun 25, 2021
The bitcoin backend plugin is inspired by #112 #214

Signed-off-by: Vincenzo Palazzo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants