PIPv1 HeadersProof Result is Empty for 'Recent Blocks'

[EBGToo]

• Parity Ethereum version: 2.1.3
• Operating system: MacOS
• Installation: built from source
• Fully synchronized: yes, archival
• Network: ethereum
• Restarted: ??

When making a PIPv1 HeadersProof request for specific block numbers, the response is empty (RLP '[]' - not a proof) for recent blocks.

Here are two requests for block 6,000,000 and 6,606,849:

ETH: LES: Send: [ PIP,   Headers Proof ] =>       127.0.0.1
ETH: SEND: L  2: [
ETH: SEND:   I  1: 0x12
ETH: SEND:   L  2: [
ETH: SEND:     I  1: 0x05
ETH: SEND:     L  1: [
ETH: SEND:       L  2: [
ETH: SEND:         I  1: 0x01
ETH: SEND:         L  1: [
ETH: SEND:           L  2: [
ETH: SEND:             I  0: 0x
ETH: SEND:             I  3: 0x5b8d80
ETH: SEND:           ]
ETH: SEND:         ]
ETH: SEND:       ]
ETH: SEND:     ]
ETH: SEND:   ]
ETH: SEND: ]
ETH: LES: Send: [ PIP,   Headers Proof ] =>       127.0.0.1
ETH: SEND: L  2: [
ETH: SEND:   I  1: 0x12
ETH: SEND:   L  2: [
ETH: SEND:     I  1: 0x06
ETH: SEND:     L  1: [
ETH: SEND:       L  2: [
ETH: SEND:         I  1: 0x01
ETH: SEND:         L  1: [
ETH: SEND:           L  2: [
ETH: SEND:             I  0: 0x
ETH: SEND:             I  3: 0x64d001
ETH: SEND:           ]
ETH: SEND:         ]
ETH: SEND:       ]
ETH: SEND:     ]
ETH: SEND:   ]
ETH: SEND: ]

and the two corresponding responses:

ETH: RECV: L  3: [
ETH: RECV:   I  1: 0x05
ETH: RECV:   I  8: <credits>
ETH: RECV:   L  1: [
ETH: RECV:     L  2: [
ETH: RECV:       I  1: 0x01
ETH: RECV:       L  3: [
ETH: RECV:         L  6: [
ETH: RECV:           I 38: 0xe58300835ba04084c38086a9b1456e9b648300f9e2470168f22beac4b03d95d3e8bed074f0b8
ETH: RECV:           I 83: 0xf8518080808080808080a010a6cd1ba8c4a57be2a094ba6270db37ff1748e56d497938c845f44ce86e907ba0f6328d1d320c6587b08506d921446c66495b6ac6cd712df5a7ea3fa11021505480808080808080
ETH: RECV:           I276: 0xf901118080808080808080a0ff15eb64d26ad0978cb4bb6cbc6e05cf7a43d1612a77fee6e915808f8a6d97fda0ca2c24a7edb80056927dcfc6d07bb782ee4b7fb6d25322aca132329c9e03e448a0587fe5a21b76c5b8c7b46c96bd37d13e4806b7fa490c418447b7a9a054b52c25a0a312a21fb72aa1f0bd7352993c24753998fbcc1364174311669a637c31177f36a0a18a50c27fbb63aec45c4c72933b49b109b05b77798397d2c3cb457c2c4f07b1a0bb8ead8afdcf9a3eceb556753df8b1e4deae85d82ff173b6fc1ce396bcbde3c5a0116a31e5204d4a728b995dc50989d918ecd85cf4a4f434d4f19559d5ceae96c5a0d264167d8a2ae995abd941ab9eaf89554dbfed258fad82ccd926699fc94b86c180
ETH: RECV:           I532: 0xf90211a0799e19407900f380fa03beef60317c5535b3fc7a99d19de91e5260ac45be52d2a0ed032576789000de4f7044338dac591015c0e93a95f0e3fba2c75d096cd18577a0ae21e63c59c6206db398bc85f7e362b15a438c64f277326b146f3d45b679b02ca01882366db5aac47c24e42eb4cd031466f6cfbcf5234d91b37315f0a93d072ccca038e75859f69c3683519edef9b427963798196b8d238db54402a78e3313a7f7a9a01532205f178581ec5348d7e2e1c2b4ec8d83eaa3477575f267b0afa450b39105a0ea0a6bcee3798ed2bcfcc91ba0e854f4e9cdebd0903702850ae62cd6974e0d64a0a1914ce8c4526fc3c5d31520043009c31ab39eda2284d63a1f1aeaaba9a6b931a0a1d61fc5c41439d607145136363dcdeb589b60fca3a4f969727d7108d4e4f7b5a00fd74bd9e94cd662cfefcfd86c5aa3dd8a92f844597600eada0072da59709343a01a61edb3499ea2c4fe97a59b5d1fea2a51b22c47a058fa9ce50c905ac130b8e5a02b1aff5b34c5979a2683254939fa86993322b75851c2c9c28381df9b50fc5beda0e4e8dae4880edac061bfada2dc7aa59b2c84d4202d60987a632b6b14b8575317a06bbe828ea668db1f5e092cd0c576bed2de65d256957aab26f6e0eaffd503126aa03174cd8035201de446ffc6855d37483a4fe40b373bc943aacb7c49863462e4cfa06b383e840a6d2ba8380dfaf409...
ETH: RECV:           I532: 0xf90211a01d05db3dea64838a4b2dd7419502f608824ad91d9fdde4441dd26c95940e96c6a02a3ca3e7c05e1beab30c456943cd80352e54f4f0342e4a86b8cc62a3dc7cd849a081524ad3f0ffb3cc46b554381557bb316e75315ebe7fe2d8c2cedaf7c8ea1202a0f5e82b00c6e39cdbbab75f3e2e4788c58afe84a70306ba64f45010a591408f60a09f43b0de95faea5da99b223bc93c86da8174a97e3cdc389636847a527713d5c7a04c773b3e4d761dbcfa23c5426541e50d9c4294185e48153ef9e7f83c858dec7aa0bb69a292f1fbd15cf192d410e726e2c47d93a7b12f66bec61b396f6ec20262cfa0c1114249fa7e800838329e9b3003da9e9615250cfcf214e5f588b7c9d7685137a0fea7ee9c4bef49699c33cd70612d5d13935628b6e3c9b440f9713fe2f757620ea071b2707a21c3ba7a03727eb05fc2490a7c09a8705f87f7de1da0bd38a13ff15da0238cc7873f3dbd1cca5cfcb4e1e3ec6b4a49af1302658097d70681c49e407973a0f1b17071bf1293ed32744cc123281e6d668682a41c56cf04e3ef2a9bc367f7f6a04f24848c5026dadd6b0770b4d8db5a4a33ca82649cce26b195e3346c3aff9d18a0cca62c3e13321dac0586b67e63b5ab6a81775a3563ece21d9f0182cc40dfc803a01a213d2ef5da28c52a6744d30a9e6e25187834859ff8675ead6dc99525ada2c4a0fd87e7af5749c4b6ace534899b...
ETH: RECV:           I 48: 0xef20adeca0be847be2bceb74e660daf96b3f0669d58f59dc9101715689a00ef864a5408f438a012950aa5d90bd65c91b
ETH: RECV:         ]
ETH: RECV:         I 32: 0xbe847be2bceb74e660daf96b3f0669d58f59dc9101715689a00ef864a5408f43
ETH: RECV:         I 10: 0x012950aa5d90bd65c91b
ETH: RECV:       ]
ETH: RECV:     ]
ETH: RECV:   ]
ETH: RECV: ]
ETH: LES: Recv: [ PIP,   Headers Proof ] <=       127.0.0.1
ETH: RECV: L  3: [
ETH: RECV:   I  1: 0x06
ETH: RECV:   I  8: <credits>
ETH: RECV:   L  0: []
ETH: RECV: ]
ETH: LES: Recv: [ PIP,        Response ] <=       127.0.0.1

These are requests to my localhost archival node. The node produces valid proofs for blocks #1 and #2, as well as numerous others. Last week I noticed the same behavior (when the block chain head was less than it is now) whereby a request for a recent block produced '[]'.

I've observed the same behavior for our corporate, cloud-accessible Parity archival node(s).

[If the CHT is rebuilt periodically, then a) it should be rebuilt on demand if a request arrives for a block not yet incorporated, or b) the interface needs to state the algorithm by which one can determine if the proof will be non-existent. I'd expect 'a' - when a block is chain/announced, it has a valid proof (after all it was chained); that proof should be provided).

[cheme]

CHT is only calculated every 2048 blocks and with a forced 2048 blocks delay, so it looks like an 'expected' behavior.

Indeed Recalculating CHT at each blocks seems a bit risky to me
(https://github.com/zsfelfoldi/go-ethereum/wiki/Canonical-Hash-Trie
expects that 2048 block is enough to avoid chain reorganization)).

If I check in recent geth code base, it is the same :
https://github.com/ethereum/go-ethereum/blob/b7bbe66b19fb19b95053977586e619a40478a7d8/les/odr_requests.go#L368
with
https://github.com/ethereum/go-ethereum/blob/b69476b372a26679e5bdb33db3d508f2c955e7ff/params/network_params.go#L49

So CHT for the last 2048 blocks should not be queryiable.

EBGToo, was your test within the latest 2048 blocks?

---

Relevant parity code :
https://github.com/paritytech/parity-ethereum/blob/59daf958595ccd064f4a67ea75120558182f379e/ethcore/light/src/client/header_chain.rs#L48

At the time the wiki is not explicit about it :
https://wiki.parity.io/The-Parity-Light-Protocol-(PIP)
In fact it should appears in the LES page, I will do a PR.

[EBGToo]

So CHT for the last 2048 blocks should not be queryiable.

EBGToo, was your test within the latest 2048 blocks?

Yes