Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty block objects constructed by upload-bin command #3630

Closed
AliceInHunterland opened this issue Oct 21, 2024 · 1 comment · Fixed by #3633
Closed

Empty block objects constructed by upload-bin command #3630

AliceInHunterland opened this issue Oct 21, 2024 · 1 comment · Fixed by #3633
Assignees
Labels
bug Something isn't working I4 No visible changes S4 Routine U1 Critically important to resolve quickly
Milestone

Comments

@AliceInHunterland
Copy link
Contributor

Current Behavior

SkipIndexFilesSearch: true

2024-10-21T12:39:15.137+0300    INFO    persisted to disk       {"blocks": 1, "keys": 20, "headerHeight": 88000, "blockHeight": 88000, "took": "17.379834ms"}
2024-10-21T12:39:45.864+0300    ERROR   failed to read block oid        {"oid": "ArKpmCjCCXFQrLWkrhzBgcHLMfd51qANurzT1ekzXNhK", "error": "EOF"}
2024-10-21T12:39:45.864+0300    INFO    shutting down NeoFS BlockFetcher service        {"force": true}
(base) ekaterinapavlova@MacBook-Air-4 neofs-node % ./bin/neofs-cli object head -r st1.storage.fs.neo.org:8080 -w /Users/ekaterinapavlova/Workplace/neo-go/panelwallet1.json --cid HmP48JvCcmiEbJemG7jFNEg2qkdCTx4thD3EwWUi288Q --oid ArKpmCjCCXFQrLWkrhzBgcHLMfd51qANurzT1ekzXNhK
ID: ArKpmCjCCXFQrLWkrhzBgcHLMfd51qANurzT1ekzXNhK
CID: HmP48JvCcmiEbJemG7jFNEg2qkdCTx4thD3EwWUi288Q
Owner: NS1QWpgWG5YWud59RdSrAgwMdD6UVkdkSh
CreatedAt: 1
Size: 0
HomoHash: 00000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001
Checksum: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Type: REGULAR
Attributes:
  block=99318
  primary=2
  hash=df52e560a0c08d19830c9f5cf1b0b9a77e7f12a0b3e91cd0fe75a8a2e908b6b3
  prevHash=15d4d8cbc4306df06429503d85d855a425b6464590dee83dba158cda3831fccf
  timestamp=1652400646552
ID signature:
  public key: 03097d44b8b933c77c52ffdda0acffb2a18ba4f092ba5de0c689d360fa55b9b00f
  signature: 0206f37e2ce2282a0ee5de9beb401c543356cd89f36c059c642668bccb7e30fca3e5b35c7bcb933fac8b273ace7b68a7485ea9877c27c0afa43293e5754a9911
2024-10-21T12:39:15.137+0300    INFO    persisted to disk       {"blocks": 1, "keys": 20, "headerHeight": 88000, "blockHeight": 88000, "took": "17.379834ms"}
2024-10-21T12:39:45.864+0300    ERROR   failed to read block oid        {"oid": "ArKpmCjCCXFQrLWkrhzBgcHLMfd51qANurzT1ekzXNhK", "error": "EOF"}
2024-10-21T12:39:45.864+0300    INFO    shutting down NeoFS BlockFetcher service        {"force": true}

Expected Behavior

not empty block object

Context

upload-bin command uploading blocks as objects.

@AliceInHunterland AliceInHunterland added bug Something isn't working U1 Critically important to resolve quickly S4 Routine I4 No visible changes labels Oct 21, 2024
@AnnaShaleva AnnaShaleva added this to the v0.106.4 milestone Oct 21, 2024
@AnnaShaleva
Copy link
Member

AnnaShaleva commented Oct 21, 2024

Uploader strategy (reupload blocks starting from the start of the oldest half-uploaded batch of blocks) with combination of empty blocks uploading leads to the fact that there can be multiple block objects corresponding to a single height (one object is a non-empty block, and another object is an empty block). Logs provided by @AliceInHunterland:

ekaterinapavlova@MacBook-Air-4 neofs-node % ./bin/neofs-cli object head -r st1.storage.fs.neo.org:8080 -w /Users/ekaterinapavlova/Workplace/neo-go/panelwallet1.json --cid 5oAvDsnVcpGEf8KM4nxNB52wqZASjbr4CSszbW9iseCk --oid 7JBanCq9oy1DptprG46Q8uN3MceqSZhDhgP4x4c1hQLU
ID: 7JBanCq9oy1DptprG46Q8uN3MceqSZhDhgP4x4c1hQLU
CID: 5oAvDsnVcpGEf8KM4nxNB52wqZASjbr4CSszbW9iseCk
Owner: NhNT6EFqrDu7hAsv2V1J3B7J1DKLvkNGRg
CreatedAt: 1
Size: 0
HomoHash: 00000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001
Checksum: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Type: REGULAR
Attributes:
  block=6500000
  primary=3
  hash=ab03ac9b20f8985e6f921ab1627e0fd5c298bfc61cd56dc1a4f84291db09c91e
  prevHash=2cdec61c03fafbf2bddfe31cc96783709f4fc52011a2c28fe396b97e1651a006
  timestamp=1728629315133
ID signature:
  public key: 02d89e1e06504a2ec3410a9ea801acfa3e45ea5d9ad3fa56ea179638a6dd72b7d2
  signature: 62ff66c17fff1e10decac963303649f18a25a55c75325cd902754ad64103d868ecdba21ccb0f427dc5506130bc59f5af1e9ee74c27f6127576c92cdfff7a618f
(base) ekaterinapavlova@MacBook-Air-4 neofs-node % ./bin/neofs-cli object head -r st1.storage.fs.neo.org:8080 -w /Users/ekaterinapavlova/Workplace/neo-go/panelwallet1.json --cid 5oAvDsnVcpGEf8KM4nxNB52wqZASjbr4CSszbW9iseCk --oid HxdTmcz9t3Wev3Kna7VXHVHzF5LoeBwwgkumPxNPgFFP
ID: HxdTmcz9t3Wev3Kna7VXHVHzF5LoeBwwgkumPxNPgFFP
CID: 5oAvDsnVcpGEf8KM4nxNB52wqZASjbr4CSszbW9iseCk
Owner: NhNT6EFqrDu7hAsv2V1J3B7J1DKLvkNGRg
CreatedAt: 1
Size: 697
HomoHash: 27e0ad6ce41b615764a3c0c5db5613e830aa69dd299e71426b538dcf0b7a01603acfa60e2c9452ebf5fb12e5a434d7a1681e326011d74876eeeee2af08e7b4b0
Checksum: f2c84bd3348b081fe86f991f9f2a7c7295013f426f5f12dbf3981883fb1ba52e
Type: REGULAR
Attributes:
  block=6500000
  primary=3
  hash=ab03ac9b20f8985e6f921ab1627e0fd5c298bfc61cd56dc1a4f84291db09c91e
  prevHash=2cdec61c03fafbf2bddfe31cc96783709f4fc52011a2c28fe396b97e1651a006
  timestamp=1728629315133
ID signature:
  public key: 02d89e1e06504a2ec3410a9ea801acfa3e45ea5d9ad3fa56ea179638a6dd72b7d2
  signature: 49c9aa9de6c87ecc14b597c5f9cf1c425f53b264c19f2c07eca33c8677fafa2b4611e807482ec26bb6797a47186a7c9b64f14ba65fcdfae42e36bb978c3b8ed9
(base) ekaterinapavlova@MacBook-Air-4 neofs-node %

It leads to invalid NeoFSBlockFetcher behaviour in "index files disabled" mode since search over range of blocks gives more results than expected:

Searching for objects with block GE 6490000 and LE 6500000...
Found 14257 objects:

AliceInHunterland added a commit that referenced this issue Oct 21, 2024
In case of uploading retry already drained bin buffer is return in `bw
.Bytes`,which leads to empty block uploading.

Close #3630

 Signed-off-by: Ekaterina Pavlova <[email protected]>

Signed-off-by: Ekaterina Pavlova <[email protected]>
AliceInHunterland added a commit that referenced this issue Oct 21, 2024
In case of uploading retry already drained bin buffer is returned in
`bw.Bytes`, which leads to empty block uploading.

Close #3630

Signed-off-by: Ekaterina Pavlova <[email protected]>
AliceInHunterland added a commit that referenced this issue Oct 21, 2024
In case of uploading retry already drained bin buffer is returned in
`bw.Bytes`, which leads to empty block uploading.

Close #3630

Signed-off-by: Ekaterina Pavlova <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working I4 No visible changes S4 Routine U1 Critically important to resolve quickly
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants