-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix!: update aggregation capabilitites to use height instead of size together with client and api #831
fix!: update aggregation capabilitites to use height instead of size together with client and api #831
Changes from 1 commit
3250e85
95bfaa1
3736b2d
11e68f0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
@@ -1,11 +1,13 @@ | ||||||||||||||
import * as Server from '@ucanto/server' | ||||||||||||||
import { CBOR } from '@ucanto/core' | ||||||||||||||
import { Node, Aggregate as AggregateBuilder } from '@web3-storage/data-segment' | ||||||||||||||
import * as Aggregate from '@web3-storage/capabilities/aggregate' | ||||||||||||||
import * as Offer from '@web3-storage/capabilities/offer' | ||||||||||||||
import * as API from '../types.js' | ||||||||||||||
|
||||||||||||||
export const MIN_SIZE = 1 + 127 * (1 << 27) | ||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure what this number is, and I wonder if it was mistaken for
Suggested change
Which would be unpadded size corresponding to 16GiB deal, that is after padding we'd get 16GiB which is probably what was the intention was.
|
||||||||||||||
export const MAX_SIZE = 127 * (1 << 28) | ||||||||||||||
// export const MAX_SIZE = 127 * (1 << 28) | ||||||||||||||
export const MAX_SIZE = AggregateBuilder.DEFAULT_DEAL_SIZE | ||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @Gozala interesting that these values are different than ones previously obtained with riba. Clues? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The former value is (unpadded) payload size, if you pad it you'll get a deal (aggregate) size which is the new value. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also I don't know if that 32GiB is the max deal size for spade, so I'm not sure we do need to impose it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am playing with values that Riba provided us back in the days. Maybe not relevant anymore, but I will confirm and iterate |
||||||||||||||
|
||||||||||||||
/** | ||||||||||||||
* @param {API.AggregateServiceContext} context | ||||||||||||||
|
@@ -39,29 +41,44 @@ export const claim = async ( | |||||||||||||
} | ||||||||||||||
|
||||||||||||||
// Validate offer content | ||||||||||||||
const size = offers.reduce((accum, offer) => accum + offer.size, 0) | ||||||||||||||
if (size < MIN_SIZE) { | ||||||||||||||
const aggregateLeafs = 2n ** BigInt(piece.height) | ||||||||||||||
const aggregateSize = aggregateLeafs * BigInt(Node.Size) | ||||||||||||||
|
||||||||||||||
if (aggregateSize < MIN_SIZE) { | ||||||||||||||
return { | ||||||||||||||
error: new AggregateOfferInvalidSizeError( | ||||||||||||||
`offer under size, offered: ${size}, minimum: ${MIN_SIZE}` | ||||||||||||||
`offer under size, offered: ${aggregateSize}, minimum: ${MIN_SIZE}` | ||||||||||||||
), | ||||||||||||||
} | ||||||||||||||
} else if (size > MAX_SIZE) { | ||||||||||||||
} else if (aggregateSize > MAX_SIZE) { | ||||||||||||||
return { | ||||||||||||||
error: new AggregateOfferInvalidSizeError( | ||||||||||||||
`offer over size, offered: ${size}, maximum: ${MAX_SIZE}` | ||||||||||||||
`offer over size, offered: ${aggregateSize}, maximum: ${MAX_SIZE}` | ||||||||||||||
), | ||||||||||||||
} | ||||||||||||||
} else if (size !== piece.size) { | ||||||||||||||
} | ||||||||||||||
|
||||||||||||||
// Validate commP of commPs | ||||||||||||||
const aggregateBuild = AggregateBuilder.build({ | ||||||||||||||
pieces: offers.map((o) => ({ | ||||||||||||||
link: o.link, | ||||||||||||||
size: 2n ** BigInt(o.height) * BigInt(Node.Size), | ||||||||||||||
})), | ||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You should pass a You also do not need to do the math there, there is Piece.fromJSON that you could use instead
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Looks like we expect pieces to be sorted, we should make note of that in the spec, if that is not the case we probably should sort them here. |
||||||||||||||
}) | ||||||||||||||
if (!aggregateBuild.link.equals(piece.link)) { | ||||||||||||||
return { | ||||||||||||||
error: new AggregateOfferInvalidSizeError( | ||||||||||||||
`offer size mismatch, specified: ${piece.size}, actual: ${size}` | ||||||||||||||
`aggregate piece CID mismatch, specified: ${piece.link}, computed: ${aggregateBuild.link}` | ||||||||||||||
), | ||||||||||||||
} | ||||||||||||||
} else if (aggregateBuild.height !== piece.height) { | ||||||||||||||
return { | ||||||||||||||
error: new AggregateOfferInvalidSizeError( | ||||||||||||||
`aggregate height mismatch, specified: ${piece.height}, computed: ${aggregateBuild.height}` | ||||||||||||||
), | ||||||||||||||
} | ||||||||||||||
} | ||||||||||||||
|
||||||||||||||
// TODO: Validate commP of commPs | ||||||||||||||
|
||||||||||||||
// Create effect for receipt | ||||||||||||||
const fx = await Offer.arrange | ||||||||||||||
.invoke({ | ||||||||||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You'll need this to use
Piece.fromJSON
directly.