feat: deterministic tasking #85

bajtos · 2024-07-22T11:29:41Z

Rework the current random task selection algorithm to a deterministic algorithm using DRAND randomness and distance between the hash of the task and the hash of the node to sort the tasks in a deterministic way that's unique to each round and each node.

Design doc:

https://www.notion.so/spacemeridian/Spark-Tasking-v3-745b0e1020bb4000ac77acafee09e683

Proof of Concept:

Proof of concept: deterministic tasking using SHA256(StationID) ^ SHA256(Task) spark-evaluate#287

Links:

Signed-off-by: Miroslav Bajtoš <[email protected]>

bajtos · 2024-07-23T09:37:27Z

lib/tasker.js

+export async function getTaskKey (task, randomness) {
+  assertEquals(typeof task, 'object', 'task must be an object')
+  assertEquals(typeof task.cid, 'string', 'task.cid must be a string')
+  assertEquals(typeof task.minerId, 'string', 'task.minerId must be a string')
+  assertEquals(typeof randomness, 'string', 'randomness must be a string')
+
+  const data = [task.cid, task.minerId, randomness].join('\n')
+  const hash = await crypto.subtle.digest('sha-256', textEncoder.encode(data))
+  return BigInt('0x' + encodeHex(hash))
+}
+
+/**
+ * @param {string} stationId
+ */
+export async function getStationKey (stationId) {
+  assertEquals(typeof stationId, 'string', 'stationId must be a string')
+
+  const hash = await crypto.subtle.digest('sha-256', textEncoder.encode(stationId))
+  return BigInt('0x' + encodeHex(hash))
+}
+
+/**
+ * @param {object} args
+ * @param {Task[]} args.tasks
+ * @param {string} args.stationId
+ * @param {string} args.randomness
+ * @param {number} args.maxTasksPerRound
+ * @returns {Promise<Task[]>}
+ */
+export async function pickTasksForNode ({ tasks, stationId, randomness, maxTasksPerRound }) {
+  assertInstanceOf(tasks, Array, 'tasks must be an array')
+  assertEquals(typeof stationId, 'string', 'stationId must be a string')
+  assertEquals(typeof randomness, 'string', 'randomness must be a string')
+  assertEquals(typeof maxTasksPerRound, 'number', 'maxTasksPerRound must be a number')
+
+  const keyedTasks = await Promise.all(tasks.map(
+    async (t) => ({ ...t, key: await getTaskKey(t, randomness) })
+  ))
+  const stationKey = await getStationKey(stationId)
+
+  /**
+   * @param {{key: bigint}} a
+   * @param {{key: bigint}} b
+   * @returns {number}
+   */
+  const comparator = (a, b) => {
+    const ad = a.key ^ stationKey
+    const bd = b.key ^ stationKey
+    return ad > bd ? 1 : ad < bd ? -1 : 0
+  }
+
+  keyedTasks.sort(comparator)
+  keyedTasks.splice(maxTasksPerRound)
+
+  return keyedTasks.map(({ key, ...t }) => (t))
+}


This code will need to stay in sync with the fraud-detection code in spark-evaluate. At the same time, I don't expect it to change often. When we do change it, we need to carefully design a migration/deployment path how to update both spark-evaluate and the checker network at the same time. Considering that, I think it's ok to keep this code duplicated between the two GitHub repos (spark and spark-evaluate).

Also in spark-evaluate, I'll need to use a more efficient version using k-closest module instead of Array.prototype.sort. If we wanted to share that code, I would need to place it into a standalone npm package or else inline k-closest module into our codebase, otherwise we won't be able to vendor it here.

Also in spark-evaluate, I'll need to use a more efficient version using k-closest module instead of Array.prototype.sort

Does this method produce different results?

It must produce the same results, otherwise the checkers will pick tasks that will not be accepted by spark-evaluate's fraud detection.

I verified this in my PoC by running Array.prototype.sort alongside k-closest and comparing the results. See here:
https://github.com/filecoin-station/spark-evaluate/pull/287/files#diff-0f9e17445925f01057ff062d7d52d2862b081121d6e722cdeda8a2b1ad106ebcR275-R283

Ok, and have you checked if Array#sort is too slow for spark-evaluate?

If we can get dependabot to work for this repo, then I think it would be nicest to share this code in a dependency. If not, inlining like this should be the simplest

Ok, and have you checked if Array#sort is too slow for spark-evaluate?

Yes, of course. I don't remember the exact number, though. It was an order of magnitude slower.

If we can get dependabot to work for this repo, then I think it would be nicest to share this code in a dependency. If not, inlining like this should be the simplest

I don't see any easy way how to enable dependabot here, let's keep this inlined for now.

If not, can we add comments to spark-evaluate to also update this?

Sure. Would you like me to add a code comment to this file too?

Yes, of course. I don't remember the exact number, though. It was an order of magnitude slower.

Order of magnitude slower doesn't mean too slow though. Eg if it's 1ms vs 10ms or even 100ms it's probably still not be worth the extra complexity.

Sure. Would you like me to add a code comment to this file too?

I don't think it's necessary since we're going to evolve the algorithm from spark-evaluate and not this repo

The k-closest version takes 3-4 seconds to finish. The sort-based version took 1-2 minutes IIRC.

I re-ran the PoC using the Array sort() version. It takes 27.38 seconds to build the list of valid tasks.

Evaluating round 10788n of contract 0x8460766edc62b525fc1fa4d628fc79229dc73031 (...) EVALUATE ROUND 10788n: using randomness f427ee008087a37142c9c22f6b227a1f915a495648bf3a3494c555c097f0880c EVALUATE ROUND 10788n: built per-node task lists in 27380ms [Tasks=1000;TN=15;Nodes=61095]

The k-closest version takes 4 seconds to complete.

EVALUATE ROUND 10788n: using randomness f427ee008087a37142c9c22f6b227a1f915a495648bf3a3494c555c097f0880c EVALUATE ROUND 10788n: built per-node task lists in 4040ms [Tasks=1000;TN=15;Nodes=61095]

For perspective: using the current spark-evaluate main, the entire dry-run evaluation of a single round takes 4.86 seconds.

I thought that increasing the time to evaluate a round from 5 seconds to 33 seconds is not acceptable. However, if you think it's fine, then I am happy to use the Array sort() version in spark-evaluate too.

Blocking the event loop for 33 seconds is super long, and the service will be unresponsive and not receive events. I agree k-closest is worth it here :)

vendor/deno-deps.js

juliangruber

great work so far!

lib/drand-client.js

lib/http-assertions.js

lib/spark.js

lib/http-assertions.js

lib/tasker.js

lib/http-assertions.js

lib/tasker.js

Co-authored-by: Julian Gruber <[email protected]>

Signed-off-by: Miroslav Bajtoš <[email protected]>

lib/drand-client.js

juliangruber · 2024-07-24T07:58:40Z

lib/tasker.js

+export async function getTaskKey (task, randomness) {
+  assertEquals(typeof task, 'object', 'task must be an object')
+  assertEquals(typeof task.cid, 'string', 'task.cid must be a string')
+  assertEquals(typeof task.minerId, 'string', 'task.minerId must be a string')
+  assertEquals(typeof randomness, 'string', 'randomness must be a string')
+
+  const data = [task.cid, task.minerId, randomness].join('\n')
+  const hash = await crypto.subtle.digest('sha-256', textEncoder.encode(data))
+  return BigInt('0x' + encodeHex(hash))
+}
+
+/**
+ * @param {string} stationId
+ */
+export async function getStationKey (stationId) {
+  assertEquals(typeof stationId, 'string', 'stationId must be a string')
+
+  const hash = await crypto.subtle.digest('sha-256', textEncoder.encode(stationId))
+  return BigInt('0x' + encodeHex(hash))
+}
+
+/**
+ * @param {object} args
+ * @param {Task[]} args.tasks
+ * @param {string} args.stationId
+ * @param {string} args.randomness
+ * @param {number} args.maxTasksPerRound
+ * @returns {Promise<Task[]>}
+ */
+export async function pickTasksForNode ({ tasks, stationId, randomness, maxTasksPerRound }) {
+  assertInstanceOf(tasks, Array, 'tasks must be an array')
+  assertEquals(typeof stationId, 'string', 'stationId must be a string')
+  assertEquals(typeof randomness, 'string', 'randomness must be a string')
+  assertEquals(typeof maxTasksPerRound, 'number', 'maxTasksPerRound must be a number')
+
+  const keyedTasks = await Promise.all(tasks.map(
+    async (t) => ({ ...t, key: await getTaskKey(t, randomness) })
+  ))
+  const stationKey = await getStationKey(stationId)
+
+  /**
+   * @param {{key: bigint}} a
+   * @param {{key: bigint}} b
+   * @returns {number}
+   */
+  const comparator = (a, b) => {
+    const ad = a.key ^ stationKey
+    const bd = b.key ^ stationKey
+    return ad > bd ? 1 : ad < bd ? -1 : 0
+  }
+
+  keyedTasks.sort(comparator)
+  keyedTasks.splice(maxTasksPerRound)
+
+  return keyedTasks.map(({ key, ...t }) => (t))
+}


Ok, and have you checked if Array#sort is too slow for spark-evaluate?

Signed-off-by: Miroslav Bajtoš <[email protected]>

sentry-io · 2024-07-28T07:30:32Z

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

‼️ ExecaError: Command failed with exit code 1: /core/modules/zinnia/zinniad spark/main.js ?(tasker) View Issue

_{Did you find this useful? React with a 👍 or 👎}

bajtos added 2 commits July 22, 2024 12:04

feat: setup drand client

e9be757

Signed-off-by: Miroslav Bajtoš <[email protected]>

feat: deterministic tasking

f9aa7ae

Signed-off-by: Miroslav Bajtoš <[email protected]>

bajtos requested a review from juliangruber July 22, 2024 11:29

bajtos commented Jul 23, 2024

View reviewed changes

vendor/deno-deps.js Outdated Show resolved Hide resolved

juliangruber requested changes Jul 23, 2024

View reviewed changes

bajtos and others added 2 commits July 24, 2024 08:19

Update lib/tasker.js

4ea71d2

Co-authored-by: Julian Gruber <[email protected]>

fixup! address review comments

b4a04af

Signed-off-by: Miroslav Bajtoš <[email protected]>

bajtos requested a review from juliangruber July 24, 2024 06:37

juliangruber approved these changes Jul 24, 2024

View reviewed changes

fixup! add a link to a GH issue

489a213

Signed-off-by: Miroslav Bajtoš <[email protected]>

bajtos merged commit 2e0527d into main Jul 24, 2024
2 checks passed

bajtos deleted the deterministic-tasking branch July 24, 2024 09:03

bajtos mentioned this pull request Jul 30, 2024

feat: deterministic task assignment CheckerNetwork/spark-evaluate#296

Merged

bajtos mentioned this pull request Aug 7, 2024

test: add tests for http assertions #88

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: deterministic tasking #85

feat: deterministic tasking #85

bajtos commented Jul 22, 2024

bajtos Jul 23, 2024

juliangruber Jul 23, 2024

bajtos Jul 23, 2024

juliangruber Jul 24, 2024

juliangruber Jul 24, 2024

bajtos Jul 24, 2024

juliangruber Jul 24, 2024

bajtos Jul 24, 2024

bajtos Jul 24, 2024

juliangruber Jul 24, 2024

juliangruber left a comment

juliangruber Jul 24, 2024

sentry-io bot commented Jul 28, 2024

feat: deterministic tasking #85

feat: deterministic tasking #85

Conversation

bajtos commented Jul 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

juliangruber left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sentry-io bot commented Jul 28, 2024

Suspect Issues