[devtool] create stream_data_sink

[Gasoonjia]

Stack from ghstack (oldest at bottom):

• -> [devtool] create stream_data_sink #8604
• [devtool] make datasink as a sepreate directory #8514
• [devtool] make ETDumpGen use datasink #8499
• [devtool] introduce datasink class to etdump #8496

this diff create StreamDataSink, a datasink that adopts array as buffer and flush to disk when buffer is full.

Differential Revision: D69936705

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8604

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 30 New Failures, 2 Unrelated Failures

As of commit 9a48bd1 with merge base 9de9ed4 ():

NEW FAILURES - The following jobs have failed:

• Build documentation / build (buck2) / Build doc (gh)
  RuntimeError: Command docker exec -t f1dde44fb739dd7491bda2cf0f8e307f842254c37b8a620bcfca515109ccdc14 /exec failed with exit code 1
• Lint / lintrunner / linux-job (gh)
  RuntimeError: Command docker exec -t 101ec808b25987b6965d3e01e5f6805570afd473ed93f29fa5cf1ce51d11e090 /exec failed with exit code 1
• pull / test-custom-ops-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 9add706a71607c680915c61d10d1e38ba7972428681c719df4eef26e187e63f9 /exec failed with exit code 1
• pull / test-eval_llama-mmlu-linux / linux-job (gh)
  RuntimeError: Command docker exec -t b7b1e2a73cc099d67447244b77d0472e85af02cd29aa3428c9fd1780d9d0dc82 /exec failed with exit code 1
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5c759afd71cda4ac0fa3cc74307196e1214fee165483a0e79269c1567f8d7be4 /exec failed with exit code 1
• pull / test-llama_runner_eager-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 016858bf455e605ffb7da4f6079547f89f3b235c1a4288df5ea96de354c56370 /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, custom) / linux-job (gh)
  RuntimeError: Command docker exec -t e7946c01ba22dd1f01f62cf93914a8adf94b0f1071fb0661482e656e259e39ba /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, portable) / linux-job (gh)
  RuntimeError: Command docker exec -t b95fc215a66adee507bf899d0fe1e0dc8ea90f8944bb6d349f5edbd799bad2a6 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, portable) / linux-job (gh)
  RuntimeError: Command docker exec -t e6b4d26c135ab18ab84116bf27646b2aab9d8eba4718d2e593abdf4b22a269ca /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom) / linux-job (gh)
  RuntimeError: Command docker exec -t cf7632e7329da9a650c338c3cb6dcdb9e778aba1b9e629139ed602ef170240d6 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe) / linux-job (gh)
  RuntimeError: Command docker exec -t 1183a20bfb4e633d09e7277ad6e9745beda189b95e8d876119181f9e40adcbf9 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv) / linux-job (gh)
  RuntimeError: Command docker exec -t 89711dc4b865574fe30d168c0ffa53ca766b86808289a44067a380de8a94934f /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv) / linux-job (gh)
  RuntimeError: Command docker exec -t c720bb54ceba2c9407aae3c468de8bbe4a5929ef8809686c7606d69b2acfe128 /exec failed with exit code 1
• pull / test-llama-runner-linux-android / linux-job (gh)
  RuntimeError: Command docker exec -t e9ff51519458a517eb231e052db84d49a8d87c1af37537eaaadf4346cd481955 /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 1fedaea38e442d629f311afd6222c66380212fdf8ed43b6ef591724933c916c2 /exec failed with exit code 2
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 7463cd3a1fd89274421466e3f4e42ada98308b1d8c054a7f369b17425ba2f345 /exec failed with exit code 2
• pull / test-llava-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 06389c07f32e5471cf29211776510474477a75ce2fa9299c20fc3d778438b96b /exec failed with exit code 1
• pull / test-models-linux (buck2, mv3, portable, linux.2xlarge, 90) / linux-job (gh)
  RuntimeError: Command docker exec -t 522bbf0d40d089f859bd11053de807363a6eb9d1906d222abb50098a87841324 /exec failed with exit code 1
• pull / test-models-linux (buck2, mv3, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d4fc047745ce4b4db8ffea365f4adff84fe2640f483fdcf2c918d220fb9d55a /exec failed with exit code 1
• pull / test-models-linux (cmake, mv3, portable, linux.2xlarge, 90) / linux-job (gh)
  RuntimeError: Command docker exec -t f9295d64a8fe8b4ee4224f62522a59d4a2fb6397213bb6401d9c4a389bd5f9d7 /exec failed with exit code 1
• pull / test-models-linux (cmake, mv3, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job (gh)
  RuntimeError: Command docker exec -t b65e30154b8dbfd586676580af7d7a5c9c0431b26a58409fb687e0743c25e2ae /exec failed with exit code 1
• pull / test-models-linux (cmake, vit, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job (gh)
  RuntimeError: Command docker exec -t 90e5b4512d8362135b88d347a26ea973aa455385f0d17b108d543c9b54493157 /exec failed with exit code 1
• pull / test-pybind-build-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 3052dd51dccab88705d6567f899d71f053b9e7ef9b1809951d059d727dc87331 /exec failed with exit code 1
• pull / test-quantized-aot-lib-linux / linux-job (gh)
  RuntimeError: Command docker exec -t bc2f03ee3564239846fb16147b04200016b3bcb21df36ae9457e14aacfbf4bee /exec failed with exit code 1
• pull / test-selective-build-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 46ef689e1d699d727a3f8a7f07e9017dd8e82e84745ed0d165e2e6e8160bc39f /exec failed with exit code 1
• pull / test-setup-linux-gcc / linux-job (gh)
  RuntimeError: Command docker exec -t b632080fbd7a89e6393c7b0a6a5956cd1cf1523d1da136b8912566f7155df675 /exec failed with exit code 1
• pull / test-static-llama-qnn-linux / linux-job (gh)
  RuntimeError: Command docker exec -t cc4cda7b3bfdce36a9c540de619e693fda4d9193904239ccfb293507ebc3fe30 /exec failed with exit code 2
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 1d193d8715d6a29a3b5dcaf03599b44e75491bf741dc284e03c96c9a047c807d /exec failed with exit code 1
• pull / unittest / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest-arm / linux-job (gh)
  RuntimeError: Command docker exec -t 1f2aaeaa18e7b6178f6fc6d00ba16edc5780b6afbf0f523853d1397ad5379db2 /exec failed with exit code 1

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• pull / test-models-linux (cmake, vit, portable, linux.2xlarge, 90) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
• pull / test-phi-3-mini-runner-linux / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D69936705

[Gasoonjia]

@pytorchbot label "topic: not user facing"