Adds parallelization between reference generation and network querying #167

p-ferreira · 2024-03-20T14:36:45Z

NOTE: INCLUDES PR #165

adds threaded_log and asyng_log utility functions
evolves async implementation from PR Features/async generate ref #118

… creation

…e_task

…o features/vllm-async-test

…ures/vllm-async-test

prompting/forward.py

prompting/utils/misc.py

tests/test_forward.py

…sor/prompting into features/vllm-async-test

steffencruz and others added 12 commits February 21, 2024 19:46

Add create_reference arg to prevent task from generating reference on…

0a6b303

… creation

Add async_generate_reference and pass create_reference=False to creat…

58e4340

…e_task

Add forward tests

efcb007

Merge remote-tracking branch 'origin/features/async-generate-ref' int…

a191412

…o features/vllm-async-test

reorganizes async functions

5c72941

modifies async implementation to a thread implementation

2f9d177

implements mixed async threading solution

707125d

Merge remote-tracking branch 'origin/features/forward_time' into feat…

64c0514

…ures/vllm-async-test

organizes decorators

b548de7

adds wait for reference gen thread

d14d80e

fix utils import

af1c183

updates log severity

4f286b4

p-ferreira requested review from steffencruz, mccrindlebrian, bkb2135 and schampoux March 20, 2024 14:36

p-ferreira changed the title ~~Features/vllm async test~~ Adds reference generation and querying network parallel Mar 20, 2024

p-ferreira changed the title ~~Adds reference generation and querying network parallel~~ Adds parallelization between reference generation and network querying Mar 20, 2024

schampoux approved these changes Mar 20, 2024

View reviewed changes

steffencruz reviewed Mar 20, 2024

View reviewed changes

normalizes forward into asyncio operation

a1cb9f6

steffencruz mentioned this pull request Mar 25, 2024

Features/async generate ref #118

Closed

p-ferreira and others added 3 commits March 25, 2024 20:39

drops unused function

d708142

updates forward unit test

774fcc6

Merge branch 'staging' into features/vllm-async-test

6c9647d

p-ferreira added the v1.2.1 label Mar 25, 2024

p-ferreira added 3 commits March 25, 2024 21:04

increase time interval for forward test

17334cf

increase eps for test_mock

cfe1a62

Merge branch 'features/vllm-async-test' of https://github.com/openten…

69c9166

…sor/prompting into features/vllm-async-test

p-ferreira requested a review from steffencruz March 26, 2024 14:58

p-ferreira mentioned this pull request Mar 26, 2024

v1.3.0 #177

Merged

updates versioning

41f471f

steffencruz approved these changes Mar 27, 2024

View reviewed changes

p-ferreira merged commit 06e9ee2 into staging Mar 27, 2024
3 checks passed

mccrindlebrian deleted the features/vllm-async-test branch April 16, 2024 13:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds parallelization between reference generation and network querying #167

Adds parallelization between reference generation and network querying #167

p-ferreira commented Mar 20, 2024 •

edited

Loading

Adds parallelization between reference generation and network querying #167

Adds parallelization between reference generation and network querying #167

Conversation

p-ferreira commented Mar 20, 2024 • edited Loading

NOTE: INCLUDES PR #165

p-ferreira commented Mar 20, 2024 •

edited

Loading