feat: Truncate context window based on session settings #381
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
whiterabbit1983
force-pushed
the
f/adaptive-context
branch
from
5b13a72 to
bc8d97d
Compare
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
There was a problem hiding this comment.
❌ Changes requested. Incremental review on bc8d97d in 1 minute and 23 seconds
More details
- Looked at
557lines of code in13files - Skipped
0files when reviewing. - Skipped posting
0drafted comments based on config settings.
Workflow ID: wflow_xAGeybtwzxVZ5KZC
Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
creatorrr
requested changes
Jun 1, 2024
whiterabbit1983
force-pushed
the
f/adaptive-context
branch
from
bc8d97d to
9f20039
Compare
There was a problem hiding this comment.
👍 Looks good to me! Incremental review on 9f20039 in 1 minute and 15 seconds
More details
- Looked at
572lines of code in13files - Skipped
0files when reviewing. - Skipped posting
1drafted comments based on config settings.
1. openapi.yaml:1301
- Draft comment:
The addition oftoken_budgetandcontext_overflowto theSessionmodel in the OpenAPI specification is correctly implemented and matches the backend changes. - Reason this comment was not posted:
Confidence changes required:0%
Thetoken_budgetandcontext_overflowfields are added to theSessionmodel in the OpenAPI specification. This is consistent with the changes made in the codebase to support these new session settings. The OpenAPI specification is correctly updated to reflect these changes, ensuring that the API documentation remains accurate and up-to-date with the backend implementation.
Workflow ID: wflow_kFeO6LOhsZJYowLW
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
whiterabbit1983
added a commit
that referenced
this pull request
Jun 3, 2024
creatorrr
added a commit
that referenced
this pull request
Jun 11, 2024
* research: Recursive summarization experiments Signed-off-by: Diwank Singh Tomer <[email protected]> * fix: Minor fix to rec-sum notebook Signed-off-by: Diwank Singh Tomer <[email protected]> * wip: Rec-sum notebook Signed-off-by: Diwank Singh Tomer <[email protected]> * entity and trim step prompt + notebook. summarise setup. * summarization step added. * wip Signed-off-by: Diwank Singh Tomer <[email protected]> * chore: Move rec_sum subpackage * feat: Summarize messages recursively * feat: Use custom model for summarization * chore: Remove commented out code * fix: Serialize pandas series objects * fix: Choose correct way to generate based on model name * fix: Add old entities message as a child for the new one instead of deleting it * chore: Add tenacity dependency * fix: Strip closing ct:trimmed tag * fix: Strip closing ct:summarized-messages tag * fix: Add a list of entries instead of only one * fix: Convert strings to UUID * fix: Stringify message content * fix: Convert new entry content to a list of JSON * fix: Strip ct:entities tag * fix: Do not add new entry explicitly * fix: Update assertion * feat: Truncate context window based on session settings (#381) * feat: Calculate tokens for image content * feat: Update SDKs to support adaptive context options * fix: Truncate entries * fix: Make truncation a background task * fix: Add truncation workflow to registry * fix: Fix deleting query * fix: Remove truncated entries * fix: Convert role to string only if needed * fix: Replace delete by rm * fix: Fix entries deleting logic * fix: Set name to None if absent * fix: Make deleting query accept UUID as a string * fix: Convert UUIDs to strings * fix: Fix query parameter name * fix: Convert input to array of arrays * fix: Make entries indices zro-based * fix: Customize summarization model via environment variable * chore: Re-arrange operations on dialog entries * deps: poetry lock on agents-api and python sdk Signed-off-by: Diwank Tomer <[email protected]> --------- Signed-off-by: Diwank Singh Tomer <[email protected]> Signed-off-by: Diwank Tomer <[email protected]> Co-authored-by: Siddharth Balyan <[email protected]> Co-authored-by: Dmitry Paramonov <[email protected]> Co-authored-by: Diwank Tomer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This PR enhances session management by introducing
token_budgetandcontext_overflowsettings, updating API models and database schema, and including a migration script.Key points:
token_budgetandcontext_overflowsettings in session models.Generated with ❤️ by ellipsis.dev