Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage schema evolution support #906

Merged
merged 34 commits into from
Jan 4, 2025
Merged

Conversation

bpkroth
Copy link
Contributor

@bpkroth bpkroth commented Jan 3, 2025

Pull Request

Title

Adds support for dynamically evolving the storage schema over time.


Description

  • Adds support for dynamic schema evolution via alembic
    • Details in included README.md, but basically uses a local stub sqlite db to automatically generate migration scripts that can be invoked by alembic.
  • Adds a --create-update-storage-schema-only flag to Launcher to allow running this independently.
    • This may be useful for mlos_benchd cases where multiple instances trying to manipulate the schema at once could be problematic. With this one could run this part separately first and then launch parallel other processes separately.
  • Adds the trial_runner_id column to the trials table (for Introduce TrialRunner Abstraction #720) (moved to a separate PR)
  • Closes Schema versioning for Sql Storage #649

Type of Change

  • ✨ New feature

Testing

  • Tests for this are somewhat hard to do in CI. I've tested this manually in this case. Details in the README.md

@bpkroth bpkroth requested a review from a team as a code owner January 3, 2025 22:26
@bpkroth bpkroth added the ready for review Ready for review label Jan 3, 2025
@bpkroth bpkroth enabled auto-merge (squash) January 4, 2025 01:53
@bpkroth bpkroth merged commit 43cc81a into microsoft:main Jan 4, 2025
15 checks passed
@bpkroth bpkroth deleted the schema-updates branch January 4, 2025 03:34
bpkroth added a commit that referenced this pull request Jan 6, 2025
…ting fixups (#910)

Minor adjustments to the template introduced in #906
bpkroth added a commit that referenced this pull request Jan 6, 2025
# Pull Request

## Title

Add trial_runner_id column to Trials table

---

## Description

- Split out from #906 in preparation for #720 

---

## Type of Change

- ✨ New feature

---

## Testing

Manually

---

## Additional Notes (optional)

To be merged after #906 

---

---------

Co-authored-by: Sergiy Matusevych <[email protected]>
Co-authored-by: Sergiy Matusevych <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for review Ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Schema versioning for Sql Storage
2 participants