Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge data access refactor feature branch #584

Merged
merged 33 commits into from
Feb 10, 2022
Merged

Conversation

jenhagg
Copy link
Collaborator

@jenhagg jenhagg commented Feb 9, 2022

Pull Request doc

Purpose

Data access refactor and blob storage integration. Someone needs to approve it but I can leave open if folks haven't had a chance to review the recent PR, just lemme know.

Testing

This was tested in #570

Usage Example/Visuals

Here is the output I get when I change the local directory from ScenarioData to an empty directory, then load a scenario from blob storage.

In [1]: from powersimdata import Scenario

In [2]: s = Scenario(1270)
Could not connect to ssh server
Initialized remote filesystem with profile_fs,scenario_fs
Transferring ScenarioList.csv.2 from scenario_fs
Transferring ExecuteList.csv.2 from scenario_fs
SCENARIO: Julia | USA2030HVDC_Design3_OB1_Mesh500x38

--> State
analyze
--> Loading grid
data/input/1270_grid.mat not found on local machine
Could not connect to ssh server
Initialized remote filesystem with profile_fs,scenario_fs
Transferring 1270_grid.mat from scenario_fs
Loading bus
Loading plant
Loading heat_rate_curve
Loading gencost_before
Loading gencost_after
Loading branch
Loading dcline
Loading sub
Loading bus2sub
Could not connect to ssh server
Initialized remote filesystem with profile_fs,scenario_fs
Transferring ScenarioList.csv.2 from scenario_fs
--> Loading ct
data/input/1270_ct.pkl not found on local machine
Transferring 1270_ct.pkl from scenario_fs

Time estimate

n/a - code has been reviewed

ahurli and others added 30 commits February 7, 2022 11:39
@jenhagg jenhagg self-assigned this Feb 9, 2022
@jenhagg jenhagg linked an issue Feb 9, 2022 that may be closed by this pull request
3 tasks
Copy link
Contributor

@kasparm kasparm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @ahurli and @jon-hagg

@jenhagg jenhagg merged commit 69c1e36 into develop Feb 10, 2022
@jenhagg jenhagg deleted the data_access_refactor branch February 10, 2022 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cleanup
3 participants