Move modules

[rouille]

Purpose

In order to integrate the REISE.jl engine to the scenario framework, we need to move some modules across packages as shown in the image below.

This PR concerns only the PowerSimData package. The const and transferdata modules have been moved from the PostREISE package to a new folder powersimdata/utility/ in the PowerSimData package. Additional PRs will follow to achieve the reorganization shown on the above image.

The PowerSimData package can now be used with no reference to PostREISE.

See issue #250 in the RenewableEnergyProject repository for additional details.

An interactive rebase will be done to keep only the most relevant commit messages after all the comments are addressed.

What is the code doing

There is no new code. Modules have been moved from the PostREISE package to the PowerSimData package. Tests have been written for the transfer_data module.

Where to look

The powersimdata/utility/ folder.

Time estimate

Hald an hour.

Note

Tests passed:

================================================================ test session starts =================================================================
platform darwin -- Python 3.6.5, pytest-4.5.0, py-1.8.0, pluggy-0.13.0 -- /Users/brdo/PyEnv/REM/v1/bin/python
cachedir: .pytest_cache
rootdir: /Users/brdo/REM/PowerSimData
collected 5 items                                                                                                                                    

test_transfer_data.py::test_setup_server_connection PASSED                                                                                     [ 20%]
test_transfer_data.py::test_get_scenario_file_from_server_type PASSED                                                                          [ 40%]
test_transfer_data.py::test_get_scenario_file_from_server_header PASSED                                                                        [ 60%]
test_transfer_data.py::test_get_execute_file_from_server_type PASSED                                                                           [ 80%]
test_transfer_data.py::test_get_execute_file_from_server_header PASSED                                                                         [100%]

Also, the LMP data frame for scenario 87 has been successfully downloaded from the server after moving the transfer_data module in PowerSimData:

>>> from powersimdata.scenario.scenario import Scenario
>>> s = Scenario('87')
SCENARIO: base | WesternBase_2016_noHVDC_Final_2019Sep

--> State
analyze
>>> s.state.get_lmp()
--> Loading LMP
87_LMP.pkl not found in /Users/brdo/ScenarioData/ on local machine
Transferring 87_LMP.pkl from server
100%|##############################################################################################################| 703M/703M [00:55<00:00, 12.7Mb/s]
                       2010001    2010002    2010003    2010004    2010005  ...    2080096    2080097    2080098    2080099    2080100
UTC                                                                         ...                                                       
2016-01-01 00:00:00  23.331623  23.331582  23.331561  23.331560  23.331560  ...  22.226513  22.226513  22.226513  22.226513  22.225539
2016-01-01 01:00:00  23.518707  23.518709  23.518713  23.518713  23.518713  ...  22.445014  22.445014  22.445014  22.445014  22.445030
2016-01-01 02:00:00  23.853874  23.853875  23.853878  23.853878  23.853878  ...  22.518730  22.518730  22.518730  22.518730  22.518311
2016-01-01 03:00:00  24.008445  24.008441  24.008441  24.008441  24.008441  ...  22.473815  22.473815  22.473815  22.473815  22.471548
2016-01-01 04:00:00  23.935515  23.935513  23.935515  23.935515  23.935515  ...  22.422640  22.422640  22.422640  22.422640  22.420606
...                        ...        ...        ...        ...        ...  ...        ...        ...        ...        ...        ...
2016-12-31 19:00:00  22.581616  22.581715  22.581777  22.581780  22.581780  ...  21.906029  21.906029  21.906029  21.906029  21.903622
2016-12-31 20:00:00  22.579543  22.579658  22.579730  22.579733  22.579733  ...  21.878081  21.878081  21.878081  21.878081  21.875458
2016-12-31 21:00:00  22.441762  22.441804  22.441829  22.441831  22.441831  ...  21.815171  21.815171  21.815171  21.815171  21.814972
2016-12-31 22:00:00  22.370087  22.370119  22.370139  22.370140  22.370140  ...  21.803841  21.803841  21.803841  21.803841  21.803840
2016-12-31 23:00:00  22.175715  22.175732  22.175743  22.175743  22.175743  ...  21.837736  21.837736  21.837736  21.837736  21.838036

[8784 rows x 10000 columns]

[dmuldrew]

There's another PowerSimData PR which moves files around, is there going to be conflict here?
#124
If so, which PR should be merged first?

[rouille]

It does not really matter. These PRs are independent. That said the one who goes second will have to rebase before merging into develop.

[kasparm]

This works fine. Some dependencies still exist:

• PostREISE

./powersimdata/input/design_transmission.py:from postreise.analyze.transmission.distance import haversine
./powersimdata/scenario/execute.py:                   '/home/EGM/v2/PostREISE/postreise/extract/extract_data.py',
./powersimdata/scenario/tests/test_scenario_info.py:from postreise.tests.mock_scenario import MockScenario

• PreREISE

/powersimdata/scenario/scenario_info.py:from prereise.gather.constants import abv2state, state2loadzone,

[rouille]

We will fix those dependencies in the next sprint.