Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: add flake8 and fix warnings #79

Merged
merged 1 commit into from
Nov 5, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 0 additions & 25 deletions .github/workflows/ci-lint.yml

This file was deleted.

21 changes: 21 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: Lint

on: push

jobs:
formatting:
if: "!contains(github.event.head_commit.message, 'skip_ci')"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- run: python -m pip install --upgrade tox
- run: tox -e checkformatting
flake8:
if: "!contains(github.event.head_commit.message, 'skip_ci')"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- run: python -m pip install --upgrade tox
- run: tox -e flake8
24 changes: 24 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Pytest

on: push

jobs:
test:
if: "!contains(github.event.head_commit.message, 'skip_ci')"
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: [3.6, 3.7, 3.8]

name: Python ${{ matrix.python-version }}
steps:
- uses: actions/checkout@v2

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- run: python -m pip install --upgrade pip tox
- run: tox -e pytest
4 changes: 1 addition & 3 deletions pyreisejl/utility/call.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
import argparse
import os
from collections import OrderedDict
from time import time

import numpy as np
import pandas as pd

from pyreisejl.utility import const
Expand Down Expand Up @@ -87,7 +85,7 @@ def launch_scenario(
# Import these within function because there is a lengthy compilation step
from julia.api import Julia

jl = Julia(compiled_modules=False)
Julia(compiled_modules=False)
from julia import REISE

start = time()
Expand Down
5 changes: 1 addition & 4 deletions pyreisejl/utility/extract_data.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
import argparse
import datetime as dt
import glob
import os
import re
import subprocess
import time
from collections import OrderedDict

import numpy as np
import pandas as pd
Expand Down Expand Up @@ -74,7 +71,7 @@ def extract_data(results):
outputs = {}

tic = time.process_time()
for filename in tqdm(results):
for i, filename in tqdm(enumerate(results)):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have an easy way to test this but I believe we reference this here (currently L88, below).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe you're right. This is an artifact of our previous load shedding strategy: on any infeasible interval, shed 5% of the original demand for that interval and re-run (as many times as necessary), so that a string of interval_num:shed_amount_... is enough to specify all load shedding details.

Breakthrough-Energy/PowerSimData#219 is related to the need to improve the summarization of load shed results in the new more-flexible load shedding method.

# For each result_#.mat file
output = load_mat73(filename)

Expand Down
2 changes: 1 addition & 1 deletion pyreisejl/utility/helpers.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from collections import OrderedDict
import os
import re
from collections import OrderedDict

import h5py
import numpy as np
Expand Down
1 change: 0 additions & 1 deletion pyreisejl/utility/tests/test_extract_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
from pyreisejl.utility.extract_data import (
_cast_keys_as_lists,
_get_pkl_path,
_update_outputs_labels,
calculate_averaged_congestion,
result_num,
)
Expand Down
1 change: 0 additions & 1 deletion pyreisejl/utility/tests/test_helpers.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
from io import StringIO

import numpy as np
import pandas as pd
import pytest

Expand Down
15 changes: 8 additions & 7 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
h5py>=2.9.0
ipython>=7.0
julia>=0.5.1
numpy>=1.16.0
pandas>=0.25.3
scipy>=1.2
tqdm>=4.29.1
h5py~=2.10.0
ipython~=7.0
julia~=0.5.1
numpy~=1.18.5
pandas~=1.1.3
scipy~=1.2
tqdm~=4.29.1
pytest~=6.1.1
21 changes: 21 additions & 0 deletions tox.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[tox]
envlist = pytest, format, flake8
skipsdist = true

[testenv]
deps =
pytest: -rrequirements.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo here? rrequirements -> requirements? Should this typo create an error in the CI check?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is actually right (otherwise I'd expect pytest to fail) - docs here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoa! TIL.

{format,checkformatting}: black
{format,checkformatting}: isort
flake8: flake8
changedir = pyreisejl
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of this file was copied from powersimdata but I added the changedir option so passing the current directory to commands still works. Also, the flake8 ignore section is customized for this repo.

commands =
pytest: pytest
format: black .
format: isort -m 3 --tc .
checkformatting: black . --check --diff
checkformatting: isort -m 3 --tc --check --diff .
flake8: flake8

[flake8]
ignore = E501,E731