Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flake8: simplify #4271

Merged
merged 1 commit into from
Jun 17, 2021
Merged

flake8: simplify #4271

merged 1 commit into from
Jun 17, 2021

Conversation

oliver-sanders
Copy link
Member

@oliver-sanders oliver-sanders commented Jun 16, 2021

More meeting code fiddling, closes #4239

List of simplify codes here: https://github.com/MartinThoma/flake8-simplify#SIM106

Used a few noqa flags where appropriate, there are a few things in the code that are hard to refactor, usually patterns which should be avoided but which aren't worth the pain of totally overhauling at this stage.

Suggest adding ?=1 to the diff URL to hide the whitespace changes.

Requirements check-list

  • I have read CONTRIBUTING.md and added my name as a Code Contributor.
  • Contains logically grouped changes (else tidy your branch by rebase).
  • Does not contain off-topic changes (use other PRs for other changes).
  • Applied any dependency changes to both setup.py and
    conda-environment.yml.
  • Already covered by existing tests.
  • No change log entry required (why? e.g. invisible to users).
  • No documentation update required.

@oliver-sanders oliver-sanders self-assigned this Jun 16, 2021
@oliver-sanders oliver-sanders added this to the cylc-8.0b2 milestone Jun 16, 2021
Comment on lines +711 to 712
with Conf('platform groups'): # noqa: SIM117 (keep same format)
with Conf('<group>'):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could combine these two with statements into a single one, however, since all of the others in this file are written individually this would break the pattern somewhat.

@@ -1538,7 +1539,7 @@ def upgrade_graph_section(cfg: Dict[str, Any], descr: str) -> None:
"""Upgrade Cylc 7 `[scheduling][dependencies][X]graph` format to
`[scheduling][graph]X`."""
# Parsec upgrader cannot do this type of move
try:
with contextlib.suppress(KeyError):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with suppress(Something):
    something()

Is generally cleaner and more readable than:

try:
    something()
except Something:
    pass

I'm not so fussed, if this causes us undue pain we can always disable the rule, however, it does have a potential advantage from the perspective of code coverage so thats nice.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I quite like the with suppress... version.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing to bear in mind is with suppress is slower, so we might want to avoid it sometimes

from contextlib import suppress

def f():
    try:
        int('a')
    except ValueError:
        pass
    
def g():
    with suppress(ValueError):
        int('a')
In [5]: %timeit f()
976 ns ± 46 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [6]: %timeit g()
1.63 µs ± 32.8 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

Comment on lines +873 to +879
if not self.cfg['scheduler']['allow implicit tasks']:
raise WorkflowConfigError(
f"{err_msg}\n\n"
"To allow implicit tasks, use "
"'flow.cylc[scheduler]allow implicit tasks'")
else:
LOG.debug(err_msg)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flake8-simplify wants you to handle the error cases first, so this:

if error_cond:
    raise Exception()
something()

Rather than this:

if not error_cond:
    something()
else:
    raise Exception()

This is generally a good idea, however, I think for pattern-matching type problems where you have a list of conditions and handlers it's a good pattern to have a else: raise catchall at the end:

if cond_a:
    something('a')
elif cond_b:
    something('b')
...
elif cond_z:
    something('z')
else:
    raise Exception()

So I've left these example in with noqa flags.

Comment on lines -33 to -34
if ',' in exclusions:
if (not exclusions.strip().startswith('(') or not
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needlessly nested if's

@@ -417,7 +423,7 @@ def __hash__(self) -> int:
pass


class ExclusionBase(object, metaclass=ABCMeta):
class ExclusionBase(metaclass=ABCMeta):
Copy link
Member Author

@oliver-sanders oliver-sanders Jun 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Python2 class.

sys.exit(1)
except IOError:
sys.exit("Workflow schd program exited")
with open(logfname, 'r') as logfile:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with open() is the better pattern.

Note that this:

with open() as h:
    h.write()

Is not equivalent to this:

h = open()
h.write()
h.close()

As the with open brings extra error handling to the table.

Comment on lines +139 to +143
return any(
message == value
for outputs_str, in res
for value in json.loads(outputs_str).values()
)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any is faster and more readable.

@@ -187,7 +192,7 @@ def get_data_elements(flow, nat_ids, element_type):
]


class BaseResolvers:
class BaseResolvers: # noqa: SIM119 (no real gain + mutable default)
Copy link
Member Author

@oliver-sanders oliver-sanders Jun 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flake8-simplify is keen on dataclasses.

I.E. this:

from dataclasses import dataclass

@dataclass
class Foo:
    a: str
    b: str

Rather than this:

class Foo:

    def __init__(self, a: str, b: str):
        self.a = a
        self.b = b

Dataclasses have additional benefits because they add other methods like __str__ and __eq__ which saves a lot of boilerplate of thekind we are often to lazy to write.

Dataclasses are generally a good idea, however, a lot of our classes (like this one) don't fit the pattern so well.

Nice to be reminded that we could use a dataclass, just stick a noqa in if not appropriate.

Copy link
Member

@hjoliver hjoliver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Quite a big change but just straightforward switch to other valid coding constructs, so one review will do on this.

@hjoliver hjoliver merged commit 3dc411a into cylc:master Jun 17, 2021
@oliver-sanders oliver-sanders deleted the flake8-simplfy branch June 17, 2021 08:24
@oliver-sanders oliver-sanders mentioned this pull request Jun 17, 2021
7 tasks
@MetRonnie
Copy link
Member

MetRonnie commented Jun 21, 2021

Is this not going to be added to GH Actions? Ah wait, it's run as part of the normal flake8 command of course

@oliver-sanders
Copy link
Member Author

(we can use the flake8 --select option to make it more explicit, otherwise flake8 runs all installed flakes)

@MetRonnie MetRonnie added the infrastructure GH Actions, Codecov etc. label Sep 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infrastructure GH Actions, Codecov etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

flake8: simplify
3 participants