Jobs: load from storage

[tseaver]

Add support for jobs loading table data from CloudStorage files.

[dhermes]

Does this have a diffbase?

[tseaver]

Does this have a diffbase?

Nope -- I rebased it before pushing. Too big?

[dhermes]

I just wasn't sure since there were 8 commits. Also, you're failing pep8 (that one runs fast, pylint is the slow one).

[dhermes]

Also ISTM a rebase won't feel very good due to the merging of #1051.

[tseaver]

I will fix it up (both the pep8 and the date stuff) and re-push.

[dhermes]

Thanks. Ping me when you push.

[tseaver]

Ugh! The pep8 fix was trivial, but working around the pylint 'too-many-branches' failures made me grind my teeth (4c14890)

[dhermes]

This thing is all ready to review?

[tseaver]

Hold off a bit -- I'm adding a system test for the "load from cloud storage" case, and it is failing.

[tseaver]

@dhermes OK, got the system test added and passing.

[dhermes]

👍 to the enums. Why not just take the jump and use enum34?

Also it looks like you had a bad merge / rebase, you've got two commits by me jammed in the middle:

[tseaver]

Why not just take the jump and use enum34?

I didn't want to eat the extra dependency. We have #1031 to track the idea, though.

[dhermes]

Why not? It's lightweight and hopefully most users already have it

>>> import enum
>>> 
>>> class RedBlue(enum.Enum):
...     RED = 'RED'
...     BLUE = 'BLUE'
... 
>>> 
>>> to_validate = 'RED'
>>> rb = RedBlue(to_validate)
>>> print(rb)
RedBlue.RED
>>> print(rb.value)
RED
>>> 
>>> to_validate = 'GREEN'
>>> try:
...     RedBlue(to_validate)
... except ValueError as exc:
...     print(repr(exc))
...     print('%s was not validated' % (to_validate,))
... 
ValueError('GREEN is not a valid RedBlue',)
GREEN was not validated

[tseaver]

Most users are on Python2, and likely won't have it unless some other library has pushed them to add it. Again, we can clean up here (and in the other APIs) when we address #1031.

[dhermes]

LGTM. I really hope the follow-up commits can spread out some of the pain here.

---

I was saying that most Python 2 only users would have lots of libraries installed that straddled Py2 and Py3, and hence would have a good chance that one of them used enum34. (I could've been clearer there.)

[dhermes]

@tseaver Please ping me on anything that will get rebased after this goes in.