Make the project packageable

[teknico]

Fixes #34. Fixes #35 (and partially #32 too).

This restructures the project to make it packageable and installable from the Package Index.

It splits the project into two packages: the decred library and the tinywallet application, the latter depending on the former. It includes the following changes:

• rename the pydecred subpackage to dcr
• use relative imports where appropriate
• create the decred package and move the library code and tests there
• complete the decred package definition and make it installable
• create the tinywallet package and move the wallet app code and tests there
• complete the tinywallet package definition and make it installable
• update the Github Action definition file
• update the documentation and examples

[rodrigondec]

@teknico include a fixes #34 too

[teknico]

@rodrigondec, is something in #34 missing here? What specifically?

[buck54321]

@teknico I think @rodrigondec is suggesting that you mention issue #34 in your PR description.

[buck54321]

PEP 8 generally recommends absolute imports, except in the case of complex package structures where absolute imports are too verbose.

Absolute imports are recommended, as they are usually more readable and tend to be better behaved (or at least give better error messages)...

However, explicit relative imports are an acceptable alternative to absolute imports, especially when dealing with complex package layouts where using absolute imports would be unnecessarily verbose

So let's have absolute imports as our default, with relative imports acceptable if the import statement is three levels or deeper. So

from tinydecred.pydecred.dcrdata import DcrdataBlockchain

can be shortened to

from .dcrdata import DcrdataBlockchain

but leave imports like

from tinydecred.util import encode

as is.

Is that fair?

[teknico]

How about a different approach?

The file decred/crypto/secp256k1/curve.py contains the following lines:

from decred.util.encode import ByteArray
from decred.crypto.rando import generateSeed
from decred.crypto.secp256k1.field import BytePoints, FieldVal

As relative imports they would become:

from ...util.encode import ByteArray
from ..rando import generateSeed
from .field import BytePoints, FieldVal

With the three-level rule all three lines would end up as relative, but only the third line is clearer as a relative import: the first one is clearer as absolute. The second line is less evident, but I'd go with absolute there too.

Therefore I'd like to suggest a different rule: only use relative imports for modules in the same subpackage or below, that is, only one dot allowed, never go up in the hierarchy.

(As an aside, the pycodestyle tool that implements the PEP8 guidelines has no checks for relative imports.)

[buck54321]

Yeah, that's even better.

[teknico]

Mmm... I overhauled the action definition file to use Poetry and it suspiciously worked on the first try, the logs look fine. Oh well...

[teknico]

Ready for final review. You can check the live docs on the proposed branch.

[buck54321]

Looking good.

• Should tinywallet get its own .coveragerc?

• I ran into an issue that other probably won't, but maybe I can save someone some time. poetry was failing for me with a complaint about a missing bz2 module. Since I had built Python from source and didn't have the library, the bz2 module was never installed. On Debian, I had to 1) get the right library with apt-get install libbz2-dev, 2) rebuild Python, and 3) reinstall poetry.

• poetry install went fine from decred, but has issues with tinywallet which required me to set a specific PyQt5 version. There were a few other issues too. Are you testing tinywallet too? My issues may be system-specific, though.

[buck54321]

I see you're not ignoring the tinywallet/poetry.lock. Is that intentional?

[teknico]

It is, well spotted. tinywallet/poetry.lock is indeed committed, decred/poetry.lock is not. The idea is to commit lock files for applications, but not for libraries.

You want to commit the lock file for apps so that a build always uses the same versions of the dependencies (and their dependencies), and therefore is reproducible.

You don't want to commit the lock file for libraries because then you would have multiple lock files for different libraries and for the app that uses them, potentially conflicting among them.

The libraries will specify the version intervals for their dependencies in their pyproject.toml or requirements.txt files; Poetry will respect those intervals, but will disregard any libraries' lock files. The only lock file that counts is the app one.

[JoeGruffins]

This is probably nothing, but using the provided tinywallet/poetry.lock file threw an error at me:

[joe@hyrule tinywallet]$ poetry install
Installing dependencies from lock file
Warning: The lock file is not up to date with the latest changes in pyproject.toml. You may be getting outdated dependencies. Run update to update them.

[SolverProblemError]
Because tinywallet depends on pyqt5 (=5.12.3) which doesn't match any versions, version solving failed.

Running poetry update solved the problem, but it gave me a rather large diff.

[teknico]

Indeed, forgot to update the dependencies and the lock file. Thanks.

[teknico]

Should tinywallet get its own .coveragerc?

It should. It should also get some tests. Until then, the coverage is probably going to be zero. 🙂

poetry install went fine from decred, but has issues with tinywallet which required me to set a specific PyQt5 version.

Interesting, it seemed to work for me with 5.9.2. I will change it to 5.12.3 as mentioned above.

There were a few other issues too. Are you testing tinywallet too?

I am. The only problem I had was the Poetry relative path bug, and I worked around it using an absolute path (that I'm not committing, of course).

[teknico]

Black requires at least Python 3.6, so I prevented it from running under 3.5 in the PR check.

Next, we're using formatted string literals in nine places and they've been introduced in Python 3.6, so pyflakes raises a SyntaxError under 3.5.

I'll remove them from the codebase, and then we'll discover the next feature unsupported by 3.5.

EDIT: removing support for Python 3.5 instead.

[JoeGruffins]

I'm new to poetry buy after reading a little bit tinywallet is running without problems for me.
python was installed via my package manager and poetry with pip3

[buck54321]

Caught one more thing while publishing to test.pypi.

https://test.pypi.org/project/decred/

[buck54321]

Noting here that publishing with Poetry

poetry config repositories.testpypi https://test.pypi.org/simple
poetry publish -r testpypi

was failing silently.

To see my error, I had to

poetry run twine upload --repository-url https://test.pypi.org/legacy/ dist/*

which is sort of roundabout. This appears to be a known issue, so hopefully only a temporary problem. Poetry is still going through some growing pains, apparently, but the goods outweigh the bads for sure.