Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add more developer docs #1089

Closed
wants to merge 3 commits into from
Closed

add more developer docs #1089

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1aa60de
add more developer docs
May 8, 2020
bc2c888
Update docs for pyqt5-dev build
emkll Nov 23, 2020
b2cf872
Use pyqt-sip 4.19.14
emkll Nov 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -357,6 +357,7 @@ Note: One of the functional tests deletes a source, so you may need to add it ba
7. Add a detached signature (with the release key) for the source tarball.
8. Submit the source tarball and signature via PR into this [repository](https://github.com/freedomofpress/securedrop-debian-packaging) along with the debian changelog addition. This tarball and changelog will be used by the package builder.


## Debugging

To use `pdb`, add these lines:
Expand All @@ -369,3 +370,73 @@ Then you can use [`pdb` commands](https://docs.python.org/3/library/pdb.html#deb

Logs can be found in the `{sdc-home}/logs`. If you are debugging a version of this application installed from a deb package in Qubes, you can debug issues by looking at the log file in `~/.securedrop_client/logs/client.log`. You can also add additional log lines in the running code in
`/opt/venvs/securedrop-client/lib/python3.7/site-packages/securedrop_client/`.

Sometimes there is a bug in Qt rather than the client, so it helps to install [a debug version of PyQt5](#build-and-install-a-debug-version-of-pyqt5) which will allow you to see Qt debug symbols in your Python tracebacks. You can use `gdb` to view a traceback as well as a core file, if you're debugging a segfault, but first you must need to install the following some additional packages in order for `gdb` to work with Python programs.

```
sudo apt-get install libclang-dev '^libxcb.*-dev' libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev libxkbcommon-dev libxkbcommon-x11-dev gperf bison flex libgl1-mesa-dev pkg-config libdbus-1-dev libnss3-dev libfontconfig1-dev libxi-dev libxcomposite-dev libxcursor-dev libxtst-dev python3-sip-dev
```

### Build and install a debug version of PyQt5

Note that the resulting artifacts should not be used in production contexts, and are used to debug issues in development environments only.

1. Build a debug version of Qt 5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also add a line about required free space for a debug build.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call.. my qt5-build directory is 1.9 GB, is that what you're seeing?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am seeing significantly more than that:

user@dev-python:~$ du -sh qt5*
6.7G	qt5
5.2G	qt5-build


Clone the Qt repo and build it from source following these instructions: https://wiki.qt.io/Building_Qt_5_from_Git#Getting_the_source_code. This defaults to creating a 'debug' build and installs the binaries in the current directory, avoiding the need for `make install`. Note that the `-developer-build` option causes more symbols to be exported in order to allow more classes and functions to be unit tested than in a regular Qt build.

```
git clone https://code.qt.io/qt/qt5.git
cd qt5
git checkout v5.11.3 # or checkout a different version
git submodule update --init --recursive
export LLVM_INSTALL_DIR=/usr/llvm
mkdir ../qt5-build && cd ../qt5-build # we don't want to build in the source code directory
../qt5/configure -developer-build -opensource -nomake examples -nomake tests -confirm-license
make -j 4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step took ~1h on my computer, perhaps worth mentioning either here or elsewhere in the docs that this takes a while

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will also need to build qttools here for the PySide2.

make -j 4 module-qttools

```

2. Prepare to build PyQt5

We are going to use the client virtual environment to build and install PyQt5. Make sure the client dependencies are installed. This will include `pyqt5-sip` which will be used when we build pyqt5 from source.

If you haven't already, download the client and set up the virtual environment.

```
git clone [email protected]:freedomofpress/securedrop-client.git
cd securedrop-client
virtualenv --python=python3.7 .venv
source .venv/bin/activate
pip install --require-hashes -r dev-requirements.txt
```

Make sure to uninstall the prepackaged pyqt5 library since we will be using our own.

```
pip uninstall pyqt5
```

3. Download the PyQt5 source tarball

```
wget http://deb.debian.org/debian/pool/main/p/pyqt5/pyqt5_5.11.3+dfsg.orig.tar.gz
```

4. Build and install PyQt5

Make sure you are still in the client virtual environment before building PyQt5.

```
tar -xzvf pyqt5_5.11.3+dfsg.orig.tar.gz
cd PyQt5_gpl-5.11.3
pip install PyQt-builder
export PATH=~/qt5-build/qtbase/bin:$PATH
export QT_PLUGIN_PATH=~/qt5-build/qtbase/plugins
python3 configure.py --verbose --qmake=~/qt5-build/qtbase/bin/qmake
make install
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step took 15-20 minutes (17M of disk space used, which is probably negligible in context)


### Find/report a Qt bug

1. Create a Qt account: https://login.qt.io/register
2. Find or report a bug here: https://bugreports.qt.io
1 change: 1 addition & 0 deletions requirements/dev-requirements.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ pyobjc-core==6.2;platform_system=="Darwin"
pyobjc==6.2;platform_system=="Darwin"
pyflakes==2.0.0
PyQt5==5.11.3
PyQt5-sip==4.19.14
pytest==5.2.1
pytest-cov==2.8.1
pytest-mock==1.10.0
Expand Down
32 changes: 14 additions & 18 deletions requirements/dev-requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -262,24 +262,20 @@ pyparsing==2.4.7 \
pyperclip==1.7.0 \
--hash=sha256:979325468ccf682104d5dcaf753f869868100631301d3e72f47babdea5700d1c \
# via mouseinfo
pyqt5-sip==4.19.19 \
--hash=sha256:304acf771b6033cb4bafc415939d227c91265d30664ed643b298d7e95f509f81 \
--hash=sha256:39d2677f4de46ed4d7aa3b612f31c74c881975efe51c6a23fbb1d9382e4cc850 \
--hash=sha256:54b99a3057e8f01b90d49cca9ca566b1ea23d8920038760f44e75b90c62b9d5f \
--hash=sha256:59f5332f86f3ccd3ac94674fe91eae6e8aca26da7c6588917cabd0fe22af106d \
--hash=sha256:72be07a21b0f379987c4ec59bc86834a9719a2f9cfb49606a4d4e34dae9aa549 \
--hash=sha256:7b3b8c015e545fa30e42205fc1115b7c6afcb6acec790ce3f330a06323730523 \
--hash=sha256:7fbb6389c20aff4c3257e89bb1787effffcaf05c32d937c00094ae45846bffd5 \
--hash=sha256:828d9911acc483672a2bae1cc1bf79f591eb3338faad1f2c798aa2f45459a318 \
--hash=sha256:a9460dac973deccc6ff2d90f18fd105cbaada147f84e5917ed79374dcb237758 \
--hash=sha256:aade50f9a1b9d20f6aabe88e8999b10db57218f5c31950160f3f7957dd64e07c \
--hash=sha256:ac9e5b282d1f0771a8310ed974afe1961ec31e9ae787d052c0e504ea46ae323a \
--hash=sha256:ba41bd21b89c6713f7077b5f7d4a1c452989190aad5704e215560a266a1ecbab \
--hash=sha256:c309dbbd6c155e961bfd6893496afa5cd184cce6f7dffd87ea68ee048b6f97e1 \
--hash=sha256:cfc21b1f80d4655ffa776c505a2576b4d148bbc52bb3e33fedbf6cfbdbc09d1b \
--hash=sha256:d7b26e0b6d81bf14c1239e6a891ac1303a7e882512d990ec330369c7269226d7 \
--hash=sha256:f8b7a3e05235ce58a38bf317f71a5cb4ab45d3b34dc57421dd8cea48e0e4023e \
# via pyqt5
pyqt5-sip==4.19.14 \
--hash=sha256:04bd0bb8b6f8fa03c2dfbdfff0c8c9bfb3f46a21dd4cac73983dae93bf949523 \
--hash=sha256:058d450c26be92193605f4628ff690d77080f599ffe381a1029cea8eeb71ab8e \
--hash=sha256:0b838ef8a55461785e78b4e347cf52ce228a5d4392c57e07cc46de51433dc8ac \
--hash=sha256:40504f96ecb834e54491ead558589bfd773056dba7f2df76599a06fdd8ed1ead \
--hash=sha256:49b2151bd0a0e439efc9d4c22c33a048d8e8ede5c7296851c221fa0988887edb \
--hash=sha256:6540b510f9436fe2d65801af55ecbf8c43bdda47294e994ed3851403a93e4a8b \
--hash=sha256:6b3063b12e700944172d57cdbeafb363229669af933f873d01c7a6d8a91c4c87 \
--hash=sha256:6b65d2b14084eb583bf4cf68b97ade295fabae5f5bf2aae0ab00ab30533f1c60 \
--hash=sha256:6ca796071b21761917ee486e57bfa2fc694580e65c462e4173cf849ed8fe201c \
--hash=sha256:6d3013a6820ea614f46fdc73cc16dd57c36a0c74bcbd38bd0b9f2d46b6e6dd16 \
--hash=sha256:84f7401afdd5f31e961de75e9c6b1610849e8883fbe0ed675bbb7d7d97286347 \
--hash=sha256:bb81cfc4d35ca59f1c419b6abeb6ca6a726a63b712cf979f2b5ab24b81c36f49 \
# via -r requirements/dev-requirements.in, pyqt5
pyqt5==5.11.3 \
--hash=sha256:517e4339135c4874b799af0d484bc2e8c27b54850113a68eec40a0b56534f450 \
--hash=sha256:ac1eb5a114b6e7788e8be378be41c5e54b17d5158994504e85e43b5fca006a39 \
Expand Down