Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc building fixes #1744

Merged
merged 5 commits into from
Apr 18, 2021
Merged

Doc building fixes #1744

merged 5 commits into from
Apr 18, 2021

Conversation

llimeht
Copy link
Contributor

@llimeht llimeht commented Jan 15, 2021

The major reason for the current failure to build the sasview docs is sasmodels being monkeypatched into sas.models. Having loaded all the classes, sphinx knows that sas.models should exist but then can't actually load any source for them. That results in the rather odd errors that we have been seeing:

WARNING: autodoc: failed to import class 'sas.models.FlexCylEllipXModel' from module 'sas.sascalc.fit'; the following exception was raised:
…
AttributeError: module 'sas.sascalc.fit' has no attribute 'sas'                                                                                                                                                                         

Note that this PR is not enough to actually make the documentation build cleanly, with the following 19 warnings/errors still emitted:

…/sasmodels/sasmodels/resolution.py:docstring of sasmodels.resolution.eval_form:3: WARNING: Unknown interpreted text role "fun".
…/sasview/src/sas/qtgui/Calculators/GenericScatteringCalculator.py:docstring of sas.qtgui.Calculators.GenericScatteringCalculator.GenericScatteringCalculator._create_default_1d_data:4: WARNING: Unexpected indentation.                                                                                                                                                                                                
…/sasview/src/sas/qtgui/Calculators/GenericScatteringCalculator.py:docstring of sas.qtgui.Calculators.GenericScatteringCalculator.GenericScatteringCalculator._create_default_1d_data:5: WARNING: Block quote ends without a blank line; unexpected unindent.                                                                                                                                                            
…/sasview/src/sas/qtgui/MainWindow/DataManager.py:docstring of sas.qtgui.MainWindow.DataManager.DataManager.__init__:3: WARNING: Unexpected indentation.
…/sasview/src/sas/qtgui/MainWindow/DataManager.py:docstring of sas.qtgui.MainWindow.DataManager.DataManager.__init__:4: WARNING: Block quote ends without a blank line; unexpected unindent.                                                                                                                                                                                                                             
…/sasview/src/sas/qtgui/MainWindow/DataManager.py:docstring of sas.qtgui.MainWindow.DataManager.DataManager.__init__:5: WARNING: Field list ends without a blank line; unexpected unindent.                                                                                                                                                                                                                              
…/sasview/src/sas/qtgui/Plotting/Arrow3D.py:docstring of sas.qtgui.Plotting.Arrow3D.Arrow3D.__init__:7: WARNING: Field list ends without a blank line; unexpected unindent.
…/sasview/src/sas/qtgui/Plotting/Binder.py:docstring of sas.qtgui.Plotting.Binder.BindArtist.__call__:34: WARNING: Unexpected indentation.
…/sasview/src/sas/qtgui/Plotting/Binder.py:docstring of sas.qtgui.Plotting.Binder.BindArtist.__call__:35: WARNING: Block quote ends without a blank line; unexpected unindent.
…/sasview/src/sas/qtgui/Plotting/Binder.py:docstring of sas.qtgui.Plotting.Binder.BindArtist.__call__:40: WARNING: Unexpected indentation.
…/sasview/src/sas/qtgui/Plotting/Binder.py:docstring of sas.qtgui.Plotting.Binder.BindArtist.__call__:53: WARNING: Unexpected indentation.
…/sasview/src/sas/qtgui/Plotting/Binder.py:docstring of sas.qtgui.Plotting.Binder.BindArtist.__call__:54: WARNING: Block quote ends without a blank line; unexpected unindent.
…/sasview/src/sas/sascalc/data_util/nxsunit.py:docstring of sas.sascalc.data_util.nxsunit._build_metric_units:3: WARNING: Unexpected indentation.
…/sasview/src/sas/sascalc/data_util/nxsunit.py:docstring of sas.sascalc.data_util.nxsunit._build_metric_units:6: WARNING: Block quote ends without a blank line; unexpected unindent.                                                                                                                                                                                                                                    
…/sasview/src/sas/sascalc/dataloader/manipulations.py:docstring of sas.sascalc.dataloader.manipulations._Sector.__init__:2: WARNING: Field list ends without a blank line; unexpected unindent.                                                                                                                                                                                                                          
…/sasview/src/sas/sascalc/fit/models.py:docstring of sas.sascalc.fit.models._check_plugin:4: WARNING: Field list ends without a blank line; unexpected unindent.
…/sasview/src/sas/sascalc/invariant/invariant.py:docstring of sas.sascalc.invariant.invariant.InvariantCalculator._fit:9: WARNING: Field list ends without a blank line; unexpected unindent.                                                                                                                                                                                                                            
…/sasview/src/sas/sascalc/invariant/invariant.py:docstring of sas.sascalc.invariant.invariant.InvariantCalculator._get_data:4: WARNING: Inline emphasis start-string without end-string.                                                                                                                                                                                                                                 
…/sasview/src/sas/sascalc/invariant/invariant.py:docstring of sas.sascalc.invariant.invariant.InvariantCalculator._get_qstar_uncertainty:6: WARNING: Literal block ends without a blank line; unexpected unindent.                                                                                 

Given that there are several PRs that touch those files already waiting to be merged, touching those files now will just create merge conflicts and so these problems need to be fixed later.

(There's also a handful of other misc fixes that I spotted while chasing this failure over the past few days.)

@wpotrzebowski wpotrzebowski self-requested a review January 15, 2021 18:46
@wpotrzebowski
Copy link
Contributor

ready for testing on OSX

@wpotrzebowski
Copy link
Contributor

ready for testing on Win

@wpotrzebowski
Copy link
Contributor

ready for testing on Linux

Copy link
Contributor

@wpotrzebowski wpotrzebowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be failing on all jenkins builds (Win, OSX, Linux) in
=== Build HTML Docs from ReST Files ===section with error:
subprocess.CalledProcessError: Command '['sphinx-build', '-v', '-b', 'html', '-d', '/Users/sasview/Jenkins/workspace/SasView-OSX10.13-ghprb/sasview/docs/sphinx-docs/build/doctrees', '-W', '--keep-going', '/Users/sasview/Jenkins/workspace/SasView-OSX10.13-ghprb/sasview/docs/sphinx-docs/source-temp', '/Users/sasview/Jenkins/workspace/SasView-OSX10.13-ghprb/sasview/docs/sphinx-docs/build/html']' returned non-zero exit status 1.

@llimeht
Copy link
Contributor Author

llimeht commented Jan 16, 2021

@wpotrzebowski, yes that is expected. The docs don't build cleanly yet -- this is just one step closer to getting them to build at all.

Your test with jenkins highlights another problem, however. The jenkins builds are using much older versions of sphinx than CI (both Travis and GitHub Actions); it's not practical to support such a wide range of sphinx versions in one project. I suspect that we're now reliant on newer features of sphinx and that's breaking the build.

Looking in build_tools I see 6 different versions of sphinx are pinned in the various build environments, with travis/actions pulling yet newer versions. Removing the version pinning everywhere in build_tools might be enough to fix this problem (or changing it to ">3.4" to make sure an old version that is already installed doesn't satisfy the requirement).

I suspect the path forward is to fix the jenkins environment to better match the development environment.

@wpotrzebowski
Copy link
Contributor

@llimeht Indeed different yaml files in build_tools folder refer to different sphinx versions (we need to clean it up!), however for build systems they all should use sphinx=1.8.5, which I believe is the latest version. I've upgraded it recently because builds were failing on keep-going flag.

@llimeht
Copy link
Contributor Author

llimeht commented Jan 16, 2021

travis and github actions are picking up the current version of sphinx: 3.4.3

"/home/travis/miniconda/bin/python" -m sphinx -b html -d _build/doctrees -D latex_elements.papersize=letter  . _build/html

Running Sphinx v3.4.3

@wpotrzebowski
Copy link
Contributor

It seems that 3.4.3 is and 1.8.5 is the same version
Screenshot 2021-01-16 at 21 01 56

@llimeht
Copy link
Contributor Author

llimeht commented Jan 17, 2021

I have no idea what conda is showing there but it looks pretty wrong. Sphinx 1.8.5 is from March 2019 and there have been 31 releases of Sphinx since then, right through to 3.4.3 in Jan 2021 (changelog).

@butlerpd
Copy link
Member

I think that was from this page https://anaconda.org/anaconda/sphinx. Am guessing that is just listing which versions are available as conda packages? A bit worrisome however how frequent the releases are coming?

@wpotrzebowski
Copy link
Contributor

Interesting with versioning. I've managed to get sphinx 3.4.3 to OSX pull request builder. Let see if this prevents build from failing

@wpotrzebowski
Copy link
Contributor

ready for testing on OSX

@wpotrzebowski
Copy link
Contributor

Failing with sphinx 3.4.3 but with different error:
Exception occurred: File "/Users/sasview/miniconda3/envs/qt5_osx/lib/python3.6/site-packages/matplotlib/sphinxext/plot_directive.py", line 268, in setup app.add_directive('plot', plot_directive, True, (0, 2, False), **options) TypeError: add_directive() got an unexpected keyword argument 'alt' The full traceback has been saved in /var/folders/s_/937ts2nn0nd_wf4cp_dcs5980000gp/T/sphinx-err-z8ednwgw.log, if you want to report the issue to the developers.
It seems like matplotlib related issue

@llimeht
Copy link
Contributor Author

llimeht commented Jan 17, 2021

I suspect that's from the old version of matplotlib that is pinned there (2.1 or 2.2 rather than 3.3 that travis/github actions is using).

@llimeht
Copy link
Contributor Author

llimeht commented Apr 5, 2021

I've dropped off the very last commit that was causing issues with ancient sphinx so that the rest of these should be safe to merge.

Note that errors in building the docs in github actions are still ignored at present and that output documentation is most likely not quite as intended. Fixing the docs so that they build correctly and making CI gate on that is still a rather important task.

For reference, most of the error are of the form:

WARNING: autodoc: failed to import class 'sas.models.BCCrystalModel' from module 'sas.sascalc.fit'; the following exception was raised:
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/sphinx/util/inspect.py", line 393, in safe_getattr
    return getattr(obj, name, *defargs)
AttributeError: module 'sas.sascalc.fit' has no attribute 'sas'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/sphinx/ext/autodoc/importer.py", line 111, in import_object
    obj = attrgetter(obj, mangled_name)
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 320, in get_attr
    return autodoc_attrgetter(self.env.app, obj, name, *defargs)
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 2604, in autodoc_attrgetter
    return safe_getattr(obj, name, *defargs)
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/sphinx/util/inspect.py", line 409, in safe_getattr
    raise AttributeError(name) from exc
AttributeError: sas

and these come from monkeypatching in sasmodels into sas.models. Telling autodoc to skip everything under sas.models is likely necessary; doing that in a fashion that is compatible with both sphinx 1.8 and 3.4 is likely a fools errand, and fixing the CI to not use such a wide range of sphinx versions would simply the remaining work.

llimeht added a commit to llimeht/sasview that referenced this pull request Apr 5, 2021
sas.models is a monkeypatched sasmodels that breaks sphinx autodoc.

Fixing this in a way that is compatible with both sphinx 1.8 and 3.x seems
difficult, so put a sphinx version check on the fix.

See SasView#1744
@wpotrzebowski
Copy link
Contributor

ready for testing on OSX

@wpotrzebowski wpotrzebowski merged commit 5fe1a0a into SasView:main Apr 18, 2021
@llimeht llimeht deleted the tmp/doc-fixes branch December 27, 2021 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants