Second attempt for easily accessible dependencies #1961

wpotrzebowski · 2021-11-04T20:44:35Z

Follow up on #1903 that broke build and was reverted

wpotrzebowski · 2021-11-04T20:47:49Z

ready for testing on OSX

wpotrzebowski · 2021-11-04T21:07:39Z

@Caddy-Jones: can you please take a look on these warnings apparently causing Jenkins build failure. Note that you need to checkout new branch becuse of (re)reverting...

/Users/sasview/Jenkins/workspace/SasView-OSX10.13-ghprb/sasview/src/sas/qtgui/MainWindow/PackageGatherer.py:docstring of sas.qtgui.MainWindow.PackageGatherer.PackageGatherer:4: WARNING: Unexpected section title.

Methods
-------
/Users/sasview/Jenkins/workspace/SasView-OSX10.13-ghprb/sasview/src/sas/qtgui/MainWindow/PackageGatherer.py:docstring of sas.qtgui.MainWindow.PackageGatherer.PackageGatherer.format_unattainable_modules_list:10: WARNING: Unexpected section title.

Parameters
----------
/Users/sasview/Jenkins/workspace/SasView-OSX10.13-ghprb/sasview/src/sas/qtgui/MainWindow/PackageGatherer.py:docstring of sas.qtgui.MainWindow.PackageGatherer.PackageGatherer.format_unattainable_modules_list:17: WARNING: Unexpected section title.

Returns
-------
/Users/sasview/Jenkins/workspace/SasView-OSX10.13-ghprb/sasview/src/sas/qtgui/MainWindow/PackageGatherer.py:docstring of sas.qtgui.MainWindow.PackageGatherer.PackageGatherer.get_imported_modules:8: WARNING: Unexpected section title.

Returns
-------
/Users/sasview/Jenkins/workspace/SasView-OSX10.13-ghprb/sasview/src/sas/qtgui/MainWindow/PackageGatherer.py:docstring of sas.qtgui.MainWindow.PackageGatherer.PackageGatherer.log_imported_modules:7: WARNING: Unexpected section title.

Returns
-------
/Users/sasview/Jenkins/workspace/SasView-OSX10.13-ghprb/sasview/src/sas/qtgui/MainWindow/PackageGatherer.py:docstring of sas.qtgui.MainWindow.PackageGatherer.PackageGatherer.log_installed_modules:7: WARNING: Unexpected section title.

Returns
-------
/Users/sasview/Jenkins/workspace/SasView-OSX10.13-ghprb/sasview/src/sas/qtgui/MainWindow/PackageGatherer.py:docstring of sas.qtgui.MainWindow.PackageGatherer.PackageGatherer.remove_duplicate_modules:10: WARNING: Unexpected section title.

Parameters
----------
/Users/sasview/Jenkins/workspace/SasView-OSX10.13-ghprb/sasview/src/sas/qtgui/MainWindow/PackageGatherer.py:docstring of sas.qtgui.MainWindow.PackageGatherer.PackageGatherer.remove_duplicate_modules:15: WARNING: Unexpected section title.

src/sas/qtgui/MainWindow/PackageGatherer.py

pkienzle · 2021-11-04T21:36:15Z

src/sas/qtgui/MainWindow/PackageGatherer.py

+        module_versions_dict = {'python': sys.version, 'SasView': sas.sasview.__version__}
+        unattainable_modules = []
+        # Generate a list of standard modules by looking at the local python library
+        standard_lib = [path.stem.split('.')[0] for path in pathlib.Path(pathlib.__file__).parent.absolute().glob('*')]


You can combine the standard lib packages, the builtin packages and sas into a single set. This simplifies the test in the loop:

standard_path = pathlib.Path(pathlib.__file__).parent.absolute() standard_lib = {path.stem.split('.')[0] for path in standard_path.glob('*')} standard_lib.update(sys.builtin_module_names) standard_lib.add('sas') # include 'sas' as standard lib

Note that this will not do what you want in the shipped application because pathlib may be in a zip file instead of on the disk. It may even fail in the application since pathlib.__file__ might not be defined. This is why I suggested hard-coding the list rather than building it on the fly.

Hi, so I've included your standard_path code in a try except block, with a hard coded list of bultin modules in the event that pathlib.file causes an error.

Could someone check if pathlib.file does indedd fail in the shipped application, as I would not know how to check.

If it does fail, then the list of builtin modules will probably have to be checked each time the python version SasView uses is updated.

May be worth having this list in an external file, making the job of updating the list easier?

src/sas/qtgui/MainWindow/PackageGatherer.py

pkienzle · 2021-11-04T22:02:53Z

src/sas/qtgui/MainWindow/PackageGatherer.py

+        installed_modules = {'python': sys.version}
+
+        # Get python modules installed locally
+        installed_modules_json = json.loads(subprocess.check_output("pip list -l --format=json"))


Fails for installed application.

src/sas/qtgui/MainWindow/PackageGatherer.py

Caddy-Jones · 2021-11-05T10:58:43Z

Hi @wpotrzebowski, The errors appear to come from the titles I used in the docstrings, causing “WARNING: Unexpected section title.” I used the NumPy/SciPy Docstring format for the docstings in the PackageGatherer class. Is there a different docstring standard used by SasView?

pkienzle · 2021-11-05T17:48:22Z

Hi @wpotrzebowski, The errors appear to come from the titles I used in the docstrings, causing “WARNING: Unexpected section title.” I used the NumPy/SciPy Docstring format for the docstings in the PackageGatherer class. Is there a different docstring standard used by SasView?

Coding standards for docstrings are here. We don't suggest (or follow) any particular style and there is wide variation across the project. So long as it works with sphinx it's okay.

Instead of underlined Section heading I've used **Section heading** to get a similar effect.

Caddy-Jones · 2021-11-08T10:50:13Z

Is there any way to add a new branch to this pull request for the code changes? If not i can set up a new PR

wpotrzebowski · 2021-11-08T14:20:02Z

@Caddy-Jones I think the easiest is to checkout this branch revert-1958-revert-1903-1869_WIP_easily_accessible_dependencie and add code there.

…ndencies' into easily_accessible_dependencies_2

Easily accessible dependencies 2

src/sas/qtgui/MainWindow/PackageGatherer.py

pkienzle · 2021-11-08T15:42:02Z

src/sas/qtgui/MainWindow/PackageGatherer.py

+                logging.error(f"{x} when attempting to access {mod} version using .version")
+                pass
+            else:
+                continue


Same comment as above re: exception handling.

And to repeat, this will probably not do what you want.

Example

>>> h5py.version <module 'h5py.version' from '/Users/pkienzle/conda/envs/py38/lib/python3.8/site-packages/h5py/version.py'>

Though in this case h5py.__version__ exists so you won't get here.

If you insist on keeping this clause at least check if isinstance(package.version, str).

Caddy-Jones · 2021-11-09T16:23:40Z

Hi @pkienzle .

Thanks for your feedback, the reason i included the .version check is because some modules, do not have a version attribute, and instead .version returns the version number. But i will include a check to see if the version number is a string or a float.

…ndencies' into easily_accessible_dependencies_2

Responding to feedback

src/sas/qtgui/MainWindow/PackageGatherer.py

…ndencies' into easily_accessible_dependencies_2

Removing .version check

wpotrzebowski · 2021-11-23T14:35:20Z

@wpotrzebowski will take a look on it

wpotrzebowski · 2021-11-24T06:46:59Z

ready for testing on OSX

wpotrzebowski · 2021-11-24T08:11:25Z

@Caddy-Jones you may consider fixing this one before we merge:
/src/sas/qtgui/MainWindow/PackageGatherer.py:docstring of sas.qtgui.MainWindow.PackageGatherer.PackageGatherer.remove_duplicate_modules:15: WARNING: Field list ends without a blank line; unexpected unindent

Caddy-Jones · 2021-11-24T10:33:57Z

Hi @wpotrzebowski , is there a way to test that the docstring is formated correctly?

wpotrzebowski · 2021-11-24T10:52:15Z

Hi @wpotrzebowski , is there a way to test that the docstring is formated correctly?

@Caddy-Jones I've just tested it by running docs build locally and looking for warnings (you can '-w' oprtion to build_sphinx.py to output warnings to file).

Caddy-Jones · 2021-11-24T11:05:56Z

Hi @wpotrzebowski , where am i supposed to run build_sphinx.py from, i keep getting errors with the script unable to find sasmodels.

smk78 · 2021-11-24T13:56:15Z

@Caddy-Jones you should be able to do python setup.py build docs from the sasview top level. Otherwise, look in \docs\sphinx-docs. But you have to build the sasmodels docs first. Go to sasmodels\doc and make clean html.

Caddy-Jones · 2021-11-24T15:12:14Z

Hi guys, I just cannot recreate the results @wpotrzebowski has gotten, can you give me the command line input you used please

smk78 · 2021-11-25T10:34:44Z

@Caddy-Jones I've pulled your branch and built the docs locally and I'm seeing the exact same doc error that @wpotrzebowski reports. I'm running Sphinx 1.8.4 (according to pip list). But nothing I've tried so far has fixed the error! :-(

Caddy-Jones · 2021-11-25T13:04:55Z

Apologies but I can’t seem to generate the sphinx documentation, anything I try gives me an error. Could somebody give me a step by step explanation of how to generate the documents, or point me to the location of such a thing. For example trying to do any make command gives me
'make' is not recognized as an internal or external command,operable program or batch file.
Trying to run the sphinx-build.py script gives me

=== Sasmodels Docs ===
Traceback (most recent call last):
  File "build_sphinx.py", line 294, in <module>
    rebuild()
  File "build_sphinx.py", line 280, in rebuild
    retrieve_user_docs()
  File "build_sphinx.py", line 134, in retrieve_user_docs
    copy_tree(SASMODELS_MODEL_SOURCE, SASMODELS_MODEL_TARGET)
  File "C:\Code_base\Anaconda3\envs\qt5_win\lib\distutils\dir_util.py", line 124, in copy_tree
    "cannot copy tree '%s': not a directory" % src)
distutils.errors.DistutilsFileError: cannot copy tree 'C:\Local_repo\sasview\docs\sphinx-docs\..\..\..\sasmodels\doc\model': not a directory

setup.py build docs Isn’t an option for sasmodels, and doesn’t give me the results wojeck had for sasview, instead i get an error:

copying src\sas\sasview\test\upcoming_formats\1000A_sphere_sm.xml -> build\lib\sas\sasview\test\upcoming_formats
running docs
========= check for sasmodels at C:\Local_repo\sasmodels\doc ============
============= Building sasmodels model documentation ===============
error: [WinError 2] The system cannot find the file specified

smk78 · 2021-11-25T13:53:02Z

@Caddy-Jones : Go to your sasmodels repo. cd doc. make clean html. If make gives you an error then presumably you don't have it installed? Once the sasmodels docs are built, cd to your sasview branch and python setup.py build docs. The sasview doc build copies the built sasmodels docs to the sasview\docs\sphinx-docs\build folder. If all has worked you should then be able to cd \html and open index.html and navigate around the docs. If you're using an Anaconda environment, remember you have to activate the environment before doing any of this! :-)

Caddy-Jones · 2021-11-25T14:32:47Z

Thanks @smk78 , turns out make wasn't installed and there was an issue with my conda environment which prevented me from installing it. I've fixed the issue with the docstring so there hopefully should be no further issues

Doc string fix

smk78 · 2021-11-25T14:38:06Z

Yes, I've just pulled your branch and re-made the docs and they pass compilation now. Good!

wpotrzebowski · 2021-11-28T21:01:52Z

ready for testing on OSX

Revert "Revert "1869 easily accessible dependencies""

215d409

pkienzle requested changes Nov 4, 2021

View reviewed changes

Responding to feedback

4fbd2ff

Variable and method name changes

e57f363

Caddy-Jones added 2 commits November 8, 2021 15:29

Merge branch 'revert-1958-revert-1903-1869_WIP_easily_accessible_depe…

fd0e071

…ndencies' into easily_accessible_dependencies_2

Merge pull request #1963 from SasView/easily_accessible_dependencies_2

72c2822

Easily accessible dependencies 2

pkienzle reviewed Nov 8, 2021

View reviewed changes

Caddy-Jones and others added 3 commits November 9, 2021 17:26

Responding to feedback

882bc40

Merge branch 'revert-1958-revert-1903-1869_WIP_easily_accessible_depe…

af0dce0

…ndencies' into easily_accessible_dependencies_2

Merge pull request #1964 from SasView/easily_accessible_dependencies_2

4a9e8b5

Responding to feedback

pkienzle reviewed Nov 9, 2021

View reviewed changes

src/sas/qtgui/MainWindow/PackageGatherer.py Outdated Show resolved Hide resolved

src/sas/qtgui/MainWindow/PackageGatherer.py Outdated Show resolved Hide resolved

src/sas/qtgui/MainWindow/PackageGatherer.py Outdated Show resolved Hide resolved

Caddy-Jones and others added 3 commits November 15, 2021 13:27

Removing .version check

cf51d47

Merge branch 'revert-1958-revert-1903-1869_WIP_easily_accessible_depe…

d7621db

…ndencies' into easily_accessible_dependencies_2

Merge pull request #1968 from SasView/easily_accessible_dependencies_2

e43dbbc

Removing .version check

butlerpd changed the base branch from main to release_5.0.5 November 20, 2021 22:38

wpotrzebowski added the Discuss At The Call Issues to be discussed at the fortnightly call label Nov 22, 2021

butlerpd removed the Discuss At The Call Issues to be discussed at the fortnightly call label Nov 23, 2021

Doc string fix

418e381

Merge pull request #1978 from SasView/easily_accessible_dependencies_2

3a2a1b0

Doc string fix

wpotrzebowski merged commit afc53ff into release_5.0.5 Nov 28, 2021

wpotrzebowski deleted the revert-1958-revert-1903-1869_WIP_easily_accessible_dependencies branch November 28, 2021 21:14

krzywon mentioned this pull request Sep 17, 2024

Replace deprecated pkg_resources check #3096

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Second attempt for easily accessible dependencies #1961

Second attempt for easily accessible dependencies #1961

wpotrzebowski commented Nov 4, 2021

wpotrzebowski commented Nov 4, 2021

wpotrzebowski commented Nov 4, 2021

pkienzle Nov 4, 2021

Caddy-Jones Nov 8, 2021

pkienzle Nov 4, 2021

Caddy-Jones Nov 8, 2021

Caddy-Jones commented Nov 5, 2021

pkienzle commented Nov 5, 2021

Caddy-Jones commented Nov 8, 2021 •

edited

Loading

wpotrzebowski commented Nov 8, 2021

pkienzle Nov 8, 2021

Caddy-Jones commented Nov 9, 2021

wpotrzebowski commented Nov 23, 2021

wpotrzebowski commented Nov 24, 2021

wpotrzebowski commented Nov 24, 2021

Caddy-Jones commented Nov 24, 2021

wpotrzebowski commented Nov 24, 2021

Caddy-Jones commented Nov 24, 2021

smk78 commented Nov 24, 2021

Caddy-Jones commented Nov 24, 2021

smk78 commented Nov 25, 2021

Caddy-Jones commented Nov 25, 2021

smk78 commented Nov 25, 2021

Caddy-Jones commented Nov 25, 2021

smk78 commented Nov 25, 2021

wpotrzebowski commented Nov 28, 2021

Second attempt for easily accessible dependencies #1961

Second attempt for easily accessible dependencies #1961

Conversation

wpotrzebowski commented Nov 4, 2021

wpotrzebowski commented Nov 4, 2021

wpotrzebowski commented Nov 4, 2021

pkienzle Nov 4, 2021

Choose a reason for hiding this comment

Caddy-Jones Nov 8, 2021

Choose a reason for hiding this comment

pkienzle Nov 4, 2021

Choose a reason for hiding this comment

Caddy-Jones Nov 8, 2021

Choose a reason for hiding this comment

Caddy-Jones commented Nov 5, 2021

pkienzle commented Nov 5, 2021

Caddy-Jones commented Nov 8, 2021 • edited Loading

wpotrzebowski commented Nov 8, 2021

pkienzle Nov 8, 2021

Choose a reason for hiding this comment

Caddy-Jones commented Nov 9, 2021

wpotrzebowski commented Nov 23, 2021

wpotrzebowski commented Nov 24, 2021

wpotrzebowski commented Nov 24, 2021

Caddy-Jones commented Nov 24, 2021

wpotrzebowski commented Nov 24, 2021

Caddy-Jones commented Nov 24, 2021

smk78 commented Nov 24, 2021

Caddy-Jones commented Nov 24, 2021

smk78 commented Nov 25, 2021

Caddy-Jones commented Nov 25, 2021

smk78 commented Nov 25, 2021

Caddy-Jones commented Nov 25, 2021

smk78 commented Nov 25, 2021

wpotrzebowski commented Nov 28, 2021

Caddy-Jones commented Nov 8, 2021 •

edited

Loading