ESS_GUI: Save/Load Corfunc and Invariant Perspectives #1675

krzywon · 2020-09-22T19:52:30Z

This allows data from the Corfunc and Invariant perspectives to be saved and loaded in project and analysis files. This also fixes a few other issues associated with loading projects. Unit tests added for all serialization methods.

Closes #1526 : All perspectives can now be saved and loaded
Fixes #1657 : Loading project files without fitting perspective data now load properly
Fixes #1655 : Corfunc and Invariant perspectives are now able to delete and swap data as needed
Fixes #1674 : The data names are now reset when loading a new project
Fixes #1539 (Unintended fix) : The name now appears the first time you send data to the Corfunc perspective

…sis save/load methods to include Invariant.

…ective.py.

…d use Fitpage swap mechanism when loading projects. refs #1657

… state.

…unc perspectives.

…e and add unit tests.

…corfunc unit tests.

…o Invariant and Corfunc perspectives. Fix bugs found from those tests.

…were failing.

…to be sure the 'isSerializable' value is set properly.

# Conflicts: # src/sas/qtgui/MainWindow/DataExplorer.py # src/sas/qtgui/Perspectives/Corfunc/CorfuncPerspective.py

…om Corfunc and add verification when swapping data in Corfunc.

…ading a project refs #1674

pkienzle

Code looks okay but I didn't test.

src/sas/qtgui/MainWindow/GuiManager.py

pkienzle · 2020-09-24T16:05:16Z

src/sas/qtgui/Perspectives/Fitting/FittingPerspective.py

-            # Tell the MdiArea to close the container
-            self.parentWidget().close()
+            # Tell the MdiArea to close the container if it is visible
+            if self.parentWidget():


Somewhat suspicious: the code requesting close gets here so the widget exists, but it is not bound to a parent so it doesn't get removed.

Is this a dangling pointer, where the widget was already removed, but some other function is saying to remove it again? Or is this a zombie window which might reappear later even though you told all windows to close?

The same code is in invariant and inversion, so update/ignore in all places.

The parentWidget for a perspective will be a widget in the main window, but only if that perspective is active. All non-active perspectives are removed from the main window and held in the self.loadedPerspectives variable of GuiManager.

src/sas/qtgui/Utilities/GuiUtils.py

…t to list when saving.

src/sas/qtgui/MainWindow/GuiManager.py

rozyczko

Works as expected.
Code is ready, too.

rozyczko · 2020-09-28T21:09:29Z

src/sas/qtgui/Perspectives/Corfunc/CorfuncPerspective.py

+        Collects all active params into a dictionary of {name: value}
+        :return: {name: value}
+        """
+        return {


This should probably query the model (self.model) rather than UI elements.

smk78 · 2020-10-30T15:22:32Z

Built and tested this branch on W10/x64.

Performed correlation function analysis on \test\1d data\ISIS_98929.TXT (load data, send to Corfunc, Calculate background, Extrapolate, Transform & Extract Parameters). Then did Save Project. Restarted SasView and did Load Project.

Everything comes back except the Real Space Correlations plot. See below.

Which is a bit odd because it has re-drawn the Scattering Data plot.

However, it does know about the transformed data, because if you do Create New it plots it:

smk78 · 2020-10-30T15:46:23Z

Performed Invariant analysis on \test\1d data\Ludox_silica.xml (load data, send to Invariant, changed background, Calculate). Then did Save Project. Restarted SasView and did Load Project.

Seemed to work as expected.

smk78 · 2020-10-30T16:19:08Z

After performing the test of Invariant project saving/loading above, I then, without closing the session (ie, SasView contained the reloaded Invariant project), loaded ISIS98929.TXT and performed Correlation Function analysis. I then saved the session to a new project. This took a while but no cause was immediately obvious. But when I closed SasView I found my Command Prompt window filled with the following messages (so many they filled the window buffer):

INFO: data cannot be serialized to json: <class 'numpy.float32'>

Closing SasView, restarting it, then re-loading the saved invariant-then-corfunc project file did work, and I could toggle between the two analyses using the Analysis menu option.

I then went 'the whole 9 yards', loading AOT_Microemulsion-Drop_Contrast.xml and fitting it with 10% Lognormal PD aswell. I then saved the session to a new project, closed SasView, restarted it, and re-loaded the invariant-then-corfunc-then-fitting project file. This also worked, though none of the fitting plots (data, residuals or size distribution) appeared until I did Show Plot in the FitPage. Interestingly the 'Create New' plot I did after doing invariant-then-corfunc was displayed.

project_files.zip

smk78

So, in summary: this PR does reinstate the ability to save correlation function or invariant analyses to projects, either singly, or in combination with one another, or in combination with fitting analyses without any apparent ill effects. For this reason I will approve the PR.

However, as my comments above demonstrate, there is some weirdness regarding what plots are redrawn and which are not.

butlerpd · 2020-11-09T23:54:16Z

So the question now seems to be whether @krzywon thinks we should merge and create new tickets based on the testing or whether some or all of these issues are "obvious" and should be fixed as part of the PR. To summarize my understanding of the issues (@smk78 can correct)

Real Space Correlation plot does not populate properly on loading project (but can be manually forced to plot)
saving PR and Invariant (at least in the sequence described) is a bit slow to save but otherwise works perfectly fine "functionally." However there are a ton of "data cannot be serialized" info messages (probably from a loop?). As info they are not errors I guess but perhaps this at least can be fixed by moving the info statement outside the loop?
Fitting plots do not show on loading fitting project at least when part of a Invariant+corfunc+fitting saved project. Question:What is the current behavior of fitting save? is this part of the problem in not seeing plots in fitting till doing a fit or asking to show plot?

…rfunc bottons where appropriate.

krzywon · 2020-11-10T19:44:48Z

The Corfunc correlation plot is now updated and the buttons are enabled properly with my latest push.
The 'data cannot be serialized' only happens when data loaded from a project is saved again and only effects the data in plots. I wasn't able to solve it here and opened a new issue Resaving Loaded Projects - 'data cannot be serialized' #1706.
Invariant plots are now plotted, but the plots are still effected by issues Invariant and the infinite multiplication of plots #1541 and Crosstalk between Corfunc and Invariant perspectives #1542.

krzywon added 17 commits August 6, 2020 09:36

Serialize/deserialize Invariant perspective. Update project and analy…

754a079

…sis save/load methods to include Invariant.

Add data removal routine and tweak state generation in InvariantPersp…

6a8e71a

…ective.py.

Close all perspectives instead of relying on data deletion schemes an…

931878a

…d use Fitpage swap mechanism when loading projects. refs #1657

Finalize invariant serialization and deserialization routines.

5bb10e3

Add data removal routine to the Corfunc perspective. refs #1655

5687d81

Serialize Corfunc perspective and allow main window to get serialized…

46a1423

… state.

Bug fixes associated with data removal methods for Invariant and Corf…

d7d738b

…unc perspectives.

Bug fixes associated with serialiation methods for Corfunc perspectiv…

03d152d

…e and add unit tests.

Send corfunc parameters to perspective when loading projects and fix …

96caf67

…corfunc unit tests.

Add Invariant serialization unit tests, bug fixing, and lint.

daf3cd2

Add unit tests for removing data and loading params from dictionary t…

54e09e8

…o Invariant and Corfunc perspectives. Fix bugs found from those tests.

Fix issues with corfunc serialization and associated unit tests that …

00afdc0

…were failing.

Bug fixes and add general unit test to all serializable perspectives …

ff3cbe4

…to be sure the 'isSerializable' value is set properly.

Merge branch 'ESS_GUI' into ESS_GUI_serialize_corfunc_invariant

e69907f

# Conflicts: # src/sas/qtgui/MainWindow/DataExplorer.py # src/sas/qtgui/Perspectives/Corfunc/CorfuncPerspective.py

Remove time-heavy routine that doesn't add much when removing data fr…

cfe995a

…om Corfunc and add verification when swapping data in Corfunc.

Fix failing unit tests

dd79ee0

Fix issue storing numpy floats and reset data name dictionary when lo…

16eab7e

…ading a project refs #1674

krzywon added this to the SasView 5.0.4 milestone Sep 22, 2020

krzywon requested review from rozyczko and smk78 September 22, 2020 19:52

pkienzle reviewed Sep 24, 2020

View reviewed changes

Add exception handling when closing/loading perspectives and set floa…

2c7269d

…t to list when saving.

pkienzle reviewed Sep 28, 2020

View reviewed changes

src/sas/qtgui/MainWindow/GuiManager.py Outdated Show resolved Hide resolved

Fix typo.

50444e8

krzywon mentioned this pull request Oct 21, 2020

Project load of Theory items causes the Fit panel to go blank #1689

Open

wpotrzebowski changed the base branch from ESS_GUI to main October 26, 2020 07:20

rozyczko approved these changes Oct 27, 2020

View reviewed changes

smk78 approved these changes Oct 30, 2020

View reviewed changes

Plot any corfunc and invariant plots when loading projects. Enable co…

cb851b1

…rfunc bottons where appropriate.

krzywon mentioned this pull request Nov 10, 2020

Resaving Loaded Projects - 'data cannot be serialized' #1706

Open

butlerpd merged commit 6f32287 into main Nov 24, 2020

krzywon deleted the ESS_GUI_serialize_corfunc_invariant branch November 24, 2020 14:27

smk78 mentioned this pull request Jun 24, 2022

Enhancements to Corfunc #1502

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ESS_GUI: Save/Load Corfunc and Invariant Perspectives #1675

ESS_GUI: Save/Load Corfunc and Invariant Perspectives #1675

krzywon commented Sep 22, 2020

pkienzle left a comment

pkienzle Sep 24, 2020

krzywon Sep 28, 2020

rozyczko left a comment

rozyczko Sep 28, 2020

smk78 commented Oct 30, 2020

smk78 commented Oct 30, 2020

smk78 commented Oct 30, 2020 •

edited

Loading

smk78 left a comment

butlerpd commented Nov 9, 2020

krzywon commented Nov 10, 2020

ESS_GUI: Save/Load Corfunc and Invariant Perspectives #1675

ESS_GUI: Save/Load Corfunc and Invariant Perspectives #1675

Conversation

krzywon commented Sep 22, 2020

pkienzle left a comment

Choose a reason for hiding this comment

pkienzle Sep 24, 2020

Choose a reason for hiding this comment

krzywon Sep 28, 2020

Choose a reason for hiding this comment

rozyczko left a comment

Choose a reason for hiding this comment

rozyczko Sep 28, 2020

Choose a reason for hiding this comment

smk78 commented Oct 30, 2020

smk78 commented Oct 30, 2020

smk78 commented Oct 30, 2020 • edited Loading

smk78 left a comment

Choose a reason for hiding this comment

butlerpd commented Nov 9, 2020

krzywon commented Nov 10, 2020

smk78 commented Oct 30, 2020 •

edited

Loading