From 61371975bf723315e0244eb8c6295cbd86853538 Mon Sep 17 00:00:00 2001 From: abuts Date: Tue, 29 Oct 2024 13:00:54 +0000 Subject: [PATCH 01/14] Re #1766 draft release notes (incomplete) --- VERSION | 2 +- documentation/release_notes/v4.0.1.md | 4 ++++ documentation/release_notes/v4.0.md | 28 ++++++++++++++++++++++++--- 3 files changed, 30 insertions(+), 4 deletions(-) create mode 100644 documentation/release_notes/v4.0.1.md diff --git a/VERSION b/VERSION index 0c89fc927e..cc868b62c3 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -4.0.0 \ No newline at end of file +4.0.1 \ No newline at end of file diff --git a/documentation/release_notes/v4.0.1.md b/documentation/release_notes/v4.0.1.md new file mode 100644 index 0000000000..180dbeac74 --- /dev/null +++ b/documentation/release_notes/v4.0.1.md @@ -0,0 +1,4 @@ +## Release Notes v4.0.1 + +Minor release, fixing some bugs, identified after release. + diff --git a/documentation/release_notes/v4.0.md b/documentation/release_notes/v4.0.md index a0f97040b9..ff0ec44a82 100644 --- a/documentation/release_notes/v4.0.md +++ b/documentation/release_notes/v4.0.md @@ -1,11 +1,33 @@ ## Release Notes v4.0 -This is major new release, providing internal changes to the sqw/dnd objects, necessary for using generic projection +This is version 4 of Horace, which contains major changes to the Horace code produced within 3-years of PACE project development. It is major upgrade from the Horace version 3.6. +Draft version of these changes were used in ISIS in cycles: 2024/1&2. + +The code is tested against Matlab versions 2019b-2023b. + +## Major changes. + +As part of major new features we would emphasise: + + - Fully changed internal structure of sqw objects. In particular: + + - allows usage of different projections (images in different coordinate system) + + - provides the same way of operating with sqw objects regardless of the way they are stored internally (the whole sqw object can stay in memory or , if the pixel data can not fit memory, the object can be stored on files). + + - New types of cuts using spherical and cylindrical projections + + - New symmetry operations and ways of symmetrizing sqw objects. + + - Extended parallel operations, new parallel algorithms for sqw_eval and Tobyfit. + + - It writes sqw objects in a new more efficient and flexible binary format though still can read of previous versions of sqw files. sqw files produced by Horace-4 can not be read by previous versions of Horace. + + - Number of less substantial additions and improvement in algorithms and design (more than 1000 competed tickets) refactoring and improving existing algorithms, providing various new features and making code much more modular to allow better maintenance and future Horace development. -## Breaking changes - - Renamed `fake_sqw` to `dummy_sqw` ## Critical changes + - Renamed `fake_sqw` to `dummy_sqw` - `DnD` objects are fully refactored and constructed from `axes_block` class (property `axes`), `aProjection` class (property `proj`) and three datasets, namely `s,e` and `npix` containing signal, error and number of pixels contributed into a bin, as before. `axes_block` defines binning of these datasets and their physical meaning, while projection defines the transformation from pixels coordinate system to the coordinate system, defined by `axes_block`. Old interface (`iax,pax,iint,p etc.`) left for compatibility with existing algorithms and set as read-only interface. - `multifit` and `tobyfit` are now able to be run in parallel through the `hpc_config` `parallel_multifit` setting. - `data_sqw_dnd` class (property `data` of `sqw` object) have been removed from `sqw` object and replaced by appropriate dimension `dnd` object. Pixels data (`PixelData` class), which were the property of `data_sqw_dnd` class are now independent property of `sqw` object (property `pix`). Old and refactored `data_sqw_dnd` class left for compatibility with old `sqw` files, allowing the loading of old data from `.mat` files. From 3f944884c4c86d9b5ee42fd7dd923ec95c37ca24 Mon Sep 17 00:00:00 2001 From: abuts Date: Tue, 29 Oct 2024 13:19:04 +0000 Subject: [PATCH 02/14] Re #1766 completed release notes --- documentation/release_notes/v4.0.1.md | 11 ++++++++- documentation/release_notes/v4.0.md | 34 +++++++++++++++++---------- 2 files changed, 31 insertions(+), 14 deletions(-) diff --git a/documentation/release_notes/v4.0.1.md b/documentation/release_notes/v4.0.1.md index 180dbeac74..65132b4566 100644 --- a/documentation/release_notes/v4.0.1.md +++ b/documentation/release_notes/v4.0.1.md @@ -1,4 +1,13 @@ ## Release Notes v4.0.1 -Minor release, fixing some bugs, identified after release. +Minor release, fixing some bugs and dealing with some optimization issues, identified after release. +## Bugs fixed: + The details of the fixed issues are available on [GitHub](https://github.com/pace-neutrons/Horace/issues/) + - #1731 invalid error reporting in load_spe. + - #1735 Issues with SymopGeneral in `cut_sym`. + - #1745 Missing dnd constructor used by spinW. + - #1748 SymIdentify takes substantial time while doing cut without symmetries. + - #1753 horace_mex and issues with `cpp_communicator`. + - #1758 horace_install install spinW integration on request. + - #1762 `horace_demo_script` is not working due to reference to fit_sqw \ No newline at end of file diff --git a/documentation/release_notes/v4.0.md b/documentation/release_notes/v4.0.md index ff0ec44a82..3a281e568c 100644 --- a/documentation/release_notes/v4.0.md +++ b/documentation/release_notes/v4.0.md @@ -23,16 +23,24 @@ As part of major new features we would emphasise: - It writes sqw objects in a new more efficient and flexible binary format though still can read of previous versions of sqw files. sqw files produced by Horace-4 can not be read by previous versions of Horace. - - Number of less substantial additions and improvement in algorithms and design (more than 1000 competed tickets) refactoring and improving existing algorithms, providing various new features and making code much more modular to allow better maintenance and future Horace development. - - -## Critical changes - - Renamed `fake_sqw` to `dummy_sqw` - - `DnD` objects are fully refactored and constructed from `axes_block` class (property `axes`), `aProjection` class (property `proj`) and three datasets, namely `s,e` and `npix` containing signal, error and number of pixels contributed into a bin, as before. `axes_block` defines binning of these datasets and their physical meaning, while projection defines the transformation from pixels coordinate system to the coordinate system, defined by `axes_block`. Old interface (`iax,pax,iint,p etc.`) left for compatibility with existing algorithms and set as read-only interface. - - `multifit` and `tobyfit` are now able to be run in parallel through the `hpc_config` `parallel_multifit` setting. - - `data_sqw_dnd` class (property `data` of `sqw` object) have been removed from `sqw` object and replaced by appropriate dimension `dnd` object. Pixels data (`PixelData` class), which were the property of `data_sqw_dnd` class are now independent property of `sqw` object (property `pix`). Old and refactored `data_sqw_dnd` class left for compatibility with old `sqw` files, allowing the loading of old data from `.mat` files. - - Added benchmarking features under the `_benchmarking` folder which will run on CI or can be run by users using either the `benchmark_horace` script or by running individual benchmark functions. Documentation included in SMG 20 - - Spawning parallel jobs with Herbert/C++ MPI no longer spawns inappropriate MATLAB threads oversubscribing the machine and causing slowdown. - - Slurm cluster handling has been rewritten to deal appropriately with multiple node requests and automatically handle threading. - - `threads` and `parallel_workers` have migrated from `hpc_config` to `parallel_config` - - `herbert_config` has been deprecated and features of `herbert_config` have migrated to `hor_config` + +## Major internal changes, wich may break compatibility with old Horace scripts + -- Renamed `fake_sqw` to `dummy_sqw` + + -- `DnD` objects are fully refactored and constructed from `axes_block` class (property `axes`), `aProjection` class (property `proj`) and three datasets, namely `s,e` and `npix` containing signal, error and number of pixels contributed into a bin, as before. `axes_block` defines binning of these datasets and their physical meaning, while projection defines the transformation from pixels coordinate system to the coordinate system, defined by `axes_block`. Old interface (`iax,pax,iint,p etc.`) left for compatibility with existing algorithms and set as read-only interface. + + -- `multifit` and `tobyfit` are now able to be run in parallel through the `hpc_config` `parallel_multifit` setting. + + -- `data_sqw_dnd` class (property `data` of `sqw` object) have been removed from `sqw` object and replaced by appropriate dimension `dnd` object. Pixels data (`PixelData` class), which were the property of `data_sqw_dnd` class are now independent property of `sqw` object (property `pix`). Old and refactored `data_sqw_dnd` class left for compatibility with old `sqw` files, allowing the loading of old data from `.mat` files. + + -- Added benchmarking features under the `_benchmarking` folder which will run on CI or can be run by users using either the `benchmark_horace` script or by running individual benchmark functions. Documentation included in SMG 20 + + -- Spawning parallel jobs with Herbert/C++ MPI no longer spawns inappropriate MATLAB threads oversubscribing the machine and causing slowdown. + + -- Slurm cluster handling has been rewritten to deal appropriately with multiple node requests and automatically handle threading. + + -- `threads` and `parallel_workers` have migrated from `hpc_config` to `parallel_config` + + -- `herbert_config` has been deprecated and features of `herbert_config` have migrated to `hor_config` + + -- Number of less substantial additions and improvement in algorithms and design (more than 1000 competed tickets) refactoring and improving existing algorithms, providing various new features and making code much more modular to allow better maintenance and future Horace development. \ No newline at end of file From 6156b1497ba5d81b8d4c7cd146a3b911dbe06117 Mon Sep 17 00:00:00 2001 From: abuts Date: Tue, 29 Oct 2024 13:26:24 +0000 Subject: [PATCH 03/14] Re #1766 minor formatting and typos --- documentation/release_notes/v4.0.1.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/documentation/release_notes/v4.0.1.md b/documentation/release_notes/v4.0.1.md index 65132b4566..d276c3430c 100644 --- a/documentation/release_notes/v4.0.1.md +++ b/documentation/release_notes/v4.0.1.md @@ -4,10 +4,10 @@ Minor release, fixing some bugs and dealing with some optimization issues, ident ## Bugs fixed: The details of the fixed issues are available on [GitHub](https://github.com/pace-neutrons/Horace/issues/) - - #1731 invalid error reporting in load_spe. - - #1735 Issues with SymopGeneral in `cut_sym`. - - #1745 Missing dnd constructor used by spinW. - - #1748 SymIdentify takes substantial time while doing cut without symmetries. - - #1753 horace_mex and issues with `cpp_communicator`. - - #1758 horace_install install spinW integration on request. - - #1762 `horace_demo_script` is not working due to reference to fit_sqw \ No newline at end of file + - #1731 invalid error reporting in `load_spe`. + - #1735 Issues with `SymopGeneral` in `cut_sym`. + - #1745 Missing `dnd` constructor used by `spinW`. + - #1748 `SymIdentify` takes substantial time while doing cut without symmetries. + - #1753 `horace_mex` and issues with `cpp_communicator`. + - #1758 `horace_install` installs `spinW` integration on request. + - #1762 `horace_demo_script` is not working due to reference to `fit_sqw` \ No newline at end of file From ab9870a1f019140c83b82ef9ca0242efb1db35d1 Mon Sep 17 00:00:00 2001 From: abuts Date: Tue, 29 Oct 2024 14:18:32 +0000 Subject: [PATCH 04/14] Update documentation/release_notes/v4.0.1.md Co-authored-by: Jacob Wilkins <46597752+oerc0122@users.noreply.github.com> --- documentation/release_notes/v4.0.1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/release_notes/v4.0.1.md b/documentation/release_notes/v4.0.1.md index d276c3430c..a1ffa5b032 100644 --- a/documentation/release_notes/v4.0.1.md +++ b/documentation/release_notes/v4.0.1.md @@ -1,6 +1,6 @@ ## Release Notes v4.0.1 -Minor release, fixing some bugs and dealing with some optimization issues, identified after release. +Minor release, fixing some bugs, and dealing with some optimization issues identified after release. ## Bugs fixed: The details of the fixed issues are available on [GitHub](https://github.com/pace-neutrons/Horace/issues/) From fa28aa9411231b748ca009da6a89a7f1b7e78beb Mon Sep 17 00:00:00 2001 From: abuts Date: Tue, 29 Oct 2024 14:19:46 +0000 Subject: [PATCH 05/14] Re #1766 Changes from review --- documentation/release_notes/v4.0.1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/release_notes/v4.0.1.md b/documentation/release_notes/v4.0.1.md index d276c3430c..6b82d65d72 100644 --- a/documentation/release_notes/v4.0.1.md +++ b/documentation/release_notes/v4.0.1.md @@ -7,7 +7,7 @@ Minor release, fixing some bugs and dealing with some optimization issues, ident - #1731 invalid error reporting in `load_spe`. - #1735 Issues with `SymopGeneral` in `cut_sym`. - #1745 Missing `dnd` constructor used by `spinW`. - - #1748 `SymIdentify` takes substantial time while doing cut without symmetries. + - #1748 Fixed `SymopIdentify` taking substantial time while doing cut without symmetries. - #1753 `horace_mex` and issues with `cpp_communicator`. - #1758 `horace_install` installs `spinW` integration on request. - #1762 `horace_demo_script` is not working due to reference to `fit_sqw` \ No newline at end of file From 7154970c55559efd80c5bee5690c34f12d4e6081 Mon Sep 17 00:00:00 2001 From: abuts Date: Tue, 29 Oct 2024 14:41:55 +0000 Subject: [PATCH 06/14] Re #1766 partial changes from review --- documentation/release_notes/v4.0.1.md | 15 ++++++--- documentation/release_notes/v4.0.md | 47 ++++++++++++++++++++------- 2 files changed, 45 insertions(+), 17 deletions(-) diff --git a/documentation/release_notes/v4.0.1.md b/documentation/release_notes/v4.0.1.md index 8244f2908c..d18d61e594 100644 --- a/documentation/release_notes/v4.0.1.md +++ b/documentation/release_notes/v4.0.1.md @@ -1,13 +1,18 @@ ## Release Notes v4.0.1 -Minor release, fixing some bugs, and dealing with some optimization issues identified after release. +Minor release, fixing some bugs, and dealing with +some optimization issues identified after release. ## Bugs fixed: - The details of the fixed issues are available on [GitHub](https://github.com/pace-neutrons/Horace/issues/) + The details of the fixed issues are available on + [GitHub](https://github.com/pace-neutrons/Horace/issues/) - #1731 invalid error reporting in `load_spe`. - #1735 Issues with `SymopGeneral` in `cut_sym`. - - #1745 Missing `dnd` constructor used by `spinW`. - - #1748 Fixed `SymopIdentify` taking substantial time while doing cut without symmetries. + - #1745 Missing `dnd` constructor used by `spinW` + and `spinW` integration tests. + - #1748 Fixed `SymopIdentify` taking substantial + time while doing cut without symmetries. - #1753 `horace_mex` and issues with `cpp_communicator`. - #1758 `horace_install` installs `spinW` integration on request. - - #1762 `horace_demo_script` is not working due to reference to `fit_sqw` \ No newline at end of file + - #1762 `horace_demo_script` is not working + due to reference to removed `fit_sqw` \ No newline at end of file diff --git a/documentation/release_notes/v4.0.md b/documentation/release_notes/v4.0.md index 3a281e568c..d4813b982e 100644 --- a/documentation/release_notes/v4.0.md +++ b/documentation/release_notes/v4.0.md @@ -1,9 +1,10 @@ ## Release Notes v4.0 -This is version 4 of Horace, which contains major changes to the Horace code produced within 3-years of PACE project development. It is major upgrade from the Horace version 3.6. -Draft version of these changes were used in ISIS in cycles: 2024/1&2. +This is version 4 of Horace, which contains major changes to the Horace code +produced within 3-years of PACE project development. +It is major upgrade from the Horace version 3.6. -The code is tested against Matlab versions 2019b-2023b. +Draft versions of these changes were used at ISIS in cycles: 2024/1&2 ## Major changes. @@ -24,23 +25,45 @@ As part of major new features we would emphasise: - It writes sqw objects in a new more efficient and flexible binary format though still can read of previous versions of sqw files. sqw files produced by Horace-4 can not be read by previous versions of Horace. -## Major internal changes, wich may break compatibility with old Horace scripts +## Major internal changes, which may break compatibility with old Horace scripts -- Renamed `fake_sqw` to `dummy_sqw` - -- `DnD` objects are fully refactored and constructed from `axes_block` class (property `axes`), `aProjection` class (property `proj`) and three datasets, namely `s,e` and `npix` containing signal, error and number of pixels contributed into a bin, as before. `axes_block` defines binning of these datasets and their physical meaning, while projection defines the transformation from pixels coordinate system to the coordinate system, defined by `axes_block`. Old interface (`iax,pax,iint,p etc.`) left for compatibility with existing algorithms and set as read-only interface. + -- `DnD` objects are fully refactored and constructed from `axes_block` class (property `axes`), + `aProjection` class (property `proj`) and three datasets, namely `s,e` and `npix` + containing signal, error and number of pixels contributed into a bin, as before. + `axes_block` defines binning of these datasets and their physical meaning, + while projection defines the transformation from pixels coordinate system to the + coordinate system, defined by `axes_block`. Old interface (`iax,pax,iint,p etc.`) + left for compatibility with existing algorithms and set as read-only interface. - -- `multifit` and `tobyfit` are now able to be run in parallel through the `hpc_config` `parallel_multifit` setting. + -- `multifit` and `tobyfit` are now able to be run in parallel through the + `hpc_config` `parallel_multifit` setting. - -- `data_sqw_dnd` class (property `data` of `sqw` object) have been removed from `sqw` object and replaced by appropriate dimension `dnd` object. Pixels data (`PixelData` class), which were the property of `data_sqw_dnd` class are now independent property of `sqw` object (property `pix`). Old and refactored `data_sqw_dnd` class left for compatibility with old `sqw` files, allowing the loading of old data from `.mat` files. + -- `data_sqw_dnd` class (property `data` of `sqw` object) have been removed from + `sqw` object and replaced by appropriate dimension `dnd` object. Pixels data + (`PixelData` class), which were the property of `data_sqw_dnd` class are now independent + property of `sqw` object (property `pix`). Old and refactored `data_sqw_dnd` class left + for compatibility with old `sqw` files, allowing the loading of old data from `.mat` files. - -- Added benchmarking features under the `_benchmarking` folder which will run on CI or can be run by users using either the `benchmark_horace` script or by running individual benchmark functions. Documentation included in SMG 20 + -- Added benchmarking features under the `_benchmarking` folder which will run on CI + or can be run by users using either the `benchmark_horace` script or by running + individual benchmark functions. Documentation included in SMG 20 - -- Spawning parallel jobs with Herbert/C++ MPI no longer spawns inappropriate MATLAB threads oversubscribing the machine and causing slowdown. + -- Spawning parallel jobs with Herbert/C++ MPI no longer spawns inappropriate MATLAB + threads oversubscribing the machine and causing slowdown. - -- Slurm cluster handling has been rewritten to deal appropriately with multiple node requests and automatically handle threading. + -- Slurm cluster handling has been rewritten to deal appropriately with multiple + node requests and automatically handle threading. -- `threads` and `parallel_workers` have migrated from `hpc_config` to `parallel_config` - -- `herbert_config` has been deprecated and features of `herbert_config` have migrated to `hor_config` + -- `herbert_config` has been deprecated and features of `herbert_config` + have migrated to `hor_config` + + -- `fit_sqw` function have been removed in favour of direct usage of `multifit_sqw` class + and this class methods. - -- Number of less substantial additions and improvement in algorithms and design (more than 1000 competed tickets) refactoring and improving existing algorithms, providing various new features and making code much more modular to allow better maintenance and future Horace development. \ No newline at end of file + -- Number of less substantial additions and improvement in algorithms and design + (more than 1000 competed tickets) refactoring and improving existing algorithms, + providing various new features and making code much more modular to allow better + maintenance and future Horace development. \ No newline at end of file From ef4d20788f5a2e215d74a9e150cbb76a802b1a16 Mon Sep 17 00:00:00 2001 From: abuts Date: Tue, 29 Oct 2024 14:42:41 +0000 Subject: [PATCH 07/14] Re #1766 Update documentation/release_notes/v4.0.md Co-authored-by: Jacob Wilkins <46597752+oerc0122@users.noreply.github.com> --- documentation/release_notes/v4.0.md | 33 +++++++++++++++++------------ 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/documentation/release_notes/v4.0.md b/documentation/release_notes/v4.0.md index 3a281e568c..49b541eacd 100644 --- a/documentation/release_notes/v4.0.md +++ b/documentation/release_notes/v4.0.md @@ -24,23 +24,30 @@ As part of major new features we would emphasise: - It writes sqw objects in a new more efficient and flexible binary format though still can read of previous versions of sqw files. sqw files produced by Horace-4 can not be read by previous versions of Horace. -## Major internal changes, wich may break compatibility with old Horace scripts - -- Renamed `fake_sqw` to `dummy_sqw` +## Breaking Changes +- Renamed `fake_sqw` to `dummy_sqw` - -- `DnD` objects are fully refactored and constructed from `axes_block` class (property `axes`), `aProjection` class (property `proj`) and three datasets, namely `s,e` and `npix` containing signal, error and number of pixels contributed into a bin, as before. `axes_block` defines binning of these datasets and their physical meaning, while projection defines the transformation from pixels coordinate system to the coordinate system, defined by `axes_block`. Old interface (`iax,pax,iint,p etc.`) left for compatibility with existing algorithms and set as read-only interface. +- Fully refactored `SQw` objects: + - `data_sqw_dnd` class (property `data` of `sqw` object) has been removed from `sqw` object and replaced by appropriately dimensioned `dnd` object. + - Pixel data (`PixelData` class), which were a property of the `data_sqw_dnd` class are now an independent property of `sqw` object (`pix` property). + - Old and refactored `data_sqw_dnd` class left for compatibility with old `sqw` files, allowing the loading of old data from `.mat` files. - -- `multifit` and `tobyfit` are now able to be run in parallel through the `hpc_config` `parallel_multifit` setting. + - Fully refactored`DnD` objects now made up of: + - `axes_block` class (`axes` property), defining binning of these datasets and their physical meaning, + - `aProjection` class (`proj` property), defining the transformation from pixels coordinate system to the coordinate system, defined by `axes_block`. + - and three datasets, namely `s,e` and `npix` containing signal, error and number of pixels contributed into a bin, as before. + - **NOTE:** Old interface (`iax`, `pax`, `iint`, `p`, etc.) left for compatibility with existing algorithms through read-only interface. - -- `data_sqw_dnd` class (property `data` of `sqw` object) have been removed from `sqw` object and replaced by appropriate dimension `dnd` object. Pixels data (`PixelData` class), which were the property of `data_sqw_dnd` class are now independent property of `sqw` object (property `pix`). Old and refactored `data_sqw_dnd` class left for compatibility with old `sqw` files, allowing the loading of old data from `.mat` files. - - -- Added benchmarking features under the `_benchmarking` folder which will run on CI or can be run by users using either the `benchmark_horace` script or by running individual benchmark functions. Documentation included in SMG 20 - - -- Spawning parallel jobs with Herbert/C++ MPI no longer spawns inappropriate MATLAB threads oversubscribing the machine and causing slowdown. +- `multifit` and `tobyfit` can be run in parallel through the `hpc_config` `parallel_multifit` setting. + +- Added benchmarking in the `_benchmarking` folder which will run on CI or can be run by users using either the `benchmark_horace` script or by running individual benchmark functions. Documentation included in SMG 20. + +- Spawning parallel jobs with Herbert/C++ MPI no longer spawns inappropriate MATLAB threads oversubscribing the machine and causing slowdown. - -- Slurm cluster handling has been rewritten to deal appropriately with multiple node requests and automatically handle threading. +- Slurm cluster interface has been rewritten to deal appropriately with multiple node requests and automatically handle threading. - -- `threads` and `parallel_workers` have migrated from `hpc_config` to `parallel_config` +- `threads` and `parallel_workers` have migrated from `hpc_config` to `parallel_config` - -- `herbert_config` has been deprecated and features of `herbert_config` have migrated to `hor_config` +- `herbert_config` has been deprecated and features of `herbert_config` have migrated to `hor_config` - -- Number of less substantial additions and improvement in algorithms and design (more than 1000 competed tickets) refactoring and improving existing algorithms, providing various new features and making code much more modular to allow better maintenance and future Horace development. \ No newline at end of file +- More than 1,000 additional tasks making substantial additions and improvements to algorithms and design: refactoring and improving existing algorithms, providing new features and making code more modular to facilitate maintenance and future Horace development. \ No newline at end of file From 6698ef9935af9a5990c965bf882cc10d93638c1b Mon Sep 17 00:00:00 2001 From: abuts Date: Tue, 29 Oct 2024 14:43:13 +0000 Subject: [PATCH 08/14] Re #1766 Update documentation/release_notes/v4.0.md Co-authored-by: Jacob Wilkins <46597752+oerc0122@users.noreply.github.com> --- documentation/release_notes/v4.0.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/documentation/release_notes/v4.0.md b/documentation/release_notes/v4.0.md index 49b541eacd..920c5d261e 100644 --- a/documentation/release_notes/v4.0.md +++ b/documentation/release_notes/v4.0.md @@ -11,17 +11,20 @@ As part of major new features we would emphasise: - Fully changed internal structure of sqw objects. In particular: - - allows usage of different projections (images in different coordinate system) +- Allows use of different projections: images (`dnd`) in different coordinate systems. + - Now provides cuts using spherical and cylindrical projections with more coming. - - provides the same way of operating with sqw objects regardless of the way they are stored internally (the whole sqw object can stay in memory or , if the pixel data can not fit memory, the object can be stored on files). +- On-file sqws for cases where they do not fit in memory, with a unified interface (no need to learn anything new, for big jobs!). - - New types of cuts using spherical and cylindrical projections +- New symmetry operations and ways of symmetrizing sqw objects. + - Multiple reductions in a single `symmetrize_sqw` call, rotationally symmetric reduction. + - Symmetrised cuts which can be more efficient and more general. - - New symmetry operations and ways of symmetrizing sqw objects. +- Extended parallel capabilities + - New parallel algorithms for `sqw_eval` and fitting. - - Extended parallel operations, new parallel algorithms for sqw_eval and Tobyfit. - - - It writes sqw objects in a new more efficient and flexible binary format though still can read of previous versions of sqw files. sqw files produced by Horace-4 can not be read by previous versions of Horace. +- New more efficient and flexible binary format. + - **NOTE:** Horace can read previous versions of sqw files, however, sqw files produced by Horace v4+ can not be read by previous versions of Horace. ## Breaking Changes From a9af0ca5ca2e15c7aace005eab1c7f414a0f2658 Mon Sep 17 00:00:00 2001 From: abuts Date: Tue, 29 Oct 2024 14:52:50 +0000 Subject: [PATCH 09/14] Re #1766 minor changes lost in merge conflicts --- documentation/release_notes/v4.0.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/documentation/release_notes/v4.0.md b/documentation/release_notes/v4.0.md index 6aa175a0d8..a2fcbb0bb0 100644 --- a/documentation/release_notes/v4.0.md +++ b/documentation/release_notes/v4.0.md @@ -15,7 +15,8 @@ As part of major new features we would emphasise: - Allows use of different projections: images (`dnd`) in different coordinate systems. - Now provides cuts using spherical and cylindrical projections with more coming. -- On-file sqws for cases where they do not fit in memory, with a unified interface (no need to learn anything new, for big jobs!). +- On-file sqws for cases where they do not fit in memory, with a unified interface + (no need to learn anything new, for big jobs!). - New symmetry operations and ways of symmetrizing sqw objects. - Multiple reductions in a single `symmetrize_sqw` call, rotationally symmetric reduction. @@ -31,12 +32,12 @@ As part of major new features we would emphasise: ## Breaking Changes - Renamed `fake_sqw` to `dummy_sqw` -- Fully refactored `SQw` objects: +- Fully refactored `SQW` objects: - `data_sqw_dnd` class (property `data` of `sqw` object) has been removed from `sqw` object and replaced by appropriately dimensioned `dnd` object. - Pixel data (`PixelData` class), which were a property of the `data_sqw_dnd` class are now an independent property of `sqw` object (`pix` property). - Old and refactored `data_sqw_dnd` class left for compatibility with old `sqw` files, allowing the loading of old data from `.mat` files. - - Fully refactored`DnD` objects now made up of: +- Fully refactored`DnD` objects now made up of: - `axes_block` class (`axes` property), defining binning of these datasets and their physical meaning, - `aProjection` class (`proj` property), defining the transformation from pixels coordinate system to the coordinate system, defined by `axes_block`. - and three datasets, namely `s,e` and `npix` containing signal, error and number of pixels contributed into a bin, as before. @@ -56,6 +57,7 @@ As part of major new features we would emphasise: - `fit_sqw` function have been removed in favour of direct usage of `multifit_sqw` class and this class methods. + - More than 1,000 additional tasks making substantial additions and improvements to algorithms and design: refactoring and improving existing algorithms, providing new features and making code more modular to facilitate maintenance and future Horace development. From b0cd751553145662998ab5c6852198089fafac40 Mon Sep 17 00:00:00 2001 From: abuts Date: Tue, 29 Oct 2024 14:58:36 +0000 Subject: [PATCH 10/14] Re #1766 minor formatting changes --- documentation/release_notes/v4.0.md | 53 ++++++++++++++++++++--------- 1 file changed, 36 insertions(+), 17 deletions(-) diff --git a/documentation/release_notes/v4.0.md b/documentation/release_notes/v4.0.md index a2fcbb0bb0..a1b378047f 100644 --- a/documentation/release_notes/v4.0.md +++ b/documentation/release_notes/v4.0.md @@ -33,31 +33,50 @@ As part of major new features we would emphasise: - Renamed `fake_sqw` to `dummy_sqw` - Fully refactored `SQW` objects: - - `data_sqw_dnd` class (property `data` of `sqw` object) has been removed from `sqw` object and replaced by appropriately dimensioned `dnd` object. - - Pixel data (`PixelData` class), which were a property of the `data_sqw_dnd` class are now an independent property of `sqw` object (`pix` property). - - Old and refactored `data_sqw_dnd` class left for compatibility with old `sqw` files, allowing the loading of old data from `.mat` files. + - `data_sqw_dnd` class (property `data` of `sqw` object) has been removed from + `sqw` object and replaced by appropriately dimensioned `dnd` object. + - Pixel data (`PixelData` class), which were a property of the `data_sqw_dnd` + class are now an independent property of `sqw` object (`pix` property). + - Old and refactored `data_sqw_dnd` class left for compatibility with old + `sqw` files, allowing the loading of old data from `.mat` files. - Fully refactored`DnD` objects now made up of: - - `axes_block` class (`axes` property), defining binning of these datasets and their physical meaning, - - `aProjection` class (`proj` property), defining the transformation from pixels coordinate system to the coordinate system, defined by `axes_block`. - - and three datasets, namely `s,e` and `npix` containing signal, error and number of pixels contributed into a bin, as before. - - **NOTE:** Old interface (`iax`, `pax`, `iint`, `p`, etc.) left for compatibility with existing algorithms through read-only interface. + - `axes_block` class (`axes` property), defining binning of + these datasets and their physical meaning, + - `aProjection` class (`proj` property), defining the transformation + from pixels coordinate system to the coordinate system, defined by `axes_block`. + - and three datasets, namely `s,e` and `npix` containing signal, error and + number of pixels contributed into a bin, as before. + - **NOTE:** Old interface (`iax`, `pax`, `iint`, `p`, etc.) left for + compatibility with existing algorithms through read-only interface. -- `multifit` and `tobyfit` can be run in parallel through the `hpc_config` `parallel_multifit` setting. +- `multifit` and `tobyfit` can be run in parallel through + the `hpc_config` `parallel_multifit` setting. -- Added benchmarking in the `_benchmarking` folder which will run on CI or can be run by users using either the `benchmark_horace` script or by running individual benchmark functions. Documentation included in SMG 20. +- Added benchmarking in the `_benchmarking` folder which will + run on CI or can be run by users using either the `benchmark_horace` + script or by running individual benchmark functions. + Documentation included in SMG 20. -- Spawning parallel jobs with Herbert/C++ MPI no longer spawns inappropriate MATLAB threads oversubscribing the machine and causing slowdown. +- Spawning parallel jobs with Herbert/C++ MPI no longer spawns + inappropriate MATLAB threads oversubscribing the machine + and causing slowdown. -- Slurm cluster interface has been rewritten to deal appropriately with multiple node requests and automatically handle threading. +- Slurm cluster interface has been rewritten to deal appropriately + with multiple node requests and automatically handle threading. -- `threads` and `parallel_workers` have migrated from `hpc_config` to `parallel_config` +- `threads` and `parallel_workers` have migrated from + `hpc_config` to `parallel_config` -- `herbert_config` has been deprecated and features of `herbert_config` have migrated to `hor_config` +- `herbert_config` has been deprecated and features of + `herbert_config` have migrated to `hor_config` -- `fit_sqw` function have been removed in favour of direct usage of `multifit_sqw` class +- `fit_sqw` function have been removed in favour of + direct usage of `multifit_sqw` class and this class methods. -- More than 1,000 additional tasks making substantial additions and improvements to algorithms and design: - refactoring and improving existing algorithms, providing new features and making code more - modular to facilitate maintenance and future Horace development. +- More than 1,000 additional tasks making substantial additions and + improvements to algorithms and design: + refactoring and improving existing algorithms, providing + new features and making code more modular to facilitate + maintenance and future Horace development. From d15b11f58ba98a341fc2f11c2ead071163d32615 Mon Sep 17 00:00:00 2001 From: abuts Date: Tue, 29 Oct 2024 15:04:01 +0000 Subject: [PATCH 11/14] Re #1766 minor changes left from review --- documentation/release_notes/v4.0.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/documentation/release_notes/v4.0.md b/documentation/release_notes/v4.0.md index a1b378047f..5a0c257206 100644 --- a/documentation/release_notes/v4.0.md +++ b/documentation/release_notes/v4.0.md @@ -8,9 +8,8 @@ Draft versions of these changes were used at ISIS in cycles: 2024/1&2 ## Major changes. -As part of major new features we would emphasise: - - - Fully changed internal structure of sqw objects. In particular: +As a major part of the new features we would like to emphasise the fully restructured sqw objects. +In particular they provide the following benefits: - Allows use of different projections: images (`dnd`) in different coordinate systems. - Now provides cuts using spherical and cylindrical projections with more coming. From 90eb67d471f9dce1e8a613af55ef6854e3fc999e Mon Sep 17 00:00:00 2001 From: abuts Date: Tue, 29 Oct 2024 15:05:37 +0000 Subject: [PATCH 12/14] Re #1766 Update documentation/release_notes/v4.0.md Co-authored-by: Jacob Wilkins <46597752+oerc0122@users.noreply.github.com> --- documentation/release_notes/v4.0.md | 1 - 1 file changed, 1 deletion(-) diff --git a/documentation/release_notes/v4.0.md b/documentation/release_notes/v4.0.md index 5a0c257206..530882e7ae 100644 --- a/documentation/release_notes/v4.0.md +++ b/documentation/release_notes/v4.0.md @@ -72,7 +72,6 @@ In particular they provide the following benefits: - `fit_sqw` function have been removed in favour of direct usage of `multifit_sqw` class - and this class methods. - More than 1,000 additional tasks making substantial additions and improvements to algorithms and design: From 36490fc3a52639ac982bb7a96722e94585eccbd5 Mon Sep 17 00:00:00 2001 From: abuts Date: Tue, 29 Oct 2024 15:05:58 +0000 Subject: [PATCH 13/14] Re #1766 Update documentation/release_notes/v4.0.md Co-authored-by: Jacob Wilkins <46597752+oerc0122@users.noreply.github.com> --- documentation/release_notes/v4.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/release_notes/v4.0.md b/documentation/release_notes/v4.0.md index 530882e7ae..2d57cc7d16 100644 --- a/documentation/release_notes/v4.0.md +++ b/documentation/release_notes/v4.0.md @@ -39,7 +39,7 @@ In particular they provide the following benefits: - Old and refactored `data_sqw_dnd` class left for compatibility with old `sqw` files, allowing the loading of old data from `.mat` files. -- Fully refactored`DnD` objects now made up of: +- Fully refactored `DnD` objects now made up of: - `axes_block` class (`axes` property), defining binning of these datasets and their physical meaning, - `aProjection` class (`proj` property), defining the transformation From 66fe00cb10833e26bbecb0c4f139d55ce83a9272 Mon Sep 17 00:00:00 2001 From: abuts Date: Tue, 29 Oct 2024 15:08:17 +0000 Subject: [PATCH 14/14] Re #1766 minor formatting --- documentation/release_notes/v4.0.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/documentation/release_notes/v4.0.md b/documentation/release_notes/v4.0.md index 2d57cc7d16..483f55007b 100644 --- a/documentation/release_notes/v4.0.md +++ b/documentation/release_notes/v4.0.md @@ -18,14 +18,16 @@ In particular they provide the following benefits: (no need to learn anything new, for big jobs!). - New symmetry operations and ways of symmetrizing sqw objects. - - Multiple reductions in a single `symmetrize_sqw` call, rotationally symmetric reduction. + - Multiple reductions in a single `symmetrize_sqw` call, + rotationally symmetric reduction. - Symmetrised cuts which can be more efficient and more general. - Extended parallel capabilities - New parallel algorithms for `sqw_eval` and fitting. - New more efficient and flexible binary format. - - **NOTE:** Horace can read previous versions of sqw files, however, sqw files produced by Horace v4+ can not be read by previous versions of Horace. + - **NOTE:** Horace can read previous versions of sqw files, however, sqw + files produced by Horace v4+ can not be read by previous versions of Horace. ## Breaking Changes @@ -65,13 +67,13 @@ In particular they provide the following benefits: with multiple node requests and automatically handle threading. - `threads` and `parallel_workers` have migrated from - `hpc_config` to `parallel_config` + `hpc_config` to `parallel_config`. - `herbert_config` has been deprecated and features of - `herbert_config` have migrated to `hor_config` + `herbert_config` have migrated to `hor_config`. - `fit_sqw` function have been removed in favour of - direct usage of `multifit_sqw` class + direct usage of `multifit_sqw` class. - More than 1,000 additional tasks making substantial additions and improvements to algorithms and design: