Merge branch 'develop' of https://github.com/PEtab-dev/PEtab into dev…

…elop
PEtab-dev · Mar 20, 2020 · 5632a11 · 5632a11
2 parents 1206185 + 8025172
commit 5632a11
Show file tree

Hide file tree

Showing 8 changed files with 136 additions and 46 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,6 +2,23 @@
 
 ## 0.1 series
 
+### 0.1.5
+
+Library:
+
+* New create empty observable function (issue 386) (#387)
+* Deprecate petab.sbml.globalize_parameters (#381)
+* Fix computing log10 likelihood (#380)
+* Documentation update and typehints for visualization  (#372)
+* Ordered result of `petab.get_output_parameters`
+* Fix missing argument to parameters.create_parameter_df
+
+Documentation:
+* Add overview of supported PEtab feature in toolboxes
+* Add contribution guide
+* Fix optional values in documentation (#378)
+
+
 ### 0.1.4
 
 Library:

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,47 @@
+# PEtab contribution guide
+
+We are happy about contributions to PEtab of any form.
+
+This includes, but is not limited to:
+
+* Extending / improving
+  [documentation and examples](https://petab.readthedocs.io/en/latest/)
+* New library functionality
+* Additional
+  [PEtab application examples](https://github.com/Benchmarking-Initiative/Benchmark-Models-PEtab)
+* Additional minimal
+  [PEtab test cases](https://github.com/PEtab-dev/petab_test_suite)
+
+
+## Extending PEtab
+
+We are aware of the fact that PEtab may not serve everybody's needs. If you
+have a suggestion of how to extend PEtab, feel free to post an
+[issue](https://github.com/PEtab-dev/PEtab/issues) at our GitHub repository.
+
+
+## Contributions to this repository
+
+General:
+
+* For code contributions, please adhere to the
+  [PEP8 style guide](https://www.python.org/dev/peps/pep-0008/)
+* All new functionality should be covered by unit tests
+* Please use Python type hints
+* Please document all modules, functions, classes, attributes, arguments,
+  return values, ... in a style consistent with the rest of the library
+* Use descriptive commit messages
+
+To contribute to this repository:
+
+* Create a pull request
+
+  *By creating a pull request you agree with your contribution being made
+  available under the license terms specified in
+  [https://github.com/PEtab-dev/PEtab/blob/master/LICENSE](https://github.com/PEtab-dev/PEtab/blob/master/LICENSE).*
+
+* Assign a reviewer, or let us know otherwise, that your pull request is ready
+  for review.
+
+* Wait for feedback (and feel free so send a gentle reminder if you did not
+  get any feedback within a week)
diff --git a/README.md b/README.md
@@ -41,51 +41,57 @@ A wide range of PEtab examples can be found in the systems biology parameter est
 
 ## PEtab support in systems biology tools
 
-Where PEtab is supported:
+Where PEtab is supported (in alphabetical order):
 
 
+  - [AMICI](https://github.com/ICB-DCM/AMICI/)
+    ([Example](https://github.com/ICB-DCM/AMICI/blob/master/python/examples/example_petab/petab.ipynb))
+
   - A PEtab -> [COPASI](http://copasi.org/)
     [converter](https://github.com/copasi/python-petab-importer)
 
-  - [d2d](https://github.com/Data2Dynamics/d2d/) ([HOWTO](https://github.com/Data2Dynamics/d2d/wiki/Support-for-PEtab))
+  - [d2d](https://github.com/Data2Dynamics/d2d/)
+    ([HOWTO](https://github.com/Data2Dynamics/d2d/wiki/Support-for-PEtab))
 
   - [dMod](https://github.com/dkaschek/dMod/)
+    ([HOWTO](https://github.com/dkaschek/dMod/wiki/Support-for-PEtab))
 
-  - [meigo](http://gingproc.iim.csic.es/meigo.html)
+  - [MEIGO](http://gingproc.iim.csic.es/meigo.html)
 
-  - [AMICI](https://github.com/ICB-DCM/AMICI/)
+  - [parPE](https://github.com/ICB-DCM/parPE/)
 
-  - [pyPESTO](https://github.com/ICB-DCM/pyPESTO/) ([Example](https://pypesto.readthedocs.io/en/latest/example/petab_import.html))
+  - [pyABC](https://github.com/ICB-DCM/pyABC/) ([Example](https://pyabc.readthedocs.io/en/latest/examples/petab.html))
 
-  - [pyABC](https://github.com/ICB-DCM/pyABC/)
+  - [pyPESTO](https://github.com/ICB-DCM/pyPESTO/)
+    ([Example](https://pypesto.readthedocs.io/en/latest/example/petab_import.html))
 
 If your project or tool is using PEtab, and you would like to have it listed
-here, please let us know.
+here, please [let us know](https://github.com/PEtab-dev/PEtab/issues).
 
 ### PEtab features supported in different tools
 
 The following list provides an overview of supported PEtab features in
 different tools, based on passed test cases of the
 [PEtab test suite](https://github.com/PEtab-dev/petab_test_suite):
 
-| ID | Test                                                           | AMICI<br>*`develop`* | AMIGO | Copasi | D2D | dMod | pyPESTO<br>*`develop`* | pyABC
-|----|----------------------------------------------------------------|-------|-------|--------|-----|------|-------|------|
-| 1  | Basic simulation                                               | +++   |       |        | +++ |      | +++   | +++  |
-| 2  | Multiple simulation conditions                                 | +++   |       |        | +++ |      | +++   | +++  |
-| 3  | Numeric observable parameter overrides in measurement table    | +++   |       |        | +++ |      | +++   | +++  |
-| 4  | Parametric observable parameter overrides in measurement table | +++   |       |        | +++ |      | +++   | +++  |
-| 5  | Parametric overrides in condition table                        | +++   |       |        | +++ |      | +++   | +++  |
-| 6  | Time-point specific overrides in the measurement table         | ---   |       |        | +++ |      | ---   | ---  |
-| 7  | Observable transformations to log10 scale                      | +-+   |       |        | +++ |      | +-+   | +-+  |
-| 8  | Replicate measurements                                         | +++   |       |        | +++ |      | +++   | +++  |
-| 9  | Pre-equilibration                                              | +++   |       |        | +++ |      | +++   | +++  |
-| 10 | Partial pre-equilibration                                      | +++   |       |        | +++ |      | +++   | +++  |
-| 11 | Numeric initial concentration in condition table               | +++   |       |        | +++ |      | +++   | +++  |
-| 12 | Numeric initial compartment sizes in condition table           | ---   |       |        | +++ |      | ---   | ---  |
-| 13 | Parametric initial concentrations in condition table           | +++   |       |        | +++ |      | +++   | +++  |
-| 14 | Numeric noise parameter overrides in measurement table         | +++   |       |        | +++ |      | +++   | +++  |
-| 15 | Parametric noise parameter overrides in measurement table      | +++   |       |        | +++ |      | +++   | +++  |
-| 16 | Observable transformations to log scale                        | +-+   |       |        | +++ |      | +-+   | +-+  |
+| ID | Test                                                           | AMICI<br>*`develop`* | Copasi | D2D | dMod | MEIGO | parPE<br>*`develop`*  | pyABC | pyPESTO<br>*`develop`* |
+|----|----------------------------------------------------------------|----------------------|--------|-----|------|-------|-----------------------|-------|------------------------|
+| 1  | Basic simulation                                               | +++                  |        | +++ |      | +++   | --+                   | +++   | +++                    |
+| 2  | Multiple simulation conditions                                 | +++                  |        | +++ |      | +++   | --+                   | +++   | +++                    |
+| 3  | Numeric observable parameter overrides in measurement table    | +++                  |        | +++ |      | +++   | --+                   | +++   | +++                    |
+| 4  | Parametric observable parameter overrides in measurement table | +++                  |        | +++ |      | +++   | --+                   | +++   | +++                    |
+| 5  | Parametric overrides in condition table                        | +++                  |        | +++ |      | +++   | --+                   | +++   | +++                    |
+| 6  | Time-point specific overrides in the measurement table         | ---                  |        | +++ |      | ---   | ---                   | ---   | ---                    |
+| 7  | Observable transformations to log10 scale                      | +-+                  |        | +++ |      | ++-   | --+                   | +-+   | +-+                    |
+| 8  | Replicate measurements                                         | +++                  |        | +++ |      | ---   | --+                   | +++   | +++                    |
+| 9  | Pre-equilibration                                              | +++                  |        | +++ |      | ---   | --+                   | +++   | +++                    |
+| 10 | Partial pre-equilibration                                      | +++                  |        | +++ |      | ---   | --+                   | +++   | +++                    |
+| 11 | Numeric initial concentration in condition table               | +++                  |        | +++ |      | ---   | --+                   | +++   | +++                    |
+| 12 | Numeric initial compartment sizes in condition table           | ---                  |        | +++ |      | ---   | ---                   | ---   | ---                    |
+| 13 | Parametric initial concentrations in condition table           | +++                  |        | +++ |      | ---   | --+                   | +++   | +++                    |
+| 14 | Numeric noise parameter overrides in measurement table         | +++                  |        | +++ |      | +++   | --+                   | +++   | +++                    |
+| 15 | Parametric noise parameter overrides in measurement table      | +++                  |        | +++ |      | +++   | --+                   | +++   | +++                    |
+| 16 | Observable transformations to log scale                        | +-+                  |        | +++ |      | ++-   | --+                   | +-+   | +-+                    |
 
     Legend:
     * First character indicates whether computing simulated data is supported and simulations are correct (+) or not (-)
@@ -124,8 +130,9 @@ further questions regarding PEtab, feel free to post an
 
 ## PEtab Python library
 
-PEtab comes with a Python package for creating, checking, and working with 
-PEtab files. This library is available on pypi and the easiest way to install 
+PEtab comes with a Python package for creating, checking, visualizing and
+working with PEtab files. This library is available on
+[pypi](https://pypi.org/project/petab/) and the easiest way to install
 it is running
 
     pip3 install petab
@@ -152,8 +159,20 @@ be:
   - `petab.create_combine_archive` to create a
     [COMBINE Archive](https://combinearchive.org/index/) from PEtab files
 
-## Extending PEtab
+### Library examples
+
+Examples for PEtab Python library usage:
+
+* [Validation](https://github.com/PEtab-dev/PEtab/blob/master/doc/example/example_petablint.ipynb)
+* [Visualization](https://github.com/PEtab-dev/PEtab/blob/master/doc/example/example_visualization.ipynb)
+
+
+## Getting help
+
+If you have any question or problems with PEtab, feel free to post them at
+our GitHub [issue tracker](https://github.com/PEtab-dev/PEtab/issues/).
+
+## Contributing to PEtab
 
-We are aware of the fact that PEtab may not serve everybody's needs. If you 
-have a suggestion of how to extend PEtab, feel free to post an issue at our 
-github repository.
+Contributions and feedback to PEtab are very welcome, see our
+[contribution guide](CONTRIBUTING.md).
diff --git a/petab/calculate.py b/petab/calculate.py
@@ -89,13 +89,11 @@ def calculate_residuals_for_table(
     # iterate over measurements, find corresponding simulations
     for irow, row in measurement_df.iterrows():
         measurement = row[MEASUREMENT]
-
         # look up in simulation df
-        masks = [simulation_df[col] == row[col] for col in compared_cols]
+        masks = [(simulation_df[col] == row[col]) | petab.is_empty(row[col])
+                 for col in compared_cols]
         mask = reduce(lambda x, y: x & y, masks)
-
         simulation = simulation_df.loc[mask][SIMULATION].iloc[0]
-
         if scale:
             # apply scaling
             observable = observable_df.loc[row[OBSERVABLE_ID]]
@@ -301,7 +299,8 @@ def calculate_llh_for_table(
         measurement = row[MEASUREMENT]
 
         # look up in simulation df
-        masks = [simulation_df[col] == row[col] for col in compared_cols]
+        masks = [(simulation_df[col] == row[col]) | petab.is_empty(row[col])
+                 for col in compared_cols]
         mask = reduce(lambda x, y: x & y, masks)
 
         simulation = simulation_df.loc[mask][SIMULATION].iloc[0]

diff --git a/petab/version.py b/petab/version.py
@@ -1,2 +1,2 @@
 """PEtab library version"""
-__version__ = '0.1.4'
+__version__ = '0.1.5'
diff --git a/petab/visualize/helper_functions.py b/petab/visualize/helper_functions.py
@@ -495,8 +495,12 @@ def handle_dataset_plot(plot_spec: pd.Series,
     ind_dataset = exp_data[DATASET_ID] == dataset_id
 
     # gather simulationConditionIds belonging to datasetId
-    uni_condition_id = np.unique(
-        exp_data[ind_dataset][SIMULATION_CONDITION_ID])
+    uni_condition_id, uind = np.unique(
+        exp_data[ind_dataset][SIMULATION_CONDITION_ID],
+        return_index=True)
+    # keep the ordering which was given by user from top to bottom
+    # (avoid ordering by names '1','10','11','2',...)'
+    uni_condition_id = uni_condition_id[np.argsort(uind)]
     col_name_unique = SIMULATION_CONDITION_ID
 
     # Case separation of independent parameter: condition, time or custom

diff --git a/petab/visualize/plotting_config.py b/petab/visualize/plotting_config.py
@@ -92,8 +92,10 @@ def plot_lowlevel(plot_spec: pd.Series,
 
         # construct errorbar-plots: noise specified above
         else:
+            scond, smean, snoise = \
+                zip(*sorted(zip(conditions, ms['mean'], ms[noise_col])))
             p = ax.errorbar(
-                conditions, ms['mean'], ms[noise_col],
+                scond, smean, snoise,
                 linestyle='-.', marker='.', label=label_base
             )
         # construct simulation plot

diff --git a/tests/test_visualization.py b/tests/test_visualization.py
@@ -1,5 +1,5 @@
 import warnings
-
+from tempfile import TemporaryDirectory
 import pytest
 from petab.C import *
 from petab.visualize import (plot_data_and_simulation,
@@ -274,8 +274,10 @@ def test_simple_visualization(data_file_Fujita, condition_file_Fujita):
 
 def test_save_plots_to_file(data_file_Isensee, condition_file_Isensee,
                             vis_spec_file_Isensee, simulation_file_Isensee):
-    plot_data_and_simulation(data_file_Isensee,
-                             condition_file_Isensee,
-                             vis_spec_file_Isensee,
-                             simulation_file_Isensee,
-                             subplot_file_path='./')
+    with TemporaryDirectory() as temp_dir:
+        plot_data_and_simulation(
+            data_file_Isensee,
+            condition_file_Isensee,
+            vis_spec_file_Isensee,
+            simulation_file_Isensee,
+            subplot_file_path=temp_dir)