Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make heuristics a simple set of "if .. elif .. else" statements and use a dictionary instead of variables. #209

Merged
merged 21 commits into from
Apr 22, 2020
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 13 additions & 16 deletions docs/heuristic.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,36 +22,34 @@ Initialisation

.. literalinclude:: ../phys2bids/heuristics/heur_tutorial.py
:linenos:
:lines: 1-4
:lines: 1-40

It's important **not to modify this part of the file**. Instead, you can copy and paste it into your own heuristic file.

This file looks like a python function, initialised by two mandatory parameters:
-``physinfo`` is the information used to label your file. **At the moment, it corresponds to the name of the input file itself**. This is what you need to build your heuristic.
-``name`` is an argument passed by the main script that contains part of the name of the file. Don't worry about this.
-``info`` is a `python dictionary <https://www.w3schools.com/python/python_dictionaries.asp>`_ passed by the main script that contains the BIDS keys, such as `sub` and `ses`, as well as all the possible keys you can add to your heuristics. This is what you will work with in creating your heuristic

The file also has another bunch of optional arguments that are empty by default. These are the labels you can add to your dictionaries, in order to construct the BIDsified name of your files.

The scripts imports ``fnmatch``, a nice python module that lets you use bash-like wildcards.
The scripts imports ``fnmatch``, a nice python module that lets you use bash-like wildcards, as well as ``logger``, a module that let us log what happens in the program execution.

Dictionaries
^^^^^^^^^^^^

.. literalinclude:: ../phys2bids/heuristics/heur_tutorial.py
:linenos:
:lines: 5-29
:lineno-start: 5
:lines: 41-63
:lineno-start: 41
:dedent: 4

This is the core of the function, and the part that should be adapted to process your files. In practice, it's the beginning of a |statement|_.
| You need an ``if`` or ``elif`` statement for each file that you want to process, that will test if the ``physinfo`` is similar to a string (first case) or exactly matches a string (second case). The content of the statement is a set of `variable initialisations as a string <https://www.w3schools.com/python/python_strings.asp>`_.
| The list of possible variables is in the comment above, and corresponds to the list of possible entities of the `BIDs specification <https://bids-specification.readthedocs.io/en/stable/04-modality-specific-files/06-physiological-and-other-continuous-recordings.html>`_:
| This is the core of the function, and the part that should be adapted to process your files. In practice, it's the beginning of a |statement|_.
| You need an ``if`` or ``elif`` statement for each file that you want to process, that will test if the ``physinfo`` is similar to a string (first case) or exactly matches a string (second case). The content of the statement is a set of `variable initialisations as a string <https://www.w3schools.com/python/python_strings.asp>`_, with the only difference that you're populating a dictionary here. This means that instead of declaring something like ``var = 'something'``, you will declare something like ``info['var'] = 'something'``
| The list of possible keys is in the comment above, and corresponds to the list of possible entities of the `BIDs specification <https://bids-specification.readthedocs.io/en/stable/04-modality-specific-files/06-physiological-and-other-continuous-recordings.html>`_:

- ``task`` stands for the name of the task. **It's the only required entity**, and it should match the task of the neuroimaging file associated to the physiological data.
- ``task`` stands for the name of the task. **It's the only required entity**, and it should match the task of the neuroimaging file associated to the physiological data. If this is missing **the program will stop running**
- ``run`` is the optional entity for the `index of the scan in a group of same modalities <https://bids-specification.readthedocs.io/en/stable/04-modality-specific-files/01-magnetic-resonance-imaging-data.html#the-run-entity>`_ (e.g. 2 resting states).
- ``rec`` is the optional entity for the `reconstruction algorithm <https://bids-specification.readthedocs.io/en/stable/04-modality-specific-files/01-magnetic-resonance-imaging-data.html#the-rec-entity>`_.
- ``acq`` is the optional entity for the `set of acquisition parameters <https://bids-specification.readthedocs.io/en/stable/04-modality-specific-files/01-magnetic-resonance-imaging-data.html#the-acq-entity>`_.
- ``direct`` is the equivalent of the ``dic`` entity, an optional entity for the phase encoding direction (see `here <https://bids-specification.readthedocs.io/en/stable/04-modality-specific-files/01-magnetic-resonance-imaging-data.html#task-including-resting-state-imaging-data>`_).
- ``dir`` is the optional entity for the phase encoding direction (see `here <https://bids-specification.readthedocs.io/en/stable/04-modality-specific-files/01-magnetic-resonance-imaging-data.html#task-including-resting-state-imaging-data>`_).

Note that one mandatory BIDs entity is missing: the **``sub`` entity**, correspondent to the subject label. This is because it has to be specified while calling ``phys2bids``, as it's explained in the tutorial section `"generating-outputs-in-bids-format" <howto.html#generating-outputs-in-bids-format>`_. The **session entity** can be specified in the same way. Moreover, if you have a **multifrequency file** there will be another entity, ``recording`` automatically added to those specified here, and containing the sample frequency of the different outputs.

Expand All @@ -70,13 +68,12 @@ Functional code

.. literalinclude:: ../phys2bids/heuristics/heur_tutorial.py
:linenos:
:lines: 27-
:lineno-start: 27
:lines: 64-
:lineno-start: 64
:dedent: 4

This part contains some code that composes the heuristic function output.
This part contains a simple warning call that will be reported to the logger if the heuristic file doesn't contain actual references to your input files.
It's important **not to modify this part of the file**. Instead, you can copy and paste it into your own heuristic file.
There's a warning that will raise if the file wasn't able to process the input file, and an error that will raise if the mandatory ``task`` entity is still empty after the dictionary attribution.

Using the heuristic file
------------------------
Expand Down
5 changes: 3 additions & 2 deletions docs/howto.rst
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,7 @@ In the second row, we see the first and last trigger (or expected first and last

.. warning::
If you have another file that was collected in an identical way, the threshold is likely to be same. However, it is *very* important to calibrate the threshold in a couple of files before assuming this. This still *won't* necessarily mean that it's the right threshold for all the files, but there's a chance that it's ok(ish) for most of them.

If for some reason ``-ntp`` and the number of timepoints found by ``phys2bids`` is not the same there are two possible reasons:

1. You didn't properly count the number of timepoints. Check this again; you can use the trigger png file to help you.
Expand Down Expand Up @@ -281,8 +282,8 @@ Alright, now the really interesting part! This section will explain how to use t

.. literalinclude:: ../phys2bids/heuristics/heur_tutorial.py
:linenos:
:lines: 4-22
:lineno-start: 4
:lines: 41-63
:lineno-start: 41

The heuristic file has to be written accordingly, with a set of rules that could work for all the files in your dataset. You can learn more about it if you check the `guide on how to set it up <heuristic.html>`_.
In this case, our heuristic file looks for a file that contains the name ``tutorial``. It corresponds to the task ``test`` and run ``00``. Note that **only the task is required**, all the other fields are optional - look them up in the BIDs documentation and see if you need them.
Expand Down
73 changes: 0 additions & 73 deletions phys2bids/heuristics/heur.py

This file was deleted.

93 changes: 93 additions & 0 deletions phys2bids/heuristics/heur_euskalibur.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
import fnmatch
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this one is an example?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, why not.
it's the one used for a dataset that will be shared at a certain point.

import logging

LGR = logging.getLogger(__name__)


def heur(physinfo, info):
"""
Set of if .. elif statements to fill BIDS names.

Requires the user (you!) to adjust it accordingly!
The `if ..` structure should always be similar to
```
if physinfo == 'somepattern':
info['var'] = 'somethingelse'
```
or, in case it's a partial match
```
if fnmatch.fnmatchcase(physinfo, '*somepattern?'):
info['var'] = 'somethingelse'
```

Where:
- `physinfo` and `info` are dedicated keywords,
- 'somepattern' is the name of the file,
- 'var' is a bids key in the list below
- 'somethingelse' is the value of the key

Parameters
----------
physinfo: str
Name of the file or partial match
info: dictionary of str
Dictionary containing BIDS keys

Returns
-------
info: dictionary of str
Populated dictionary containing BIDS keys
"""
# ################################# #
# ## Modify here! ## #
# ## ## #
# ## Possible variables are: ## #
# ## -info['task'] (required) ## #
# ## -info['run'] ## #
# ## -info['rec'] ## #
# ## -info['acq'] ## #
# ## -info['dir'] ## #
# ## ## #
# ## Remember that they are ## #
# ## dictionary keys ## #
# ## See example below ## #
# ################################# #

if physinfo == 'origfilename1':
info['task'] = 'newname1'
elif physinfo == 'origfilename2':
info['task'] = 'newname2'
info['run'] = 'runnum'
elif physinfo == 'BH4':
info['task'] = 'breathhold'
elif fnmatch.fnmatchcase(physinfo, 'MOTOR?'):
info['task'] = 'motor'
elif fnmatch.fnmatchcase(physinfo, 'LOCALIZER?'):
info['task'] = 'pinel'
elif fnmatch.fnmatchcase(physinfo, 'SIMON?'):
info['task'] = 'simon'
elif physinfo == 'RS1':
info['task'] = 'rest'
info['run'] = '01'
elif physinfo == 'RS2':
info['task'] = 'rest'
info['run'] = '02'
elif physinfo == 'RS3':
info['task'] = 'rest'
info['run'] = '03'
elif physinfo == 'RS4':
info['task'] = 'rest'
info['run'] = '04'
# ############################## #
# ## Don't modify below this! ## #
# ############################## #
else:
# #!# Transform sys.exit in debug warnings or raiseexceptions!
# #!# Make all of the above a dictionary
LGR.warning(f'The heuristic {__file__} could not deal with {physinfo}')

# filename spec: sub-<label>[_ses-<label>]_task-<label>[_acq-<label>] ...
# ... [_ce-<label>][_dir-<label>][_rec-<label>] ...
# ... [_run-<index>][_recording-<label>]_physio

return info
57 changes: 0 additions & 57 deletions phys2bids/heuristics/heur_ex.py

This file was deleted.

Loading