diff --git a/CLUSTER_SETUP b/CLUSTER_SETUP new file mode 100644 index 0000000..8986138 --- /dev/null +++ b/CLUSTER_SETUP @@ -0,0 +1,14 @@ +module load miniforge3 + +conda create -n myenv_arbor python=3.10 + +source activate myenv_arbor + +conda install -y scipy matplotlib pandas cmake svgwrite git boost svgwrite nano hyperfine expect pybind11 +conda install -y gcc=13.2.0 g++=13.2.0 +conda install -y nvidia/label/cuda-12.4.0::cuda-toolkit -c nvidia/label/cuda-12.4.0 + +python3 -m pip install pybind11 pybind11-stubgen +python3 -m pip install pytest pytest-cov coverage + +conda deactivate diff --git a/README.md b/README.md index fc356d4..a474dae 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,20 @@ -# Arbor simulation of memory consolidation in recurrent spiking neural networks based on synaptic tagging and capture +# Arbor simulation of memory formation and consolidation in recurrent spiking neural networks with synaptic tagging and capture -This package serves to simulate recurrent spiking neural networks, consisting of leaky integrate-and-fire neurons connected via current-based plastic synapses, with the Arbor simulator library. +This package serves to simulate recurrent spiking neural networks, consisting of single-compartment (approximate leaky integrate-and-fire) neurons connected via current-based plastic synapses, with the Arbor simulator library. The long-term plasticity model that is employed features a calcium-based early phase and a late phase that is based on synaptic tagging-and-capture mechanisms. -The underlying model has been described in detail in [Luboeinski and Tetzlaff (2021)](https://doi.org/10.1038/s42003-021-01778-y) and has previously been implemented with a [stand-alone simulator](https://github.com/jlubo/memory-consolidation-stc). The code provided here can reproduce the previous results with the Arbor simulator library. -An implementation that employs Arbor to simulate similar networks with morphological neurons will be made available soon. +The underlying model is described in detail in [Luboeinski and Tetzlaff (2021)](https://doi.org/10.1038/s42003-021-01778-y) and has also been implemented with point neurons using a [stand-alone simulator](https://github.com/jlubo/memory-consolidation-stc) as well as [Brian 2](https://github.com/jlubo/brian_network_plasticity). The code provided here serves to reproduce the previous results with the Arbor simulator library. + +Compared to the stand-alone implementation, using Arbor has several advantages: +* facilitated use of multi-compartment neurons, +* optimized computing on a variety of hardware backends, especially, on high-performance computing setups, +* easier entry for users who are already using Arbor or open to use it. + +An implementation that employs Arbor to simulate networks with multi-compartment neurons, derived from the present single-compartment implementation, can be found [here](https://github.com/jlubo/arbor_network_consolidation_mc). ### Provided code -The main simulation code is found in `arborNetworkConsolidation.py`. The code in `plotResults.py` is used to plot the results and is called automatically after a simulation is finished (but it can also be used on its own). The file `outputUtilities.py` provides additional utility functions needed. The parameter configurations for different types of simulations are provided by means of `*.json` files. The neuron and synapse mechanisms are provided in the [Arbor NMODL format](https://docs.arbor-sim.org/en/v0.10.0/fileformat/nmodl.html) in the files `mechanisms/*.mod`. +The main simulation code is found in `arborNetworkConsolidation.py`. The code in `plotResults.py` is used to plot the results and is called automatically after a simulation is finished (but it can also be used on its own). The file `outputUtilities.py` provides additional utility functions that are needed. The parameter configurations for different types of simulations are provided by means of `*.json` files. The neuron and synapse mechanisms are provided in the [Arbor NMODL format](https://docs.arbor-sim.org/en/v0.10.0/fileformat/nmodl.html) in the files `mechanisms/*.mod`. -To achieve viable runtimes even for long biological timescales, the program determines a "schedule" for each simulation (e.g., "learn - consolidate"). This causes the simulation to either run in one piece, or to be split up into different phases which are computed using different timesteps (small timesteps for detailed dynamics and substantially longer timesteps for phases of plasticity consolidation; in the latter case, the spiking and calcium dynamics are neglected, and only the late-phase dynamics and the exponential decay of the early-phase weights are computed, the validty of which has been shown [here](https://doi.org/10.53846/goediss-463)). The plasticity mechanism (`expsyn_curr_early_late_plasticity` or `expsyn_curr_early_late_plasticity_ff`) is chosen accordingly. +To achieve viable runtimes even for long biological timescales, the program determines a "schedule" for each simulation (e.g., "learn - consolidate"). This causes the simulation to either run in one piece, or to be split up into different phases which are computed using different timesteps (small timesteps for detailed dynamics and substantially longer timesteps for phases of plasticity consolidation; in the latter case, the spiking and calcium dynamics are neglected, and only the late-phase dynamics and the exponential decay of the early-phase weights are computed, the validity of which has been shown [here](https://doi.org/10.53846/goediss-463)). The plasticity mechanism (`expsyn_curr_early_late_plasticity` or `expsyn_curr_early_late_plasticity_ff`) is chosen accordingly. Different simulation protocols can easily be run using the following bash script files: * `run_basic_early` @@ -46,37 +52,39 @@ Different simulation protocols can easily be run using the following bash script - one exc. neuron receives a learning stimulus; - the same exc. neurons receives a recall stimulus 8 hours later -Besides these scripts, there are also scripts with the filename suffix `_gwdg-*` (instead of `_desktop`). Those scripts are intended to run simulations on a SLURM compute cluster (as operated by [GWDG](https://gwdg.de/)). +Besides these scripts, there are also scripts with the filename suffix `_gwdg-*` (instead of `_desktop`). Those scripts are intended to run simulations on a SLURM compute cluster (as operated by [GWDG](https://gwdg.de/)). See the file `CLUSTER_SETUP` on how to set up a suitable miniforge environment for this. ### Tests Integration tests are defined in `test_arborNetworkConsolidation.py` and can be run via the bash script file `run_tests`. ### Arbor installation -The code has been tested with Arbor version v0.10.0. +The latest version of the code has been tested with the Arbor development version [v0.10.1-dev-b8b768d](https://github.com/arbor-sim/arbor/commit/b8b768d6aed3aa1e72b91912753d98bbc17fb44c). Use this version if you want to be sure that everything runs correctly. + +Previous versions of the code have also been tested with the relase version [v0.10.0](https://github.com/arbor-sim/arbor/releases/v0.10.0) and with several development versions of v0.9.1 and v0.10.1. #### Default version -Most conveniently, you can install this Arbor version via +You can, most conveniently, install the latest Arbor release version via ``` -python3 -m pip install arbor==0.10.0 +python3 -m pip install arbor ``` -To install this Arbor version from source code (default version, without SIMD support), you can run the following: +To install a specific Arbor version from source code (default mode, without SIMD support), you can run the following (gcc/g++ v13 are recommended): ``` git clone --recursive https://github.com/arbor-sim/arbor/ arbor_source_repo mkdir arbor_source_repo/build && cd arbor_source_repo/build -git checkout 6b6cc900b85fbf833fae94817b9406a0d690dc28 -b arbor_v0.10.0 -cmake -DARB_WITH_PYTHON=ON -DARB_USE_BUNDLED_LIBS=ON -DCMAKE_INSTALL_PREFIX=$(readlink -f ~/arbor_v0.9.1-dev-plastic_arbor_v2-nosimd) -DPYTHON_EXECUTABLE:FILEPATH=`which python3.10` -S .. -B . +git checkout b8b768d6aed3aa1e72b91912753d98bbc17fb44c -b arbor_v0.10.1-dev-b8b768d +CC=gcc-13 CXX=g++-13 cmake -DARB_WITH_PYTHON=ON -DARB_USE_BUNDLED_LIBS=ON -DCMAKE_INSTALL_PREFIX=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-nosimd) -DPYTHON_EXECUTABLE:FILEPATH=`which python3.10` -S .. -B . #make tests && ./bin/unit # optionally: testing make install ``` #### SIMD version -To install this Arbor version from source code (with SIMD support), you can run the following: +To install a specific Arbor version from source code (with SIMD support), you can run the following (gcc/g++ v13 are recommended): ``` git clone --recursive https://github.com/arbor-sim/arbor/ arbor_source_repo mkdir arbor_source_repo/build && cd arbor_source_repo/build -git checkout 6b6cc900b85fbf833fae94817b9406a0d690dc28 -b arbor_v0.10.0 -cmake -DARB_WITH_PYTHON=ON -DARB_USE_BUNDLED_LIBS=ON -DARB_VECTORIZE=ON -DCMAKE_INSTALL_PREFIX=$(readlink -f ~/arbor_v0.9.1-dev-plastic_arbor_v2-simd) -DPYTHON_EXECUTABLE:FILEPATH=`which python3.10` -S .. -B . +git checkout b8b768d6aed3aa1e72b91912753d98bbc17fb44c -b arbor_v0.10.1-dev-b8b768d +CC=gcc-13 CXX=g++-13 cmake -DARB_WITH_PYTHON=ON -DARB_USE_BUNDLED_LIBS=ON -DARB_VECTORIZE=ON -DCMAKE_INSTALL_PREFIX=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-simd) -DPYTHON_EXECUTABLE:FILEPATH=`which python3.10` -S .. -B . #make tests && ./bin/unit # optionally: testing make install ``` @@ -86,19 +94,20 @@ CMAKE_ARGS="-DARB_VECTORIZE=ON" python3 -m pip install ./arbor_source_repo ``` #### CUDA version -To install this Arbor version from source code (with CUDA GPU support), you can run the following (cudatoolkit v11.5 and gcc/g++ v10 are recommended): +To install a specific Arbor version from source code (with CUDA GPU support), you can run the following (cudatoolkit v12 and gcc/g++ v13 are recommended): ``` git clone --recursive https://github.com/arbor-sim/arbor/ arbor_source_repo mkdir arbor_source_repo/build && cd arbor_source_repo/build -git checkout 6b6cc900b85fbf833fae94817b9406a0d690dc28 -b arbor_v0.10.0 -CC=gcc-10 CXX=g++-10 cmake -DARB_WITH_PYTHON=ON -DARB_USE_BUNDLED_LIBS=ON -DARB_GPU=cuda -DARB_USE_GPU_RNG=ON -DCMAKE_INSTALL_PREFIX=$(readlink -f ~/arbor_v0.9.1-dev-plastic_arbor_v2-cuda_use_gpu_rng) -DPYTHON_EXECUTABLE:FILEPATH=`which python3.10` -S .. -B . +git checkout b8b768d6aed3aa1e72b91912753d98bbc17fb44c -b arbor_v0.10.1-dev-b8b768d +CC=gcc-13 CXX=g++-13 cmake -DARB_WITH_PYTHON=ON -DARB_USE_BUNDLED_LIBS=ON -DARB_GPU=cuda -DARB_USE_GPU_RNG=ON -DCMAKE_INSTALL_PREFIX=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu_use_gpu_rng) -DPYTHON_EXECUTABLE:FILEPATH=`which python3.10` -S .. -B . #make tests && ./bin/unit # optionally: testing make install ``` You may also take this shortcut with `pip`: ``` -CMAKE_ARGS="-DARB_GPU=cuda" python3 -m pip install ./arbor_source_repo +CMAKE_ARGS="-DARB_GPU=cuda -DARB_USE_GPU_RNG=ON" python3 -m pip install ./arbor_source_repo ``` +Note that you may set `ARB_USE_GPU_RNG` to `OFF` to avoid issues in certain software setups (but this may increase the runtime). #### Environment setting diff --git a/arborNetworkConsolidation.py b/arborNetworkConsolidation.py index 2ca9f73..4fbc7c2 100755 --- a/arborNetworkConsolidation.py +++ b/arborNetworkConsolidation.py @@ -2,9 +2,10 @@ # Arbor simulation of memory consolidation in recurrent spiking neural networks, consisting of leaky integrate-and-fire neurons that are # connected via current-based, plastic synapses (early phase based on calcium dynamics and late phase based on synaptic tagging and capture). -# Intended to reproduce the results of Luboeinski and Tetzlaff, Commun. Biol., 2021 (https://doi.org/10.1038/s42003-021-01778-y). +# Intended to reproduce the results of Luboeinski and Tetzlaff, Commun. Biol., 2021 (https://doi.org/10.1038/s42003-021-01778-y). For further +# details, see https://doi.org/10.48550/arXiv.2411.16445. -# Copyright 2021-2023 Jannik Luboeinski +# Copyright 2021-2024 Jannik Luboeinski # License: Apache-2.0 (http://www.apache.org/licenses/LICENSE-2.0) # Contact: mail[at]jlubo.net @@ -21,7 +22,7 @@ from outputUtilities import getTimestamp, getFormattedTime, getTimeDiff, \ setDataPathPrefix, getDataPath, \ openLog, writeLog, writeAddLog, flushLog, closeLog, \ - getSynapseId + getPresynapticId setDataPathPrefix("data_") @@ -286,7 +287,7 @@ def __init__(self, config, adap_dt, plasticity_mechanism, store_latest_state): self.recall_prot = completeProt(config["simulation"]["recall_protocol"]) # protocol for recall stimulation as a dictionary with the keys "time_start" (starting time), "scheme" (scheme of pulses), and "freq" (stimulation frequency) self.bg_prot = completeProt(config["simulation"]["bg_protocol"]) # protocol for background input as a dictionary with the keys "time_start" (starting time), "scheme" (scheme of pulses), "I_0" (mean), and "sigma_WN" (standard deviation) self.sample_gid_list = config['simulation']['sample_gid_list'] # list of the neurons that are to be probed (given by number/gid) - self.sample_syn_list = config['simulation']['sample_syn_list'] # list of the synapses that are to be probed (one synapse per neuron only, given by its internal number respective to a neuron in sample_gid); -1: none + self.sample_pre_list = config['simulation']['sample_pre_list'] # list of the presynaptic neurons for probing synapses (one value for each value in `sample_gid`; -1: no synapse probing) if config['populations']['conn_file']: # if a connections file is specified, load the connectivity matrix from that file self.conn_matrix = np.loadtxt(config['populations']['conn_file']).T @@ -360,7 +361,10 @@ def cell_description(self, gid): # initialize decor decor = arbor.decor() - decor.discretization(arbor.cv_policy("(max-extent 100)")) + + # set CV policy + cv_policy = arbor.cv_policy("(max-extent 100)") + #decor.discretization(cv_policy) # since v0.10.1-dev-76120d1, this has to be done in `cable_cell()` (see below) # neuronal dynamics decor.set_property(Vm=self.neuron_config["V_init"]*U.mV, cm=c_mem) @@ -390,7 +394,7 @@ def cell_description(self, gid): decor.set_ion("sps_", int_con=0*U.mM, diff=self.D) # signal to trigger protein synthesis decor.set_ion("pc_", int_con=0*U.mM, diff=self.D) # proteins - # excitatory neurons + # excitatory (postsynaptic) neurons if gid < self.N_exc: # set neuronal state variables to values loaded from previous state of the network if self.p is not None: @@ -404,7 +408,7 @@ def cell_description(self, gid): writeAddLog("c_mem =", c_mem, "F/m^2") writeAddLog("tau_mem =", tau_mem, "ms") - # create plastic excitatory exponential synapse + # initialize plastic exponential mechanism for excitatory mech_expsyn_exc = arbor.mechanism(self.plasticity_mechanism) # set standard parameters @@ -421,57 +425,60 @@ def cell_description(self, gid): mech_expsyn_exc.set('theta_d', self.syn_plasticity_config["theta_d"]) mech_expsyn_exc.set('sigma_pl', self.syn_plasticity_config["sigma_pl"]) - exc_connections = np.array(self.conn_matrix[gid][0:self.N_exc], dtype=bool) # array of booleans indicating all incoming excitatory connections - exc_pre_neurons = np.arange(self.N_exc)[exc_connections] # array of excitatory presynaptic neurons indicated by their gid - #inc_exc_connections = np.sum(self.conn_matrix[gid][0:self.N_exc], dtype=int) # number of incoming excitatory connections - inc_exc_connections = len(exc_pre_neurons) + exc_exc_connections = np.array(self.conn_matrix[gid][0:self.N_exc], dtype=bool) # array of booleans defining all incoming excitatory connections + exc_exc_pre_neurons = np.arange(self.N_exc)[exc_exc_connections] # array of excitatory presynaptic neurons given by their gid - # place synapses; set synaptic state variables to values loaded from previous state of the network - for n in reversed(range(inc_exc_connections)): ### NOTE THE REVERSED ORDER (SEE BELOW) !!! + # place exc.->exc. synapses; set synaptic state variables to values loaded from previous state of the network + for presyn_gid in exc_exc_pre_neurons: if (self.h is not None and self.z is not None): - mech_expsyn_exc.set('h_init', self.h[gid][exc_pre_neurons[n]]) - mech_expsyn_exc.set('z_init', self.z[gid][exc_pre_neurons[n]]) + mech_expsyn_exc.set('h_init', self.h[gid][presyn_gid]) + mech_expsyn_exc.set('z_init', self.z[gid][presyn_gid]) if self.N_exc <= 4: # only for small networks - writeAddLog(f"Setting loaded values for synapse {exc_pre_neurons[n]}->{gid} (neuron {gid}, inc. " + - f"synapse {n})..." + - f"\n h = {self.h[gid][exc_pre_neurons[n]]}" + - f"\n z = {self.z[gid][exc_pre_neurons[n]]}") + writeAddLog(f"Setting loaded values for synapse {presyn_gid}->{gid}...\n" + + f" h = {self.h[gid][presyn_gid]}\n" + + f" z = {self.z[gid][presyn_gid]}") # add synapse - decor.place('"center"', arbor.synapse(mech_expsyn_exc), "syn_ee_calcium_plasticity") # place synapse at the center of the soma (because: point neuron) - #writeLog("Placed", inc_exc_connections, "incoming E->E synapses for neuron", gid) - - # non-plastic inhibitory exponential synapse + decor.place('"center"', arbor.synapse(mech_expsyn_exc), f"syn_ee_calcium_plasticity_{presyn_gid}_{gid}") # place synapse at the center of the soma + # (because of point neuron approximation) + # initialize non-plastic exponential mechanism for inhibitory synapse mech_expsyn_inh = arbor.mechanism('expsyn_curr') mech_expsyn_inh.set('w', -self.w_ie.value_as(U.mV) * self.h_0.value_as(U.mV)) mech_expsyn_inh.set('R_mem', R_leak.value_as(U.MOhm)) mech_expsyn_inh.set('tau', self.syn_config["tau_syn"]) - inc_inh_connections = np.sum(self.conn_matrix[gid][self.N_exc:self.N_tot], dtype=int) # number of incoming inhibitory connections - for i in range(inc_inh_connections): - decor.place('"center"', arbor.synapse(mech_expsyn_inh), "syn_ie") # place synapse at the center of the soma (because: point neuron) + # place inh.->exc. synapses + inh_exc_connections = np.array(self.conn_matrix[gid][self.N_exc:self.N_tot], dtype=bool) # array of booleans defining all incoming inhibitory connections + inh_exc_pre_neurons = np.arange(self.N_exc, self.N_tot)[inh_exc_connections] # array of inhibitory presynaptic neurons given by their gid + for presyn_gid in inh_exc_pre_neurons: + decor.place('"center"', arbor.synapse(mech_expsyn_inh), f"syn_ie_{presyn_gid}_{gid}") # place synapse at the center of the soma (because: point neuron) self.exc_neurons_counter += 1 # for testing - # inhibitory neurons + # inhibitory (postsynaptic) neurons else: - # non-plastic excitatory exponential synapse + # initialize non-plastic exponential mechanism for excitatory synapse mech_expsyn_exc = arbor.mechanism('expsyn_curr') mech_expsyn_exc.set('w', self.w_ei.value_as(U.mV) * self.h_0.value_as(U.mV)) mech_expsyn_exc.set('R_mem', R_leak.value_as(U.MOhm)) mech_expsyn_exc.set('tau', self.syn_config["tau_syn"]) - inc_exc_connections = np.sum(self.conn_matrix[gid][0:self.N_exc], dtype=int) # number of incoming excitatory connections - for i in range(inc_exc_connections): - decor.place('"center"', arbor.synapse(mech_expsyn_exc), "syn_ei") # place synapse at the center of the soma (because: point neuron) + + # place exc.->inh. synapses + exc_inh_connections = np.array(self.conn_matrix[gid][0:self.N_exc], dtype=bool) # array of booleans defining all incoming excitatory connections + exc_inh_pre_neurons = np.arange(self.N_exc)[exc_inh_connections] # array of inhibitory presynaptic neurons given by their gid + for presyn_gid in exc_inh_pre_neurons: + decor.place('"center"', arbor.synapse(mech_expsyn_exc), f"syn_ei_{presyn_gid}_{gid}") # place synapse at the center of the soma (because: point neuron) - # non-plastic inhibitory exponential synapse + # initialize non-plastic exponential mechanism for inhibitory synapse mech_expsyn_inh = arbor.mechanism('expsyn_curr') mech_expsyn_inh.set('w', -self.w_ii.value_as(U.mV) * self.h_0.value_as(U.mV)) mech_expsyn_inh.set('R_mem', R_leak.value_as(U.MOhm)) mech_expsyn_inh.set('tau', self.syn_config["tau_syn"]) - inc_inh_connections = np.sum(self.conn_matrix[gid][self.N_exc:self.N_tot], dtype=int) # number of incoming inhibitory connections - for i in range(inc_inh_connections): - decor.place('"center"', arbor.synapse(mech_expsyn_inh), "syn_ii") # place synapse at the center of the soma (because: point neuron) + # place inh.->inh. synapses + inh_inh_connections = np.array(self.conn_matrix[gid][self.N_exc:self.N_tot], dtype=bool) # array of booleans defining all incoming inhibitory connections + inh_inh_pre_neurons = np.arange(self.N_exc, self.N_tot)[inh_inh_connections] # array of inhibitory presynaptic neurons given by their gid + for presyn_gid in inh_inh_pre_neurons: + decor.place('"center"', arbor.synapse(mech_expsyn_inh), f"syn_ii_{presyn_gid}_{gid}") # place synapse at the center of the soma (because: point neuron) self.inh_neurons_counter += 1 # for testing @@ -522,7 +529,7 @@ def cell_description(self, gid): # paint neuron mechanism decor.paint('(all)', arbor.density(mech_neuron)) - return arbor.cable_cell(tree, decor, labels) + return arbor.cable_cell(tree, decor, labels, discretization=cv_policy) # connections_on # Defines the list of incoming synaptic connections to the neuron given by gid @@ -556,28 +563,28 @@ def connections_on(self, gid): #d1 = self.syn_plasticity_config["t_Ca_delay"] # delay time of the calcium increase in ms (only for plastic synapses) d1 = max(self.adap_dt.value_as(U.ms), self.syn_plasticity_config["t_Ca_delay"])*U.ms # delay time of the calcium increase in ms (only for plastic synapses) - # excitatory neurons + # excitatory (postsynaptic) neurons if gid < self.N_exc: # incoming excitatory synapses for src in exc_pre_neurons: - #connections_list.append(arbor.connection((src, "spike_detector"), ("syn_ee_calcium_plasticity", rr), 1, d0)) # for postsynaptic potentials - connections_list.append(arbor.connection((src, "spike_detector"), ("syn_ee_calcium_plasticity", rr_halt), 1, d0)) # for postsynaptic potentials - connections_list.append(arbor.connection((src, "spike_detector"), ("syn_ee_calcium_plasticity", rr), -1, d1)) # for plasticity-related calcium dynamics - + syn_target = f"syn_ee_calcium_plasticity_{src}_{gid}" + connections_list.append(arbor.connection((src, "spike_detector"), (syn_target, rr_halt), 1, d0)) # for postsynaptic potentials + connections_list.append(arbor.connection((src, "spike_detector"), (syn_target, rr), -1, d1)) # for plasticity-related calcium dynamics + # incoming inhibitory synapses for src in inh_pre_neurons: - connections_list.append(arbor.connection((src,"spike_detector"), ("syn_ie", rr), 1, d0)) + connections_list.append(arbor.connection((src,"spike_detector"), (f"syn_ie_{src}_{gid}", rr), 1, d0)) - # inhibitory neurons + # inhibitory (postsynaptic) neurons else: # incoming excitatory synapses for src in exc_pre_neurons: - connections_list.append(arbor.connection((src,"spike_detector"), ("syn_ei", rr), 1, d0)) + connections_list.append(arbor.connection((src,"spike_detector"), (f"syn_ei_{src}_{gid}", rr), 1, d0)) # incoming inhibitory synapses for src in inh_pre_neurons: - connections_list.append(arbor.connection((src,"spike_detector"), ("syn_ii", rr), 1, d0)) + connections_list.append(arbor.connection((src,"spike_detector"), (f"syn_ii_{src}_{gid}", rr), 1, d0)) if self.N_exc <= 4: # only for small networks writeAddLog(f"Setting connections for gid = {gid}...") @@ -675,7 +682,7 @@ def get_max_num_latest_state_probes(self): return self.max_num_latest_state_probes # probes - # Sets the probes to measure neuronal and synaptic state -- WARNING: the indexing here is (for CPU) reversed to that used by 'sim.sample((gid, index), sched)' + # Sets the probes to measure neuronal and synaptic state # - gid: global identifier of the cell # - return: the probes on the given cell def probes(self, gid): @@ -687,13 +694,12 @@ def probes(self, gid): latest_state_probes = [] if self.store_latest_state and gid < self.N_exc: latest_state_probes.extend([arbor.cable_probe_density_state('"center"', self.neuron_config["mechanism"], "pc", "tag_pc_latest")]) - num_sample_syn = 0 - for j in range(self.N_exc): - if self.conn_matrix[gid][j]: - latest_state_probes.extend([arbor.cable_probe_point_state(num_sample_syn, self.plasticity_mechanism, "h", f"tag_h_latest_{num_sample_syn}"), - arbor.cable_probe_point_state(num_sample_syn, self.plasticity_mechanism, "z", f"tag_z_latest_{num_sample_syn}")]) - #arbor.cable_probe_point_state(num_sample_syn, self.plasticity_mechanism, "pc", f"tag_pc_{num_sample_syn}")]) - num_sample_syn += 1 + for presyn_gid in range(self.N_exc): + if self.conn_matrix[gid][presyn_gid]: + syn_target = f"syn_ee_calcium_plasticity_{presyn_gid}_{gid}" + latest_state_probes.extend([arbor.cable_probe_point_state(syn_target, self.plasticity_mechanism, "h", f"tag_h_latest_{presyn_gid}"), + arbor.cable_probe_point_state(syn_target, self.plasticity_mechanism, "z", f"tag_z_latest_{presyn_gid}")]) + #arbor.cable_probe_point_state(syn_target, self.plasticity_mechanism, "pc", f"tag_pc_{presyn_gid}")]) self.set_max_num_latest_state_probes(len(latest_state_probes)) # loop over all synapses whose traces are to be probed (for each 'gid' in 'sample_gid_list') @@ -709,23 +715,24 @@ def probes(self, gid): arbor.cable_probe_point_state_cell("ou_input", "I_ou", "tag_I_ou")]) # gets the synapse identifier for the corresponding neuron identifier in 'sample_gid_list' - sample_syn = getSynapseId(self.sample_syn_list, i) + presyn_gid = getPresynapticId(self.sample_pre_list, i) # for synapses to be probed: additionally get synaptic calcium concentration, early- and late-phase weight, and concentration of PRPs - if sample_syn >= 0: - - trace_probes.extend([arbor.cable_probe_point_state(sample_syn, self.plasticity_mechanism, "Ca", f"tag_Ca_{sample_syn}"), - arbor.cable_probe_point_state(sample_syn, self.plasticity_mechanism, "h", f"tag_h_{sample_syn}"), - arbor.cable_probe_point_state(sample_syn, self.plasticity_mechanism, "z", f"tag_z_{sample_syn}"), - #arbor.cable_probe_ion_diff_concentration('"center"', "sps_", f"tag_sps__{sample_syn}"), - #arbor.cable_probe_point_state(sample_syn, self.plasticity_mechanism, "sps", f"tag_sps_{sample_syn}"), - arbor.cable_probe_density_state('"center"', self.neuron_config["mechanism"], "spsV", f"tag_spsV_{sample_syn}"), - #arbor.cable_probe_point_state(sample_syn, self.plasticity_mechanism, "pc", f"tag_pc_{sample_syn}"), - #arbor.cable_probe_ion_diff_concentration('"center"', "pc_", f"tag_pc__{sample_syn}"), - arbor.cable_probe_density_state('"center"', self.neuron_config["mechanism"], "pc", f"tag_pc__{sample_syn}")]) + if presyn_gid >= 0: + syn_target = f"syn_ee_calcium_plasticity_{presyn_gid}_{gid}" + + trace_probes.extend([arbor.cable_probe_point_state(syn_target, self.plasticity_mechanism, "Ca", f"tag_Ca_{presyn_gid}"), + arbor.cable_probe_point_state(syn_target, self.plasticity_mechanism, "h", f"tag_h_{presyn_gid}"), + arbor.cable_probe_point_state(syn_target, self.plasticity_mechanism, "z", f"tag_z_{presyn_gid}"), + #arbor.cable_probe_ion_diff_concentration('"center"', "sps_", f"tag_sps__{presyn_gid}"), + #arbor.cable_probe_point_state(syn_target, self.plasticity_mechanism, "sps", f"tag_sps_{presyn_gid}"), + arbor.cable_probe_density_state('"center"', self.neuron_config["mechanism"], "spsV", f"tag_spsV_{presyn_gid}"), + #arbor.cable_probe_point_state(syn_target, self.plasticity_mechanism, "pc", f"tag_pc_{presyn_gid}"), + #arbor.cable_probe_ion_diff_concentration('"center"', "pc_", f"tag_pc__{presyn_gid}"), + arbor.cable_probe_density_state('"center"', self.neuron_config["mechanism"], "pc", f"tag_pc__{presyn_gid}")]) if self.N_exc <= 4: # only for small networks - writeAddLog(f" set probes for sample_syn = {sample_syn}") + writeAddLog(f" set probes for synapse {presyn_gid}->{gid}") self.set_max_num_trace_probes(len(trace_probes)) all_trace_probes.extend(trace_probes) @@ -754,7 +761,7 @@ def runSimPhase(phase, recipe, config, rseed, t_final_red, adap_dt, platform, pl # initialize s_desc = config['simulation']['short_description'] # short description of the simulation, including hashtags for benchmarking sample_gid_list = config['simulation']['sample_gid_list'] # list of the neurons that are to be probed (given by number/gid) - sample_syn_list = config['simulation']['sample_syn_list'] # list of the synapses that are to be probed (one synapse per neuron only, given by its internal number respective to a neuron in sample_gid); -1: none + sample_pre_list = config['simulation']['sample_pre_list'] # list of the presynaptic neurons for probing synapses (one value for each value in `sample_gid`; -1: no synapse probing) sample_curr = config['simulation']['sample_curr'] # pointer to current data (0: total membrane current, 1: OU stimulation current, 2: OU background noise current) output_period = config['simulation']['output_period'] # sampling size in timesteps (every "output_period-th" timestep, data will be recorded for plotting) loc = 0 # for single-compartment neurons, there is only one location @@ -815,7 +822,7 @@ def runSimPhase(phase, recipe, config, rseed, t_final_red, adap_dt, platform, pl #sampl_policy = arbor.sampling_policy.lax # use exact policy, just to be safe!? ############################################### - # set handles -- WARNING: the indexing here is (for CPU) reversed to that used by the probe setting mechanism !!! + # set handles handle_mem, handle_tot_curr, handle_stim_curr, handle_Ca_specific, handle_h_specific, handle_z_specific, handle_sps_specific, handle_p_specific = [], [], [], [], [], [], [], [] handle_h_syn_latest, handle_z_syn_latest, handle_p_comp_latest = [], [], [] synapse_mapping = [[] for i in range(recipe.N_exc)] # list that contains a list of matrix indices indicating the presynaptic neurons for each (excitatory) neuron -> sparse representation @@ -828,36 +835,35 @@ def runSimPhase(phase, recipe, config, rseed, t_final_red, adap_dt, platform, pl handle_p_comp_latest.append(sim.sample(i, "tag_pc_latest", final_sched)) # neuronal protein amount if recipe.N_exc <= 4: # only for small networks - writeAddLog(f"Setting handles for latest state of neuron {i}..." + - f"\n handle(p) = {handle_p_comp_latest[-1]} " + str(sim.probe_metadata(i, "tag_pc_latest"))) + writeAddLog(f"Setting handles for latest state of neuron {i}...\n" + + f" handle(p) = {handle_p_comp_latest[-1]} " + str(sim.probe_metadata(i, "tag_pc_latest"))) - for j in reversed(range(recipe.N_exc)): ### NOTE THE REVERSED ORDER !!! + for j in range(recipe.N_exc): # check if synapse is supposed to exist if recipe.conn_matrix[i][j]: - num_sample_syn = len(synapse_mapping[i]) # number of synapses that have so far been found for postsynaptic neuron 'i' + #num_sample_syn = len(synapse_mapping[i]) # number of synapses that have so far been found for postsynaptic neuron 'i' - handle_h_syn_latest.append(sim.sample(i, f"tag_h_latest_{num_sample_syn}", final_sched)) # early-phase weight - handle_z_syn_latest.append(sim.sample(i, f"tag_z_latest_{num_sample_syn}", final_sched)) # late-phase weight + handle_h_syn_latest.append(sim.sample(i, f"tag_h_latest_{j}", final_sched)) # early-phase weight + handle_z_syn_latest.append(sim.sample(i, f"tag_z_latest_{j}", final_sched)) # late-phase weight if recipe.N_exc <= 4: # only for small networks - writeAddLog(f"Setting handles for latest state of synapse {j}->{i} (neuron {i}, inc. " + - f"synapse {num_sample_syn})..." + - f"\n handle(h) = {handle_h_syn_latest[-1]} " + str(sim.probe_metadata(i, f"tag_h_latest_{num_sample_syn}")) + - f"\n handle(z) = {handle_z_syn_latest[-1]} " + str(sim.probe_metadata(i, f"tag_z_latest_{num_sample_syn}"))) + writeAddLog(f"Setting handles for latest state of synapse {j}->{i}...\n" + + f" handle(h) = {handle_h_syn_latest[-1]} " + str(sim.probe_metadata(i, f"tag_h_latest_{j}")) + "\n" + + f" handle(z) = {handle_z_syn_latest[-1]} " + str(sim.probe_metadata(i, f"tag_z_latest_{j}"))) synapse_mapping[i].append(j) - # loop over elements in 'sample_gid_list' (and thereby, 'sample_syn_list') to set handles for specific sampling + # loop over elements in 'sample_gid_list' (and thereby, 'sample_pre_list') to set handles for specific sampling for i in range(len(sample_gid_list)): # retrieve the current neuron index sample_gid = sample_gid_list[i] # retrieve the synapse index by the corresponding neuron identifier - sample_syn = getSynapseId(sample_syn_list, i) + sample_presyn_gid = getPresynapticId(sample_pre_list, i) writeAddLog(f"Setting handles for specific sample #{i} (neuron {sample_gid}, " + - f"synapse {sample_syn})...") + f"synapse {sample_presyn_gid}->{sample_gid})...") # for all neurons: get membrane potential and current(s) handle_mem.append(sim.sample(sample_gid, "tag_v", reg_sched)) # membrane potential @@ -868,17 +874,17 @@ def runSimPhase(phase, recipe, config, rseed, t_final_red, adap_dt, platform, pl f"\n handle(I_stim) = {handle_stim_curr[-1]} " + str(sim.probe_metadata(sample_gid, "tag_I_ou"))) # for excitatory neurons with synapses to be probed: get synapse data - if sample_syn >= 0: # synapse probes exist only if this condition is true - handle_Ca_specific.append(sim.sample(sample_gid, f'tag_Ca_{sample_syn}', reg_sched)) # calcium amount - handle_h_specific.append(sim.sample(sample_gid, f'tag_h_{sample_syn}', reg_sched)) # early-phase weight - handle_z_specific.append(sim.sample(sample_gid, f'tag_z_{sample_syn}', reg_sched)) # late-phase weight - handle_sps_specific.append(sim.sample(sample_gid, f'tag_spsV_{sample_syn}', reg_sched)) # signal triggering protein synthesis - handle_p_specific.append(sim.sample(sample_gid, f'tag_pc__{sample_syn}', reg_sched)) # neuronal protein amount - writeAddLog(f" handle(Ca) = {handle_Ca_specific[-1]} " + str(sim.probe_metadata(sample_gid, f"tag_Ca_{sample_syn}")) + - f"\n handle(h) = {handle_h_specific[-1]} " + str(sim.probe_metadata(sample_gid, f"tag_h_{sample_syn}")) + - f"\n handle(z) = {handle_z_specific[-1]} " + str(sim.probe_metadata(sample_gid, f"tag_z_{sample_syn}")) + - f"\n handle(sps) = {handle_sps_specific[-1]} " + str(sim.probe_metadata(sample_gid, f"tag_spsV_{sample_syn}")) + - f"\n handle(p) = {handle_p_specific[-1]} " + str(sim.probe_metadata(sample_gid, f"tag_pc__{sample_syn}"))) + if sample_presyn_gid >= 0: # synapse probes exist only if this condition is true + handle_Ca_specific.append(sim.sample(sample_gid, f'tag_Ca_{sample_presyn_gid}', reg_sched)) # calcium amount + handle_h_specific.append(sim.sample(sample_gid, f'tag_h_{sample_presyn_gid}', reg_sched)) # early-phase weight + handle_z_specific.append(sim.sample(sample_gid, f'tag_z_{sample_presyn_gid}', reg_sched)) # late-phase weight + handle_sps_specific.append(sim.sample(sample_gid, f'tag_spsV_{sample_presyn_gid}', reg_sched)) # signal triggering protein synthesis + handle_p_specific.append(sim.sample(sample_gid, f'tag_pc__{sample_presyn_gid}', reg_sched)) # neuronal protein amount + writeAddLog(f" handle(Ca) = {handle_Ca_specific[-1]} " + str(sim.probe_metadata(sample_gid, f"tag_Ca_{sample_presyn_gid}")) + + f"\n handle(h) = {handle_h_specific[-1]} " + str(sim.probe_metadata(sample_gid, f"tag_h_{sample_presyn_gid}")) + + f"\n handle(z) = {handle_z_specific[-1]} " + str(sim.probe_metadata(sample_gid, f"tag_z_{sample_presyn_gid}")) + + f"\n handle(sps) = {handle_sps_specific[-1]} " + str(sim.probe_metadata(sample_gid, f"tag_spsV_{sample_presyn_gid}")) + + f"\n handle(p) = {handle_p_specific[-1]} " + str(sim.probe_metadata(sample_gid, f"tag_pc__{sample_presyn_gid}"))) writeAddLog("Number of set handles:" + f"\n len(handle_h_syn_latest) = {len(handle_h_syn_latest)}" + @@ -939,7 +945,7 @@ def runSimPhase(phase, recipe, config, rseed, t_final_red, adap_dt, platform, pl for num_sample_syn in range(len(synapse_mapping[i])): j = synapse_mapping[i][num_sample_syn] if recipe.N_exc <= 4: # only for small networks - writeAddLog(f"Getting latest state of synapse {j}->{i} (neuron {i}, incoming synapse {num_sample_syn})...") + writeAddLog(f"Getting latest state of synapse {j}->{i}...") if recipe.conn_matrix[i][j]: h_samples = sim.samples(handle_h_syn_latest[n]) z_samples = sim.samples(handle_z_syn_latest[n]) @@ -958,7 +964,7 @@ def runSimPhase(phase, recipe, config, rseed, t_final_red, adap_dt, platform, pl n += 1 else: - raise Exception(f"Entry with i = {i}, j = {j} not found in connectivity matrix.") + raise Exception(f"Entry with i={i}, j={j} not found in connectivity matrix.") np.savetxt(os.path.join(state_path, "h_synapses.txt"), h_synapses.T, fmt="%.4f") np.savetxt(os.path.join(state_path, "z_synapses.txt"), z_synapses.T, fmt="%.4f") @@ -974,14 +980,15 @@ def runSimPhase(phase, recipe, config, rseed, t_final_red, adap_dt, platform, pl data_z, data_sps, data_p = [], [], [] dimin = 0 - # loop over elements in 'sample_gid_list' (and thereby, 'sample_syn_list') to get traces + # loop over elements in 'sample_gid_list' (and thereby, 'sample_pre_list') to get traces for i in range(len(sample_gid_list)): # retrieve the current neuron index sample_gid = sample_gid_list[i] - # retrieve the synapse index by the corresponding neuron identifier - sample_syn = getSynapseId(sample_syn_list, i) - writeLog(f"Getting traces of sample #{i} (neuron {sample_gid}, synapse {sample_syn})...") + # retrieve the presynaptic neuron identifier by the corresponding index + sample_presyn_gid = getPresynapticId(sample_pre_list, i) + writeLog(f"Getting traces of sample #{i} (neuron {sample_gid}, " + + f"synapse {sample_presyn_gid}->{sample_gid})...") # get neuronal data data, _ = sim.samples(handle_mem[i])[loc] @@ -1004,7 +1011,7 @@ def runSimPhase(phase, recipe, config, rseed, t_final_red, adap_dt, platform, pl writeAddLog(f" Read from handle {handle_tot_curr[i]} (I_tot, {bool(sim.samples(handle_tot_curr[i]))})") # get synaptic data - if sample_syn >= 0: # synapse handles exist only if this condition is true + if sample_presyn_gid >= 0: # synapse handles exist only if this condition is true if len(sim.samples(handle_Ca_specific[i-dimin])) > 0: data, _ = sim.samples(handle_Ca_specific[i-dimin])[loc] data_Ca.append(data[:, 1]) @@ -1077,16 +1084,16 @@ def arborNetworkConsolidation(config, add_info = None, platform = "", plotting = std_dt = config['simulation']['dt']*U.ms # duration of one timestep for rich computation in ms ff_dt = config['simulation']['dt_ff']*U.ms # duration of one timestep for fast-forward computation in ms sample_gid_list = config['simulation']['sample_gid_list'] # list of the neurons that are to be probed (given by number/gid) - sample_syn_list = config['simulation']['sample_syn_list'] # list of the synapses that are to be probed (one synapse per neuron only, given by its internal number respective to a neuron in sample_gid); -1: none + sample_pre_list = config['simulation']['sample_pre_list'] # list of the presynaptic neurons for probing synapses (one value for each value in `sample_gid`; -1: no synapse probing) learn_prot = completeProt(config['simulation']['learn_protocol']) # protocol for learning stimulation as a dictionary with the keys "time" (starting time), "scheme" (scheme of pulses), and "freq" (stimulation frequency) recall_prot = completeProt(config['simulation']['recall_protocol']) # protocol for recall stimulation as a dictionary with the keys "time" (starting time), "scheme" (scheme of pulses), and "freq" (stimulation frequency) bg_prot = completeProt(config['simulation']['bg_protocol']) # protocol for background input as a dictionary with the keys "time" (starting time), "scheme" (scheme of pulses), "I_0" (mean), and "sigma_WN" (standard deviation) rich_comp_window = 5000*U.ms # time window of rich computation to consider before and after learning and recall stimulation - if len(sample_gid_list) != len(sample_syn_list) and len(sample_syn_list) > 1: + if len(sample_gid_list) != len(sample_pre_list) and len(sample_pre_list) > 1: raise ValueError("List of sample synapses has to either have the same length as the list of sample neurons, or to contain maximally one element.") - elif len(sample_syn_list) == 0: - sample_syn_list.append(-1) + elif len(sample_pre_list) == 0: + sample_pre_list.append(-1) ##################################### # initialize structures to store the data @@ -1586,23 +1593,24 @@ def initBackgroundPhase(phase, store_latest_state): data_header = "Time, " data_stacked = times # start with times, then add the data columns in the right order - # loop over elements in 'sample_gid_list' (and thereby, 'sample_syn_list') to assemble the retrieved data + # loop over elements in 'sample_gid_list' (and thereby, 'sample_pre_list') to assemble the retrieved data for i in range(len(sample_gid_list)): # retrieve the current neuron index sample_gid = sample_gid_list[i] - # retrieve the synapse index by the corresponding neuron identifier - sample_syn = getSynapseId(sample_syn_list, i) + # retrieve the presynaptic neuron identifier by the corresponding index + sample_presyn_gid = getPresynapticId(sample_pre_list, i) # retrieve the synapse index by finding out how often the current neuron index has occurred already - sample_syn_alt = sample_gid_list[:i].count(sample_gid) + #sample_syn_alt = sample_gid_list[:i].count(sample_gid) - writeAddLog(f"Assembling data for sample #{i} " + - f"(sample_gid = {sample_gid}, sample_syn = {sample_syn}, sample_syn_alt = {sample_syn_alt})...") + writeAddLog(f"Assembling data for sample #{i}" + + f" (gid = {sample_gid}" + + f", presyn_gid = {sample_presyn_gid})...") # prepare strings for header output neur = str(sample_gid) - syn = str(sample_syn) + pre_neur = str(sample_presyn_gid) datacol_mem = np.hstack((data_mem1[i], data_mem2[i], data_mem3[i])) datacol_curr = np.hstack((data_curr1[i], data_curr2[i], data_curr3[i])) @@ -1619,8 +1627,8 @@ def initBackgroundPhase(phase, store_latest_state): data_stacked = np.column_stack([data_stacked, datacol_mem, datacol_curr]) - data_header += f"h({neur},,{syn}), z({neur},,{syn}), Ca({neur},,{syn}), " + \ - f"spsV({neur},,{syn}), p^C({neur},,{syn}), " + data_header += f"h({pre_neur},{neur}), z({pre_neur},{neur}), Ca({pre_neur},{neur}), " + \ + f"spsV({pre_neur},{neur}), p^C({pre_neur},{neur}), " data_stacked = np.column_stack([data_stacked, datacol_h, datacol_z, datacol_Ca, datacol_spsV, datacol_p]) @@ -1638,14 +1646,14 @@ def initBackgroundPhase(phase, store_latest_state): # retrieve the current neuron index sample_gid = sample_gid_list[i] - # retrieve the synapse index by the corresponding neuron identifier - sample_syn = getSynapseId(sample_syn_list, i) + # retrieve the presynaptic neuron identifier by the corresponding index + sample_presyn_gid = getPresynapticId(sample_pre_list, i) # retrieve the synapse index by finding out how often the current neuron index has occurred already #sample_syn_alt = sample_gid_list[:i].count(sample_gid) # call the plot function for neuron and synapse traces, depending on whether reduced or all data shall be plotted - plotResults(config, data_stacked, getTimestamp(), i, mem_dyn_data = True, neuron=sample_gid, synapse=sample_syn, store_path=getDataPath(s_desc), figure_fmt = 'svg') + plotResults(config, data_stacked, getTimestamp(), i, mem_dyn_data = True, neuron=sample_gid, pre_neuron=sample_presyn_gid, store_path=getDataPath(s_desc), figure_fmt = 'svg') # call the plot function for spike raster plotRaster(config, spike_times.T, getTimestamp(), store_path=getDataPath(s_desc), figure_fmt = 'png') @@ -1683,8 +1691,8 @@ def initBackgroundPhase(phase, store_latest_state): parser.add_argument('-N_CA', type=int, help="number of neurons in the cell assembly") parser.add_argument('-learn', type=str, help="protocol for learning stimulation") parser.add_argument('-recall', type=str, help="protocol for recall stimulation") - parser.add_argument('-sample_gid', type=int, nargs='+', help="numbers of the neurons that shall be probed") - parser.add_argument('-sample_syn', type=int, nargs='+', help="relative numbers of the synapses that shall be probed") + parser.add_argument('-sample_gid', type=int, nargs='+', help="identifiers of the neurons that shall be probed") + parser.add_argument('-sample_presyn_gid', type=int, nargs='+', help="identifiers of the presynaptic neurons for synapse probes") parser.add_argument('-sample_curr', type=int, help="current that shall be probed") parser.add_argument('-w_ei', type=float, help="excitatory to inhibitory coupling strength in units of h_0") parser.add_argument('-w_ie', type=float, help="inhibitory to excitatory coupling strength in units of h_0") @@ -1703,7 +1711,7 @@ def initBackgroundPhase(phase, store_latest_state): if (args.learn is not None): config['simulation']['learn_protocol'] = json.loads(args.learn) # convert "learn" argument string into dictionary if (args.recall is not None): config['simulation']['recall_protocol'] = json.loads(args.recall) # convert "recall" argument string into dictionary if (args.sample_gid is not None): config['simulation']['sample_gid_list'] = args.sample_gid - if (args.sample_syn is not None): config['simulation']['sample_syn_list'] = args.sample_syn + if (args.sample_presyn_gid is not None): config['simulation']['sample_pre_list'] = args.sample_presyn_gid if (args.sample_curr is not None): config['simulation']['sample_curr'] = args.sample_curr if (args.w_ei is not None): config['populations']['w_ei'] = args.w_ei if (args.w_ie is not None): config['populations']['w_ie'] = args.w_ie diff --git a/build_catalogue_gpu b/build_catalogue_gpu index 2df17c5..26bac0d 100644 --- a/build_catalogue_gpu +++ b/build_catalogue_gpu @@ -5,6 +5,4 @@ source set_arbor_env_gpu # compile the mechanisms and build the catalogue #rm -R -f ./tmp -#CC=gcc-10 CXX=g++-10 arbor-build-catalogue custom mechanisms --debug "./tmp" -#CC=gcc-10 CXX=g++-10 arbor-build-catalogue custom mechanisms arbor-build-catalogue custom mechanisms diff --git a/build_catalogue_gpu_testing b/build_catalogue_gpu-use_gpu_rng similarity index 52% rename from build_catalogue_gpu_testing rename to build_catalogue_gpu-use_gpu_rng index 8d48804..38ea3c7 100644 --- a/build_catalogue_gpu_testing +++ b/build_catalogue_gpu-use_gpu_rng @@ -1,10 +1,8 @@ #!/bin/bash # set environment variables required to run custom Arbor installation -source set_arbor_env_gpu_testing +source set_arbor_env_gpu-use_gpu_rng # compile the mechanisms and build the catalogue #rm -R -f ./tmp -#CC=gcc-10 CXX=g++-10 arbor-build-catalogue custom mechanisms --debug "./tmp" -#CC=gcc-10 CXX=g++-10 arbor-build-catalogue custom mechanisms arbor-build-catalogue custom mechanisms diff --git a/config_defaultnet-like.json b/config_defaultnet-like.json index bb487f0..3be5a9d 100755 --- a/config_defaultnet-like.json +++ b/config_defaultnet-like.json @@ -22,7 +22,7 @@ "sigma_WN" : 0.05 }, "sample_gid_list": [6, 68, 68, 1615], - "sample_syn_list": [-1, 0, 153, -1], + "sample_pre_list": [-1, 6, 1528, -1], "sample_curr": 0 }, "populations" : diff --git a/config_defaultnet.json b/config_defaultnet.json index b67d4a4..2e23582 100755 --- a/config_defaultnet.json +++ b/config_defaultnet.json @@ -22,7 +22,7 @@ "sigma_WN" : 0.05 }, "sample_gid_list": [6, 68, 68, 1615], - "sample_syn_list": [-1, 0, 153, -1], + "sample_pre_list": [-1, 6, 1528, -1], "sample_curr": 0 }, "populations" : diff --git a/config_defaultnet_det_max_activity.json b/config_defaultnet_det_max_activity.json index 44d6c88..16b7203 100755 --- a/config_defaultnet_det_max_activity.json +++ b/config_defaultnet_det_max_activity.json @@ -17,7 +17,7 @@ "recall_protocol": { }, "bg_protocol": { }, "sample_gid_list": [0], - "sample_syn_list": [0], + "sample_pre_list": [0], "sample_curr": 0 }, "populations" : diff --git a/config_defaultnet_no_bg.json b/config_defaultnet_no_bg.json index eaea40d..d670356 100755 --- a/config_defaultnet_no_bg.json +++ b/config_defaultnet_no_bg.json @@ -17,7 +17,7 @@ "recall_protocol": { }, "bg_protocol": { }, "sample_gid_list": [ ], - "sample_syn_list": [ ], + "sample_pre_list": [ ], "sample_curr": 0 }, "populations" : diff --git a/config_smallnet2_basic_early.json b/config_smallnet2_basic_early.json index c4b35d1..0041a5f 100755 --- a/config_smallnet2_basic_early.json +++ b/config_smallnet2_basic_early.json @@ -9,7 +9,7 @@ "output_period" : 1, "reduced_data" : 0, "sample_gid_list" : [1], - "sample_syn_list" : [0], + "sample_pre_list" : [0], "learn_protocol": { "time_start": 0, "scheme": "EXPLICIT", diff --git a/config_smallnet2_basic_early_det.json b/config_smallnet2_basic_early_det.json index 2180b44..c26e420 100755 --- a/config_smallnet2_basic_early_det.json +++ b/config_smallnet2_basic_early_det.json @@ -9,7 +9,7 @@ "output_period" : 1, "reduced_data" : 0, "sample_gid_list" : [1], - "sample_syn_list" : [0], + "sample_pre_list" : [0], "learn_protocol": { "time_start": 0, "scheme": "EXPLICIT", diff --git a/config_smallnet2_basic_late.json b/config_smallnet2_basic_late.json index 0293a93..8b8bcb5 100755 --- a/config_smallnet2_basic_late.json +++ b/config_smallnet2_basic_late.json @@ -9,7 +9,7 @@ "output_period" : 100, "reduced_data" : 0, "sample_gid_list" : [1], - "sample_syn_list" : [0], + "sample_pre_list" : [0], "learn_protocol": { "scheme": "RECT", "time_start": 0, diff --git a/config_smallnet2_basic_late_STET.json b/config_smallnet2_basic_late_STET.json index 091c69b..502eaaf 100755 --- a/config_smallnet2_basic_late_STET.json +++ b/config_smallnet2_basic_late_STET.json @@ -9,7 +9,7 @@ "output_period" : 10, "reduced_data" : 0, "sample_gid_list" : [1], - "sample_syn_list" : [0], + "sample_pre_list" : [0], "learn_protocol": { "scheme": "STET", "time_start": 300, diff --git a/config_smallnet2_det.json b/config_smallnet2_det.json index e62baa4..898c7ab 100755 --- a/config_smallnet2_det.json +++ b/config_smallnet2_det.json @@ -17,7 +17,7 @@ "recall_protocol": { }, "bg_protocol": { }, "sample_gid_list": [0, 1, 2, 2, 3, 3], - "sample_syn_list": [0, 0, 1, 0, 0, 1], + "sample_pre_list": [0, 0, 0, 3, 0, 2], "sample_curr": 0 }, "populations" : diff --git a/config_smallnet2_det_small_tau_mem.json b/config_smallnet2_det_small_tau_mem.json index d4b80c3..90135d2 100755 --- a/config_smallnet2_det_small_tau_mem.json +++ b/config_smallnet2_det_small_tau_mem.json @@ -17,7 +17,7 @@ "recall_protocol": { }, "bg_protocol": { }, "sample_gid_list": [0, 1, 2, 2, 3, 3], - "sample_syn_list": [0, 0, 0, 1, 0, 1] + "sample_pre_list": [0, 0, 0, 3, 0, 2] }, "populations" : { diff --git a/config_smallnet3_det.json b/config_smallnet3_det.json index a5518ac..054073f 100755 --- a/config_smallnet3_det.json +++ b/config_smallnet3_det.json @@ -17,7 +17,7 @@ "recall_protocol": { }, "bg_protocol": { }, "sample_gid_list": [ 0, 1, 2, 2, 3, 3, 4], - "sample_syn_list": [-1, 0, 1, 0, 0, 1, -1] + "sample_pre_list": [-1, 0, 0, 3, 0, 2, -1] }, "populations" : { diff --git a/outputUtilities.py b/outputUtilities.py index 759bf63..f1cedb8 100644 --- a/outputUtilities.py +++ b/outputUtilities.py @@ -2,7 +2,7 @@ # Utility functions for displaying and storing data -# Copyright 2021-2023 Jannik Luboeinski +# Copyright 2021-2024 Jannik Luboeinski # License: Apache-2.0 (http://www.apache.org/licenses/LICENSE-2.0) # Contact: mail[at]jlubo.net @@ -159,20 +159,20 @@ def closeLog(): logf.close() ############################################################################### -# getSynapseId -# If provided, returns the synapse identifier for the corresponding neuron identifier in 'sample_gid_list', -# otherwise, returns the generic identifier conveyed by 'sample_gid_list' of type int. -# - sample_syn_list: the list of synapse indices (or value -1 to disable reading) -# - index: pointing at the list element to be considered -# - return: the synapse identifier -def getSynapseId(sample_syn_list, index): - - # get synapse id provided for corresponding neuron in 'sample_gid_list' - if len(sample_syn_list) > 1: - sample_syn = sample_syn_list[index] - - # get generic synapse id that has been provided +# getPresynapticId +# If a list has been specified, returns the identifier of the postsynaptic neuron from 'sample_pre_list' +# corresponding to the given index. Otherwise, returns the generic identifier. +# - sample_pre_list: the list of presynaptic neuron identifiers (or value -1 to disable considering the synapse) +# - index: index pointing at the list element to be considered +# - return: identifier of the presynaptic neuron +def getPresynapticId(sample_pre_list, index): + + # identifier of the postsynaptic neuron from 'sample_pre_list' corresponding to the given index + if len(sample_pre_list) > 1: + sample_syn = sample_pre_list[index] + + # generic identifier else: - sample_syn = sample_syn_list[0] + sample_syn = sample_pre_list[0] return sample_syn diff --git a/plotResults.py b/plotResults.py index f1ef26c..8bf40a1 100644 --- a/plotResults.py +++ b/plotResults.py @@ -1,8 +1,8 @@ #!/bin/python3 -# Plot functions for neuronal and synaptic variables as wells as spike raster plots +# Plot functions for neuronal and synaptic variables as well as spike raster plots -# Copyright 2021-2023 Jannik Luboeinski +# Copyright 2021-2024 Jannik Luboeinski # License: Apache-2.0 (http://www.apache.org/licenses/LICENSE-2.0) # Contact: mail[at]jlubo.net @@ -11,7 +11,7 @@ import json import os from pathlib import Path -from outputUtilities import getSynapseId +from outputUtilities import getPresynapticId ##################################### # plotResults @@ -22,10 +22,10 @@ # dataset: number of the dataset (usually one neuron/synapse pair) that shall be considered # mem_dyn_data [optional]: specifies if membrane potential and current shall be plotted # neuron [optional]: number of the neuron that shall be considered [default: 0] -# synapse [optional]: number of the synapse that shall be considered [default: 0] +# pre_neuron [optional]: number of the presynaptic neuron that shall be considered for the synapse [default: 0] # store_path [optional]: path to store resulting graphics file # figure_fmt [optional]: format of resulting graphics file -def plotResults(config, data_stacked, timestamp, dataset, mem_dyn_data = False, neuron=0, synapse=0, store_path = ".", figure_fmt = 'png'): +def plotResults(config, data_stacked, timestamp, dataset, mem_dyn_data = False, neuron=0, pre_neuron=0, store_path = ".", figure_fmt = 'png'): h_0 = config["synapses"]["syn_exc_calcium_plasticity"]["h_0"] # reference value (initial value) of synaptic weight time = data_stacked[:,0] / 60 / 1000 # time values, in min @@ -100,7 +100,7 @@ def plotResults(config, data_stacked, timestamp, dataset, mem_dyn_data = False, #axes[num_rows-1].legend(loc="center left") # save figure in given format (e.g., 'png' or 'svg') - fig.savefig(os.path.join(store_path, f"{timestamp}_traces_neuron_{neuron}_synapse_{synapse}.{figure_fmt}")) + fig.savefig(os.path.join(store_path, f"{timestamp}_traces_neuron_{neuron}_synapse_{pre_neuron}to{neuron}.{figure_fmt}")) plt.close() ##################################### @@ -164,12 +164,12 @@ def plotRaster(config, spikes_stacked, timestamp, store_path = ".", figure_fmt = #plotResults(config, data_stacked, timestamp, 0, mem_dyn_data = True, figure_fmt = 'svg') sample_gid_list = config['simulation']['sample_gid_list'] # list of the neurons that are to be probed (given by number/gid) - sample_syn_list = config['simulation']['sample_syn_list'] # list of the synapses that are to be probed (one synapse per neuron only, given by its internal number respective to a neuron in sample_gid); -1: none + sample_pre_list = config['simulation']['sample_pre_list'] # list of the presynaptic neurons for probing synapses (one value for each value in `sample_gid`; -1: no synapse probing) for i in range(len(sample_gid_list)): sample_gid = sample_gid_list[i] - sample_syn = getSynapseId(sample_syn_list, i) - plotResults(config, data_stacked, timestamp, i, mem_dyn_data = True, neuron=sample_gid, synapse=sample_syn, figure_fmt = 'svg') + sample_presyn_gid = getPresynapticId(sample_pre_list, i) + plotResults(config, data_stacked, timestamp, i, mem_dyn_data = True, neuron=sample_gid, pre_neuron=sample_presyn_gid, figure_fmt = 'svg') if spikes_stacked.size != 0: plotRaster(config, spikes_stacked, timestamp, figure_fmt = 'png') diff --git a/run_benchmark_desktop b/run_benchmark_desktop index 54771dd..07d0310 100644 --- a/run_benchmark_desktop +++ b/run_benchmark_desktop @@ -1,7 +1,7 @@ #!/bin/sh # build necessary catalogue (also sets the environment to run custom Arbor installation) -source build_catalogue_simd +source build_catalogue benchmark () { # name of the benchmark diff --git a/run_defaultnet_bg_only_desktop b/run_defaultnet_bg_only_desktop index 7d93377..2531145 100644 --- a/run_defaultnet_bg_only_desktop +++ b/run_defaultnet_bg_only_desktop @@ -4,5 +4,5 @@ source build_catalogue # run the simulation -python3 ./arborNetworkConsolidation.py -s_desc="bg no stim" -runtime=1 -config_file="config_defaultnet.json" -learn='{}' -recall='{}' -sample_gid=68 -sample_syn=153 -output_period=1 -sample_curr=0 +python3 ./arborNetworkConsolidation.py -s_desc="bg no stim" -runtime=1 -config_file="config_defaultnet.json" -learn='{}' -recall='{}' -sample_gid=68 -sample_presyn_gid=6 -output_period=1 -sample_curr=0 diff --git a/run_tests b/run_tests index f394fc7..e047a2c 100644 --- a/run_tests +++ b/run_tests @@ -4,17 +4,17 @@ #source build_catalogue #source build_catalogue_simd #source build_catalogue_gpu -source build_catalogue_gpu_testing +source build_catalogue_gpu-use_gpu_rng -# run specific tests with console output -#python3 -m pytest ./test_arborNetworkConsolidation.py::test_smallnet2_basic_early -s -#python3 -m pytest ./test_arborNetworkConsolidation.py::test_smallnet2_basic_late -s -#python3 -m pytest ./test_arborNetworkConsolidation.py::test_generated_connectivity -s -#python3 -m pytest ./test_arborNetworkConsolidation.py::test_smallnetX_onespike -s -#python3 -m pytest ./test_arborNetworkConsolidation.py::test_defn_max_activity -s -#python3.10 -m pytest ./test_arborNetworkConsolidation.py::test_smallnet3_schedules -s -#python3 -m pytest ./test_arborNetworkConsolidation.py::test_defn_onespike -s -#python3.10 -m pytest ./test_arborNetworkConsolidation.py::test_smallnetX_onespike -s +# run specific tests (add `-s` for console output) +#python3 -m pytest ./test_arborNetworkConsolidation.py::test_smallnet2_basic_early +#python3 -m pytest ./test_arborNetworkConsolidation.py::test_smallnet2_basic_late +#python3 -m pytest ./test_arborNetworkConsolidation.py::test_generated_connectivity +#python3 -m pytest ./test_arborNetworkConsolidation.py::test_smallnetX_onespike +#python3 -m pytest ./test_arborNetworkConsolidation.py::test_defn_max_activity +#python3 -m pytest ./test_arborNetworkConsolidation.py::test_smallnet3_schedules +#python3 -m pytest ./test_arborNetworkConsolidation.py::test_defn_onespike +#python3 -m pytest ./test_arborNetworkConsolidation.py::test_smallnetX_onespike # run all tests with coverage (send output to terminal and file, using tee with unbuffer) #python3.10 -m coverage run -m pytest ./test_arborNetworkConsolidation.py --report-log=test_result.txt diff --git a/set_arbor_env b/set_arbor_env index 3671f88..dd484f5 100755 --- a/set_arbor_env +++ b/set_arbor_env @@ -4,24 +4,24 @@ # check if PATH exists and set environment variable accordingly if [[ ! -v PATH ]]; then - export PATH=$(readlink -f ~/arbor_v0.10.0-nosimd/bin) + export PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-nosimd/bin) else - export PATH=$(readlink -f ~/arbor_v0.10.0-nosimd/bin):$PATH + export PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-nosimd/bin):$PATH fi # check if LD_LIBRARY_PATH exists and set environment variable accordingly if [[ ! -v LD_LIBRARY_PATH ]]; then - export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.0-nosimd/lib) + export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-nosimd/lib) else - export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.0-nosimd/lib):$LD_LIBRARY_PATH + export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-nosimd/lib):$LD_LIBRARY_PATH fi # check if PYTHONPATH exists and set environment variable accordingly if [[ ! -v PYTHONPATH ]]; then - export PYTHONPATH=$(readlink -f ~/arbor_v0.10.0-nosimd/lib/python3.10/site-packages) # or: .../python3.8/..., etc. + export PYTHONPATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-nosimd/lib/python3.10/site-packages) # or: .../python3.8/..., etc. else - export PYTHONPATH=$(readlink -f ~/arbor_v0.10.0-nosimd/lib/python3.10/site-packages):$PYTHONPATH # or: .../python3.8/..., etc. + export PYTHONPATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-nosimd/lib/python3.10/site-packages):$PYTHONPATH # or: .../python3.8/..., etc. fi # set CMAKE_PREFIX_PATH environment variable -export CMAKE_PREFIX_PATH=$(readlink -f ~/arbor_v0.10.0-nosimd) +export CMAKE_PREFIX_PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-nosimd) diff --git a/set_arbor_env_gpu b/set_arbor_env_gpu index 6a2597e..4da03fc 100755 --- a/set_arbor_env_gpu +++ b/set_arbor_env_gpu @@ -4,24 +4,24 @@ # check if PATH exists and set environment variable accordingly if [[ ! -v PATH ]]; then - export PATH=$(readlink -f ~/arbor_v0.10.0-gpu-use_gpu_rng/bin) + export PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu/bin) else - export PATH=$(readlink -f ~/arbor_v0.10.0-gpu-use_gpu_rng/bin):$PATH + export PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu/bin):$PATH fi # check if LD_LIBRARY_PATH exists and set environment variable accordingly if [[ ! -v LD_LIBRARY_PATH ]]; then - export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.0-gpu-use_gpu_rng/lib) + export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu/lib) else - export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.0-gpu-use_gpu_rng/lib):$LD_LIBRARY_PATH + export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu/lib):$LD_LIBRARY_PATH fi # check if PYTHONPATH exists and set environment variable accordingly if [[ ! -v PYTHONPATH ]]; then - export PYTHONPATH=$(readlink -f ~/arbor_v0.10.0-gpu-use_gpu_rng/lib/python3.10/site-packages) # or: .../python3.8/..., etc. + export PYTHONPATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu/lib/python3.10/site-packages) # or: .../python3.8/..., etc. else - export PYTHONPATH=$(readlink -f ~/arbor_v0.10.0-gpu-use_gpu_rng/lib/python3.10/site-packages):$PYTHONPATH # or: .../python3.8/..., etc. + export PYTHONPATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu/lib/python3.10/site-packages):$PYTHONPATH # or: .../python3.8/..., etc. fi # set CMAKE_PREFIX_PATH environment variable -export CMAKE_PREFIX_PATH=$(readlink -f ~/arbor_v0.10.0-gpu-use_gpu_rng) +export CMAKE_PREFIX_PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu) diff --git a/set_arbor_env_gpu-use_gpu_rng b/set_arbor_env_gpu-use_gpu_rng new file mode 100755 index 0000000..f96968f --- /dev/null +++ b/set_arbor_env_gpu-use_gpu_rng @@ -0,0 +1,27 @@ +#!/bin/bash + +# Script that sets environment variables required to run custom Arbor installation + +# check if PATH exists and set environment variable accordingly +if [[ ! -v PATH ]]; then + export PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu-use_gpu_rng/bin) +else + export PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu-use_gpu_rng/bin):$PATH +fi + +# check if LD_LIBRARY_PATH exists and set environment variable accordingly +if [[ ! -v LD_LIBRARY_PATH ]]; then + export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu-use_gpu_rng/lib) +else + export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu-use_gpu_rng/lib):$LD_LIBRARY_PATH +fi + +# check if PYTHONPATH exists and set environment variable accordingly +if [[ ! -v PYTHONPATH ]]; then + export PYTHONPATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu-use_gpu_rng/lib/python3.10/site-packages) # or: .../python3.8/..., etc. +else + export PYTHONPATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu-use_gpu_rng/lib/python3.10/site-packages):$PYTHONPATH # or: .../python3.8/..., etc. +fi + +# set CMAKE_PREFIX_PATH environment variable +export CMAKE_PREFIX_PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-gpu-use_gpu_rng) diff --git a/set_arbor_env_gpu_testing b/set_arbor_env_gpu_testing deleted file mode 100755 index ec25c66..0000000 --- a/set_arbor_env_gpu_testing +++ /dev/null @@ -1,27 +0,0 @@ -#!/bin/bash - -# Script that sets environment variables required to run custom Arbor installation - -# check if PATH exists and set environment variable accordingly -if [[ ! -v PATH ]]; then - export PATH=$(readlink -f ~/arbor_v0.10.1-mem-allocation-fix-15a282d-gpu_use_gpu_rng/bin) -else - export PATH=$(readlink -f ~/arbor_v0.10.1-mem-allocation-fix-15a282d-gpu_use_gpu_rng/bin):$PATH -fi - -# check if LD_LIBRARY_PATH exists and set environment variable accordingly -if [[ ! -v LD_LIBRARY_PATH ]]; then - export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.1-mem-allocation-fix-15a282d-gpu_use_gpu_rng/lib) -else - export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.1-mem-allocation-fix-15a282d-gpu_use_gpu_rng/lib):$LD_LIBRARY_PATH -fi - -# check if PYTHONPATH exists and set environment variable accordingly -if [[ ! -v PYTHONPATH ]]; then - export PYTHONPATH=$(readlink -f ~/arbor_v0.10.1-mem-allocation-fix-15a282d-gpu_use_gpu_rng/lib/python3.10/site-packages) # or: .../python3.8/..., etc. -else - export PYTHONPATH=$(readlink -f ~/arbor_v0.10.1-mem-allocation-fix-15a282d-gpu_use_gpu_rng/lib/python3.10/site-packages):$PYTHONPATH # or: .../python3.8/..., etc. -fi - -# set CMAKE_PREFIX_PATH environment variable -export CMAKE_PREFIX_PATH=$(readlink -f ~/arbor_v0.10.1-mem-allocation-fix-15a282d-gpu_use_gpu_rng) diff --git a/set_arbor_env_simd b/set_arbor_env_simd index b400e39..430dd6d 100755 --- a/set_arbor_env_simd +++ b/set_arbor_env_simd @@ -4,24 +4,24 @@ # check if PATH exists and set environment variable accordingly if [[ ! -v PATH ]]; then - export PATH=$(readlink -f ~/arbor_v0.10.0-simd/bin) + export PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-simd/bin) else - export PATH=$(readlink -f ~/arbor_v0.10.0-simd/bin):$PATH + export PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-simd/bin):$PATH fi # check if LD_LIBRARY_PATH exists and set environment variable accordingly if [[ ! -v LD_LIBRARY_PATH ]]; then - export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.0-simd/lib) + export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-simd/lib) else - export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.0-simd/lib):$LD_LIBRARY_PATH + export LD_LIBRARY_PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-simd/lib):$LD_LIBRARY_PATH fi # check if PYTHONPATH exists and set environment variable accordingly if [[ ! -v PYTHONPATH ]]; then - export PYTHONPATH=$(readlink -f ~/arbor_v0.10.0-simd/lib/python3.10/site-packages) # or: .../python3.8/..., etc. + export PYTHONPATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-simd/lib/python3.10/site-packages) # or: .../python3.8/..., etc. else - export PYTHONPATH=$(readlink -f ~/arbor_v0.10.0-simd/lib/python3.10/site-packages):$PYTHONPATH # or: .../python3.8/..., etc. + export PYTHONPATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-simd/lib/python3.10/site-packages):$PYTHONPATH # or: .../python3.8/..., etc. fi # set CMAKE_PREFIX_PATH environment variable -export CMAKE_PREFIX_PATH=$(readlink -f ~/arbor_v0.10.0-simd) +export CMAKE_PREFIX_PATH=$(readlink -f ~/arbor_v0.10.1-dev-b8b768d-simd) diff --git a/test_arborNetworkConsolidation.py b/test_arborNetworkConsolidation.py index 9e6a3a2..0b32672 100644 --- a/test_arborNetworkConsolidation.py +++ b/test_arborNetworkConsolidation.py @@ -2,7 +2,7 @@ # Tests for the arborNetworkConsolidation module -# Copyright 2022-2023 Jannik Luboeinski +# Copyright 2022-2024 Jannik Luboeinski # License: Apache-2.0 (http://www.apache.org/licenses/LICENSE-2.0) # Contact: mail[at]jlubo.net @@ -28,7 +28,7 @@ def test_generated_connectivity(config_file): config = json.load(open(config_file, "r")) # load JSON file config["simulation"]["runtime"] = 0.1 # very short runtime (we are only interested in the initial connectivity) config["simulation"]["sample_gid_list"] = [ ] # no neuron probes - config["simulation"]["sample_syn_list"] = [ ] # no synapse probes + config["simulation"]["sample_pre_list"] = [ ] # no synapse probes func_name = inspect.getframeinfo(inspect.currentframe()).function # run simulation @@ -50,7 +50,7 @@ def test_defn_connectivity(config_file): config = json.load(open(config_file, "r")) # load JSON file config["simulation"]["runtime"] = 0.1 # very short runtime (we are only interested in the initial connectivity) config["simulation"]["sample_gid_list"] = [ ] # no neuron probes - config["simulation"]["sample_syn_list"] = [ ] # no synapse probes + config["simulation"]["sample_pre_list"] = [ ] # no synapse probes func_name = inspect.getframeinfo(inspect.currentframe()).function # run simulation @@ -126,7 +126,7 @@ def printValues(data_stacked, tb_PSP_begin, tb_Ca_begin, dt): n_stim = len(learn_prot["explicit_input"]["stim_times"]) # the number of stimulus pulses config["simulation"]["sample_curr"] = 1 # retrieve Ornstein-Uhlenbeck stimulation current sample_gid = config["simulation"]["sample_gid_list"][0] # gid of the neuron to be probed (e.g., 68) - sample_syn = config["simulation"]["sample_syn_list"][0] # internal number (relative to neuron determined by sample_gid) of the synapse to be probed (e.g., 154 for 6->68) + sample_presyn_gid = config["simulation"]["sample_pre_list"][0] # gid of the presynaptic neuron for probing synapses (e.g., 6 for synapse 6->68) func_name = inspect.getframeinfo(inspect.currentframe()).function # run simulation @@ -146,7 +146,7 @@ def printValues(data_stacked, tb_PSP_begin, tb_Ca_begin, dt): assert data_stacked[tb_before_last_spike][1] == pytest.approx(config["neuron"]["V_rev"]) # before PSP: membrane potential equal V_rev if (test_type == "onespike_ee"): assert sample_gid == 68 - assert sample_syn == 153 + assert sample_presyn_gid == 6 assert data_stacked[tb_before_last_spike+2][1] > config["neuron"]["V_rev"] + epsilon # upon PSP onset: membrane potential greater than V_rev assert data_stacked[tb_before_last_Ca_increase+2][5] \ == pytest.approx(config["synapses"]["syn_exc_calcium_plasticity"]["Ca_pre"], 1e-2) # calcium amount approx. equal to Ca_pre @@ -174,7 +174,7 @@ def test_defn_background_noise(config_file): config['simulation']['learn_protocol'] = anc.completeProt({ }) # no learning protocol config['simulation']['recall_protocol'] = anc.completeProt({ }) # no recall protocol config['simulation']['sample_gid_list'] = [0] # probe neuron 0 - config['simulation']['sample_syn_list'] = [0] # probe first synapse incoming to neuron 0 + config['simulation']['sample_pre_list'] = [27] # probe synapse incoming from neuron 27 config['simulation']['sample_curr'] = 2 # retrieve Ornstein-Uhlenbeck background noise current func_name = inspect.getframeinfo(inspect.currentframe()).function @@ -203,10 +203,10 @@ def test_defn_max_activity(config_file, num_spikes_max_activity): s_desc = config['simulation']['short_description'] # short description of the simulation if config["populations"]["p_c"] > epsilon: # in the case that there is connectivity config['simulation']['sample_gid_list'] = [862] # neuron to consider - config['simulation']['sample_syn_list'] = [155] # synapse to consider (0->862) + config['simulation']['sample_pre_list'] = [0] # synapse incoming from neuron 0 else: # in the case that there is no connectivity config['simulation']['sample_gid_list'] = [862] # neuron to consider - config['simulation']['sample_syn_list'] = [-1] # no synapse to consider + config['simulation']['sample_pre_list'] = [-1] # no synapse to consider config['simulation']['sample_curr'] = 1 # retrieve Ornstein-Uhlenbeck stimulation current func_name = inspect.getframeinfo(inspect.currentframe()).function @@ -242,7 +242,7 @@ def test_def_const_conv_relax(config_file): s_desc = config['simulation']['short_description'] # short description of the simulation I_0 = 0.15 # constant input current (in nA) config['simulation']['sample_gid_list'] = [0] # probe neuron 0 - config['simulation']['sample_syn_list'] = [-1] # no synapse to consider + config['simulation']['sample_pre_list'] = [-1] # no synapse to consider config['simulation']['sample_curr'] = 0 # retrieve total current config['simulation']['bg_protocol']['I_0'] = I_0 func_name = inspect.getframeinfo(inspect.currentframe()).function @@ -278,7 +278,7 @@ def test_smallnetX_connectivity(network, config_file): # configuration config = json.load(open(config_file, "r")) # load JSON file config['simulation']['sample_gid_list'] = [ ] # no neuron probes - config['simulation']['sample_syn_list'] = [ ] # no synapse probes + config['simulation']['sample_pre_list'] = [ ] # no synapse probes config['simulation']['sample_curr'] = 0 # retrieve total current config['simulation']['runtime'] = 0.1 # very short runtime (we are only interested in the initial connectivity) config['simulation']['learn_protocol'] = anc.completeProt({ }) # no learning protocol @@ -350,7 +350,7 @@ def test_smallnetX_max_activity(network, config_file): s_desc = config['simulation']['short_description'] # short description of the simulation learn_prot = anc.completeProt(config["simulation"]["learn_protocol"]) config['simulation']['sample_gid_list'] = [0] # neuron probes - config['simulation']['sample_syn_list'] = [-1] # no synapse to consider + config['simulation']['sample_pre_list'] = [-1] # no synapse to consider config['simulation']['sample_curr'] = 0 # retrieve total current config['populations']['conn_file'] = f"connections_{network}.txt" # connectivity matrix if network == "smallnet3": @@ -391,10 +391,10 @@ def test_smallnetX_onespike(network, config_file, platform): learn_prot = anc.completeProt(config["simulation"]["learn_protocol"]) if network == "smallnet2": config['simulation']['sample_gid_list'] = [2,3] # neurons to consider (3 is stimulated and projects to 2) - config['simulation']['sample_syn_list'] = [0,0] # synapses to consider + config['simulation']['sample_pre_list'] = [3,2] # presynaptic neurons to consider for synapse probing elif network == "smallnet3": config['simulation']['sample_gid_list'] = [2,3,4] # neurons to consider (3 is stimulated and projects to 2 and 4) - config['simulation']['sample_syn_list'] = [0,0,0] # synapses to consider + config['simulation']['sample_pre_list'] = [3,2,3] # presynaptic neurons to consider for synapse probing config['simulation']['sample_curr'] = 0 # retrieve total current config['populations']['conn_file'] = f"connections_{network}.txt" # connectivity matrix h_0 = config['synapses']['syn_exc_calcium_plasticity']['h_0'] @@ -469,7 +469,7 @@ def test_smallnet3_schedules(runtime, learn, recall, config_file, platform): s_desc += f", {runtime}, {platform}" config['simulation']['short_description'] = s_desc config['simulation']['sample_gid_list'] = [1] # probe neuron 1 - config['simulation']['sample_syn_list'] = [0] # probe first synapse incoming to neuron 1 + config['simulation']['sample_pre_list'] = [0] # probe synapse incoming from neuron 0 config['simulation']['sample_curr'] = 1 # retrieve Ornstein-Uhlenbeck stimulation current func_name = inspect.getframeinfo(inspect.currentframe()).function @@ -508,8 +508,8 @@ def test_smallnet3_schedules(runtime, learn, recall, config_file, platform): assert recipe.z[1][0] > 0.4 # late-phase weight has built up at synapse 0->1 (NOTE: no `h_0` normalization here!) assert recipe.p[1] > epsilon # protein has built up at neuron 1 - # get spikes & test the number of spikes (unless the schedule is "consolidate", where no spikes are computed) - if not schedule == "consolidate": + # get spikes & test the number of spikes (unless the schedule is "consolidate" or "background", where no or few spikes occur) + if not schedule == "consolidate" and not schedule == "background": spike_data = np.loadtxt(getDataPath(s_desc, "spikes.txt")).transpose() spikes_0 = spike_data[0][spike_data[1] == 0] # spike times of neuron 0 (exc.) spikes_1 = spike_data[0][spike_data[1] == 1] # spike times of neuron 1 (exc.) @@ -571,13 +571,10 @@ def test_smallnet2_basic_early(prot, config_file): ############################################################################### # Test a basic protocol for late-phase plasticity in a small pre-defined network ('smallnet2') -#@pytest.mark.parametrize("config_file, platform", [("config_smallnet2_basic_late.json", "CPU"), -# ("test_config_smallnet2_basic_late_noff.json", "CPU")]) @pytest.mark.parametrize("config_file, platform", [("config_smallnet2_basic_late.json", "CPU"), - ("config_smallnet2_basic_late.json", "GPU"), + #("config_smallnet2_basic_late.json", "GPU"), # takes long ("test_config_smallnet2_basic_late_noff.json", "CPU")]) -#@pytest.mark.parametrize("config_file, platform", [("config_smallnet2_basic_late.json", "GPU")]) -#@pytest.mark.parametrize("config_file, platform", [("test_config_smallnet2_basic_late_noff.json", "GPU")]) +#@pytest.mark.parametrize("config_file, platform", [("test_config_smallnet2_basic_late_noff.json", "GPU")]) # takes very long def test_smallnet2_basic_late(config_file, platform): # configuration (with learning protocol) diff --git a/test_config_defaultnet_bg.json b/test_config_defaultnet_bg.json index c6d9dbf..dfe8f58 100755 --- a/test_config_defaultnet_bg.json +++ b/test_config_defaultnet_bg.json @@ -17,7 +17,7 @@ "sigma_WN" : 0.05 }, "sample_gid_list": [ ], - "sample_syn_list": [ ], + "sample_pre_list": [ ], "sample_curr": 0 }, "populations" : diff --git a/test_config_defaultnet_no_bg_noplast.json b/test_config_defaultnet_no_bg_noplast.json index 7d5bb6e..434f622 100755 --- a/test_config_defaultnet_no_bg_noplast.json +++ b/test_config_defaultnet_no_bg_noplast.json @@ -9,7 +9,7 @@ "output_period" : 1, "reduced_data" : 0, "sample_gid_list" : [68], - "sample_syn_list" : [153], + "sample_pre_list" : [6], "learn_protocol": { "time_start": 0, "scheme": "EXPLICIT", diff --git a/test_config_defaultnet_onespike_ee.json b/test_config_defaultnet_onespike_ee.json index 9345bc7..7129e86 100755 --- a/test_config_defaultnet_onespike_ee.json +++ b/test_config_defaultnet_onespike_ee.json @@ -9,7 +9,7 @@ "output_period" : 1, "reduced_data" : 0, "sample_gid_list" : [68], - "sample_syn_list" : [153], + "sample_pre_list" : [6], "learn_protocol": { "time_start": 0, "scheme": "EXPLICIT", diff --git a/test_config_defaultnet_onespike_ee_noplast.json b/test_config_defaultnet_onespike_ee_noplast.json index 4188173..3b15002 100755 --- a/test_config_defaultnet_onespike_ee_noplast.json +++ b/test_config_defaultnet_onespike_ee_noplast.json @@ -9,7 +9,7 @@ "output_period" : 1, "reduced_data" : 0, "sample_gid_list" : [68], - "sample_syn_list" : [153], + "sample_pre_list" : [6], "learn_protocol": { "time_start": 0, "scheme": "EXPLICIT", diff --git a/test_config_defaultnet_onespike_ei.json b/test_config_defaultnet_onespike_ei.json index d572586..825eb70 100755 --- a/test_config_defaultnet_onespike_ei.json +++ b/test_config_defaultnet_onespike_ei.json @@ -9,7 +9,7 @@ "output_period" : 1, "reduced_data" : 0, "sample_gid_list" : [1760], - "sample_syn_list" : [-1], + "sample_pre_list" : [-1], "learn_protocol": { "time_start": 0, "scheme": "EXPLICIT", diff --git a/test_config_defaultnet_onespike_ei_lowered.json b/test_config_defaultnet_onespike_ei_lowered.json index bb3f000..4ebafe6 100755 --- a/test_config_defaultnet_onespike_ei_lowered.json +++ b/test_config_defaultnet_onespike_ei_lowered.json @@ -9,7 +9,7 @@ "output_period" : 1, "reduced_data" : 0, "sample_gid_list" : [1760], - "sample_syn_list" : [-1], + "sample_pre_list" : [-1], "learn_protocol": { "time_start": 0, "scheme": "EXPLICIT", diff --git a/test_config_defaultnet_onespike_ie.json b/test_config_defaultnet_onespike_ie.json index b461759..824c0c3 100755 --- a/test_config_defaultnet_onespike_ie.json +++ b/test_config_defaultnet_onespike_ie.json @@ -9,7 +9,7 @@ "output_period" : 1, "reduced_data" : 0, "sample_gid_list" : [17], - "sample_syn_list" : [-1], + "sample_pre_list" : [-1], "learn_protocol": { "time_start": 0, "scheme": "EXPLICIT", diff --git a/test_config_defaultnet_onespike_ie_lowered.json b/test_config_defaultnet_onespike_ie_lowered.json index 2d74feb..7f3ec4f 100755 --- a/test_config_defaultnet_onespike_ie_lowered.json +++ b/test_config_defaultnet_onespike_ie_lowered.json @@ -9,7 +9,7 @@ "output_period" : 1, "reduced_data" : 0, "sample_gid_list" : [17], - "sample_syn_list" : [-1], + "sample_pre_list" : [-1], "learn_protocol": { "time_start": 0, "scheme": "EXPLICIT", diff --git a/test_config_defaultnet_onespike_ii.json b/test_config_defaultnet_onespike_ii.json index dccaba6..6396739 100755 --- a/test_config_defaultnet_onespike_ii.json +++ b/test_config_defaultnet_onespike_ii.json @@ -9,7 +9,7 @@ "output_period" : 1, "reduced_data" : 0, "sample_gid_list" : [1690], - "sample_syn_list" : [-1], + "sample_pre_list" : [-1], "learn_protocol": { "time_start": 0, "scheme": "EXPLICIT", diff --git a/test_config_defaultnet_onespike_ii_lowered.json b/test_config_defaultnet_onespike_ii_lowered.json index 68f5f2d..fc5979e 100755 --- a/test_config_defaultnet_onespike_ii_lowered.json +++ b/test_config_defaultnet_onespike_ii_lowered.json @@ -9,7 +9,7 @@ "output_period" : 1, "reduced_data" : 0, "sample_gid_list" : [1690], - "sample_syn_list" : [-1], + "sample_pre_list" : [-1], "learn_protocol": { "time_start": 0, "scheme": "EXPLICIT", diff --git a/test_config_smallnet2_basic_early_recall.json b/test_config_smallnet2_basic_early_recall.json index 9e3506e..51ba094 100644 --- a/test_config_smallnet2_basic_early_recall.json +++ b/test_config_smallnet2_basic_early_recall.json @@ -9,7 +9,7 @@ "output_period" : 1, "reduced_data" : 0, "sample_gid_list" : [1], - "sample_syn_list" : [0], + "sample_pre_list" : [0], "learn_protocol": { "time_start": 0, "scheme": "EXPLICIT", diff --git a/test_config_smallnet2_basic_late_noff.json b/test_config_smallnet2_basic_late_noff.json index 041b258..e6b641f 100755 --- a/test_config_smallnet2_basic_late_noff.json +++ b/test_config_smallnet2_basic_late_noff.json @@ -9,7 +9,7 @@ "output_period" : 10, "reduced_data" : 0, "sample_gid_list" : [1], - "sample_syn_list" : [0], + "sample_pre_list" : [0], "learn_protocol": { "scheme": "RECT", "time_start": 0, diff --git a/test_config_smallnet2_det_onespike.json b/test_config_smallnet2_det_onespike.json index d62c9c3..aabd044 100644 --- a/test_config_smallnet2_det_onespike.json +++ b/test_config_smallnet2_det_onespike.json @@ -31,7 +31,7 @@ "sigma_WN" : 0 }, "sample_gid_list": [0, 1, 2, 2, 3, 3], - "sample_syn_list": [0, 0, 0, 1, 0, 1] + "sample_pre_list": [0, 0, 0, 3, 0, 2] }, "populations" : { diff --git a/test_config_smallnet3_det_learn_2s-recall.json b/test_config_smallnet3_det_learn_2s-recall.json index 87ce94c..8bd81e7 100755 --- a/test_config_smallnet3_det_learn_2s-recall.json +++ b/test_config_smallnet3_det_learn_2s-recall.json @@ -27,7 +27,7 @@ "sigma_WN" : 0.05 }, "sample_gid_list": [ 0, 1, 2, 2, 3, 3, 4], - "sample_syn_list": [-1, 0, 1, 0, 0, 1, -1] + "sample_pre_list": [-1, 0, 0, 3, 0, 2, -1] }, "populations" : { diff --git a/test_config_smallnet3_det_learn_8h-recall.json b/test_config_smallnet3_det_learn_8h-recall.json index 5a92d9f..4ee16ac 100755 --- a/test_config_smallnet3_det_learn_8h-recall.json +++ b/test_config_smallnet3_det_learn_8h-recall.json @@ -27,7 +27,7 @@ "sigma_WN" : 0.05 }, "sample_gid_list": [ 0, 1, 2, 2, 3, 3, 4], - "sample_syn_list": [-1, 0, 1, 0, 0, 1, -1] + "sample_pre_list": [-1, 0, 0, 3, 0, 2, -1] }, "populations" : { diff --git a/test_config_smallnet3_det_onespike.json b/test_config_smallnet3_det_onespike.json index e3f372d..224d15f 100644 --- a/test_config_smallnet3_det_onespike.json +++ b/test_config_smallnet3_det_onespike.json @@ -31,7 +31,7 @@ "sigma_WN" : 0 }, "sample_gid_list": [0, 1, 2, 2, 3, 3], - "sample_syn_list": [0, 0, 0, 1, 0, 1] + "sample_pre_list": [0, 0, 0, 3, 0, 2] }, "populations" : { diff --git a/test_coverage.txt b/test_coverage.txt index 953f7a8..04c30d8 100644 --- a/test_coverage.txt +++ b/test_coverage.txt @@ -1,7 +1,7 @@ Name Stmts Miss Cover Missing ----------------------------------------------------------------- -arborNetworkConsolidation.py 913 93 90% 77, 176-205, 787, 902, 936, 951, 957, 961, 989, 998-999, 1033-1034, 1040-1041, 1087, 1200, 1303, 1554, 1635-1653, 1667-1724 -outputUtilities.py 62 1 98% 20 -test_arborNetworkConsolidation.py 399 3 99% 163, 225, 233 +arborNetworkConsolidation.py 914 93 90% 78, 177-206, 794, 908, 942, 957, 963, 967, 996, 1005-1006, 1040-1041, 1047-1048, 1094, 1207, 1310, 1561, 1643-1661, 1675-1732 +outputUtilities.py 62 2 97% 20, 61 +test_arborNetworkConsolidation.py 399 4 99% 163, 225, 233, 537 ----------------------------------------------------------------- -TOTAL 1374 97 93% +TOTAL 1375 99 93% diff --git a/test_result.txt b/test_result.txt index a88ce3f..9763dca 100644 --- a/test_result.txt +++ b/test_result.txt @@ -2,8 +2,8 @@ platform linux -- Python 3.10.12, pytest-8.3.2, pluggy-1.5.0 rootdir: /home/jannik/Arbor_Network_GitLab_Repository plugins: cov-5.0.0, reportlog-0.4.0 -collecting ...  collected 43 items  +collecting ...  collected 42 items  -test_arborNetworkConsolidation.py ........................................... [100%] +test_arborNetworkConsolidation.py .......................................... [100%] -===================================================================================== 43 passed in 6354.06s (1:45:54) ===================================================================================== +===================================================================================== 42 passed in 850.22s (0:14:10) ====================================================================================== diff --git a/track_allocated_memory b/track_allocated_memory index ae2e367..a9a2e11 100644 --- a/track_allocated_memory +++ b/track_allocated_memory @@ -1,6 +1,6 @@ #!/bin/bash -# Copyright 2023 Jannik Luboeinski +# Copyright 2023-2024 Jannik Luboeinski # License: Apache-2.0 (http://www.apache.org/licenses/LICENSE-2.0) # Contact: mail[at]jlubo.net