Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updating #292

Merged
merged 55 commits into from
May 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
5afc352
Make tsd, tsd-tensot, and tsd-frame lazy
alejoe91 Apr 9, 2024
ec0ffe8
Add lazy flag and set it to True for NWB interface
alejoe91 Apr 9, 2024
58bfbd3
black
alejoe91 Apr 10, 2024
3da0d46
add TsGroup.merge_group method
qian-chu May 6, 2024
60719f5
doc aesthetics
qian-chu May 6, 2024
b3874a1
Update ts_group.py
qian-chu May 6, 2024
d24f331
Simplify tsgroup.merge()
qian-chu May 7, 2024
04a0053
Update pynapple/core/ts_group.py
qian-chu May 7, 2024
c642068
Update pynapple/core/ts_group.py
qian-chu May 7, 2024
f3508d5
Merge pull request #284 from pynapple-org/signal
gviejo May 17, 2024
2d08c33
Update ts_group.py
qian-chu May 17, 2024
26f6333
correctly access nap_config
qian-chu May 19, 2024
5c49a0e
add test for merging groups
qian-chu May 19, 2024
58c3558
Merge branch 'main' into merge_tsgroup
qian-chu May 19, 2024
a65f5f5
Update tests/test_ts_group.py
qian-chu May 20, 2024
6bf20a7
Update pynapple/core/ts_group.py
qian-chu May 20, 2024
6988758
Update pynapple/core/ts_group.py
qian-chu May 20, 2024
41e0dfc
Move docstring to merge()
qian-chu May 20, 2024
e08c0e4
Fix regex error
qian-chu May 20, 2024
e1a8eb7
fixed merge conflict
BalzaniEdoardo May 20, 2024
2f9ded5
changed label name
BalzaniEdoardo May 20, 2024
2809448
added a couple of tests
BalzaniEdoardo May 20, 2024
bc761df
Merge branch 'dev' into lazy-loading
BalzaniEdoardo May 20, 2024
e02a423
tested methods
BalzaniEdoardo May 20, 2024
3651421
added test for value from
BalzaniEdoardo May 20, 2024
91d14f1
added tsd specific methods
BalzaniEdoardo May 20, 2024
6995aea
save lazy
BalzaniEdoardo May 20, 2024
9487728
added test for nwb lazy
BalzaniEdoardo May 20, 2024
a985a72
Revert "Signal"
gviejo May 20, 2024
651d3cc
Merge pull request #286 from pynapple-org/revert-284-signal
gviejo May 20, 2024
af6b53f
compatibility
BalzaniEdoardo May 20, 2024
cffb134
Merge branch 'dev' into lazy-loading
BalzaniEdoardo May 20, 2024
468fb18
test fixed
BalzaniEdoardo May 20, 2024
bd01724
add test for get
BalzaniEdoardo May 20, 2024
809c140
close hdf5 after opening
BalzaniEdoardo May 20, 2024
2225a2a
fixed docstrings
BalzaniEdoardo May 20, 2024
b409409
close the nwb
BalzaniEdoardo May 20, 2024
235d06e
add checks for pr to dev
BalzaniEdoardo May 20, 2024
92e0df4
change to codecov
BalzaniEdoardo May 21, 2024
e6c9bc5
updated actions
BalzaniEdoardo May 21, 2024
ff231e0
Merge pull request #275 from qian-chu/merge_tsgroup
gviejo May 21, 2024
2904cba
changed var name
BalzaniEdoardo May 21, 2024
183a951
improved docstrings
BalzaniEdoardo May 21, 2024
57b9a1e
renamed flag
BalzaniEdoardo May 21, 2024
3b02d27
added a lazy load flag to load nwb
BalzaniEdoardo May 21, 2024
acf8f44
fix on interval set
BalzaniEdoardo May 21, 2024
4100806
linted
BalzaniEdoardo May 21, 2024
10d1e27
Merge pull request #264 from alejoe91/lazy-loading
gviejo May 21, 2024
f05f574
Fixing t loading
gviejo May 21, 2024
b828581
added a test for catching warnings in different lazy-loadings
BalzaniEdoardo May 21, 2024
6db7390
test for wanrns
BalzaniEdoardo May 21, 2024
efaad74
linted
BalzaniEdoardo May 21, 2024
7aa66aa
added test tsgroup
BalzaniEdoardo May 22, 2024
1e82780
Testings test
gviejo May 22, 2024
e207800
Merge pull request #289 from pynapple-org/dev
gviejo May 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 8 additions & 10 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
branches: [ main, dev ]

jobs:
lint:
Expand Down Expand Up @@ -40,9 +40,9 @@ jobs:
# - os: windows-latest
# python-version: 3.7
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
Expand All @@ -52,14 +52,12 @@ jobs:
- name: Test
run: |
coverage run --source=pynapple --branch -m pytest tests/
coverage report -m
coverage report -m

- name: Coveralls
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
pip install coveralls
coveralls --service=github
- name: Upload coverage reports to Codecov
uses: codecov/[email protected]
with:
token: ${{ secrets.CODECOV_TOKEN }}
check:
if: always()
needs:
Expand Down
4 changes: 2 additions & 2 deletions pynapple/core/_core_functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,6 @@ def _threshold(time_array, data_array, starts, ends, thr, method):
if get_backend() == "jax":
from pynajax.jax_core_threshold import threshold

return threshold(time_array, data_array, starts, ends, thr, method)
return threshold(time_array, data_array[:], starts, ends, thr, method)
else:
return jitthreshold(time_array, data_array, starts, ends, thr, method)
return jitthreshold(time_array, data_array[:], starts, ends, thr, method)
11 changes: 8 additions & 3 deletions pynapple/core/_jitted_functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -312,11 +312,16 @@ def jitthreshold(time_array, data_array, starts, ends, thr, method="above"):
return (new_time_array, new_data_array, new_starts, new_ends)


@jit(nopython=True)
def jitbin_array(time_array, data_array, starts, ends, bin_size):
"""Slice first for compatibility with lazy loading."""
idx, countin = jitrestrict_with_count(time_array, starts, ends)
time_array = time_array[idx]
data_array = data_array[idx]
return _jitbin_array(
countin, time_array[idx], data_array[idx], starts, ends, bin_size
)


@jit(nopython=True)
def _jitbin_array(countin, time_array, data_array, starts, ends, bin_size):

m = starts.shape[0]
f = data_array.shape[1:]
Expand Down
54 changes: 38 additions & 16 deletions pynapple/core/time_series.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,10 +68,18 @@ class BaseTsd(Base, NDArrayOperatorsMixin, abc.ABC):
Implement most of the shared functions across concrete classes `Tsd`, `TsdFrame`, `TsdTensor`
"""

def __init__(self, t, d, time_units="s", time_support=None):
def __init__(self, t, d, time_units="s", time_support=None, load_array=True):
super().__init__(t, time_units, time_support)

self.values = convert_to_array(d, "d")
if load_array:
self.values = convert_to_array(d, "d")
else:
if not is_array_like(d):
raise TypeError(
"Data should be array-like, i.e. be indexable, iterable and, have attributes "
"`shape`, `ndim` and, `dtype`)."
)
self.values = d

assert len(self.index) == len(
self.values
Expand Down Expand Up @@ -220,36 +228,38 @@ def __array_function__(self, func, types, args, kwargs):

def as_array(self):
"""
Return the data as a numpy.ndarray
Return the data.

Returns
-------
out: numpy.ndarray
out: array-like
_
"""
return self.values

def data(self):
"""
Return the data as a numpy.ndarray
Return the data.

Returns
-------
out: numpy.ndarray
out: array-like
_
"""
return self.values

def to_numpy(self):
"""
Return the data as a numpy.ndarray. Mostly useful for matplotlib plotting when calling `plot(tsd)`
Return the data as a numpy.ndarray.

Mostly useful for matplotlib plotting when calling `plot(tsd)`.
"""
return self.values
return np.asarray(self.values)

def copy(self):
"""Copy the data, index and time support"""
return self.__class__(
t=self.index.copy(), d=self.values.copy(), time_support=self.time_support
t=self.index.copy(), d=self.values[:].copy(), time_support=self.time_support
)

def value_from(self, data, ep=None):
Expand Down Expand Up @@ -662,7 +672,9 @@ class TsdTensor(BaseTsd):
The time support of the time series
"""

def __init__(self, t, d, time_units="s", time_support=None, **kwargs):
def __init__(
self, t, d, time_units="s", time_support=None, load_array=True, **kwargs
):
"""
TsdTensor initializer

Expand All @@ -677,7 +689,7 @@ def __init__(self, t, d, time_units="s", time_support=None, **kwargs):
time_support : IntervalSet, optional
The time support of the TsdFrame object
"""
super().__init__(t, d, time_units, time_support)
super().__init__(t, d, time_units, time_support, load_array)

assert (
self.values.ndim >= 3
Expand Down Expand Up @@ -831,7 +843,15 @@ class TsdFrame(BaseTsd):
The time support of the time series
"""

def __init__(self, t, d=None, time_units="s", time_support=None, columns=None):
def __init__(
self,
t,
d=None,
time_units="s",
time_support=None,
columns=None,
load_array=True,
):
"""
TsdFrame initializer
A pandas.DataFrame can be passed directly
Expand Down Expand Up @@ -859,7 +879,7 @@ def __init__(self, t, d=None, time_units="s", time_support=None, columns=None):
else:
assert d is not None, "Missing argument d when initializing TsdFrame"

super().__init__(t, d, time_units, time_support)
super().__init__(t, d, time_units, time_support, load_array)

assert self.values.ndim <= 2, "Data should be 1 or 2 dimensional."

Expand Down Expand Up @@ -1084,7 +1104,7 @@ def save(self, filename):
np.savez(
filename,
t=self.index.values,
d=self.values,
d=self.values[:],
start=self.time_support.start,
end=self.time_support.end,
columns=cols_name,
Expand All @@ -1108,7 +1128,9 @@ class Tsd(BaseTsd):
The time support of the time series
"""

def __init__(self, t, d=None, time_units="s", time_support=None, **kwargs):
def __init__(
self, t, d=None, time_units="s", time_support=None, load_array=True, **kwargs
):
"""
Tsd Initializer.

Expand All @@ -1129,7 +1151,7 @@ def __init__(self, t, d=None, time_units="s", time_support=None, **kwargs):
else:
assert d is not None, "Missing argument d when initializing Tsd"

super().__init__(t, d, time_units, time_support)
super().__init__(t, d, time_units, time_support, load_array)

assert self.values.ndim == 1, "Data should be 1 dimensional"

Expand Down
Loading
Loading