many new tools developed from napari plugins, linting, type checking,…

… no Python 3.12

CHANGELOG.md

@@ -0,0 +1,32 @@
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+### Added
+* Types and functions related to `geometry`
+* Types and functions related to paths (`pathtools`)
+* ZARR tools (`zarr_tools`)
+* Some generally useful data types for genome biology (`types`)
+
+### Changed
+* Adopted `ruff` for formatting (rather than `black`) and for linting (rather than `pylint`).
+
+### Removed
+* All custom error types were removed; namely, absence of TensorFlow now will give `ModuleNotFoundError` (built-in) rather than a narrower error type.
+
+## [2024-03-13] - 2024-03-13
+* Made it so that `PathWrapperException` is always what arises when construction of a value of a refined path type fails, rather than having direct construction with a `Path` giving a `TypeError` as it was previously.
+
+## [2023-08-01] - 2023-08-01
+
+### Changed
+* Exposed names, mainly from `pathtools` and `exceptions`, at the package level, for more stable use in dependent projects.
+
+## [2023-07-14] - 2023-07-14
+
+### Added
+* This package, first release

LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) [year] [fullname]
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+

README.md

@@ -3,12 +3,14 @@ General GERlich group uTILities
 
 These tools are organised by use case at the module level; that is, tools that are used in a similar context will tend to be defined in the same module, with the module name reflecting that shared usage context. If you see that something's not well placed, please open an issue and/or a pull request.
 
-## Index
-- [collection_extras.py](collection_extras.py) -- tools for working with generic containers / collections
-- [environments.py](./gertils/environments.py) -- tools for working with `conda` and `pip` environments
-- [gpu.py](./gertils/gpu.py) -- tools for running computations on GPUs, especially with TensorFlow
-- [group_paths.py](./gertils/group_paths.py) -- tools for working with Gerlich group paths on filesystems
-- [pathtools.py](./gertils/pathtools.py) -- tools for working with filesystem paths generally
+## Index: modules and packages
+- [collection_extras](collection_extras.py) -- tools for working with generic containers / collections
+- [environments](./gertils/environments.py) -- tools for working with `conda` and `pip` environments
+- [geometry](./gertils/geometry.py) -- tools for working with entities in space
+- [gpu](./gertils/gpu.py) -- tools for running computations on GPUs, especially with TensorFlow
+- [pathtools](./gertils/pathtools.py) -- tools for working with filesystem paths generally
+- [types](./gertils/pathtools.py) -- data types for working with genome biology, especially imaging
+- [zarr_tools](./gertils/zarr_tools.py) -- functions and types for working with ZARR-stored data
 
 ## Development
 This project is configured to use Nix for a shell/environment with dependencies, and Nox to make common development commands/workflows easier. Start always with `nix-shell`. If it takes a long time to build, try...
@@ -24,7 +26,6 @@ From the Nix shell, run `nox --list` to see a list of available commands, notabl
 
 NB: To pass arguments through `nox` to `pytest`, separate the argument strings with `--`, e.g.:
 ```shell
-nox -s tests-3.12 -- -vv
+nox -s tests-3.11 -- -vv
 ```
 to run the tests with additional verbosity (e.g., `pytest -vv`)
-

gertils/__init__.py

@@ -2,6 +2,13 @@
 
 # mypy: ignore-errors
 
-from .collection_extras import *
-from .exceptions import *
-from .pathtools import *
+# Make things from various modules available at package level.
+from .pathtools import (
+    ExtantFile,
+    ExtantFolder,
+    NonExtantPath,
+    PathWrapperException,
+    find_multiple_paths_by_fov,
+    find_single_path_by_fov,
+    get_fov_sort_key,
+)

gertils/collection_extras.py

@@ -1,28 +1,22 @@
 """Extra tools for working with collections"""
 
-from collections import OrderedDict
-from typing import *
+from collections import Counter, OrderedDict
+from collections.abc import Hashable, Iterable, MutableMapping
+from typing import Optional, TypeVar
 
 __all__ = ["count_repeats", "listify", "uniquify"]
-__author__ = "Vince Reuter"
-__email__ = "[email protected]"
-
-# Very core abstractions related to collections are apt to use very generic
-# names deliberately to underscore the generality of the logic, so disable
-# the invalid name warnings that often result, for all code in this module.
-# pylint: disable=invalid-name
 
 
 AnyT = TypeVar("AnyT")
 HashT = TypeVar("HashT", bound=Hashable)
 
 
-def count_repeats(xs: Iterable[HashT]) -> List[Tuple[HashT, int]]:
+def count_repeats(xs: Iterable[HashT]) -> list[tuple[HashT, int]]:
     """Count the instance of repeated elements and their number"""
     return [(x, n) for x, n in Counter(xs).items() if n > 1]
 
 
-def listify(maybe_items: Optional[Iterable[AnyT]]) -> List[AnyT]:
+def listify(maybe_items: Optional[Iterable[AnyT]]) -> list[AnyT]:
     """Convert an optional iterable into a list, or wrap a string as a singleton list."""
     if maybe_items is None:
         return []

gertils/environments.py

@@ -3,9 +3,9 @@
 import string
 from dataclasses import dataclass
 from pathlib import Path
-from typing import *
+from typing import Optional, Union
 
-from .collection_extras import count_repeats, listify, uniquify
+from .collection_extras import count_repeats, uniquify
 
 __all__ = [
     "CondaEnvironmentSpecification",
@@ -14,23 +14,19 @@
     "RepeatedEnvironmentElementException",
     "combine_pip_environments",
     "conda2pip",
-    "pip2conda",
-    "pipfile_to_condafile",
     "read_pip_env_file",
     "write_env_file",
 ]
-__author__ = "Vince Reuter"
-__email__ = "[email protected]"
 
 
 @dataclass
 class CondaEnvironmentSpecification:
     """Specification of an environment to be managed by conda"""
 
     python_spec: str
-    channels: List[str]
-    conda_dependencies: List[str]
-    pip_dependencies: List[str]
+    channels: list[str]
+    conda_dependencies: list[str]
+    pip_dependencies: list[str]
     name: Optional[str]
 
     def __post_init__(self) -> None:
@@ -52,9 +48,7 @@ def __post_init__(self) -> None:
             raise RepeatedEnvironmentElementException(
                 f"Repeated elements by section for conda environment: {repeats}"
             )
-        python_likes = [
-            d for d in self.conda_dependencies if _is_python_like_dependency(d)
-        ]
+        python_likes = [d for d in self.conda_dependencies if _is_python_like_dependency(d)]
         if python_likes:
             raise RepeatedEnvironmentElementException(
                 "Python(s) specified among other conda environment dependencies; "
@@ -77,7 +71,7 @@ def _is_python_like_dependency(dep: str) -> bool:
 class PipEnvironmentSpecification:
     """Specification of an environment to be managed by pip"""
 
-    dependencies: List[str]
+    dependencies: list[str]
     name: Optional[str]
 
     def __post_init__(self) -> None:
@@ -100,15 +94,15 @@ def combine_pip_environments(
     *environments: PipEnvironmentSpecification,
 ) -> PipEnvironmentSpecification:
     """Combine multiple pip environment specifications."""
-    names = set(e.name for e in environments if e.name)
-    if len(names) == 0:
-        name = None
-    elif len(names) == 1:
-        name = list(names)[0]
-    else:
+    names = {e.name for e in environments if e.name}
+    if len(names) > 1:
         raise IllegalEnvironmentOperationException(
             f"Cannot combine pip environments with different names: {', '.join(names)}"
         )
+    try:
+        name = next(iter(names))
+    except StopIteration:
+        name = None
     deps = list(uniquify(dep for env in environments for dep in env.dependencies))
     return PipEnvironmentSpecification(name=name, dependencies=deps)
 
@@ -122,48 +116,11 @@ def conda2pip(env: CondaEnvironmentSpecification) -> PipEnvironmentSpecification
     return PipEnvironmentSpecification(dependencies=env.pip_dependencies, name=env.name)
 
 
-def pip2conda(
-    env: PipEnvironmentSpecification,
-    python_spec: str,
-    channels: Iterable[str] = (),
-    conda_dependencies: Iterable[str] = (),
-) -> CondaEnvironmentSpecification:
-    """Convert a pip environment to a conda one"""
-    return CondaEnvironmentSpecification(
-        python_spec=python_spec,
-        channels=listify(channels),
-        conda_dependencies=listify(conda_dependencies),
-        pip_dependencies=env.dependencies,
-        name=env.name,
-    )
-
-
-def pipfile_to_condafile(
-    pip_path: Path,
-    python_spec: str,
-    conda_path: Path,
-    overwrite: bool = False,
-    **kwargs: List[str],
-) -> Path:
-    """Write a pip environment specification file as a conda one."""
-    if conda_path.is_file():
-        if not overwrite:
-            raise FileExistsError(
-                f"Conda environment file already exists: {conda_path}"
-            )
-        print(f"Existing conda env file will be overwritten: {conda_path}")
-    pipenv = read_pip_env_file(pip_path)
-    condaenv = pip2conda(env=pipenv, python_spec=python_spec, **kwargs)
-    return write_env_file(env=condaenv, path=conda_path)
-
-
 def read_pip_env_file(path: Path) -> PipEnvironmentSpecification:
     """Parse a pip requirements.txt style file."""
-    with open(path, "r") as envfile:
-        lines = [l for l in envfile if l.strip()]
-    return PipEnvironmentSpecification(
-        name=None, dependencies=[l.strip() for l in lines]
-    )
+    with path.open() as envfile:
+        lines = [line for line in envfile if line.strip()]
+    return PipEnvironmentSpecification(name=None, dependencies=[line.strip() for line in lines])
 
 
 def write_env_file(
@@ -189,7 +146,7 @@ def write_env_file(
         raise TypeError(
             f"Environment to write to file is not a supported type: {type(env).__name__}"
         )
-    with open(path, "w") as envfile:
+    with path.open(mode="w") as envfile:
         for line in lines:
             envfile.write(line + "\n")
     return path

gertils/geometry.py

@@ -0,0 +1,87 @@
+"""Geometric types and tools"""
+
+from abc import abstractmethod
+from dataclasses import dataclass
+from typing import Protocol, Union
+
+import numpy as np
+from numpydoc_decorator import doc  # type: ignore[import]
+
+ZCoordinate = Union[int, float, np.float64]  # int to accommodate notion of "z-slice"
+
+
+class LocatableXY(Protocol):
+    """Something that admits x- and y-coordinate."""
+
+    @abstractmethod
+    def get_x_coordinate(self) -> float:
+        """Getter of position in x dimension"""
+        raise NotImplementedError
+
+    @abstractmethod
+    def get_y_coordinate(self) -> float:
+        """Getter of position in y dimension"""
+        raise NotImplementedError
+
+
+class LocatableZ(Protocol):
+    """Something that admits z-coordinate."""
+
+    @abstractmethod
+    def get_z_coordinate(self) -> ZCoordinate:
+        """Getter of position in z dimension"""
+        raise NotImplementedError
+
+
+@doc(
+    summary="Bundle x and y position to create point in 2D space.",
+    parameters=dict(
+        x="Position in x",
+        y="Position in y",
+    ),
+)
+@dataclass(kw_only=True, frozen=True)
+class ImagePoint2D(LocatableXY):  # noqa: D101
+    x: float
+    y: float
+
+    def __post_init__(self) -> None:
+        if not all(isinstance(c, float) for c in [self.x, self.y]):
+            raise TypeError(f"At least one coordinate isn't floating-point! {self}")
+        if any(c < 0 for c in [self.x, self.y]):
+            raise ValueError(f"At least one coordinate is negative! {self}")
+
+    @doc(summary="Get the x-coordinate for this point.")
+    def get_x_coordinate(self) -> float:  # noqa: D102
+        return self.x
+
+    @doc(summary="Get the x-coordinate for this point.")
+    def get_y_coordinate(self) -> float:  # noqa: D102
+        return self.y
+
+
+@doc(
+    summary="Bundle x and y position to create point in 2D space.",
+    parameters=dict(
+        x="Position in x",
+        y="Position in y",
+        z="Position in z",
+    ),
+    see_also=dict(
+        ImagePoint2D="Simpler, non-z implementation of an image point",
+    ),
+)
+@dataclass(kw_only=True, frozen=True)
+class ImagePoint3D(ImagePoint2D, LocatableZ):  # noqa: D101
+    z: ZCoordinate
+
+    def __post_init__(self) -> None:
+        super().__post_init__()
+        if not isinstance(self.z, int | float | np.float64):
+            raise TypeError(f"Bad z ({type(self.z).__name__}: {self.z}")
+        if any(c < 0 for c in [self.x, self.y]):
+            raise ValueError(f"z-coordinate is negative! {self}")
+
+    @doc(summary="Get the x-coordinate for this point.")
+    def get_z_coordinate(self) -> ZCoordinate:  # noqa: D102
+        return self.z