Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement WsiAnnotations in C++ #252

Merged
merged 98 commits into from
Sep 26, 2024
Merged
Show file tree
Hide file tree
Changes from 59 commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
2a468bf
Updated workflows
jonasteuwen Aug 7, 2024
1d26585
Adding C++ TIFF writer
jonasteuwen Aug 4, 2024
103ea69
Add ZSTD compression, reformat some files
jonasteuwen Aug 10, 2024
004fad7
First commit with geometries
jonasteuwen Aug 10, 2024
625847a
First writing of Polygon
jonasteuwen Aug 10, 2024
aa76501
Some additional changes to cast the Polygon.
jonasteuwen Aug 11, 2024
0773971
Commiting to not lose work
jonasteuwen Aug 11, 2024
96962b8
Updated for initial version of WsiAnnotations
jonasteuwen Aug 11, 2024
b4aa87e
updates
jonasteuwen Aug 12, 2024
c5a7656
Some updates during development
jonasteuwen Aug 12, 2024
a784f79
merge
jonasteuwen Aug 12, 2024
9e76c78
Fix scaling issues
jonasteuwen Aug 12, 2024
25cd927
Added functionality to remove polygons and points and to rebuild the …
jonasteuwen Aug 13, 2024
7e589e7
Implement complete initial version of geometry module
jonasteuwen Aug 13, 2024
f7a3e1c
We actually need to iterate over all layers, not just polys
jonasteuwen Aug 13, 2024
8af648d
Implement scaling and offset functions
jonasteuwen Aug 13, 2024
d0a1ada
Adding functionality to reindex polygons
jonasteuwen Aug 13, 2024
958ad8e
Created output container for results
jonasteuwen Aug 13, 2024
56deac8
Sensible updates
jonasteuwen Aug 13, 2024
19d644d
Add masking utility to AnnotationRegion
jonasteuwen Aug 13, 2024
4943171
Memory optimizations
jonasteuwen Aug 13, 2024
8d2b241
Adding sorting capabilities
jonasteuwen Aug 13, 2024
9e99882
Add bounding box
jonasteuwen Aug 13, 2024
8cb01a6
Added color LUT
jonasteuwen Aug 13, 2024
999fa95
Refactor
jonasteuwen Aug 14, 2024
2be4b87
Extend dunder methods and code simplification for annotations (#251)
BPdeRooij Aug 13, 2024
87c361d
Merge branch 'main' into feature/geometry
jonasteuwen Aug 17, 2024
7f0db0e
Fixing the scaling, mypy and adding tests
jonasteuwen Aug 17, 2024
b7c2698
minor changes in annotations
jonasteuwen Aug 17, 2024
cc3da2e
Merge branch 'feature/geometry' of github.com:NKI-AI/dlup into featur…
jonasteuwen Aug 17, 2024
707ceb4
Updating geometry module, and its tests.
jonasteuwen Aug 17, 2024
354ef87
Rename modules, improve CI/CD
jonasteuwen Aug 18, 2024
36d1772
Almost done with the geometry module
jonasteuwen Aug 18, 2024
efcfdc6
Disable a few tests, squash pylint errors
jonasteuwen Aug 18, 2024
5dede83
Fixing some bugs when trees get invalidated
jonasteuwen Aug 18, 2024
b82d93b
Do an allclose rather than equal
jonasteuwen Aug 18, 2024
d3d61b1
Weirdly enough different values on github
jonasteuwen Aug 18, 2024
35858e9
Maybe a sudo install won't hurt
jonasteuwen Aug 18, 2024
a0377eb
Remove assert
jonasteuwen Aug 18, 2024
c5c3857
Restructure C++ code
jonasteuwen Aug 18, 2024
5dcf06e
Restructure C++ code
jonasteuwen Aug 18, 2024
eaa0683
Refactored code.
jonasteuwen Aug 19, 2024
2750029
Refactoring
jonasteuwen Aug 19, 2024
d8ddb80
Refactor code
jonasteuwen Aug 19, 2024
549f54a
Rename functions, remove functions
jonasteuwen Aug 19, 2024
5467ca3
Adhere to google C++ style guide
jonasteuwen Aug 19, 2024
92b6145
Maybe include stdexcept so that github understandS?
jonasteuwen Aug 19, 2024
84e53a9
Reducing a few compiler warnings
jonasteuwen Aug 19, 2024
ac343a8
Make sure mypy runs.
jonasteuwen Aug 19, 2024
3223071
Convenient helper function
jonasteuwen Aug 19, 2024
66d8544
Implement XML export
jonasteuwen Aug 19, 2024
223efc2
Add xsdata to requirements
jonasteuwen Aug 19, 2024
d095bb3
XML Schema reading now works
jonasteuwen Aug 19, 2024
c2a5284
Order can be 0...
jonasteuwen Aug 19, 2024
fff73f4
Several changes to support HaloXML
jonasteuwen Aug 20, 2024
b656d78
Improve test coverage
jonasteuwen Aug 21, 2024
5f6efe9
Improve test coverage to 90%
jonasteuwen Aug 21, 2024
81cf038
Streamlining
jonasteuwen Aug 21, 2024
55b0932
Trying to get better CI/CD coverage
jonasteuwen Aug 21, 2024
ba9851f
Fix bug in annotations_experimental.py
jonasteuwen Aug 22, 2024
34feb81
Add more tests, and fix bug in process
jonasteuwen Aug 22, 2024
d75a072
Updating tests, updating schema, add box
jonasteuwen Aug 23, 2024
3aec384
Add more boxes functionality.
jonasteuwen Aug 23, 2024
5236a52
Refactor factory methods to factory.h
jonasteuwen Aug 24, 2024
1311d55
Refactor
jonasteuwen Aug 24, 2024
ea1eff9
Code simplifications
jonasteuwen Aug 24, 2024
4d5b066
Some final memory improvements
jonasteuwen Aug 24, 2024
096d18d
Adding boxes to the geometry module
jonasteuwen Aug 25, 2024
263ec7a
Improve CI/CD
jonasteuwen Aug 25, 2024
5d73af0
Adding deprecation warnings
jonasteuwen Aug 25, 2024
3a3e66e
Streamline meson
jonasteuwen Aug 25, 2024
fa862d1
Attempt lazy initialization of the geometry collection in the region …
jonasteuwen Aug 25, 2024
a01429e
Update tests for lazy evaluation
jonasteuwen Aug 25, 2024
1b25250
Fixing memory error in region.h. Fix tox
jonasteuwen Aug 26, 2024
4f5ed1c
Improve workflows
jonasteuwen Aug 27, 2024
1001fee
Improve workflows
jonasteuwen Aug 27, 2024
3155040
Handle tox merge
jonasteuwen Aug 27, 2024
ca3cbe4
Update tox.ini
jonasteuwen Aug 27, 2024
1ff5552
Remove line that doesn't work on github (?!)
jonasteuwen Aug 27, 2024
0ecc899
Cleanup code
jonasteuwen Aug 27, 2024
4d40c71
Try to run tests non-parallel
jonasteuwen Aug 27, 2024
0c4b06e
Squash errors for now
jonasteuwen Aug 27, 2024
c0f6aca
Fixes for ubuntu
Aug 28, 2024
d1e4dcc
Improving annotations to mask conversion
jonasteuwen Aug 31, 2024
b9405b4
Merge branches 'feature/geometry' and 'feature/geometry' of github.co…
jonasteuwen Aug 31, 2024
a0ffb8a
Reformat
jonasteuwen Aug 31, 2024
438f293
Add Lazy Array
jonasteuwen Sep 1, 2024
7e44d55
Added some extra functionality to lazy array
jonasteuwen Sep 1, 2024
eb2e16b
Added lazy evaluation for PolygonCollection
jonasteuwen Sep 2, 2024
e3fc083
Added lazy evaluation for PolygonCollection
jonasteuwen Sep 2, 2024
bb8ba91
Merge branch 'feature/geometry' of github.com:NKI-AI/dlup into featur…
jonasteuwen Sep 2, 2024
85f90f9
Make precommit run
jonasteuwen Sep 2, 2024
001c3e9
Updates to CI
jonasteuwen Sep 2, 2024
c46f90f
Fix mypy
jonasteuwen Sep 2, 2024
5e24d82
Add ROI to the GeometryCollection
jonasteuwen Sep 3, 2024
8dbcffc
Added ROIs to DLUP XML
jonasteuwen Sep 3, 2024
7a86527
Added ROIs to output xml and parsing
jonasteuwen Sep 4, 2024
0009ba5
move declarations to headers
JorenB Sep 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .clang-format
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
---
BasedOnStyle: Google
IndentWidth: 2
Language: Cpp
ColumnLimit: 120
AccessModifierOffset: -4
Expand Down Expand Up @@ -69,7 +71,6 @@ IncludeCategories:
IncludeIsMainRegex: '(Test)?$'
IndentCaseLabels: false
IndentPPDirectives: None
IndentWidth: 4
IndentWrappedFunctionNames: false
JavaScriptQuotes: Leave
JavaScriptWrapImports: true
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
run: |
sudo apt update
sudo apt install -y meson libgl1-mesa-glx libcairo2-dev libgdk-pixbuf2.0-dev libglib2.0-dev libjpeg-dev libpng-dev libtiff5-dev libxml2-dev libopenjp2-7-dev libsqlite3-dev zlib1g-dev libzstd-dev
sudo apt install -y libfftw3-dev libexpat1-dev libgsf-1-dev liborc-0.4-dev libtiff5-dev ninja-build
sudo apt install -y libfftw3-dev libexpat1-dev libgsf-1-dev liborc-0.4-dev libtiff5-dev ninja-build libboost-all-dev libopencv-dev
- name: Build and install OpenSlide
run: |
git clone https://github.com/openslide/openslide.git
Expand Down Expand Up @@ -57,7 +57,7 @@ jobs:
echo "Current directory: $PWD"
meson setup builddir
meson compile -C builddir
meson install -C builddir
sudo meson install -C builddir
- name: Run coverage
run: |
mv dlup _dlup # This is needed because otherwise it won't find the compiled libraries
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/mypy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
run: |
sudo apt update
sudo apt install -y meson libgl1-mesa-glx libcairo2-dev libgdk-pixbuf2.0-dev libglib2.0-dev libjpeg-dev libpng-dev libtiff5-dev libxml2-dev libopenjp2-7-dev libsqlite3-dev zlib1g-dev libzstd-dev
sudo apt install -y libfftw3-dev libexpat1-dev libgsf-1-dev liborc-0.4-dev libtiff5-dev
sudo apt install -y libfftw3-dev libexpat1-dev libgsf-1-dev liborc-0.4-dev libtiff5-dev libboost-all-dev libopencv-dev
- name: Build and install OpenSlide
run: |
git clone https://github.com/openslide/openslide.git
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pylint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
run: |
sudo apt update
sudo apt install -y meson libgl1-mesa-glx libcairo2-dev libgdk-pixbuf2.0-dev libglib2.0-dev libjpeg-dev libpng-dev libtiff5-dev libxml2-dev libopenjp2-7-dev libsqlite3-dev zlib1g-dev libzstd-dev
sudo apt install -y libfftw3-dev libexpat1-dev libgsf-1-dev liborc-0.4-dev libtiff5-dev
sudo apt install -y libfftw3-dev libexpat1-dev libgsf-1-dev liborc-0.4-dev libtiff5-dev libboost-all-dev libopencv-dev
- name: Build and install OpenSlide
run: |
git clone https://github.com/openslide/openslide.git
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tox.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
run: |
sudo apt update
sudo apt install -y meson libgl1-mesa-glx libcairo2-dev libgdk-pixbuf2.0-dev libglib2.0-dev libjpeg-dev libpng-dev libtiff5-dev libxml2-dev libopenjp2-7-dev libsqlite3-dev zlib1g-dev libzstd-dev
sudo apt install -y libfftw3-dev libexpat1-dev libgsf-1-dev liborc-0.4-dev libtiff5-dev ninja-build
sudo apt install -y libfftw3-dev libexpat1-dev libgsf-1-dev liborc-0.4-dev libtiff5-dev ninja-build libboost-all-dev libopencv-dev
- name: Build and install OpenSlide
run: |
git clone https://github.com/openslide/openslide.git
Expand Down
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@
_background.c
_background.cpp
_background.html
_background.cpython-*-darwin.so
_background.cpython-*.so
_libtiff_tiff_writer.cpython-*.so
_geometry.cython-*.so

# Output files
*.tif
Expand Down
27 changes: 17 additions & 10 deletions .spin/cmds.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,28 @@ def test(verbose, tests):
cmd = ["pytest"]
if verbose:
cmd.append("-v")
if coverage:
cmd.extend(["--cov=dlup --cov=tests --cov-report=html --cov-report=term"])
if tests:
cmd.extend(tests)
subprocess.run(cmd, check=True)


@cli.command()
@click.option("-v", "--verbose", is_flag=True, help="Verbose output")
@click.argument("tests", nargs=-1)
def coverage(verbose, tests):
"""🧪 Run tests and generate coverage report"""
cmd = ["pytest", "--cov=dlup", "--cov=tests", "--cov-report=html", "--cov-report=term"]
if verbose:
cmd.append("-v")
if tests:
cmd.extend(tests)
subprocess.run(cmd, check=True)
coverage_path = Path.cwd() / "htmlcov" / "index.html"
webbrowser.open(f"file://{coverage_path.resolve()}")


@cli.command()
def mypy():
"""🦆 Run mypy for type checking"""
Expand Down Expand Up @@ -133,16 +150,6 @@ def clean():
path.unlink()


@cli.command()
def coverage():
"""🧪 Run tests and generate coverage report"""
subprocess.run(["coverage", "run", "--source", "dlup", "-m", "pytest"], check=True)
subprocess.run(["coverage", "report", "-m"], check=True)
subprocess.run(["coverage", "html"], check=True)
coverage_path = Path.cwd() / "htmlcov" / "index.html"
webbrowser.open(f"file://{coverage_path.resolve()}")


@cli.command()
def release():
"""📦 Package and upload a release"""
Expand Down
66 changes: 66 additions & 0 deletions dlup/_geometry.pyi
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
from typing import Callable, overload

from dlup._types import GenericNumber
from dlup.geometry import Point as Point_
from dlup.geometry import Polygon as Polygon_

class Polygon:
@property
def fields(self) -> list[str]: ...
def get_exterior(self) -> list[tuple[float, float]]: ...
def get_interiors(self) -> list[list[tuple[float, float]]]: ...
def contains(self, other: Polygon_) -> bool: ...
def correct_orientation(self) -> None: ...
@property
def area(self) -> float: ...

def set_polygon_factory(factory: Callable[[Polygon_], Polygon_]) -> None: ...

class Point:
@property
def fields(self) -> list[str]: ...
def scale(self, scaling: float) -> None: ...
def get_coordinates(self) -> tuple[float, float]: ...
@property
def x(self) -> float: ...
@property
def y(self) -> float: ...

def set_point_factory(factory: Callable[[Point_], Point_]) -> None: ...

class AnnotationRegion:
@property
def polygons(self) -> list[Polygon_]: ...
@property
def points(self) -> list[Point_]: ...

class GeometryCollection:
def add_polygon(self, polygon: Polygon) -> None: ...
def add_point(self, point: Point_) -> None: ...
@property
def polygons(self) -> list[Polygon_]: ...
@property
def points(self) -> list[Point_]: ...
def set_offset(self, offset: tuple[float, float]) -> None: ...
def rebuild_rtree(self) -> None: ...
def reindex_polygons(self, index_map: dict[str, int]) -> None: ...
@overload
def remove_point(self, index: int) -> None: ...
@overload
def remove_point(self, point: Point_) -> None: ...
@overload
def remove_polygon(self, index: int) -> None: ...
@overload
def remove_polygon(self, polygon: Polygon_) -> None: ...
def sort_polygons(self, key: Callable[[Polygon_], int | float | str | None], reverse: bool) -> None: ...
@property
def bounding_box(self) -> tuple[tuple[float, float], tuple[float, float]]: ...
def size(self) -> int: ...
def simplify(self, tolerance: float) -> None: ...
def scale(self, scaling: float) -> None: ...
def read_region(
self,
coordinates: tuple[GenericNumber, GenericNumber],
scaling: float,
size: tuple[GenericNumber, GenericNumber],
) -> AnnotationRegion: ...
Loading
Loading