Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement of the blockwise function for CO3D data scaling #697

Open
adebardo opened this issue Mar 3, 2025 · 0 comments
Open

Enhancement of the blockwise function for CO3D data scaling #697

adebardo opened this issue Mar 3, 2025 · 0 comments
Labels
[POC] Conception To review Tickets needs approval about it conception

Comments

@adebardo
Copy link
Contributor

adebardo commented Mar 3, 2025

Context:
To enable efficient processing of CO3D data, which consists of multiple gigabytes, we need to refine the blockwise functionality, particularly the apply section, to integrate our RANSAC filter. Additionally, several modifications must be implemented to optimize processing using our latest advancements in tiling and the implementation of a multiprocessing cluster.

Modifications to be made:

  1. Replacement of the subdivision grid creation

    • Integration of the new tiling method.
    • Generation of coordinates and the required number of tiles for an efficient iteration in multiprocessing.
  2. Implementation of reprojection in multiprocessing

    • Modification of the _preprocess_coreg_fit_raster_raster function to integrate multiprocessing from GeoUtils.
    • Use of the previously created tile grid.
    • Creation and management of a multiprocessing cluster.
    • Reprojection of data on the tiles.
  3. Reading and aligning the reference DEM

    • Loading the DEM in tiling mode.
    • Processing in multiprocessing using the appropriate class.
    • Each worker will execute the .fit function on the tiles provided by the tiling function.
  4. Replacing the call to stat with get_stats

    • Removal of the stat call in blockwise.
    • Replacement with get_stats, which is now available for each raster.
@adebardo adebardo added the [POC] Conception To review Tickets needs approval about it conception label Mar 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[POC] Conception To review Tickets needs approval about it conception
Projects
None yet
Development

No branches or pull requests

1 participant