tiledbsoma 1.16.0rc1 pre-check,main branch

[johnkerl]

Following our established procedure

single-cell-data/TileDB-SOMA#3720
[sc-63626]

Sibling of #284. Replays #275.

[jdblischak]

1.16.0 requires all the updates to support C++20, plus it would be ideal to add the flag -DSPDLOG_LINK_SHARED=ON. See #275 and #276

[johnkerl]

Derp, thanks for the reminders you had been so thorough to set for me, and which I had misplaced. Thank you @jdblischak !

[jdblischak]

Not a deal breaker, but it would be nice to add -DSPDLOG_LINK_SHARED=ON to build-libtiledbsoma.sh to finally fix the dynamic linking to spdlog

[johnkerl]

Yup, still iterating with commits -- thanks @jdblischak !

[johnkerl]

I'm baffled by the CI fail here:
https://dev.azure.com/TileDB-Inc/CI/_build/results?buildId=42007&view=logs&jobId=e0208569-136d-54da-4ec5-14d0e8771cbc&j=e0208569-136d-54da-4ec5-14d0e8771cbc&t=25992e30-3e2a-5788-2b3e-e373b0534bf7

conda_libmamba_solver.conda_build_exceptions.ExplainedDependencyNeedsBuildingError: Unsatisfiable dependencies for platform osx-64: {MatchSpec("somacore==1.0.28=pyh4471522_0")}
Encountered problems while solving:
  - package somacore-1.0.28-pyh4471522_0 is excluded by strict repo priority

-- I'm googling that one now as this is a new one to me

[johnkerl]

Baffled by
https://dev.azure.com/TileDB-Inc/CI/_build/results?buildId=42007&view=logs&jobId=e0208569-136d-54da-4ec5-14d0e8771cbc&j=e0208569-136d-54da-4ec5-14d0e8771cbc&t=25992e30-3e2a-5788-2b3e-e373b0534bf7

conda_libmamba_solver.conda_build_exceptions.ExplainedDependencyNeedsBuildingError: Unsatisfiable dependencies for platform osx-64: {MatchSpec("somacore==1.0.28=pyh4471522_0")}
Encountered problems while solving:
  - package somacore-1.0.28-pyh4471522_0 is excluded by strict repo priority

-- I have never seen this before.

As a first attempt, I'll rerun the CI just in case the issue involves somacore 1.0.28 being too recently published. (Again, I've never seen it fail in this way when a CI here is run too soon, but, it's an easy first thing to try.)

[johnkerl]

$ conda smithy rerender --commit auto
INFO:conda_smithy.configure_feedstock:Re-rendered with conda-build 25.1.1, conda-smithy 3.46.1, and conda-forge-pinning 2025.03.05.16.19.52
INFO:conda_smithy.configure_feedstock:No changes made. This feedstock is up-to-date.

[johnkerl]

I think I found the issue: https://github.com/conda-forge/somacore-feedstock now (newly) exists and is still at 1.0.26 while https://github.com/TileDB-Inc/somacore-feedstock is at 1.0.28.

[johnkerl]

Regardless we do still want our own channel as higher-pri

[johnkerl]

conda-forge/somacore-feedstock#3

[johnkerl]

Changing channel priority is bad b/c now a new fail with

  - package tiledb-2.27.2-he4dccf3_0 is excluded by strict repo priority

So: now that https://github.com/conda-forge/somacore-feedstock exists, I need to add a new release-blocking step to
https://github.com/single-cell-data/TileDB-SOMA/wiki/Branches-and-releases
namely: we must upgrade conda-forge/somacore-feedstock, and this is now a new and blocking step in our release process.

[johnkerl]

Closing per our established procedure

[jdblischak]

So: now that https://github.com/conda-forge/somacore-feedstock exists, I need to add a new release-blocking step to
https://github.com/single-cell-data/TileDB-SOMA/wiki/Branches-and-releases
namely: we must upgrade conda-forge/somacore-feedstock, and this is now a new and blocking step in our release process.

Nice troubleshooting @johnkerl! I hadn't considered this potential problem when somacore and tiledbsoma-py were added upstream to conda-forge. This means that the somacore installed in these feedstock builds is the one from conda-forge.

In light of this:

• Do we need to continue maintaining our own TileDB-Inc/somacore-feedstock? If there is nothing special about it, it seems like an unnecessary step now that we can obtain the conda binary from conda-forge
• Given the complexity of this repo, I'm open to switching the channel_priority to "flexible" if we want to be able to install somacore from the tiledb channel

[jdblischak]

Changing channel priority is bad b/c now a new fail with - package tiledb-2.27.2-he4dccf3_0 is excluded by strict repo priority

This puzzled me at first glance, but now I understand it. The tiledb channel is for special binaries like "for-cloud" and "nightlies". The most recent standard version we uploaded with the default label of "main" was 2.17.1 over a year ago.

[johnkerl]

Do we need to continue maintaining our own TileDB-Inc/somacore-feedstock? If there is nothing special about it, it seems like an unnecessary step now that we can obtain the conda binary from conda-forge

Agreed! I've marked this as a follow-up item for myself after we get 1.16.0 out.

Given the complexity of this repo, I'm open to switching the channel_priority to "flexible" if we want to be able to install somacore from the tiledb channel

I'm open to Conda-expert advice (I am not a Conda expert) but the googling I did do told me strict priority is important for solve times, and I don't want slower solve times.

[jdblischak]

strict priority is important for solve times, and I don't want slower solve times.

Good point! If we're fine with installing somacore from conda-forge, let's keep the strict channel priority.

Personally, the reason strict channel priority is important is because otherwise users that install packages from many different channels end up with "frankenstein" conda environments where their packages come from a mishmash of different places. This leads to hard-to-decipher errors (because packages from different channels won't necessarily be compatible), and also makes it hard to ever reproduce that conda environment. It's much safer to install almost everything from a single channel (eg conda-forge) and then only diverge from this for specific packages you need from other special channels.

[johnkerl]

frankenstein

💯

[johnkerl]

Need to test-drive spatial dependencies for [sc-64100]

[johnkerl]

Current blocking path to get us spatialdata >= 0.2.5 on conda-forge:

• spatialdata v0.3.0 conda-forge/spatialdata-feedstock#14 (comment)
• multiscale-spatial-image v2.0.2 conda-forge/multiscale-spatial-image-feedstock#5
• xarray-dataclasses v1.8.0 conda-forge/xarray-dataclasses-feedstock#3

Current is 0.2.3 https://anaconda.org/conda-forge/spatialdata/files

[johnkerl]

Re-closing. The spatialdata experiment can wait.