[BUG] 71.x fails with backports.tarfile with conda-installed backports present

[austin3dickey]

setuptools version

setuptools==71.0.4

Python version

Python 3.8.19

OS

macOS, Linux

Additional environment information

No response

Description

I am running pip install . to install pyarrow during Apache Arrow benchmarking, and it is failing due to ImportError: cannot import name 'tarfile' from 'backports'.

This works for setuptools versions <71, but fails for anything >=71. It happened on Linux in CI and I was able to reproduce on my Mac.

Expected behavior

The package should be installed successfully.

How to Reproduce

First clone the arrow repo. We create and activate a conda environment, pip install some dependencies, and then try to pip install the project.

git clone [email protected]:apache/arrow.git
cd arrow/python
conda create -yn test_env -c conda-forge 'autoconf' 'ccache' 'orc' 'pkg-config' 'aws-sdk-cpp==1.11.68' "azure-core-cpp[version='>=1.10.3']" "azure-identity-cpp[version='>=1.6.0']" "azure-storage-blobs-cpp[version='>=12.10.0']" "azure-storage-common-cpp[version='>=12.5.0']" "azure-storage-files-datalake-cpp[version='>=12.9.0']" "benchmark[version='>=1.6.0,!=1.8.4']" "boost-cpp[version='>=1.68.0']" 'brotli' 'bzip2' 'c-ares' 'cmake' 'gflags' 'glog' "gmock[version='>=1.10.0']" "google-cloud-cpp[version='>=1.34.0']" "grpc-cpp[version='<=1.50.1']" "gtest[version='>=1.10.0']" 'libprotobuf' 'libutf8proc' 'lz4-c' 'make' 'ninja' 'nodejs' 'orc' 'pkg-config' 'python' 'rapidjson' 're2' 'snappy' "thrift-cpp[version='>=0.11.0']" 'xsimd' 'zlib' 'zstd' 'boto3' 'cffi' "cython[version='>=0.29.31']" 'cloudpickle' 'fsspec' 'hypothesis' "numpy[version='>=1.16.6']" 'pytest' 'pytest-faulthandler' "s3fs[version='>=2023.10.0']" "setuptools[version='>=64']" "setuptools_scm[version='>=8']" 'compilers' 'python=3.8' 'pandas' 'r'
conda activate test_env
pip install 'oldest-supported-numpy>=0.14'
pip install --no-deps --no-build-isolation -vv .

Output

bash-5.2$ git clone [email protected]:apache/arrow.git
Cloning into 'arrow'...
remote: Enumerating objects: 246694, done.
remote: Counting objects: 100% (234/234), done.
remote: Compressing objects: 100% (179/179), done.
remote: Total 246694 (delta 85), reused 146 (delta 47), pack-reused 246460
Receiving objects: 100% (246694/246694), 190.10 MiB | 12.62 MiB/s, done.
Resolving deltas: 100% (173529/173529), done.
Updating files: 100% (7758/7758), done.
bash-5.2$ cd arrow/python
bash-5.2$ conda create -yn test_env -c conda-forge 'autoconf' 'ccache' 'orc' 'pkg-config' 'aws-sdk-cpp==1.11.68' "azure-core-cpp[version='>=1.10.3']" "azure-identity-cpp[version='>=1.6.0']" "azure-storage-blobs-cpp[version='>=12.10.0']" "azure-storage-common-cpp[version='>=12.5.0']" "azure-storage-files-datalake-cpp[version='>=12.9.0']" "benchmark[version='>=1.6.0,!=1.8.4']" "boost-cpp[version='>=1.68.0']" 'brotli' 'bzip2' 'c-ares' 'cmake' 'gflags' 'glog' "gmock[version='>=1.10.0']" "google-cloud-cpp[version='>=1.34.0']" "grpc-cpp[version='<=1.50.1']" "gtest[version='>=1.10.0']" 'libprotobuf' 'libutf8proc' 'lz4-c' 'make' 'ninja' 'nodejs' 'orc' 'pkg-config' 'python' 'rapidjson' 're2' 'snappy' "thrift-cpp[version='>=0.11.0']" 'xsimd' 'zlib' 'zstd' 'boto3' 'cffi' "cython[version='>=0.29.31']" 'cloudpickle' 'fsspec' 'hypothesis' "numpy[version='>=1.16.6']" 'pytest' 'pytest-faulthandler' "s3fs[version='>=2023.10.0']" "setuptools[version='>=64']" "setuptools_scm[version='>=8']" 'compilers' 'python=3.8' 'pandas' 'r'
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /opt/miniconda3/envs/test_env

  added / updated specs:
    - autoconf
    - aws-sdk-cpp==1.11.68
    - azure-core-cpp[version='>=1.10.3']
    - azure-identity-cpp[version='>=1.6.0']
    - azure-storage-blobs-cpp[version='>=12.10.0']
    - azure-storage-common-cpp[version='>=12.5.0']
    - azure-storage-files-datalake-cpp[version='>=12.9.0']
    - benchmark[version='>=1.6.0,!=1.8.4']
    - boost-cpp[version='>=1.68.0']
    - boto3
    - brotli
    - bzip2
    - c-ares
    - ccache
    - cffi
    - cloudpickle
    - cmake
    - compilers
    - cython[version='>=0.29.31']
    - fsspec
    - gflags
    - glog
    - gmock[version='>=1.10.0']
    - google-cloud-cpp[version='>=1.34.0']
    - grpc-cpp[version='<=1.50.1']
    - gtest[version='>=1.10.0']
    - hypothesis
    - libprotobuf
    - libutf8proc
    - lz4-c
    - make
    - ninja
    - nodejs
    - numpy[version='>=1.16.6']
    - orc
    - pandas
    - pkg-config
    - pytest
    - pytest-faulthandler
    - python=3.8
    - r
    - rapidjson
    - re2
    - s3fs[version='>=2023.10.0']
    - setuptools[version='>=64']
    - setuptools_scm[version='>=8']
    - snappy
    - thrift-cpp[version='>=0.11.0']
    - xsimd
    - zlib
    - zstd


The following NEW packages will be INSTALLED:

  _r-mutex           conda-forge/noarch::_r-mutex-1.0.1-anacondar_1
  aiobotocore        conda-forge/noarch::aiobotocore-2.13.1-pyhd8ed1ab_0
  aiohttp            conda-forge/osx-arm64::aiohttp-3.9.5-py38h336bac9_0
  aioitertools       conda-forge/noarch::aioitertools-0.11.0-pyhd8ed1ab_0
  aiosignal          conda-forge/noarch::aiosignal-1.3.1-pyhd8ed1ab_0
  async-timeout      conda-forge/noarch::async-timeout-4.0.3-pyhd8ed1ab_0
  attrs              conda-forge/noarch::attrs-23.2.0-pyh71513ae_0
  autoconf           conda-forge/osx-arm64::autoconf-2.71-pl5321hcd07c0c_1
  aws-c-auth         conda-forge/osx-arm64::aws-c-auth-0.7.3-hab8b942_3
  aws-c-cal          conda-forge/osx-arm64::aws-c-cal-0.6.2-he70778a_1
  aws-c-common       conda-forge/osx-arm64::aws-c-common-0.9.0-hb547adb_0
  aws-c-compression  conda-forge/osx-arm64::aws-c-compression-0.2.17-he70778a_2
  aws-c-event-stream conda-forge/osx-arm64::aws-c-event-stream-0.3.2-hcf14f3f_0
  aws-c-http         conda-forge/osx-arm64::aws-c-http-0.7.12-he297698_1
  aws-c-io           conda-forge/osx-arm64::aws-c-io-0.13.32-h3c776e5_3
  aws-c-mqtt         conda-forge/osx-arm64::aws-c-mqtt-0.9.5-h83b98fe_1
  aws-c-s3           conda-forge/osx-arm64::aws-c-s3-0.3.14-h24e141d_3
  aws-c-sdkutils     conda-forge/osx-arm64::aws-c-sdkutils-0.1.12-he70778a_1
  aws-checksums      conda-forge/osx-arm64::aws-checksums-0.1.17-he70778a_1
  aws-crt-cpp        conda-forge/osx-arm64::aws-crt-cpp-0.23.0-h04fc39a_5
  aws-sdk-cpp        conda-forge/osx-arm64::aws-sdk-cpp-1.11.68-h0092a47_13
  azure-core-cpp     conda-forge/osx-arm64::azure-core-cpp-1.12.0-hd01fc5c_0
  azure-identity-cpp conda-forge/osx-arm64::azure-identity-cpp-1.8.0-h0a11218_1
  azure-storage-blo~ conda-forge/osx-arm64::azure-storage-blobs-cpp-12.11.0-h77cc766_1
  azure-storage-com~ conda-forge/osx-arm64::azure-storage-common-cpp-12.6.0-h7024f69_1
  azure-storage-fil~ conda-forge/osx-arm64::azure-storage-files-datalake-cpp-12.10.0-h64d02d0_1
  backports          conda-forge/noarch::backports-1.0-pyhd8ed1ab_3
  backports.zoneinfo conda-forge/osx-arm64::backports.zoneinfo-0.2.1-py38hb192615_8
  benchmark          conda-forge/osx-arm64::benchmark-1.8.5-h00cdb27_0
  boost-cpp          conda-forge/osx-arm64::boost-cpp-1.85.0-hca5e981_2
  boto3              conda-forge/noarch::boto3-1.34.131-pyhd8ed1ab_0
  botocore           conda-forge/noarch::botocore-1.34.131-pyge38_1234567_0
  brotli             conda-forge/osx-arm64::brotli-1.1.0-hb547adb_1
  brotli-bin         conda-forge/osx-arm64::brotli-bin-1.1.0-hb547adb_1
  brotli-python      conda-forge/osx-arm64::brotli-python-1.1.0-py38he333c0f_1
  bwidget            conda-forge/osx-arm64::bwidget-1.9.14-hce30654_1
  bzip2              conda-forge/osx-arm64::bzip2-1.0.8-h99b78c6_7
  c-ares             conda-forge/osx-arm64::c-ares-1.32.2-h99b78c6_0
  c-compiler         conda-forge/osx-arm64::c-compiler-1.7.0-h6aa9301_1
  ca-certificates    conda-forge/osx-arm64::ca-certificates-2024.7.4-hf0a4a13_0
  cairo              conda-forge/osx-arm64::cairo-1.18.0-hc6c324b_2
  ccache             conda-forge/osx-arm64::ccache-4.10.1-hbe278c5_0
  cctools            conda-forge/osx-arm64::cctools-986-h4faf515_0
  cctools_osx-arm64  conda-forge/osx-arm64::cctools_osx-arm64-986-h62378fb_0
  cffi               conda-forge/osx-arm64::cffi-1.16.0-py38h73f40f7_0
  clang              conda-forge/osx-arm64::clang-16.0.6-default_h675cc0c_11
  clang-16           conda-forge/osx-arm64::clang-16-16.0.6-default_h5c12605_11
  clang_impl_osx-ar~ conda-forge/osx-arm64::clang_impl_osx-arm64-16.0.6-hc421ffc_18
  clang_osx-arm64    conda-forge/osx-arm64::clang_osx-arm64-16.0.6-h54d7cd3_18
  clangxx            conda-forge/osx-arm64::clangxx-16.0.6-default_h675cc0c_11
  clangxx_impl_osx-~ conda-forge/osx-arm64::clangxx_impl_osx-arm64-16.0.6-hcd7bac0_18
  clangxx_osx-arm64  conda-forge/osx-arm64::clangxx_osx-arm64-16.0.6-h54d7cd3_18
  click              conda-forge/noarch::click-8.1.7-unix_pyh707e725_0
  cloudpickle        conda-forge/noarch::cloudpickle-3.0.0-pyhd8ed1ab_0
  cmake              conda-forge/osx-arm64::cmake-3.30.1-had79d8f_0
  colorama           conda-forge/noarch::colorama-0.4.6-pyhd8ed1ab_0
  compiler-rt        conda-forge/osx-arm64::compiler-rt-16.0.6-h3808999_2
  compiler-rt_osx-a~ conda-forge/noarch::compiler-rt_osx-arm64-16.0.6-h3808999_2
  compilers          conda-forge/osx-arm64::compilers-1.7.0-hce30654_1
  curl               conda-forge/osx-arm64::curl-8.8.0-h653d890_1
  cxx-compiler       conda-forge/osx-arm64::cxx-compiler-1.7.0-h2ffa867_1
  cython             conda-forge/osx-arm64::cython-3.0.10-py38hb9fa5a8_0
  exceptiongroup     conda-forge/noarch::exceptiongroup-1.2.2-pyhd8ed1ab_0
  expat              conda-forge/osx-arm64::expat-2.6.2-hebf3989_0
  font-ttf-dejavu-s~ conda-forge/noarch::font-ttf-dejavu-sans-mono-2.37-hab24e00_0
  font-ttf-inconsol~ conda-forge/noarch::font-ttf-inconsolata-3.000-h77eed37_0
  font-ttf-source-c~ conda-forge/noarch::font-ttf-source-code-pro-2.038-h77eed37_0
  font-ttf-ubuntu    conda-forge/noarch::font-ttf-ubuntu-0.83-h77eed37_2
  fontconfig         conda-forge/osx-arm64::fontconfig-2.14.2-h82840c6_0
  fonts-conda-ecosy~ conda-forge/noarch::fonts-conda-ecosystem-1-0
  fonts-conda-forge  conda-forge/noarch::fonts-conda-forge-1-0
  fortran-compiler   conda-forge/osx-arm64::fortran-compiler-1.7.0-hafb19e3_1
  freetype           conda-forge/osx-arm64::freetype-2.12.1-hadb7bae_2
  fribidi            conda-forge/osx-arm64::fribidi-1.0.10-h27ca646_0
  frozenlist         conda-forge/osx-arm64::frozenlist-1.4.1-py38h336bac9_0
  fsspec             conda-forge/noarch::fsspec-2024.6.1-pyhff2d567_0
  gflags             conda-forge/osx-arm64::gflags-2.2.2-hc88da5d_1004
  gfortran           conda-forge/osx-arm64::gfortran-12.3.0-h1ca8e4b_1
  gfortran_impl_osx~ conda-forge/osx-arm64::gfortran_impl_osx-arm64-12.3.0-hbbb9e1e_1
  gfortran_osx-arm64 conda-forge/osx-arm64::gfortran_osx-arm64-12.3.0-h57527a5_1
  glog               conda-forge/osx-arm64::glog-0.7.1-heb240a5_0
  gmock              conda-forge/osx-arm64::gmock-1.15.0-hce30654_0
  gmp                conda-forge/osx-arm64::gmp-6.3.0-h7bae524_2
  google-cloud-cpp   conda-forge/osx-arm64::google-cloud-cpp-2.5.0-hc461e41_0
  graphite2          conda-forge/osx-arm64::graphite2-1.3.13-hebf3989_1003
  grpc-cpp           conda-forge/osx-arm64::grpc-cpp-1.49.1-h44b9a77_1
  gsl                conda-forge/osx-arm64::gsl-2.7-h6e638da_0
  gtest              conda-forge/osx-arm64::gtest-1.15.0-h420ef59_0
  harfbuzz           conda-forge/osx-arm64::harfbuzz-9.0.0-h1836168_0
  hypothesis         conda-forge/noarch::hypothesis-6.108.3-pyha770c72_0
  icu                conda-forge/osx-arm64::icu-73.2-hc8870d7_0
  idna               conda-forge/noarch::idna-3.7-pyhd8ed1ab_0
  iniconfig          conda-forge/noarch::iniconfig-2.0.0-pyhd8ed1ab_0
  isl                conda-forge/osx-arm64::isl-0.25-h9a09cb3_0
  jmespath           conda-forge/noarch::jmespath-1.0.1-pyhd8ed1ab_0
  krb5               conda-forge/osx-arm64::krb5-1.21.3-h237132a_0
  ld64               conda-forge/osx-arm64::ld64-711-h634c8be_0
  ld64_osx-arm64     conda-forge/osx-arm64::ld64_osx-arm64-711-ha4bd21c_0
  lerc               conda-forge/osx-arm64::lerc-4.0.0-h9a09cb3_0
  libabseil          conda-forge/osx-arm64::libabseil-20220623.0-cxx17_h28b99d4_6
  libasprintf        conda-forge/osx-arm64::libasprintf-0.22.5-h8fbad5d_2
  libblas            conda-forge/osx-arm64::libblas-3.9.0-22_osxarm64_openblas
  libboost           conda-forge/osx-arm64::libboost-1.85.0-h17eb2be_2
  libboost-devel     conda-forge/osx-arm64::libboost-devel-1.85.0-hf450f58_2
  libboost-headers   conda-forge/osx-arm64::libboost-headers-1.85.0-hce30654_2
  libbrotlicommon    conda-forge/osx-arm64::libbrotlicommon-1.1.0-hb547adb_1
  libbrotlidec       conda-forge/osx-arm64::libbrotlidec-1.1.0-hb547adb_1
  libbrotlienc       conda-forge/osx-arm64::libbrotlienc-1.1.0-hb547adb_1
  libcblas           conda-forge/osx-arm64::libcblas-3.9.0-22_osxarm64_openblas
  libclang-cpp16     conda-forge/osx-arm64::libclang-cpp16-16.0.6-default_h5c12605_11
  libcrc32c          conda-forge/osx-arm64::libcrc32c-1.1.2-hbdafb3b_0
  libcurl            conda-forge/osx-arm64::libcurl-8.8.0-h7b6f9a7_1
  libcxx             conda-forge/osx-arm64::libcxx-18.1.8-h167917d_0
  libdeflate         conda-forge/osx-arm64::libdeflate-1.20-h93a5062_0
  libedit            conda-forge/osx-arm64::libedit-3.1.20191231-hc8eb9b7_2
  libev              conda-forge/osx-arm64::libev-4.33-h93a5062_2
  libevent           conda-forge/osx-arm64::libevent-2.1.12-h2757513_1
  libexpat           conda-forge/osx-arm64::libexpat-2.6.2-hebf3989_0
  libffi             conda-forge/osx-arm64::libffi-3.4.2-h3422bc3_5
  libgettextpo       conda-forge/osx-arm64::libgettextpo-0.22.5-h8fbad5d_2
  libgfortran        conda-forge/osx-arm64::libgfortran-5.0.0-13_2_0_hd922786_3
  libgfortran-devel~ conda-forge/noarch::libgfortran-devel_osx-arm64-12.3.0-hc62be1c_3
  libgfortran5       conda-forge/osx-arm64::libgfortran5-13.2.0-hf226fd6_3
  libglib            conda-forge/osx-arm64::libglib-2.80.3-h59d46d9_1
  libgoogle-cloud    conda-forge/osx-arm64::libgoogle-cloud-2.5.0-hc461e41_0
  libgrpc            conda-forge/osx-arm64::libgrpc-1.49.1-h55edf5b_1
  libhiredis         conda-forge/osx-arm64::libhiredis-1.0.2-hbec66e7_0
  libiconv           conda-forge/osx-arm64::libiconv-1.17-h0d3ecfb_2
  libintl            conda-forge/osx-arm64::libintl-0.22.5-h8fbad5d_2
  libjpeg-turbo      conda-forge/osx-arm64::libjpeg-turbo-3.0.0-hb547adb_1
  liblapack          conda-forge/osx-arm64::liblapack-3.9.0-22_osxarm64_openblas
  libllvm16          conda-forge/osx-arm64::libllvm16-16.0.6-haab561b_3
  libnghttp2         conda-forge/osx-arm64::libnghttp2-1.58.0-ha4dd798_1
  libopenblas        conda-forge/osx-arm64::libopenblas-0.3.27-openmp_h517c56d_1
  libpng             conda-forge/osx-arm64::libpng-1.6.43-h091b4b1_0
  libprotobuf        conda-forge/osx-arm64::libprotobuf-3.21.12-ha614eb4_2
  libsqlite          conda-forge/osx-arm64::libsqlite-3.46.0-hfb93653_0
  libssh2            conda-forge/osx-arm64::libssh2-1.11.0-h7a5bd25_0
  libthrift          conda-forge/osx-arm64::libthrift-0.20.0-h33edb3e_0
  libtiff            conda-forge/osx-arm64::libtiff-4.6.0-h07db509_3
  libutf8proc        conda-forge/osx-arm64::libutf8proc-2.8.0-h1a8c8d9_0
  libuv              conda-forge/osx-arm64::libuv-1.48.0-h93a5062_0
  libwebp-base       conda-forge/osx-arm64::libwebp-base-1.4.0-h93a5062_0
  libxml2            conda-forge/osx-arm64::libxml2-2.12.7-h9a80f22_3
  libzlib            conda-forge/osx-arm64::libzlib-1.3.1-hfb2fe0b_1
  llvm-openmp        conda-forge/osx-arm64::llvm-openmp-18.1.8-hde57baf_0
  llvm-tools         conda-forge/osx-arm64::llvm-tools-16.0.6-haab561b_3
  lz4-c              conda-forge/osx-arm64::lz4-c-1.9.4-hb7217d7_0
  m4                 conda-forge/osx-arm64::m4-1.4.18-h642e427_1001
  make               conda-forge/osx-arm64::make-4.3-he57ea6c_1
  mpc                conda-forge/osx-arm64::mpc-1.3.1-h91ba8db_0
  mpfr               conda-forge/osx-arm64::mpfr-4.2.1-h41d338b_1
  multidict          conda-forge/osx-arm64::multidict-6.0.5-py38hb991d35_0
  ncurses            conda-forge/osx-arm64::ncurses-6.5-hb89a1cb_0
  ninja              conda-forge/osx-arm64::ninja-1.12.1-h420ef59_0
  nlohmann_json      conda-forge/osx-arm64::nlohmann_json-3.11.3-hebf3989_0
  nodejs             conda-forge/osx-arm64::nodejs-22.5.1-h3fe1c63_0
  numpy              conda-forge/osx-arm64::numpy-1.24.4-py38ha84db1f_0
  openssl            conda-forge/osx-arm64::openssl-3.3.1-hfb2fe0b_2
  orc                conda-forge/osx-arm64::orc-1.9.0-ha98e9e8_1
  packaging          conda-forge/noarch::packaging-24.1-pyhd8ed1ab_0
  pandas             conda-forge/osx-arm64::pandas-2.0.3-py38hefb543e_1
  pango              conda-forge/osx-arm64::pango-1.54.0-h9ee27a3_1
  pcre2              conda-forge/osx-arm64::pcre2-10.44-h297a79d_0
  perl               conda-forge/osx-arm64::perl-5.32.1-7_h4614cfb_perl5
  pip                conda-forge/noarch::pip-24.0-pyhd8ed1ab_0
  pixman             conda-forge/osx-arm64::pixman-0.43.4-hebf3989_0
  pkg-config         conda-forge/osx-arm64::pkg-config-0.29.2-hde07d2e_1009
  pluggy             conda-forge/noarch::pluggy-1.5.0-pyhd8ed1ab_0
  pycparser          conda-forge/noarch::pycparser-2.22-pyhd8ed1ab_0
  pysocks            conda-forge/noarch::pysocks-1.7.1-pyha2e5f31_6
  pytest             conda-forge/noarch::pytest-8.3.1-pyhd8ed1ab_0
  pytest-faulthandl~ conda-forge/noarch::pytest-faulthandler-2.0.1-py_0
  python             conda-forge/osx-arm64::python-3.8.19-h2469fbe_0_cpython
  python-dateutil    conda-forge/noarch::python-dateutil-2.9.0-pyhd8ed1ab_0
  python-tzdata      conda-forge/noarch::python-tzdata-2024.1-pyhd8ed1ab_0
  python_abi         conda-forge/osx-arm64::python_abi-3.8-4_cp38
  pytz               conda-forge/noarch::pytz-2024.1-pyhd8ed1ab_0
  r                  conda-forge/noarch::r-4.3-r43hd8ed1ab_1007
  r-base             conda-forge/osx-arm64::r-base-4.3.3-h8112bfe_10
  r-boot             conda-forge/noarch::r-boot-1.3_30-r43hc72bb7e_0
  r-class            conda-forge/osx-arm64::r-class-7.3_22-r43h07cda29_2
  r-cluster          conda-forge/osx-arm64::r-cluster-2.1.6-r43h7807725_1
  r-codetools        conda-forge/noarch::r-codetools-0.2_20-r43hc72bb7e_1
  r-foreign          conda-forge/osx-arm64::r-foreign-0.8_87-r43h07cda29_1
  r-kernsmooth       conda-forge/osx-arm64::r-kernsmooth-2.23_24-r43h30ed03b_1
  r-lattice          conda-forge/osx-arm64::r-lattice-0.22_6-r43h07cda29_1
  r-mass             conda-forge/osx-arm64::r-mass-7.3_60.0.1-r43h07cda29_1
  r-matrix           conda-forge/osx-arm64::r-matrix-1.6_5-r43hded8dfa_1
  r-mgcv             conda-forge/osx-arm64::r-mgcv-1.9_1-r43h83b88f1_1
  r-nlme             conda-forge/osx-arm64::r-nlme-3.1_165-r43h7807725_1
  r-nnet             conda-forge/osx-arm64::r-nnet-7.3_19-r43h07cda29_2
  r-recommended      conda-forge/noarch::r-recommended-4.3-r43hd8ed1ab_1006
  r-rpart            conda-forge/osx-arm64::r-rpart-4.1.23-r43h07cda29_1
  r-spatial          conda-forge/osx-arm64::r-spatial-7.3_17-r43h07cda29_1
  r-survival         conda-forge/osx-arm64::r-survival-3.7_0-r43h07cda29_1
  rapidjson          conda-forge/osx-arm64::rapidjson-1.1.0.post20240409-h00cdb27_1
  re2                conda-forge/osx-arm64::re2-2022.06.01-h9a09cb3_1
  readline           conda-forge/osx-arm64::readline-8.2-h92ec313_1
  rhash              conda-forge/osx-arm64::rhash-1.4.4-hb547adb_0
  s3fs               conda-forge/noarch::s3fs-2024.6.1-pyhd8ed1ab_0
  s3transfer         conda-forge/noarch::s3transfer-0.10.2-pyhd8ed1ab_0
  setuptools         conda-forge/noarch::setuptools-71.0.4-pyhd8ed1ab_0
  setuptools-scm     conda-forge/noarch::setuptools-scm-8.1.0-pyhd8ed1ab_0
  setuptools_scm     conda-forge/noarch::setuptools_scm-8.1.0-hd8ed1ab_0
  sigtool            conda-forge/osx-arm64::sigtool-0.1.3-h44b9a77_0
  six                conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
  snappy             conda-forge/osx-arm64::snappy-1.1.10-hd04f947_1
  sortedcontainers   conda-forge/noarch::sortedcontainers-2.4.0-pyhd8ed1ab_0
  tapi               conda-forge/osx-arm64::tapi-1100.0.11-he4954df_0
  thrift-compiler    conda-forge/osx-arm64::thrift-compiler-0.20.0-hebf3989_0
  thrift-cpp         conda-forge/osx-arm64::thrift-cpp-0.20.0-hce30654_0
  tk                 conda-forge/osx-arm64::tk-8.6.13-h5083fa2_1
  tktable            conda-forge/osx-arm64::tktable-2.10-h1e387b8_6
  tomli              conda-forge/noarch::tomli-2.0.1-pyhd8ed1ab_0
  typing-extensions  conda-forge/noarch::typing-extensions-4.12.2-hd8ed1ab_0
  typing_extensions  conda-forge/noarch::typing_extensions-4.12.2-pyha770c72_0
  tzdata             conda-forge/noarch::tzdata-2024a-h0c530f3_0
  urllib3            conda-forge/noarch::urllib3-1.26.19-pyhd8ed1ab_0
  wheel              conda-forge/noarch::wheel-0.43.0-pyhd8ed1ab_1
  wrapt              conda-forge/osx-arm64::wrapt-1.16.0-py38h336bac9_0
  xsimd              conda-forge/osx-arm64::xsimd-13.0.0-h420ef59_0
  xz                 conda-forge/osx-arm64::xz-5.2.6-h57fd34a_0
  yarl               conda-forge/osx-arm64::yarl-1.9.4-py38h336bac9_0
  zlib               conda-forge/osx-arm64::zlib-1.3.1-hfb2fe0b_1
  zstd               conda-forge/osx-arm64::zstd-1.5.6-hb46c0d2_0


Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate test_env
#
# To deactivate an active environment, use
#
#     $ conda deactivate

bash-5.2$ conda activate test_env

(test_env) bash-5.2$ pip install 'oldest-supported-numpy>=0.14'
Collecting oldest-supported-numpy>=0.14
  Using cached oldest_supported_numpy-2023.12.21-py3-none-any.whl.metadata (9.8 kB)
Collecting numpy==1.21.0 (from oldest-supported-numpy>=0.14)
  Using cached numpy-1.21.0-cp38-cp38-macosx_11_0_arm64.whl.metadata (2.0 kB)
Using cached oldest_supported_numpy-2023.12.21-py3-none-any.whl (4.9 kB)
Using cached numpy-1.21.0-cp38-cp38-macosx_11_0_arm64.whl (12.1 MB)
Installing collected packages: numpy, oldest-supported-numpy
  Attempting uninstall: numpy
    Found existing installation: numpy 1.24.4
    Uninstalling numpy-1.24.4:
      Successfully uninstalled numpy-1.24.4
Successfully installed numpy-1.21.0 oldest-supported-numpy-2023.12.21
(test_env) bash-5.2$ pip install --no-deps --no-build-isolation -vv .
Using pip 24.0 from /opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip (python 3.8)
Non-user install because site-packages writeable
Created temporary directory: /private/var/folders/fz/ydy9vm4d6qbgxw11ys6j_kfh0000gn/T/pip-build-tracker-dgxj782e
Initialized build tracking at /private/var/folders/fz/ydy9vm4d6qbgxw11ys6j_kfh0000gn/T/pip-build-tracker-dgxj782e
Created build tracker: /private/var/folders/fz/ydy9vm4d6qbgxw11ys6j_kfh0000gn/T/pip-build-tracker-dgxj782e
Entered build tracker: /private/var/folders/fz/ydy9vm4d6qbgxw11ys6j_kfh0000gn/T/pip-build-tracker-dgxj782e
Created temporary directory: /private/var/folders/fz/ydy9vm4d6qbgxw11ys6j_kfh0000gn/T/pip-install-a52um_oj
Created temporary directory: /private/var/folders/fz/ydy9vm4d6qbgxw11ys6j_kfh0000gn/T/pip-ephem-wheel-cache-sc513tbd
Processing /Users/austin/Desktop/arrow/python
  Added file:///Users/austin/Desktop/arrow/python to build tracker '/private/var/folders/fz/ydy9vm4d6qbgxw11ys6j_kfh0000gn/T/pip-build-tracker-dgxj782e'
  Created temporary directory: /private/var/folders/fz/ydy9vm4d6qbgxw11ys6j_kfh0000gn/T/pip-modern-metadata-vqqu5kpa
  Running command Preparing metadata (pyproject.toml)
  Preparing metadata (pyproject.toml) ... done
ERROR: Exception:
Traceback (most recent call last):
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
    return func(self, options, args)
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 377, in run
    requirement_set = resolver.resolve(
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 76, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 534, in collect_root_requirements
    reqs = list(
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 490, in _make_requirements_from_install_req
    cand = self._make_base_candidate_from_link(
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 228, in _make_base_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 290, in __init__
    super().__init__(
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
    self.dist = self._prepare()
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 222, in _prepare
    dist = self._prepare_distribution()
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 301, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 525, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 640, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 71, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 67, in prepare_distribution_metadata
    self.req.prepare_metadata()
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 579, in prepare_metadata
    self.metadata_directory = generate_metadata(
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/operations/build/metadata.py", line 35, in generate_metadata
    distinfo_dir = backend.prepare_metadata_for_build_wheel(metadata_dir)
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_internal/utils/misc.py", line 766, in prepare_metadata_for_build_wheel
    return super().prepare_metadata_for_build_wheel(
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 186, in prepare_metadata_for_build_wheel
    return self._call_hook('prepare_metadata_for_build_wheel', {
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 321, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
    obj = import_module(mod_path)
  File "/opt/miniconda3/envs/test_env/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/setuptools/__init__.py", line 21, in <module>
    from .dist import Distribution
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/setuptools/dist.py", line 29, in <module>
    from . import _entry_points
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/setuptools/_entry_points.py", line 6, in <module>
    from jaraco.text import yield_lines
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/setuptools/_vendor/jaraco/text/__init__.py", line 12, in <module>
    from jaraco.context import ExceptionTrap
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/setuptools/_vendor/jaraco/context.py", line 17, in <module>
    from backports import tarfile
ImportError: cannot import name 'tarfile' from 'backports' (/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/backports/__init__.py)

Remote version of pip: 24.1.2
Local version of pip:  24.0
Was pip installed by pip? False
Removed file:///Users/austin/Desktop/arrow/python from build tracker '/private/var/folders/fz/ydy9vm4d6qbgxw11ys6j_kfh0000gn/T/pip-build-tracker-dgxj782e'
Removed build tracker: '/private/var/folders/fz/ydy9vm4d6qbgxw11ys6j_kfh0000gn/T/pip-build-tracker-dgxj782e'

[jaraco]

I used the continuumio/anaconda3 docker image to replicate the issue. Thanks for the instructions. I noted that the issue can be more succinctly encountered by simply importing setuptools instead of invoking pip:

- pip install --no-deps --no-build-isolation -vv .
+ python -c "import setuptools"

The problem appears to be rooted in how the backports package is installed by conda. It has installed the backports.zoneinfo package, which I confirm uses pkgutil-style namespace packages, but when installed by conda, the namespace functionality has been removed:

(test_env) root@57192a6ec5fa:/arrow/python# ls -la /opt/conda/envs/test_env/lib/python3.8/site-packages/backports/__init__.py
-rw-r--r-- 2 root root 0 Nov 22  2022 /opt/conda/envs/test_env/lib/python3.8/site-packages/backports/__init__.py

The file is empty. That means that any other backports package, even if present on sys.path, will be masked (the machinery has been removed that enables subpackages of backports to exist in multiple places), as documented in the backports project.

This seems like a bug in conda or in how it's packaged up the backports or backports-zoneinfo packages.

Both workarounds mentioned in #4476 (comment) should bypass the issue. I confirmed that doing pip install backports.tarfile, setuptools no longer fails with the error.

Another workaround, closer to a solution, would be for conda to package Setuptools with its dependencies (those declared in setuptools[core]), such that when installing Setuptools 71 or later, all of its dependencies are installed naturally instead of relying on the vendored fallback.

Please raise the issue with conda, but let me know if there's something more that you or the conda team think Setuptools should be doing differently.

[glatterf42]

Just passed it on to conda, please feel free to jump in with any corrections/additional information :)

[Avasam]

Would jaraco/jaraco.text#15 just sidestep the vendoring issue entirely ?

I know it's more attacking the symptoms than the root cause, but reducing extraneous dependencies isn't a bad thing in the first place.

[jaraco]

Would jaraco/jaraco.text#15 just sidestep the vendoring issue entirely ?

I know it's more attacking the symptoms than the root cause, but reducing extraneous dependencies isn't a bad thing in the first place.

It would sidestep the issue, but it would also mask the problem and leave it as a lingering implicit constraint that's liable to be violated in the future. Let's solve the root problem.

[glatterf42]

@jaraco this is just to confirm: with conda-forge/backports-feedstock#7 merged, I don't experience the issue described here anymore :)