Skip to content

Commit

Permalink
Stop setting CPATH and PKG_CONFIG_PATH at launch time
Browse files Browse the repository at this point in the history
For parity with the automatic build related env vars set by `lifecycle`,
are only set at build time, and not also at launch time:
https://github.com/buildpacks/spec/blob/main/buildpack.md#layer-paths
  • Loading branch information
edmorley committed Jul 29, 2024
1 parent 9fb8fd5 commit 47cd56d
Show file tree
Hide file tree
Showing 3 changed files with 13 additions and 18 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed

- Stopped manually creating a `src` directory inside the Pip dependencies layer. Pip will create the directory itself if needed (when there are editable VCS dependencies). ([#228](https://github.com/heroku/buildpacks-python/pull/228))
- Stopped setting `CPATH` and `PKG_CONFIG_PATH` at launch time. ([#231](https://github.com/heroku/buildpacks-python/pull/231))

## [0.12.1] - 2024-07-15

Expand Down
28 changes: 12 additions & 16 deletions src/layers/python.rs
Original file line number Diff line number Diff line change
Expand Up @@ -292,10 +292,6 @@ fn cache_invalidation_reasons(

/// Environment variables that will be set by this layer.
fn generate_layer_env(layer_path: &Path, python_version: &PythonVersion) -> LayerEnv {
// Several of the env vars below are technically build-time only vars, however, we use
// `Scope::All` instead of `Scope::Build` to reduce confusion if pip install commands
// are used at runtime when debugging.
//
// Remember to force invalidation of the cached layer if these env vars ever change.
LayerEnv::new()
// We have to set `CPATH` explicitly, since:
Expand All @@ -304,15 +300,15 @@ fn generate_layer_env(layer_path: &Path, python_version: &PythonVersion) -> Laye
// - Older setuptools cannot find this directory without `CPATH` being set:
// https://github.com/pypa/setuptools/issues/3657
.chainable_insert(
Scope::All,
Scope::Build,
ModificationBehavior::Prepend,
"CPATH",
layer_path.join(format!(
"include/python{}.{}",
python_version.major, python_version.minor
)),
)
.chainable_insert(Scope::All, ModificationBehavior::Delimiter, "CPATH", ":")
.chainable_insert(Scope::Build, ModificationBehavior::Delimiter, "CPATH", ":")
// Ensure Python uses a Unicode locate, to prevent the issues described in:
// https://github.com/docker-library/python/pull/570
.chainable_insert(
Expand All @@ -324,7 +320,7 @@ fn generate_layer_env(layer_path: &Path, python_version: &PythonVersion) -> Laye
// We use a curated Pip version, so disable the update check to speed up Pip invocations,
// reduce build log spam and prevent users from thinking they need to manually upgrade.
// This uses an env var (rather than the `--disable-pip-version-check` arg) so that it also
// takes effect for any pip invocations in later buildpacks or when debugging at runtime.
// takes effect for any pip invocations in later buildpacks or when debugging at run-time.
.chainable_insert(
Scope::All,
ModificationBehavior::Override,
Expand All @@ -334,26 +330,26 @@ fn generate_layer_env(layer_path: &Path, python_version: &PythonVersion) -> Laye
// We have to set `PKG_CONFIG_PATH` explicitly, since the automatic path set by lifecycle/libcnb
// is `<layer>/pkgconfig/`, whereas Python's pkgconfig files are at `<layer>/lib/pkgconfig/`.
.chainable_insert(
Scope::All,
Scope::Build,
ModificationBehavior::Prepend,
"PKG_CONFIG_PATH",
layer_path.join("lib/pkgconfig"),
)
.chainable_insert(
Scope::All,
Scope::Build,
ModificationBehavior::Delimiter,
"PKG_CONFIG_PATH",
":",
)
// Our Python runtime is relocated (installed into a different location to which is was
// originally compiled) which Python itself handles well, since it recalculates its actual
// location at startup:
// https://docs.python.org/3.11/library/sys_path_init.html
// https://docs.python.org/3/library/sys_path_init.html
// However, the uWSGI package uses the wrong `sysconfig` APIs so tries to reference the old
// compile location, unless we override that by setting `PYTHONHOME`:
// https://github.com/unbit/uwsgi/issues/2525
// In addition, some legacy apps have `PYTHONHOME` set to an invalid value, so if we did not
// set it explicitly here, Python would fail to run both during the build and at runtime.
// set it explicitly here, Python would fail to run both during the build and at run-time.
.chainable_insert(
Scope::All,
ModificationBehavior::Override,
Expand All @@ -379,7 +375,7 @@ fn generate_layer_env(layer_path: &Path, python_version: &PythonVersion) -> Laye
// files to a fixed value in order to improve the determinism of builds:
// https://buildpacks.io/docs/features/reproducibility/#consequences-and-caveats
//
// At runtime, this then means the timestamps embedded in the `.pyc` files no longer match
// At run-time, this then means the timestamps embedded in the `.pyc` files no longer match
// the timestamps of the original `.py` files, causing Python to have to regenerate the
// bytecode, and so losing any benefit of having kept the `.pyc` files in the image.
//
Expand All @@ -392,12 +388,12 @@ fn generate_layer_env(layer_path: &Path, python_version: &PythonVersion) -> Laye
//
// Instead, we use the hash-based cache files mode added in Python 3.7+, which embeds a hash
// of the original `.py` file in the `.pyc` file instead of the timestamp:
// https://docs.python.org/3.11/reference/import.html#pyc-invalidation
// https://docs.python.org/3/reference/import.html#pyc-invalidation
// https://peps.python.org/pep-0552/
//
// This mode can be enabled by passing `--invalidation-mode checked-hash` to `compileall`,
// or via the `SOURCE_DATE_EPOCH` env var:
// https://docs.python.org/3.11/library/compileall.html#cmdoption-compileall-invalidation-mode
// https://docs.python.org/3/library/compileall.html#cmdoption-compileall-invalidation-mode
//
// Note: Both the CLI args and the env var only apply to usages of `compileall` or `py_compile`,
// and not `.pyc` generation as part of Python importing a file during normal operation.
Expand Down Expand Up @@ -568,10 +564,10 @@ mod tests {
assert_eq!(
utils::environment_as_sorted_vector(&layer_env.apply(Scope::Launch, &base_env)),
[
("CPATH", "/layer-dir/include/python3.11:/base"),
("CPATH", "/base"),
("LANG", "C.UTF-8"),
("PIP_DISABLE_PIP_VERSION_CHECK", "1"),
("PKG_CONFIG_PATH", "/layer-dir/lib/pkgconfig:/base"),
("PKG_CONFIG_PATH", "/base"),
("PYTHONHOME", "/layer-dir"),
("PYTHONUNBUFFERED", "1"),
]
Expand Down
2 changes: 0 additions & 2 deletions tests/pip_test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -59,12 +59,10 @@ fn pip_basic_install_and_cache_reuse() {
assert_contains!(
command_output.stdout,
&formatdoc! {"
CPATH=/layers/heroku_python/python/include/python3.12
LANG=C.UTF-8
LD_LIBRARY_PATH=/layers/heroku_python/python/lib:/layers/heroku_python/dependencies/lib
PATH=/layers/heroku_python/python/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PIP_DISABLE_PIP_VERSION_CHECK=1
PKG_CONFIG_PATH=/layers/heroku_python/python/lib/pkgconfig
PYTHONHOME=/layers/heroku_python/python
PYTHONUNBUFFERED=1
PYTHONUSERBASE=/layers/heroku_python/dependencies
Expand Down

0 comments on commit 47cd56d

Please sign in to comment.