Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure private PyPi repositories in the environment #471

5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,14 +156,15 @@ The default category is `main`.
[environment.yml][envyaml], using a vendored copy of [Poetry's][poetry] dependency solver.

### private pip repositories
Right now `conda-lock` only supports [legacy](https://warehouse.pypa.io/api-reference/legacy.html) pypi repos with basic auth. Most self-hosted repositories like Nexus, Artifactory etc. use this. To use this feature, add your private repo into Poetry's config _including_ the basic auth in the url:
Right now `conda-lock` only supports [legacy](https://warehouse.pypa.io/api-reference/legacy.html) pypi repos with basic auth. Most self-hosted repositories like Nexus, Artifactory etc. use this. You can configure private repositories by setting the `CONDA_LOCK_PYPI_REPOSITORY_<custom-repository>` environment variable to the URL of the private repo, _including_ any basic auth. For example:

```bash
poetry config repositories.foo https://username:password@foo.repo/simple/
export CONDA_LOCK_PYPI_REPOSITORY_EXAMPLE="https://username:password@example.repo/simple"
```

The private repo will be used in addition to `pypi.org`. For projects using `pyproject.toml`, it is possible to [disable `pypi.org` entirely](#disabling-pypiorg).


### --dev-dependencies/--no-dev-dependencies

By default conda-lock will include dev dependencies in the specification of the lock (if the files that the lock
Expand Down
16 changes: 16 additions & 0 deletions conda_lock/pypi_solver.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
import os
import re
import sys

from functools import lru_cache
from pathlib import Path
from typing import TYPE_CHECKING, Dict, List, Optional
from urllib.parse import urldefrag, urlsplit, urlunsplit
Expand Down Expand Up @@ -373,6 +375,9 @@ def _prepare_repositories_pool(allow_pypi_requests: bool) -> Pool:
factory = Factory()
config = factory.create_config()
repos = [
factory.create_legacy_repository({"name": name, "url": url}, config)
for name, url in _parse_repositories_from_environment().items()
] + [
Comment on lines +378 to +380
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please write a test for _prepare_repositories_pool to exercise this functionality?

factory.create_legacy_repository(
{"name": source[0], "url": source[1]["url"]}, config
)
Comment on lines 381 to 383
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've left the previous method here so this is not a breaking change.

However since this is exposing an implementation detail as an interface, it might be worth adding a deprecation warning?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if we're ready yet to deprecate it. I'm very eager to ditch anything Poetry-related, but setting up environment variables can be a pain, and until we get something better in place, some people may prefer the Poetry interface.

Expand All @@ -383,10 +388,21 @@ def _prepare_repositories_pool(allow_pypi_requests: bool) -> Pool:
return Pool(repositories=[*repos])


@lru_cache(maxsize=1)
def _parse_repositories_from_environment() -> Dict[str, str]:
env_prefix = "CONDA_LOCK_PYPI_REPOSITORY_"
return {
key[len(env_prefix) :].lower(): value
for key, value in os.environ.items()
if key.startswith(env_prefix)
}


def _strip_auth(url: str) -> str:
"""Strip HTTP Basic authentication from a URL."""
parts = urlsplit(url, allow_fragments=True)
# Remove everything before and including the last '@' character in the part
# between 'scheme://' and the subsequent '/'.
netloc = parts.netloc.split("@")[-1]
return urlunsplit((parts.scheme, netloc, parts.path, parts.query, parts.fragment))