Updating requirements.in with minimal changes to requirements.txt

[cneberg]

What's the problem this feature will solve?

I'd like to know the process of adding a single new dependency to requirements.in and requirements.txt with minimal version changes to support the new dependency. ie I don't want all requirements upgraded - just make the new module work in the least invasive way possible.

Related to #1358 and pull request !1369 - although I actually want to change the requirements.txt.

I've tried just running pip-compile again after updating requirements.in and it adds the new requirements.txt so it might do what I want by default - but I can't tell. Is it trying to upgrade everything, or do minimal upgrade to add the changes? Please clarify and possibly update the docs of what should happen in this instance. So in short does a new entry or modified entry in requirements.in requirement imply the --upgrade flag or just something more similar to an --upgrade-module flag for just the modified lines?

Additional context

The use case for this is the python app I'm working on has dozens of dependencies, and anyone of of much larger team can create a new module, and add a new dependency to support their module. They are not experts in the whole system, just their piece - so I'd like to limit unintended side effects/breakage from upgrading all dependencies. A more seasoned team member can upgrade all dependencies on a different schedule.

[atugushev]

I've tried just running pip-compile again after updating requirements.in and it adds the new requirements.txt so it might do what I want by default - but I can't tell. Is it trying to upgrade everything, or do minimal upgrade to add the changes?

If you add a dependency and run pip-compile it'll add a new dependency and its sub-dependencies to the requirements.txt as well as preserve versions for the rest of the dependencies. So, as you've said, it does minimal upgrade to add the changes.

We have a section in README for updating dependencies. Would be nice to improve our docs and clarify adding new dependencies process. Feel free to open a PR.

[tmarice]

This is something I'm also wondering, and which isn't completely clear from the documentation.

So, as you've said, it does minimal upgrade to add the changes.

Let's look at an example:
At time t0, we have the following state:

# requirements.in
A==1.0.0  # depends on A_B>=1.0.0<2


# requirements.txt
A==1.0.0
A_B==1.0.0

At time t1, a version 1.1.0 of package A_B is released.

At time t2, we add a new package C to our requirements.in:

# requirements.in
A==1.0.0  # depends on A_B>=1.0.0<2
C==1.0.0  # depends on C_D>=1.0.0<2

If we run pip-compile after this change, will we get:

# requirements.txt

A==1.0.0
A_B==1.0.0
C==1.0.0  # newly added
C_D==1.0.0  # newly added

or

# requirements.txt

A==1.0.0
A_B==1.1.0  # upgraded from 1.0.0
C==1.0.0  # newly added
C_D==1.0.0  # newly added

?

[AndydeCleyre]

@tmarice It should be the former. Here's a real test-able example:

$ echo yamlpath >reqs.in
$ pip-compile --no-header reqs.in

ruamel.yaml.clib==0.2.2
    # via ruamel.yaml
ruamel.yaml==0.17.4
    # via yamlpath
yamlpath==3.5.0
    # via -r reqs.in

$ sed -i 's/0\.17\.4/0\.17\.3/' reqs.txt
$ cat reqs.txt

ruamel.yaml.clib==0.2.2
    # via ruamel.yaml
ruamel.yaml==0.17.3
    # via yamlpath
yamlpath==3.5.0
    # via -r reqs.in

$ pip-compile --no-header reqs.in

ruamel.yaml.clib==0.2.2
    # via ruamel.yaml
ruamel.yaml==0.17.3
    # via yamlpath
yamlpath==3.5.0
    # via -r reqs.in

$ echo plumbum >>reqs.in
$ pip-compile --no-header reqs.in

plumbum==1.7.0
    # via -r reqs.in
ruamel.yaml.clib==0.2.2
    # via ruamel.yaml
ruamel.yaml==0.17.3
    # via yamlpath
yamlpath==3.5.0
    # via -r reqs.in

That said, things will get upgraded if they need to be, such as an old version being pulled from PyPI, or new packages requiring newer versions of existing deps.

[AndydeCleyre]

@tmarice @cneberg

Do the doc changes from @mikepqr satisfactorily address this issue?

[AndydeCleyre]

As what's being asked for is already the default behavior, and it's been >6 months since any new info, and the docs have improved on this point, I'm going to close this, but please do write again if the heart of the issue is not fully addressed.

Thanks!