poetry lock (no poetry.lock) pins different versions than poetry update

[JacobHayes]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS Monterey

• Poetry version: Poetry (version 1.2.0)

• Contents of your pyproject.toml file:

pyproject.toml

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "debug-lock"
version = "0.1.0"
description = ""
authors = ["Jacob Hayes <[email protected]>"]
readme = "README.md"
packages = [{include = "debug_lock"}]

[tool.poetry.dependencies]
coiled = "*"
python = "^3.9"

[tool.poetry.group.dev.dependencies]
dask-labextension = "*"
jupyter = "*"
jupyterlab = "*"

Issue

With some sets of dependencies poetry lock and poetry update result in different poetry.lock files with vastly different versions. Running poetry lock without a poetry.lock will pin some version (eg: coiled==0.0.56), then an immediate poetry update will pick different versions (eg: coiled==0.2.27), and then a subsequent poetry lock will pin the old version again (eg: coiled==0.0.56).

With the pyproject embedded above, I can reproduce this with:

Test script

#!/usr/bin/env bash

set -o errexit
set -o nounset
set -o pipefail
set -o xtrace

rm poetry.lock
poetry lock
grep -U1 'name = "coiled"' poetry.lock
poetry update
grep -U1 'name = "coiled"' poetry.lock
poetry lock
grep -U1 'name = "coiled"' poetry.lock

$ rm poetry.lock
$ poetry lock
Updating dependencies
Resolving dependencies... (6.2s)

Writing lock file
$ grep -U1 'name = "coiled"' poetry.lock
[[package]]
name = "coiled"
version = "0.0.56"
$ poetry update
Updating dependencies
Resolving dependencies... (3.8s)

Writing lock file

No dependencies to install or update
$ grep -U1 'name = "coiled"' poetry.lock
[[package]]
name = "coiled"
version = "0.2.27"
$ poetry lock
Updating dependencies
Resolving dependencies... (6.2s)

Writing lock file
$ grep -U1 'name = "coiled"' poetry.lock
[[package]]
name = "coiled"
version = "0.0.56"

I know I can set explicit version ranges in my pyproject.toml, but the instability also causes issues when different team members update deps with poetry lock vs poetry update. Additionally, the lock docs say:

By default, this will lock all dependencies to the latest available compatible versions.

but this must not be true if a poetry update gives newer packages without erroring about any incompatibilities.

[dimbleby]

The latest versions of all the packages that you've asked for are not compatible. In such cases it's a coin-flip whether you get recent a and old b, or vice versa.

Per #2883 (comment), IMO this is not a bug and if you care about the difference you can add an explicit requirement to say so.

Anyway this is duplicate of lots of others, some still open: #2883, #3551, #3779, #4243

[JacobHayes]

In such cases it's a coin-flip whether you get recent a and old b, or vice versa.

If it is a coin flip (I'm reading that as "random"), then why does lock reliably give one result (and not waiver on subsequent runs) when update gives another?

I would expect the code paths to be equivalent and poetry update to basically be a fancy poetry lock && poetry install, though I acknowledge that's probably naive.

this is duplicate of lots of others

What the heck, apparently my Google and GitHub searching wasn't effective (perhaps drowned out by a million other things having those keywords - how'd you find them again?).

I'll close this one and chime in on others, but despite perhaps "not being a bug" in poetry, it caused a bug when a poetry lock downgraded to some things from >1 year ago (but yes, they were unpinned as the project post-dates the downgraded dep version). I acknowledge this is an incredibly hard problem, but the instability made it harder to debug and fix.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.