Allow locking nim version in nimble.lock #1017

yyoncho · 2022-08-08T13:20:09Z

Fixes nimble install doesn't respect newly downloaded packages.json #953

Allow having nim as locked dependency.

I will add unit tests once we agree on the approach and once nimble related
changes in nim are merged (I will link the PR in comment). Ditto for the
documentation.

Here it is the flow:

nimble develop nim
nimble lock

After that nimble install and nimble build commands will use the locked
nim version

dom96

Seems fair. Though can you elaborate what will end up in the lock file when nimble lock is called?

dom96 · 2022-08-08T16:33:34Z

src/nimblepkg/packageparser.nim

+    if not pkgInfo.basicInfo.name.isNim:
+      if dir.len == 0:


Nit: instead of adding another nesting to this if, please just put the condition into one of the existing if statements.

In this case, I will have to add one if in the else clause as well. It will be:

if not foo and bar: ... elif not foo ...

To me this looks less "clean. Let me know what you think.

ahh, in that case if you wouldn't mind refactoring it out into its own proc that would be the best of both worlds :)

Can you elaborate?

Move the whole if dir.len == 0: if statement (its body and the else) into a validatePackageStructure proc, then only call it under the if not pkgInfo.basicInfo.name.isNim:

Ok, will do.

dom96 · 2022-08-08T16:34:29Z

src/nimblepkg/packageinfo.nim

@@ -530,6 +528,9 @@ proc hash*(x: PackageInfo): Hash =
 proc getNameAndVersion*(pkgInfo: PackageInfo): string =
  &"{pkgInfo.basicInfo.name}@{pkgInfo.basicInfo.version}"

+proc isNim*(name: string): bool =
+  result = name == "nim" or name == "nimrod"


Nim hasn't been known by that name in long enough to exclude it IMO

Ok, will do. I will rename a few functions using nimrod in their name as well.

Nim hasn't been known by that name in long enough to exclude it IMO

I reverted that part - nimrod is used in the tests so I don't think that this PR is the place to go and fix them.

dom96 · 2022-08-08T16:35:38Z

src/nimblepkg/tools.nim

 proc getNimrodVersion*(options: Options): Version =
-  let vOutput = doCmdEx(getNimBin(options).quoteShell & " -v").output
+  let vOutput = doCmdEx(getNimBin(options) & " -v").output


if you're gonna do that then rename getNimBin to getNimBinQuoted

alternatively just revert this change, quoteShell is much better closest to doCmdExe to ensure we use it (plus other programs like choosenim might be using getNimBin and won't be expecting quoteShell)

I will revert this refactoring and keep the quoteShell in the caller.

src/nimble.nim

yyoncho · 2022-08-09T07:15:27Z

Seems fair. Though can you elaborate what will end up in the lock file when nimble lock is called?

Here is what it looks like(I have patched packages.json locally to make that work until we have an installable version in main repo)

  {
  "version": 1,
  "packages": {
    "nim": {
      "version": "1.7.1",
      "vcsRevision": "afc3c24ff713453aea869df0df93e591e9947f24",
      "url": "https://github.com/yyoncho/Nim",
      "downloadMethod": "git",
      "dependencies": [],
      "checksums": {
        "sha1": "33781958ab4835a413519059a03217ee81e56dc3"
      }
    }
  }
}

yyoncho · 2022-08-09T07:28:22Z

@dom96 please disregard this PR - refer to nim-lang/Nim#20179 (comment)

I have to revisit the implementation.

dom96 · 2022-08-09T12:28:03Z

Suggested a solution :)

yyoncho · 2022-08-09T12:50:06Z

@dom96 applied your comments(hopefully I haven't missed anything).

dom96 · 2022-08-10T12:33:41Z

Just for reference why I'm not accepting this yet: We need to wait for @Araq to give his approval on the Nim PR. So I would say this PR is blocked on that.

But also would be nice for @zah to take a look.

- see nim-lang/nimble#1017 and nim-lang/Nim#20179

arnetheduck · 2022-08-11T11:38:47Z

Here it is the flow:

why is nimble develop needed? nim should simply always be part of the lock file, just like any other dep?

yyoncho · 2022-08-11T12:00:45Z

Here it is the flow:

why is nimble develop needed? nim should simply always be part of the lock file, just like any other dep?

I don't have a strong preference for that. The idea was to avoid managing nim by accident and having it more explicit. Zah suggestion was to do that with flag, but I did it with nimble develop which will help fixing the version as well.

* Rename compiler to nim - see nim-lang/nimble#1017 and nim-lang/Nim#20179 * Better description of the Nim package Co-authored-by: zah <[email protected]>

yyoncho · 2022-09-02T09:16:47Z

As per my discussion with @arnetheduck I was trying to add flag to make locking possible with nimble --lock-nim lock but that will download the latest tagged version which does not have the proper package structure.

@arnetheduck I think that this feature should be held on till we tag nim version, WDYT?

arnetheduck · 2022-09-02T15:24:42Z

@arnetheduck I think that this feature should be held on till we tag nim version, WDYT?

I believe this is the case for any nimble dependency, ie nimble will happily download any random version of anything, depending on what is in the repo at the time and what the user happens to have in their global cache.

I think the two issues can be addressed separately - in particular, I imagine that if an application wants a specific nim version, it would indicate so in its nimble file - until nimble has a proper dep resolver and a nimble update package command that allows updating a particular dependency to a particular version, users can set nim#somehash as their version requirement and it should hopefully force nimble to do the right thing (or die trying)

yyoncho · 2023-01-30T13:08:16Z

Depends on nim-lang/Nim#21313 and nim-lang/Nim#21314

This caused the lockfiles to ignore certain dependencies

Removes previous hack that needed to read current nimble file state This changes the lock file structure, but compaitability shouldn't be affected

Using nimlangserver since that was that project I first noticed it with

Will now investigate other tests

Force yes in tests Fix tests Go back to non destructive uninstall for deps

Error says it is due to stdlib but it works a few lines later

- Fixes nim-lang#953 Allow having nim as locked dependency. - I will add unit tests once we agree on the approach and once nimble related changes in nim are merged (I will link the PR in comment). Ditto for the documentation. In order that change to work we have to add nim package in nimble packages repo and also add alias compiler -> nim to avoid breaking backward compatibility. Here it is the flow: ``` bash nimble develop nim nimble lock ``` After that `nimble install` and `nimble build` commands will use the locked `nim` version

yyoncho · 2023-02-01T11:51:48Z

Fails with:

Verifying dependencies for [email protected] /home/runner/work/nimble/nimble/src/nimble.nim(2158) nimble /home/runner/work/nimble/nimble/src/nimble.nim(2111) doAction /home/runner/work/nimble/nimble/src/nimble.nim(1695) lock /home/runner/work/nimble/nimble/src/nimble.nim(1528) validateDevelopDependenciesVersionRanges
Error:  Some of the develop mode dependencies are with versions which are not in the required by other package's Nimble file range.

@Araq It is good now.

dom96 reviewed Aug 8, 2022

View reviewed changes

yyoncho force-pushed the nim-as-dep branch 4 times, most recently from cf302e2 to 045134d Compare August 9, 2022 12:49

yyoncho mentioned this pull request Aug 9, 2022

Rename compiler to nim nim-lang/packages#2317

Merged

yyoncho force-pushed the nim-as-dep branch 2 times, most recently from 3f8d087 to b4b2ace Compare August 10, 2022 12:00

yyoncho force-pushed the nim-as-dep branch 2 times, most recently from 72cd1c3 to 11f85b7 Compare August 10, 2022 13:36

yyoncho added a commit to yyoncho/packages that referenced this pull request Aug 11, 2022

Rename compiler to nim

7dddf12

- see nim-lang/nimble#1017 and nim-lang/Nim#20179

yyoncho mentioned this pull request Aug 11, 2022

move -d:nimRawSetjmp to nim.cfg vacp2p/nim-libp2p#732

Merged

yyoncho force-pushed the nim-as-dep branch from 11f85b7 to 567f6a5 Compare August 11, 2022 10:07

Araq pushed a commit to nim-lang/packages that referenced this pull request Aug 24, 2022

Rename compiler to nim (#2317)

7b40703

* Rename compiler to nim - see nim-lang/nimble#1017 and nim-lang/Nim#20179 * Better description of the Nim package Co-authored-by: zah <[email protected]>

yyoncho force-pushed the nim-as-dep branch 3 times, most recently from 8c4c6e3 to 28ca4ea Compare September 12, 2022 10:46

yyoncho force-pushed the nim-as-dep branch from 28ca4ea to a1bda0f Compare December 21, 2022 13:58

yyoncho force-pushed the nim-as-dep branch from b5e7daa to 1c80809 Compare January 3, 2023 09:43

yyoncho marked this pull request as draft January 30, 2023 08:18

yyoncho force-pushed the nim-as-dep branch 5 times, most recently from 8041276 to 5f4813a Compare January 30, 2023 09:57

yyoncho force-pushed the nim-as-dep branch from 5f4813a to 0ad1a70 Compare January 30, 2023 13:09

yyoncho mentioned this pull request Feb 1, 2023

Do not use system.NimVersion for nim.nimble nim-lang/Nim#21313

Closed

ire4ever1190 and others added 12 commits February 1, 2023 12:22

Fix problem were URL's weren't being resolved

2e0e4ef

This caused the lockfiles to ignore certain dependencies

Tasks are now stored as subgraphs in lock files

92bb332

Removes previous hack that needed to read current nimble file state This changes the lock file structure, but compaitability shouldn't be affected

Add test case for regression

57f6695

Using nimlangserver since that was that project I first noticed it with

Task dependency tests are passing again

451b63a

Will now investigate other tests

Properly merge lock files

45d5ed5

Force yes in tests Fix tests Go back to non destructive uninstall for deps

Fix package list getting reset after every call

d1a007d

Check satisisfaction

aae8561

Remove unused variable

8354713

Turn off ProveInit for line

0f6656e

Error says it is due to stdlib but it works a few lines later

Force using nim defined in the lock file for tasks

dc7c3db

Add --use-system-nim setting

b5fa809

yyoncho force-pushed the nim-as-dep branch from 0ad1a70 to b5fa809 Compare February 1, 2023 10:22

Fix the tests on Windows

5dc23bc

yyoncho marked this pull request as ready for review February 1, 2023 11:10

Implement support for getPaths()/getPathsClause

8e5501b

yyoncho closed this Feb 13, 2023

yyoncho reopened this Feb 13, 2023

zah merged commit edd6b3e into nim-lang:master Feb 13, 2023

zah mentioned this pull request Feb 13, 2023

Fix lock file regression #1047

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow locking nim version in nimble.lock #1017

Allow locking nim version in nimble.lock #1017

yyoncho commented Aug 8, 2022

dom96 left a comment

dom96 Aug 8, 2022

yyoncho Aug 8, 2022

dom96 Aug 8, 2022

yyoncho Aug 9, 2022

dom96 Aug 9, 2022

yyoncho Aug 9, 2022

dom96 Aug 8, 2022

yyoncho Aug 8, 2022

yyoncho Aug 10, 2022

dom96 Aug 8, 2022

dom96 Aug 8, 2022

yyoncho Aug 8, 2022 •

edited

Loading

yyoncho commented Aug 9, 2022

yyoncho commented Aug 9, 2022

dom96 commented Aug 9, 2022

yyoncho commented Aug 9, 2022

dom96 commented Aug 10, 2022 •

edited

Loading

arnetheduck commented Aug 11, 2022

yyoncho commented Aug 11, 2022

yyoncho commented Sep 2, 2022

arnetheduck commented Sep 2, 2022

yyoncho commented Jan 30, 2023

yyoncho commented Feb 1, 2023

Allow locking nim version in nimble.lock #1017

Allow locking nim version in nimble.lock #1017

Conversation

yyoncho commented Aug 8, 2022

dom96 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yyoncho Aug 8, 2022 • edited Loading

Choose a reason for hiding this comment

yyoncho commented Aug 9, 2022

yyoncho commented Aug 9, 2022

dom96 commented Aug 9, 2022

yyoncho commented Aug 9, 2022

dom96 commented Aug 10, 2022 • edited Loading

arnetheduck commented Aug 11, 2022

yyoncho commented Aug 11, 2022

yyoncho commented Sep 2, 2022

arnetheduck commented Sep 2, 2022

yyoncho commented Jan 30, 2023

yyoncho commented Feb 1, 2023

yyoncho Aug 8, 2022 •

edited

Loading

dom96 commented Aug 10, 2022 •

edited

Loading