[VCPKG-ARTIFACTS] Fixes to make Espressif IDF work (with artifacts for dependencies) #604

fearthecowboy · 2022-06-24T20:42:11Z

Includes work from @BillyONeal 's fork for espressif.

when a registry is updated, the zip file will check to see how deep to strip off folders that should not be there
made registry loading async
fix git clone to be smarter/faster
make the 'localname' for a repo a bit cleaner for github and aka.ms registries, otherwise generate a smaller name

To test this:

# ensure we have vcpkg stable
iex (iwr -useb https://aka.ms/vcpkg-init.ps1)

#clone the blink repo
git clone https://github.com/some-example/blink
cd blink

# run the devel version of vcpkg:
C:\work\2022\vcpkg-tool\out\build\x64-Debug\vcpkg.ps1 activate

output

warning: vcpkg-ce ('configure environment') is experimental and may change at any time.
Using in-development vcpkg-artifacts built at: C:/work/2022/vcpkg-tool/ce/ce
Updating registry data from https://github.com/fearthecowboy/registries/releases/download/espressif-idf-1.10/Espressif.zip
Updating registry data from https://aka.ms/vcpkg-ce-default
 Artifact                                                                                                             Version                Status        Dependency  Summary
 espressif:idf                                                                                                        1.10.2                 will install  *           Espressif IDF
 microsoft:tools/kitware/cmake                                                                                        3.20.1                 will install  *           Kitware's cmake tool
 microsoft:tools/ninja-build/ninja                                                                                    1.10.2                 will install  *           Ninja is a small build system with a focus on speed.
 [https://github.com/fearthecowboy/registries/releases/download/espressif-idf-1.10/Espressif.zip]:ccache              4.3.0                  will install  *
 [https://github.com/fearthecowboy/registries/releases/download/espressif-idf-1.10/Espressif.zip]:dfu_util            0.9.0                  will install  *
 [https://github.com/fearthecowboy/registries/releases/download/espressif-idf-1.10/Espressif.zip]:riscv32-esp-elf     8.4.0-patch3           will install  *
 [https://github.com/fearthecowboy/registries/releases/download/espressif-idf-1.10/Espressif.zip]:openocd             0.11.0-esp32-20211120  will install  *
 [https://github.com/fearthecowboy/registries/releases/download/espressif-idf-1.10/Espressif.zip]:xtensa-esp32s2-elf  8.4.0-patch3           will install  *
 [https://github.com/fearthecowboy/registries/releases/download/espressif-idf-1.10/Espressif.zip]:xtensa-esp32s3-elf  8.4.0-patch3           will install  *
 [https://github.com/fearthecowboy/registries/releases/download/espressif-idf-1.10/Espressif.zip]:xtensa-esp32-elf    8.4.0-patch3           will install  *
 [https://github.com/fearthecowboy/registries/releases/download/espressif-idf-1.10/Espressif.zip]:esp32s2ulp-elf      2.28.51-esp20191205    will install  *
 [https://github.com/fearthecowboy/registries/releases/download/espressif-idf-1.10/Espressif.zip]:esp32ulp-elf        2.28.51-esp20191205    will install  *

@fearthecowboy

Thanks to @fearthecowboy for help on this one. I considered ripping out the rush dependency, since it seems like we are only ever going to have one meaningful "project", and the dependency deduplication we need seems to be already done by pnpm (rather than npm), but after talking with @fearthecowboy I've decided to not go there since we still have rush linking the test project in. Unfortunately, there does not appear to be an effiicent way to build the typescript parts out-of-source, since they depend on node_modules which is put into the source tree. This adds a vcpkg.ps1 which does the same "environment hacking" as the in-development ce.ps1, teaches CMakeLists.txt to invoke rush and the typescript compiler as necessary, and teaches vcpkg.exe to use a hard-coded-into-the-binary path to the source tree when that in-development setting is turned on. The previous "always download latest ce bits" behavior is retained for folks who build vcpkg.exe from source and don't want to arrange for node and rush to be available.

…egistry zip files

…spressif

BillyONeal · 2022-06-24T20:51:29Z

ROFL it's actually "github.com/some-example" 🤣😂

# Conflicts: # CMakeLists.txt # src/vcpkg-in-development.ps1

fearthecowboy · 2022-06-24T20:56:45Z

Yeah, I got tired of not having a place for 1-off examples of stuff so I made a new org for myself.

BillyONeal · 2022-06-24T20:58:40Z

I merged 426593e then merged main into this to get the different-PR content out.

ce/ce/amf/installer.ts

ce/ce/archivers/ZipUnpacker.ts

ce/ce/archivers/git.ts

ce/ce/installers/espidf.ts

ce/ce/archivers/git.ts

BillyONeal · 2022-06-24T21:52:36Z

ce/ce/artifacts/artifact.ts

@@ -232,6 +234,17 @@ export class Artifact extends ArtifactBase {
      this.session.activation.addExports(exportsBlock, this.targetLocation);
    }

+    // if espressif install
+    if (this.metadata.info.flags.has('espidf')) {


Is that get/set property earlier in the review used now? Since this is reaching into flags directly.

Ah! no, it's not -- the espidf flag there was when we had it nested in the git clone ..

I've deleted that code several times, but it keeps creeping back in when I pull from someone else 😢

lemme remove that

This one doesn't look changed?

This is the correct usage, the other has been removed.

ce/ce/archivers/git.ts

Co-authored-by: Billy O'Neal <[email protected]>

…cpkg-tool into espressif-update

…strip

BillyONeal · 2022-06-28T17:14:50Z

ce/ce/artifacts/artifact.ts

@@ -232,6 +234,17 @@ export class Artifact extends ArtifactBase {
      this.session.activation.addExports(exportsBlock, this.targetLocation);
    }

+    // if espressif install
+    if (this.metadata.info.flags.has('espidf')) {


This one doesn't look changed?

BillyONeal and others added 10 commits June 13, 2022 13:07

format

c28bf00

Add better messages during build.

ae2705e

Merge remote-tracking branch 'origin/main' into espressif

8005516

Update vcpkg-scripts-sha to include python3_with_venv

b7f36c1

Restore espidf handling backed out in microsoft@323242c .

e71aeea

Fix python fetch in another place.

068fc0a

Update espressif support

eaff4ab

Updated epressif support to do better git clone and properly handle r…

efd8157

…egistry zip files

Merge branch 'main' of https://github.com/microsoft/vcpkg-tool into e…

3286b17

…spressif

fearthecowboy changed the title ~~Fixes to make Espressif IDF work (with artifacts for dependencies)~~ [VCPKG-ARTIFACTS] Fixes to make Espressif IDF work (with artifacts for dependencies) Jun 24, 2022

Merge remote-tracking branch 'origin/main' into espressif-update

e15b659

# Conflicts: # CMakeLists.txt # src/vcpkg-in-development.ps1

Undo accidential CMakeLists.txt changes broken by merge.

c5a4773

Fix removal instructions for PowerShell (microsoft#603)

35c8e29

BillyONeal requested changes Jun 24, 2022

View reviewed changes

fearthecowboy and others added 8 commits June 24, 2022 15:15

Update ce/ce/installers/espidf.ts

dfbaf27

Co-authored-by: Billy O'Neal <[email protected]>

Update ce/ce/archivers/git.ts

cd8835d

Co-authored-by: Billy O'Neal <[email protected]>

Update ce/ce/archivers/git.ts

306729a

Co-authored-by: Billy O'Neal <[email protected]>

Update ce/ce/archivers/git.ts

aab7117

Co-authored-by: Billy O'Neal <[email protected]>

Merge branch 'espressif-update' of https://github.com/fearthecowboy/v…

d414ae2

…cpkg-tool into espressif-update

add strip=-1 option to tar to allow detection of how many folders to …

e899fe8

…strip

simplify heartbeat during git

111501a

added comment with reasoning behind git clone changes

1305559

BillyONeal approved these changes Jun 28, 2022

View reviewed changes

fearthecowboy merged commit 0060c5c into microsoft:main Jun 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[VCPKG-ARTIFACTS] Fixes to make Espressif IDF work (with artifacts for dependencies) #604

[VCPKG-ARTIFACTS] Fixes to make Espressif IDF work (with artifacts for dependencies) #604

fearthecowboy commented Jun 24, 2022 •

edited

Loading

BillyONeal commented Jun 24, 2022

fearthecowboy commented Jun 24, 2022

BillyONeal commented Jun 24, 2022

BillyONeal Jun 24, 2022

fearthecowboy Jun 24, 2022

BillyONeal Jun 28, 2022

fearthecowboy Jun 28, 2022

BillyONeal Jun 28, 2022

[VCPKG-ARTIFACTS] Fixes to make Espressif IDF work (with artifacts for dependencies) #604

[VCPKG-ARTIFACTS] Fixes to make Espressif IDF work (with artifacts for dependencies) #604

Conversation

fearthecowboy commented Jun 24, 2022 • edited Loading

output

BillyONeal commented Jun 24, 2022

fearthecowboy commented Jun 24, 2022

BillyONeal commented Jun 24, 2022

BillyONeal Jun 24, 2022

Choose a reason for hiding this comment

fearthecowboy Jun 24, 2022

Choose a reason for hiding this comment

BillyONeal Jun 28, 2022

Choose a reason for hiding this comment

fearthecowboy Jun 28, 2022

Choose a reason for hiding this comment

BillyONeal Jun 28, 2022

Choose a reason for hiding this comment

fearthecowboy commented Jun 24, 2022 •

edited

Loading