Try using rtools43 on windows with R 4.2

[gadenbuie]

There is, apparently, an incompatibility with httpuv and rtools42 in R 4.2 that can be seen in recent GHA runs. Compiling httpuv fails with many undefined reference errors, e.g.

  C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: uvutil.o:uvutil.cpp:(.text+0x844): undefined reference to `uv_buf_init'
  C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: uvutil.o:uvutil.cpp:(.text+0x8c7): undefined reference to `uv_buf_init'
  C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: uvutil.o:uvutil.cpp:(.text+0x923): undefined reference to `uv_write'
  C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: websockets.o:websockets.cpp:(.text+0x9aa): undefined reference to `uv_timer_start'
  C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: websockets.o:websockets.cpp:(.text+0x8bb): undefined reference to `uv_timer_start'
  C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: websockets.o:websockets.cpp:(.text$_ZN19WebSocketConnectionD1Ev[_ZN19WebSocketConnectionD1Ev]+0x6c): undefined reference to `uv_close'
  collect2.exe: error: ld returned 1 exit status
  no DLL was created
  ERROR: compilation failed for package 'httpuv'

As a test, we tried updating the R CMD Check workflow to use rtools43 instead, and compilation succeeded.

Likely related, rtools43 upgraded libuv from 1.44.2 to 1.47.0.

[wch]

I think the error indicates that it's not finding the library during the static linking step.

Note: I believe that with Rtools42 and above, it uses the Makevars.ucrt file instead of Makevars.win.

With Rtools43 and R 4.2, the linking succeeds. This is the command (https://github.com/rstudio/httpuv/actions/runs/13014853474/job/36301447772#step:6:55):

g++ -std=gnu++14 -shared -s -static-libgcc -o httpuv.dll tmp.def RcppExports.o callback.o callbackqueue.o
  filedatasource-unix.o filedatasource-win.o fs.o gzipdatasource.o http.o httprequest.o httpresponse.o httpuv.o
  md5.o mime.o socket.o staticpath.o thread.o timegm.o utils.o uvutil.o webapplication.o websockets-base.o
  websockets-hixie76.o websockets-hybi03.o websockets-ietf.o websockets.o winutils.o
  ./http-parser/http_parser.o ./sha1/sha1.o ./base64/base64.o
  -LC:/rtools43/x86_64-w64-mingw32.static.posix/lib 
  -lz -luv -ldl -lws2_32 -lpsapi -liphlpapi -luserenv -luser32 -ldbghelp -lole32 -luuid -lshell32 
  -Lc:/rtools42/x86_64-w64-mingw32.static.posix/lib/x64 
  -Lc:/rtools42/x86_64-w64-mingw32.static.posix/lib
  -LC:/R/bin/x64 -lR

With Rtools42 and R 4.2, the linking fails. This is the command that gives the errors quoted above (https://github.com/rstudio/httpuv/actions/runs/12627511329/job/35182260888#step:3:2029):

 g++ -std=gnu++14 -shared -s -static-libgcc -o httpuv.dll tmp.def RcppExports.o callback.o callbackqueue.o 
  filedatasource-unix.o filedatasource-win.o fs.o gzipdatasource.o http.o httprequest.o httpresponse.o httpuv.o
  md5.o mime.o socket.o staticpath.o thread.o timegm.o utils.o uvutil.o webapplication.o websockets-base.o
  websockets-hixie76.o websockets-hybi03.o websockets-ietf.o websockets.o winutils.o
  ./http-parser/http_parser.o ./sha1/sha1.o ./base64/base64.o 
  -Lc:/rtools42/x86_64-w64-mingw32.static.posix/lib/x64 
  -Lc:/rtools42/x86_64-w64-mingw32.static.posix/lib
  -LC:/R/bin/x64 -lR

The big difference I see is that in the Rtools43 case, which succeeds, there are these flags, which tell it which libraries to link to:

  -lz -luv -ldl -lws2_32 -lpsapi -liphlpapi -luserenv -luser32 -ldbghelp -lole32 -luuid -lshell32 

And in the Rtools42 case, it doesn't have those flags.

In the old Makevars.win (which again, is not used with Rtools42 and up), it explicitly specifies those libraries in the PKG_LIBS environment variable:

httpuv/src/Makevars.win

Lines 1 to 2 in ff00e6b

	PKG_LIBS = ./libuv/libuv.a ./http-parser/http_parser.o ./sha1/sha1.o ./base64/base64.o \
	-lpthread -lws2_32 -lkernel32 -lpsapi -liphlpapi -lshell32 -luserenv -lz

In the Makrvars.ucrt (which is used with Rtools42 and Rtools43), we add the -l flags to PKG_LIBS conditionally, here. This code is originally from https://cran.r-project.org/bin/windows/base/howto-R-4.3.html.

httpuv/src/Makevars.ucrt

Lines 3 to 9 in ff00e6b

	ifeq (,$(shell pkg-config --version 2>/dev/null))
	PKG_LIBS += \
	-luv -lpthread -lws2_32 -lkernel32 -lpsapi -liphlpapi -lshell32 -luserenv -lz \
	-ldbghelp -luuid -lole32
	else
	PKG_LIBS += $(shell pkg-config --libs zlib libuv)
	endif

When the pkg-config program is present (which is true of Rtools43), then it takes the else code path and lets pkg-config figure out which libraries need to be listed with the -l flags.

When the pkg-config program is NOT present (which is supposed to be true of Rtools42), then it takes the first code path and explicitly sets all the -l flags.

---

My theory of what is happening is that on this build system with Rtools42, there is a copy of pkg-config present, but it's not one that comes with Rtools, because Rtools42 doesn't include it. Then when it runs that pkg-config command on line 8, it's not finding zlib or libuv at all.

The error messages seem to agree with this (https://github.com/rstudio/httpuv/actions/runs/12627511329/job/35182260888#step:3:2017):

  Can't find zlib.pc in any of /usr/lib/pkgconfig /usr/share/pkgconfig
  use the PKG_CONFIG_PATH environment variable, or
  specify extra search paths via 'search_paths'

I think it should be looking somewhere in c:/rtools42/, not in /usr/lib or /usr/share.

If I'm right about this, then the question is, why is there another copy of pkg-config, and where did it come from? And maybe the way to fix it is to set PKG_CONFIG_PATH?

[gaborcsardi]

Comments from Slack:

Are you sure that Rtools42 has recent enough versions of all libs for current httpuv? Plus the pkg-config config files, etc?

FWIW if I run R CMD INSTALL httpuv on Windows it fails the same way as on the GHA VM.

Btw. it is easier to debug these issues interactively. You can use this action to start a shell, and then ssh into the VM: https://github.com/r-hub/actions/tree/main/debug-shell

Btw.2. Just checked, Rtools42 indeed does not have pkg-config