Investigate why Clang/LLVM UBSan doesn't link

[StephanTLavavej]

While merging #3452 (internal MSVC-PR-457627), I had to disable the Clang/LLVM UBSan configurations for the internal test harness, as they fail to link (with linker errors about support machinery). We use Clang/LLVM via an internally captured "NuGet package" with a subset of binaries (but most of the support libs, AFAIK) to avoid excessive space consumption. At the time, I am unsure why UBSan doesn't work in the internal environment. It could be that we need to add more libs to the NuGet package, or it could be due to PATH ordering issues where we're mixing up MSVC and Clang/LLVM's support libs.

Thanks to @zacklj89 for reminding me to file this followup issue instead of being a bad kitty 😼 and silently working around it forever.

[StephanTLavavej]

I'm now seeing this with VS 2022 17.7 Preview 1 and Clang 16:

code: FAIL
name: std :: tests/GH_002334_branchless_clamp:27
output: Build setup steps:
Build steps:
Build step failed unexpectedly.
Command: "C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\Llvm\x64\bin\clang-cl.EXE" "D:\GitHub\STL\tests\std\tests\GH_002334_branchless_clamp\test.cpp" "-ID:\GitHub\STL\out\x64\out\inc" "-ID:\GitHub\STL\llvm-project\libcxx\test\support" "-ID:\GitHub\STL\tests\std\include" "/nologo" "/Od" "/W4" "/w14061" "/w14242" "/w14265" "/w14582" "/w14583" "/w14587" "/w14588" "/w14749" "/w14841" "/w14842" "/w15038" "/w15214" "/w15215" "/w15216" "/w15217" "/w15262" "/sdl" "/WX" "/D_ENABLE_STL_INTERNAL_CHECK" "/bigobj" "/FIforce_include.hpp" "/w14365" "/D_ENFORCE_FACET_SPECIALIZATIONS=1" "/D_STL_CALL_ABORT_INSTEAD_OF_INVALID_PARAMETER" "/w14640" "/Zc:threadSafeInit-" "-fno-ms-compatibility" "-fno-delayed-template-parsing" "-Wno-unqualified-std-cast-call" "/EHsc" "/MT" "/std:c++latest" "/permissive-" "/fp:strict" "-fsanitize=undefined" "-fno-sanitize-recover=undefined" "-m64" "-FeD:\GitHub\STL\out\x64\tests\std\tests\GH_002334_branchless_clamp\Output\27\GH_002334_branchless_clamp.exe" "-link" "-LIBPATH:D:\GitHub\STL\out\x64\out\lib\amd64" "-LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\MSVC\14.37.32705\lib\x64" "/MANIFEST:EMBED"
Exit Code: 1120
Standard Output:
--
   Creating library D:\GitHub\STL\out\x64\tests\std\tests\GH_002334_branchless_clamp\Output\27\GH_002334_branchless_clamp.lib and object D:\GitHub\STL\out\x64\tests\std\tests\GH_002334_branchless_clamp\Output\27\GH_002334_branchless_clamp.exp
clang_rt.ubsan_standalone-x86_64.lib(sanitizer_printf.cpp.obj) : error LNK2019: unresolved external symbol "bool __cdecl __coe_win::ContinueOnError(void)" (?ContinueOnError@__coe_win@@YA_NXZ) referenced in function "void __cdecl __sanitizer::SharedPrintfCodeNoBuffer(bool,char *,int,char const *,char *)" (?SharedPrintfCodeNoBuffer@__sanitizer@@YAX_NPEADHPEBD1@Z)
clang_rt.ubsan_standalone-x86_64.lib(sanitizer_common.cpp.obj) : error LNK2001: unresolved external symbol "bool __cdecl __coe_win::ContinueOnError(void)" (?ContinueOnError@__coe_win@@YA_NXZ)
clang_rt.ubsan_standalone-x86_64.lib(sanitizer_symbolizer_win.cpp.obj) : error LNK2001: unresolved external symbol "bool __cdecl __coe_win::ContinueOnError(void)" (?ContinueOnError@__coe_win@@YA_NXZ)
clang_rt.ubsan_standalone-x86_64.lib(sanitizer_stacktrace_libcdep.cpp.obj) : error LNK2001: unresolved external symbol "bool __cdecl __coe_win::ContinueOnError(void)" (?ContinueOnError@__coe_win@@YA_NXZ)
clang_rt.ubsan_standalone-x86_64.lib(sanitizer_printf.cpp.obj) : error LNK2019: unresolved external symbol "void __cdecl __coe_win::RawWrite(char const *)" (?RawWrite@__coe_win@@YAXPEBD@Z) referenced in function "void __cdecl __sanitizer::SharedPrintfCodeNoBuffer(bool,char *,int,char const *,char *)" (?SharedPrintfCodeNoBuffer@__sanitizer@@YAX_NPEADHPEBD1@Z)
clang_rt.ubsan_standalone-x86_64.lib(sanitizer_stacktrace_libcdep.cpp.obj) : error LNK2019: unresolved external symbol "void __cdecl __coe_win::PrintStack(struct __sanitizer::StackTrace const *)" (?PrintStack@__coe_win@@YAXPEBUStackTrace@__sanitizer@@@Z) referenced in function "public: void __cdecl __sanitizer::StackTrace::Print(void)const " (?Print@StackTrace@__sanitizer@@QEBAXXZ)
D:\GitHub\STL\out\x64\tests\std\tests\GH_002334_branchless_clamp\Output\27\GH_002334_branchless_clamp.exe : fatal error LNK1120: 3 unresolved externals
--
Standard Error:
--
clang-cl: error: linker command failed with exit code 1120 (use -v to see invocation)
--

[Benjins]

For anyone else running into this issue: I don't get any linker errors when using lld, via -fuse-ld=lld. This is when compiling LLVM from source on Windows, with top-of-tree. I'm not sure if this is a general workaround, but it at least lets me compile and run w/ UBSAN on my code

[timabroad]

Not sure if it's the same issue that you're seeing here, but I've been trying to resolve a similar issue and my problem seems to be due to VS shipping with a (presumably custom) version of clang_rt.ubsan_standalone-x86_64.lib that doesn't work:

$ cat src.c
int main(void) {int x=1; return x/1;}


# Single step compile and link works fine (I guess in this case clang finds its own version of the lib)
$ clang-cl -fsanitize=undefined src.c
   Creating library src.lib and object src.exp


# Compile and link separately doesn't work (with either link or lld-link)
$ clang-cl -fsanitize=undefined -c src.c && lld-link src.obj
<snipped...>
lld-link: error: undefined symbol: __declspec(dllimport) getenv
>>> referenced by D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\ubsan\ubsan_flags.cpp:29
>>>               clang_rt.ubsan_standalone-x86_64.lib(ubsan_flags.cpp.obj):(char const * __cdecl __ubsan::GetFlag(char const *))

lld-link: error: undefined symbol: bool __cdecl __coe_win::ContinueOnError(void)
>>> referenced by D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_printf.cpp:294
>>>               clang_rt.ubsan_standalone-x86_64.lib(sanitizer_printf.cpp.obj):(void __cdecl __sanitizer::SharedPrintfCodeNoBuffer(bool, char *, int, char const *, char *))
<snipped...>


# Explicitly specifying clang's version of libubsan works (with both link and lld-link)
$ lld-link src.obj /c/Program\ Files/Microsoft\ Visual\ Studio/2022/Professional/VC/Tools/Llvm/x64/lib/clang/16/lib/windows/clang_rt.ubsan_standalone-x86_64.lib


# Explicitly specifying MSVC's version of libusan doesn't work (with either link or lld-link)
$ lld-link src.obj /c/Program\ Files/Microsoft\ Visual\ Studio/2022/Professional/VC/Tools/MSVC/14.37.32822/lib/x64/clang_rt.ubsan_standalone-x86_64.lib
<snipped...>
lld-link: error: undefined symbol: __declspec(dllimport) getenv
>>> referenced by D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\ubsan\ubsan_flags.cpp:29
>>>               clang_rt.ubsan_standalone-x86_64.lib(ubsan_flags.cpp.obj):(char const * __cdecl __ubsan::GetFlag(char const *))

lld-link: error: undefined symbol: bool __cdecl __coe_win::ContinueOnError(void)
>>> referenced by D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_printf.cpp:294
>>>               clang_rt.ubsan_standalone-x86_64.lib(sanitizer_printf.cpp.obj):(void __cdecl __sanitizer::SharedPrintfCodeNoBuffer(bool, char *, int, char const *, char *))
<snipped...>