LLVM comes with a really useful set of options LLVM_USE_CRT_ which allows you to specify a different C RunTime (CRT) when compiling with Visual Studio. If you want to be able to compile LLVM as a release build, but compile some code that uses it in debug (EG. our ComputeAorta product that allows customers to implement OpenCL/Vulkan on their hardware insanely quickly), Visual Studio will complain about mixing the differing version of the CRT. By using the LLVM_USE_CRT_ option, we can specify that LLVM compiles in a release build, but using a debug CRT.

There is one annoying catch with this though - compiling LLVM can be expensive to build. We’ll average 10 minutes build time for a full build of LLVM. We don’t want to recompile LLVM, and we don’t want to be constantly building different copies of LLVM everytime we pull in the latest commits for 2-4 different versions of the CRT. We want to be able to change a ComputeAorta build from debug/release without having to rebuild LLVM, and we want all this to just work™ without any manual input from a developer.

Changing LLVM⌗

So what we need to do is detect which CRT LLVM was built against. My first thought was to allow LLVM to export which CRT it was built against into an LLVM install. LLVM already outputs an LLVMConfig.cmake during its install process, so why not just record what CRT was used too? I contacted the LLVM mailing list asking... and got no response. I’ve found in general if you are not a super active contributor **and **located in the bay area this is a common occurrence. Not wanting to be that guy that nags on the mailing list about things no-one else clearly cares about, how else could I solve it?

Detecting the CRT⌗

So I reasoned that since the Visual Studio linker could detect and give me a good error message when I was accidentally mixing CRT versions, there must be some information recorded in the library files produced from Visual Studio that said which CRT the library was linked against. Using dumpbin.exe (which is included with Visual Studio) I first called:

$ dumpbin /? 
Microsoft (R) COFF/PE Dumper Version 14.00.24215.1 
Copyright (C) Microsoft Corporation. All rights reserved. 
 
usage: DUMPBIN [options] [files] 
 
 options: 
 
 /ALL 
 /ARCHIVEMEMBERS 
 /CLRHEADER 
 /DEPENDENTS 
 /DIRECTIVES 
 /DISASM[:{BYTES|NOBYTES}] 
 /ERRORREPORT:{NONE|PROMPT|QUEUE|SEND} 
 /EXPORTS 
 /FPO 
 /HEADERS 
 /IMPORTS[:filename] 
 /LINENUMBERS 
 /LINKERMEMBER[:{1|2}] 
 /LOADCONFIG 
 /NOLOGO 
 /OUT:filename 
 /PDATA 
 /PDBPATH[:VERBOSE] 
 /RANGE:vaMin[,vaMax] 
 /RAWDATA[:{NONE|1|2|4|8}[,#]] 
 /RELOCATIONS 
 /SECTION:name 
 /SUMMARY 
 /SYMBOLS 
 /TLS 
 /UNWINDINFO

And through a process of elimination I ran the ‘/DIRECTIVES’ command against one of the .lib files in LLVM which gave:

$ dumpbin /DIRECTIVES LLVMCore.lib
Microsoft (R) COFF/PE Dumper Version 14.00.24215.1
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file LLVMCore.lib

File Type: LIBRARY

   Linker Directives
   -----------------
   /FAILIFMISMATCH:_MSC_VER=1900
   /FAILIFMISMATCH:_ITERATOR_DEBUG_LEVEL=2
   /FAILIFMISMATCH:RuntimeLibrary=MDd_DynamicDebug
   /DEFAULTLIB:msvcprtd
   /FAILIFMISMATCH:_CRT_STDIO_ISO_WIDE_SPECIFIERS=0
   /FAILIFMISMATCH:LLVM_ENABLE_ABI_BREAKING_CHECKS=1
   /DEFAULTLIB:MSVCRTD
   /DEFAULTLIB:OLDNAMES

...

And what do you know ‘/FAILIFMISMATCH:RuntimeLibrary=MDd_DynamicDebug’ is telling the linker to output an error message if the CRT is not the dynamic debug variant! So now I have a method of detecting the CRT from one of LLVM’s libraries, how to incorporate that in our build?

CMake Integration⌗

LLVM uses CMake for its builds, and thus we also use CMake for our builds. We already include LLVM by specifying the location of an LLVM install like:

$ cmake -DCA_LLVM_INSTALL_DIR=<directory> .
-- Overriding option 'CA_LLVM_INSTALL_DIR' to '<directory>' (default was '').

And then within our CMake we do:

# Setup LLVM/Clang search paths.
list(APPEND CMAKE_MODULE_PATH
  ${CA_LLVM_INSTALL_DIR}/lib/cmake/llvm
  ${CA_LLVM_INSTALL_DIR}/lib/cmake/clang)

# Include LLVM.
include(LLVMConfig)

# Include Clang.
include(ClangTargets)

So I added a new DetectLLVMMSVCCRT.cmake to our CMake modules and included it just after the ClangTargets include. This does the following:

  • Get the directory of CMAKE_C_COMPILER (always cl.exe in our case).
  • Look for dumpbin.exe in the same directory.
  • Get the location of LLVMCore.lib.
    • My reasoning is that most libraries in LLVM could change over time, but the core library of LLVM is unlikely to be moved (I hope!).
  • Run dumpbin /DIRECTIVES LLVMCore.lib
    • Find the first usage of ‘/FAILIFMISMATCH:RuntimeLibrary='
    • Get the string that occurs between ‘/FAILIFMISMATCH:RuntimeLibrary=’ and the next ‘_'

And then we’ve got the CRT we need to use to build with. To actually set the CRT to use, we can just call LLVM’s ChooseMSVCCRT.cmake (that ships in an LLVM install), specifying the LLVM_USE_CRT_ variables and voila, we’ll be using the same CRT as LLVM, and get no linker errors!

The full CMake script is:

if(NOT CMAKE_SYSTEM_NAME STREQUAL Windows)
  return()
endif()

# Get the directory of cl.exe
get_filename_component(tools_dir "${CMAKE_C_COMPILER}" DIRECTORY)

# Find the dumpbin.exe executable in the directory of cl.exe
find_program(dumpbin "dumpbin.exe" PATHS "${tools_dir}" NO_DEFAULT_PATH)

if("${dumpbin}" STREQUAL "dumpbin-NOTFOUND")
  message(WARNING "Could not detect which CRT LLVM was built against - "
                  "could not find 'dumpbin.exe'.")
  return()
endif()

# Get the location in the file-system of LLVMCore.lib
get_target_property(llvmcore LLVMCore LOCATION)

if("${llvmcore}" STREQUAL "llvmcore-NOTFOUND")
  message(WARNING "Could not detect which CRT LLVM was built against - "
                  "could not find location of 'LLVMCore.lib'.")
  return()
endif()

# Get the directives that LLVMCore.lib contains
execute_process(COMMAND "${dumpbin}" "/DIRECTIVES" "${llvmcore}"
  OUTPUT_VARIABLE output)

# Find the first directive specifying what CRT to use
string(FIND "${output}" "/FAILIFMISMATCH:RuntimeLibrary=" position)

# Strip away everything but the directive we want to examine
string(SUBSTRING "${output}" ${position} 128 output)

# Remove the directive prefix which we don't need
string(REPLACE "/FAILIFMISMATCH:RuntimeLibrary=" "" output "${output}")

# Get the position of the '_' character that breaks the CRT from all else
string(FIND "${output}" "_" position)

# Substring output to be one of the four CRT values: MDd MD MTd MT
string(SUBSTRING "${output}" 0 ${position} output)

# Set all possible CMAKE_BUILD_TYPE's to the CRT that LLVM was linked against
set(LLVM_USE_CRT_DEBUG "${output}")
set(LLVM_USE_CRT_RELWITHDEBINFO "${output}")
set(LLVM_USE_CRT_MINSIZEREL "${output}")
set(LLVM_USE_CRT_RELEASE "${output}")

# Include the LLVM cmake module to choose the correct CRT
include(ChooseMSVCCRT)

Conclusion⌗

We’ve been able to do what we set out to do - auto-magically make our project that uses an LLVM install work reliably even with mixed Debug/Release builds. This has reduced the number of LLVM compiles I do daily by 2x (yay) and also allowed me to stop tracking (and caring) about CRT conflicts and how to avoid them.