Finding MSVC with CMake
9 minutes
It would appear that my concept of time continues to be an ephemeral or
completely non-existent experience on this plane of reality. I apologize for
the long delay, given that I last wrote in June that this very post was Coming
Soon™. Today, I’ll be discussing in more detail one of the ways you can
write a CMake Toolchain File so that you no longer have to use vcvarsall.bat
,
devenv.cmd
, or other “shortcut” when you want to target MSVC. This is a more
technical post than what I usually write, so apologies if you’re on mobile.
Some of the code snippets might be a pain to read. I’m aware my site’s design
has bitrotted to a degree. I’ll hopefully spend some time redesigning some
parts of it in the coming months, as I would like to better organize my
technical posts, articles, and one-offs.
In addition to relieving yourself of having to run “yet another shell” to
extract information, this will let us cache the INCLUDE
and LIBPATH
directories under CMake, which means that you won’t have to worry about passing
flags in or having the correct environment variables on every run.
To kick this off, we need to discuss the meat of where we’ll be acquiring our
information: vswhere
. The link will take you to the github project, which
will explain the details of vswhere
, however to quickly sum it up:
- It is installed with all Visual Studio versions
- It can find all Visual Studio installations
- It can provide said information to us in JSON.
This is ideal, as CMake now has JSON parsing support via string(JSON)
.
However, this only gets us the information to select from the possible set of
toolchains. The primary work comes from selecting the correct set of options.
After all, not everyone wants to select “latest” when they’re building (I
sure as shit can’t at work given our target specifications, but I digress), but
luckily CMake does provide us some Visual Studio specific variables that
MSBuild users will be using anyhow. Because of this, we can effectively
hijack them for the Makefile or Ninja based generators:
CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE
CMAKE_VS_PLATFORM_NAME
These variables specifically permit us to set the host and target system architectures, which do matter when we’re selecting a cross compiler of sorts.
We’ll also need a way to specify the edition of CMake (e.g., Community,
Professional, BuildTools), as well as the product year (e.g., 2022, 2019,
2017). Technically speaking, these variables can be anything. They are not
standard. I would suggest selecting a useful name. For the purpose of this
post, however, we’ll be using IZ_MSVS_VERSION
, IZ_MSVS_EDITION
, and
IZ_MSVS_TOOLSET
. And if I see anyone using these exact variables out in the
wild (and I’ll be able to tell because GitHub’s search is decent now!), I’m
gonna post a single issue on your repository with a newspaper (🗞) emoji to
digitally bop you one, because you shouldn’t be copying these variable names.
At all. Please pick something different.
Searching, Searching, Searching
With all of that out of the way, we’re free to get started writing out
toolchain file. The very first thing we want is to make sure our custom
variables get copied through to any try_compile
calls. This matters a great
deal, as our toolchain info would be lost when isolated otherwise:
include_guard(GLOBAL)
set(CMAKE_TRY_COMPILE_PLATFORM_VARIABLES
IZ_MSVS_VERSION
IZ_MSVS_EDITION
IZ_MSVS_TOOLSET)
Next, we need to shim in the value detection for the target and host for
Visual Studio. If the VS variables are not set, we can fall back onto the
CMAKE_HOST_SYSTEM_PROCESSOR
and CMAKE_SYSTEM_PROCESSOR
variables.
if (NOT CMAKE_GENERATOR STREQUAL "^Visual Studio")
if (NOT DEFINED CMAKE_SYSTEM_PROCESSOR)
set(CMAKE_SYSTEM_PROCESSOR "${CMAKE_HOST_SYSTEM_PROCESSOR}")
endif()
if (NOT DEFINED CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE)
set(CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE "x86")
if (CMAKE_HOST_SYSTEM_PROCESSOR STREQUAL "AMD64")
set(CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE "x64")
elseif (CMAKE_HOST_SYSTEM_PROCESSOR STREQUAL "ARM64")
set(CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE "arm64")
endif()
endif()
if (NOT DEFINED CMAKE_VS_PLATFORM_NAME)
set(CMAKE_VS_PLATFORM_NAME "x86")
if (CMAKE_SYSTEM_PROCESSOR STREQUAL "AMD64")
set(CMAKE_VS_PLATFORM_NAME "x64")
elseif (CMAKE_SYSTEM_PROCESSOR STREQUAL "ARM64")
set(CMAKE_VS_PLATFORM_NAME "arm64")
endif()
endif()
Next, we need to do 5 things in quick succession:
- Find
vswhere
- Get the Visual Studio installation path
- Get the Root Windows SDK Directory
- Find the MSVC Tools Version directory
- Find the Windows SDK Version Directory
Where is VSWhere?
The reason we want to find vswhere
is so that it can be overridden by users
at configure time. Letting them set -DVSWHERE_EXECUTABLE=<path>
gives a lot
more customization to developers, as you can then override the information with
whatever you want, allowing for a user-controlled hook over their environment.
# Block was added in CMake 3.25, and lets us create variable scopes without
# using a function.
block(SCOPE_FOR VARIABLES)
cmake_path(
CONVERT "$ENV{ProgramFiles\(x86\)}/Microsoft Visual Studio Installer"
TO_CMAKE_PATH_LIST vswhere.dir
NORMALIZE)
# This only temporarily affects the variable since we're inside a block.
list(APPEND CMAKE_SYSTEM_PROGRAM_PATH "${vswhere.dir}")
find_program(VSWHERE_EXECUTABLE NAMES vswhere DOC "Visual Studio Locator" REQUIRED)
endblock()
Visual Studio Installation Search
Next we need to find the installation path for the Visual Studio edition that
we want to use. If you’re on CI, I would personally set your equivalent
IZ_MSVS_EDITION
to BuildTools
, as this is a smaller install and doesn’t
install the Visual Studio directory itself. This is a bit of a doozy, but it
can be simplified if you’re willing to always select the latest installation
(which is whatever version was most recently updated by the installer not the
latest version itself)
if (DEFINED IZ_MSVS_EDITION)
set(product "Microsoft.VisualStudio.Product.${IZ_MSVS_EDITION}")
else()
set(product "*")
endif()
message(CHECK_START "Searching for Visual Studio ${IZ_MSVS_EDITION}")
execute_process(COMMAND "${VSWHERE_EXECUTABLE}" -nologo -nocolor
-format json
-products "${product}"
-utf8
-sort
ENCODING UTF-8
OUTPUT_VARIABLE candidates
OUTPUT_STRIP_TRAILING_WHITESPACE)
string(JSON candidates.length LENGTH "${candidates}")
string(JOIN " " error "Could not find Visual Studio"
"${IZ_MSVS_VERSION}"
"${IZ_MSVS_EDITION}")
if (candidates.length EQUAL 0)
message(CHECK_FAIL "no products")
# You can choose to either hard fail here, or continue
message(FATAL_ERROR "${error}")
endif()
if (NOT IZ_MSVS_VERSION)
string(JSON candidate.install.path GET "${candidates}" 0 "installationPath")
else()
# Unfortunately, range operations are inclusive in CMake for god knows why
math(EXPR stop "${candidates.length} - 1")
foreach (idx RANGE 0 ${stop})
string(JSON version GET "${candidates}" ${idx} "catalog" "productLineVersion")
if (version VERSION_EQUAL IZ_MSVS_VERSION)
string(JSON candidate.install.path
GET "${candidates}" ${idx} "installationPath")
break()
endif()
endforeach()
endif()
if (NOT candidate.install.path)
message(CHECK_FAIL "no install path found")
message(FATAL_ERROR "${error}")
endif()
cmake_path(
CONVERT "${candidate.install.path}"
TO_CMAKE_PATH_LIST candidate.install.path
NORMALIZE)
message(CHECK_PASS "found : ${candidate.install.path}")
set(IZ_MSVS_INSTALL_PATH "${candidate.install.path}"
CACHE PATH "Visual Studio Installation Path")
endblock()
Windows SDK Root Discovery
Next, we need to find the Windows SDK Root directory. This involves using some of the newer features of the CMake API regarding registry values under windows.
message(CHECK_START "Searching for Windows SDK Root Directory")
cmake_host_system_information(RESULT IZ_MSVS_WINDOWS_SDK_ROOT QUERY
WINDOWS_REGISTRY "HKLM/SOFTWARE/Microsoft/Windows Kits/Installed Roots"
VALUE "KitsRoot10"
VIEW BOTH
ERROR_VARIABLE error
PATH)
if (error)
message(CHECK_FAIL "not found : ${error}")
else()
cmake_path(CONVERT "${IZ_MSVS_WINDOWS_SDK_ROOT}"
TO_CMAKE_PATH_LIST IZ_MSVS_WINDOWS_SDK_ROOT
NORMALIZE)
message(CHECK_PASS "found : ${IZ_MSVS_WINDOWS_SDK_ROOT}")
endif()
Version Directories
With that out of the way (phew!), we can now move on to the next few steps. Luckily, these two steps are more or less the same. We just want to compare them to a specific value for each case. For this reason, I’ll be writing this as a function, but you can choose to do it inline:
function (msvs::directory out-var)
if (${out-var})
return()
endif()
cmake_parse_arguments(ARG "" "VARIABLE;PATH;DOC" "" ${ARGN})
message(CHECK_START "Searching for ${ARG_DOC}")
# We want to get the list of options, but *not* the full path string, hence
# the use of `RELATIVE`
file(GLOB candidates
LIST_DIRECTORIES YES
RELATIVE "${ARG_PATH}"
"${ARG_PATH}/*")
list(SORT candidates COMPARE NATURAL ORDER DESCENDING)
if (NOT DEFINED ARG_VARIABLE)
list(GET candidates 0 ${out-var})
else()
foreach (candidate IN LISTS candidates)
if ("${ARG_VARIABLE}" VERSION_EQUAL candidate)
set(${out-var} "${candidate}")
break()
endif()
endforeach()
endif()
if (NOT ${out-var})
message(CHECK_FAIL "not found")
else()
message(CHECK_PASS "found : ${${out-var}}")
set(${out-var} "${out-var}" CACHE INTERNAL "${out-var} value")
endif()
endfunction()
With this function, it’s now as simple as selecting the paths we want:
cmake_language(CALL msvs::directory msvc.tools.version
IZ_MSVS_TOOLS_VERSION
DIRECTORY "${IZ_MSVS_INSTALL_PATH}"
VARIABLE IZ_MSVS_TOOLSET
DOC "MSVC Toolset")
# Your CMAKE_SYSTEM_VERSION should line up with the minimum SDK version you're
# targeting exactly.
cmake_language(CALL msvs::directory IZ_MSVS_WINDOWS_SDK_VERSION
DIRECTORY "${IZ_MSVS_WINDOWS_SDK_ROOT}/Include"
VARIABLE CMAKE_SYSTEM_VERSION
DOC "Windows SDK")
Finding the Actual Tools
Now that we’ve found all the initial paths we need, we can now extrapolate a ton of additional paths that are needed for finding tools, include paths, and libraries. Hang tight, you just gotta trust the process here:
set(windows.sdk.host "Host${CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE}")
set(windows.sdk.target "${CMAKE_VS_PLATFORM_NAME}")
set(msvc.tools.dir "${IZ_MSVS_INSTALL_PATH}/VS/Tools/MSVC/${IZ_MSVS_TOOLS_VERSION}")
block(SCOPE_FOR VARIABLES)
list(PREPEND CMAKE_SYSTEM_PROGRAM_PATH
"${msvc.tools.dir}/bin/${windows.sdk.host}/${windows.sdk.target}"
"${IZ_MSVS_WINDOWS_SDK_ROOT}/bin/${IZ_MSVS_WINDOWS_SDK_VERSION}/${windows.sdk.target}"
"${IZ_MSVS_WINDOWS_SDK_ROOT}/bin")
find_program(CMAKE_MASM_ASM_COMPILER NAMES ml64 ml DOC "MSVC ASM Compiler")
find_program(CMAKE_CXX_COMPILER NAMES cl REQUIRED DOC "MSVC C++ Compiler")
find_program(CMAKE_RC_COMPILER NAMES rc REQUIRED DOC "MSVC Resource Compiler")
find_program(CMAKE_C_COMPILER NAMES cl REQUIRED DOC "MSVC C Compiler")
find_program(CMAKE_LINKER NAMES link REQUIRED DOC "MSVC Linker")
find_program(CMAKE_AR NAMES lib REQUIRED DOC "MSVC Archiver")
find_program(CMAKE_MT NAMES mt REQUIRED DOC "MSVC Manifest Tool")
endblock()
As long as these calls succeed, you’ll know you’ve got a proper toolchain discovered. But! We’re not quite done yet. The last bits of setup needed include setting the correct linker directories and include directories.
Finalizing the ✨Experience✨
This is effectively the time where you could start calling find_library
and
importing specific operating system libraries you’d like to link to without
listing their names simply. It’s up to you how you would like to handle it, but
for brevity, I’ll just be calling include_directories
and link_directories
.
Some of this could also be done entirely with generator expressions, but I’ll leave that as an exercise for the reader.
set(includes ucrt shared um winrt cppwinrt)
set(libs ucrt um)
list(TRANSFORM includes PREPEND "${IZ_MSVS_WINDOWS_SDK_ROOT}/Include/${IZ_MSVS_WINDOWS_SDK_VERSION}/")
list(TRANSFORM lib PREPEND "${IZ_MSVS_WINDOWS_SDK_ROOT}/Lib/${IZ_MSVS_WINDOWS_SDK_VERSION}/")
list(TRANSFORM lib APPEND "/${windows.sdk.target}")
# We could technically set `CMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES` and others,
# but not for the library paths.
include_directories(BEFORE SYSTEM "${msvc.tools.dir}/include" ${includes})
link_directories(BFORE "${msvc.tools.dir}/lib/${windows.sdk.target}" ${lib})
Wrapping Up
And just like that, we’ve now got a custom toolchain file for automatically
finding MSVC. It’s quite a bit of work involved, and as you will most
undoubtedly discover cannot cover every edge case, which is most likely why
autodetection isn’t setup under CMake if you pass -DCMAKE_CXX_COMPILER=cl
.
It also cannot resolve the main issue that I have with MSVC: it’s absolute
dogwater. I have never seen a single instance where MSVC outshines the
competition. Any point where it does the right thing, there are 3-4 places
where it does the wrong thing, and then causes a myriad of compatibility issues
(e.g., [[no_unique_address]]
)
Frankly, if I had my way, I’d recommend you stick to using clang
or
clang
-likes (such as zig cc
). It’ll work with the Windows SDK, it has
better code gen, and your life will be easier.
CMake
Build Systems
C++