Tutorial: Preparing libraries for CMake FetchContent

If you’re working on an executable project in C++, as opposed to a C++ library, using a package manager to get your dependencies might be overkill: If all you need is to get the source code of a library, include in your CMake project, and have it compiled from source with the rest of your project, CMake’s FetchContent module can do it for you.

If you’re a library writer, there are ways you can structure your CMake project to improve the experience for end users that use FetchContent: hide developer targets like tests, provide a zip archive that contains only the source files relevant downstream, and use GitHub actions to create it automatically.

Let’s see how.

Basic FetchContent usage

FetchContent is a CMake module that makes downloading or “fetching” dependencies really trivial. All you need is to let CMake know where the sources are with a call to FetchContent_Declare() and then include them as a subproject with FetchContent_MakeAvailable(). This will automatically download the project and make the targets available so you can link against them and have them built as necessary.

FetchContent can clone git repositories,

include(FetchContent) # once in the project to include the module

FetchContent_Declare(googletest
                     GIT_REPOSITORY https://github.com/google/googletest.git
                     GIT_TAG        703bd9caab50b139428cea1aaff9974ebee5742e # release-1.10.0)
FetchContent_MakeAvailable(googletest)

# Link against googletest's CMake targets now.

individual files,

FetchContent_Declare(doctest URL https://raw.githubusercontent.com/doctest/doctest/v2.4.9/doctest/doctest.h)
FetchContent_MakeAvailable(doctest)

# Add ${doctest_SOURCE_DIR} to the project's include paths

or zipped folders.

FetchContent_Declare(lexy URL https://lexy.foonathan.net/download/lexy-src.zip)
FetchContent_MakeAvailable(lexy)

# Link against lexy's targets now.

Very simple and straightforward, refer to CMake’s documentation for more details. Let’s look at the library side of things for the remainder of the post.

Designing projects for FetchContent

If a project is used via FetchContent, CMake will automatically call add_subdirectory(). This makes all targets of the project available in the parent, so you can link against them and use them.

However, this includes targets that are not useful for downstream consumers like unit tests, documentation builders, and so on. Crucially, this includes the dependencies of those targets – when using a library, I don’t want CMake to download that libraries testing framework! It is therefore a good idea to prevent that by only exposing those helper targets when not used as a subdirectory.

In the library’s root CMakeLists.txt, it can be detected by comparing CMAKE_CURRENT_SOURCE_DIR with CMAKE_SOURCE_DIR: they’re only the same if it is the real root of the source tree. As such, we only define test targets, when this is not the case:

project(my_project LANGUAGES CXX)

# define build options useful for all use
…

# define the library targets
add_subdirectory(src)

if(CMAKE_CURRENT_SOURCE_DIR STREQUAL CMAKE_SOURCE_DIR)
    # We're in the root, define additional targets for developers.
    option(MY_PROJECT_BUILD_EXAMPLES   "whether or not examples should be built" ON)
    option(MY_PROJECT_BUILD_TESTS      "whether or not tests should be built" ON)

    if(MY_PROJECT_BUILD_EXAMPLES)
        add_subdirectory(examples)
    endif()
    if(MY_PROJECT_BUILD_TESTS)
        enable_testing()
        add_subdirectory(tests)
    endif()

    …
endif()

By bifurcating the CMakeLists.txt in that way, we can even use different CMake versions for downstream consumers and library developers. For example, lexy requires version 3.8 to consume it, but 3.18 to develop it. This is done by calling cmake_minimum_required(VERSION 3.18) inside the if() block.

What to download?

FetchContent_Declare can download the project from many different sources, but not all sources take the same time. At least from GitHub, cloning the git repository takes a lot longer than downloading and extracting the zipped sources:

# slow
FetchContent_Declare(lexy GIT_REPOSITORY https://github.com/foonathan/lexy)
FetchContent_MakeAvailable(lexy)

# fast
FetchContent_Declare(lexy URL https://github.com/foonathan/lexy/archive/refs/heads/main.zip)
FetchContent_MakeAvailable(lexy)

However, downloading all sources can be too much. In the case of lexy, for example, it includes many tests, examples, and benchmarks – none of which are necessary to actually consume the project as a downstream user. This is especially true, because lexy disables most functionality when used as a subproject as explained above.

So instead, for lexy, you’re meant to download a prepackaged zip file that only contains the necessary files: the header files, source files of the library, and top-level CMakeLists.txt. That way, you don’t waste bandwidth or disk space on unnecessary stuff

# really fast
FetchContent_Declare(lexy URL https://lexy.foonathan.net/download/lexy-src.zip)
FetchContent_MakeAvailable(lexy)

If you’re maintaining a library meant for use with FetchContent, I highly recommend you do that as well – especially, because the process can be completely automated.

Automatically creating and publishing packaged source files

For that, we first need to define a custom CMake target that will create the package:

set(package_files include/ src/ CMakeLists.txt LICENSE)
add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}-src.zip
    COMMAND ${CMAKE_COMMAND} -E tar c ${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}-src.zip --format=zip -- ${package_files}
    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
    DEPENDS ${package_files})
add_custom_target(${PROJECT_NAME}_package DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}-src.zip)

This is done in three steps.

1. We define a list of all files and folders that need to be included in the package. This always needs to include the root CMakeLists.txt and the include and source files of the library.
2. We define a custom command to create the zip file: it needs to invoke cmake -E tar to create an archive. It has a dependency on the list of package files, so that CMake knows it needs to rebuild the zip archive when those files change.
3. We define a custom target. In order to build it (which itself does nothing), we’ve instructed CMake that we need the zip file. So building the target will execute the custom command and create the archive.

Of course, this targets is only defined if the project is not used as a subdirectory!

With that done, we just need a GitHub action that is triggered when we create a new release and adds the packaged source files as an artifact:

name: Release
permissions:
  contents: write

on:
  release:
    types: [published]

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Create build environment
        run: cmake -E make_directory build
      - name: Configure
        working-directory: build/
        run: cmake $GITHUB_WORKSPACE
      - name: Package source code
        working-directory: build/
        run: cmake --build . --target my_project_package

      - name: Add packaged source code to release
        uses: svenstaro/upload-release-action@v2
        with:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
          file: build/my_project-src.zip
          tag: ${{ github.ref }}

Now we just need to create a new release in GitHub’s UI, wait for everything to finish execute, and automatically have a packaged source file that people can download via FetchContent.

Conclusion

FetchContent is a really convenient way of managing dependencies. But you as a library authors can do a couple of things to make it even easier for the end user:

1. Only define minimal targets when the project is included as a subdirectory.
2. Provide minimal zipped archive of sources that users can download instead of the entire repository.
3. Use GitHub actions to automatically create the archive for each release.

If you want to check the techniques out in more detail, lexy uses them.