SpECTRE Documentation Coverage Report
Current view: top level - __w/spectre/spectre/docs/DevGuide - BuildSystem.md Hit Total Coverage
Commit: f1ddee3e40d81480e49140855d2b0e66fafaa908 Lines: 0 1 0.0 %
Date: 2020-12-02 17:35:08
Legend: Lines: hit not hit

          Line data    Source code
       1           0 : \cond NEVER
       2             : Distributed under the MIT License.
       3             : See LICENSE.txt for details.
       4             : \endcond
       5             : # Build System {#spectre_build_system}
       6             : 
       7             : # CMake {#cmake}
       8             : 
       9             : SpECTRE uses [CMake](https://cmake.org/) for the build system. In this
      10             : section of the guide we outline how to [add new source
      11             : files](#adding_source_files), [libraries](#adding_libraries), [unit
      12             : tests](#adding_unit_tests), [executables](#adding_executables), and
      13             : [external dependencies](#adding_external_dependencies).  We also
      14             : describe [commonly used CMake flags](#common_cmake_flags).
      15             : 
      16             : Note that in editing `CMakeLists.txt` files, it is conventional to
      17             : indent multiline commands by two spaces (except for the first line),
      18             : and to separate most commands by blank lines.
      19             : 
      20             : ## Adding Source Files {#adding_source_files}
      21             : 
      22             : SpECTRE organizes source files into subdirectories of `src` that are
      23             : compiled into libraries.  To add a new source file `FILE.cpp` to an
      24             : existing library in `src/PATH/DIR`, just edit
      25             : `src/PATH/DIR/CMakeLists.txt` and add `FILE.cpp` to the list of files
      26             : in
      27             : ```
      28             : set(LIBRARY_SOURCES
      29             :   <list_of_files>
      30             :   )
      31             : ```
      32             : such that the resulting `<list_of_files>` is in alphabetical order.
      33             : 
      34             : ### Adding Libraries {#adding_libraries}
      35             : 
      36             : To add a source file `FILE.cpp` that is compiled into a new library `LIB` in a
      37             : directory `src/PATH/DIR` (either in a new directory, or in an existing
      38             : directory that either does not have a `CMakeLists.txt` file, or does
      39             : not create a library in the existing `CMakeLists.txt`):
      40             : - Create (if necessary) a `CMakeLists.txt` file in `DIR`, with the following
      41             : two lines at the top:
      42             : ```
      43             : # Distributed under the MIT License.
      44             : # See LICENSE.txt for details.
      45             : ```
      46             : - In the parent directory (i.e. `src/PATH`), (if necessary) add the
      47             : following line to its `CMakeLists.txt` file (if necessary, recursively
      48             : do the previous step and this one until you reach a `CMakeLists.txt` that
      49             : adds the appropriate subdirectory):
      50             : ```
      51             : add_subdirectory(DIR)
      52             : ```
      53             : If there are already other `add_subdirectory()` lines in the file, place
      54             : the new one so that the subdirectories are in alphabetical order.
      55             : - Add the line:
      56             : ```
      57             : set(LIBRARY LIB)
      58             : ```
      59             : where convention is that `LIB` = `DIR`.  As library names must be
      60             : unique, this is not always possible, in which case the convention is to
      61             : prepend the parent directory to `DIR`.
      62             : - Add the lines
      63             : ```
      64             : set(LIBRARY_SOURCES
      65             :   FILE.cpp
      66             :   )
      67             : 
      68             : add_spectre_library(${LIBRARY} ${LIBRARY_SOURCES})
      69             : 
      70             : target_link_libraries(
      71             :   ${LIBRARY}
      72             :   <list_of_interfaces>
      73             :   )
      74             : ```
      75             : where `<list_of_interfaces>` is an alphabetized list of lines of the form
      76             : ```
      77             :   INTERFACE SOME_LIBRARY
      78             : ```
      79             : where `SOME_LIBRARY` is a library that must be linked in order for the
      80             : new library to be successfully linked in an executable.
      81             : 
      82             : ## Adding Unit Tests {#adding_unit_tests}
      83             : 
      84             : We use the [Catch](https://github.com/philsquared/Catch) testing
      85             : framework for unit tests. All unit tests are housed in `tests/Unit`
      86             : with subdirectories for each subdirectory of `src`. Add the `cpp` file
      87             : to the appropriate subdirectory and also to the `CMakeLists.txt` in
      88             : that subdirectory. Inside the source file you can create a new test by
      89             : adding a `SPECTRE_TEST_CASE("Unit.Dir.Component",
      90             : "[Unit][Dir][Tag]")`. The `[Tag]` is optional and you can have more
      91             : than one, but the tags should be used quite sparingly.  The purpose of
      92             : the tags is to be able to run all unit tests or all tests of a
      93             : particular set of components, e.g. `ctest -L Data` to run all tests
      94             : inside the `Data` directory. Please see \ref writing_unit_tests
      95             : "writing unit tests", other unit tests and the [Catch
      96             : documentation](https://github.com/philsquared/Catch) for more help on
      97             : writing tests. Unit tests should take as short a time as possible,
      98             : with a goal of less than two seconds.  Please also limit the number of
      99             : distinct cases (by using `SECTION`s).
     100             : 
     101             : You can check the unit test coverage of your code by installing all the optional
     102             : components and then running `make unit-test-coverage` (after re-running CMake).
     103             : This will create the
     104             : directory `BUILD_DIR/docs/html/unit-test-coverage/` which is where the coverage
     105             : information is located. Open the `index.html` file in your browser and make
     106             : sure that your tests are indeed checking all lines of your code. Your pull
     107             : requests might not be merged until your line coverage is over 90% (we are aiming
     108             : for 100% line coverage wherever possible). Unreachable lines of code can be
     109             : excluded from coverage analysis by adding the inline comment `LCOV_EXCL_LINE`
     110             : or a block can be excluded using `LCOV_EXCL_START` and `LCOV_EXCL_STOP`.
     111             : However, this should be used extremely sparingly since unreachable code paths
     112             : should be removed from the code base altogether.
     113             : 
     114             : ## Adding Executables {#adding_executables}
     115             : 
     116             : All general executables are found in `src/Executables`, while those
     117             : for specific evolution (elliptic) systems are found in
     118             : `src/Evolution/Executables` (`src/Elliptic/Executables`).  See \ref
     119             : dev_guide_creating_executables "how to create executables".
     120             : 
     121             : ## Adding External Dependencies {#adding_external_dependencies}
     122             : 
     123             : To add an external dependency, first add a `SetupDEPENDENCY.cmake`
     124             : file to the `cmake` directory. You should model this after one of the
     125             : existing ones for `Catch` or `Brigand` if you're adding a header-only
     126             : library and `yaml-cpp` if the library is not header-only. If CMake
     127             : does not already support `find_package` for the library you're adding
     128             : you can write your own. These should be modeled after `FindBrigand` or
     129             : `FindCatch` for header-only libraries, and `FindYAMLCPP` for compiled
     130             : libraries. The `SetupDEPENDENCY.cmake` file must then be included in
     131             : the root `spectre/CMakeLists.txt`. Be sure to test both that setting
     132             : `LIBRARY_ROOT` works correctly for your library, and also that if the
     133             : library is required that CMake fails gracefully if the library is not
     134             : found.
     135             : 
     136             : ## Commonly Used CMake flags {#common_cmake_flags}
     137             : The following are the most common flags used to control building with
     138             : `CMake`. They are used by
     139             : ```
     140             : cmake -D FLAG1=OPT1 ... -D FLAGN=OPTN <SPECTRE_ROOT>
     141             : ```
     142             : - ASAN
     143             :   - Whether or not to turn on the address sanitizer compile flags
     144             :     (`-fsanitize=address`) (default is `OFF`)
     145             : - BUILD_PYTHON_BINDINGS
     146             :   - Build python libraries to call SpECTRE C++ code from python
     147             :     (default is `OFF`)
     148             : - BUILD_SHARED_LIBS
     149             :   - Whether shared libraries are built instead of static libraries
     150             :     (default is `OFF`)
     151             : - CHARM_ROOT
     152             :   - The path to the build directory of `Charm++`
     153             : - CMAKE_BUILD_TYPE
     154             :   - Sets the build type.  Common options:
     155             :     - `Debug` (the default if the flag is not specified): sets flags
     156             :       that trigger additional error checking
     157             :     - `Release`
     158             : - CMAKE_C_COMPILER
     159             :   - The `C` compiler used (defaults to whatever is determined by
     160             :     `CMake/Modules/CMakeDetermineCCompiler.cmake`, usually `cc`)
     161             : - CMAKE_CXX_COMPILER
     162             :   - The `C++` compiler used (defaults to whatever is determined by
     163             :     `CMake/Modules/CMakeDetermineCXXCompiler.cmake`, usually `c++`)
     164             : - CMAKE_Fortran_COMPILER
     165             :   - The `Fortran` compiler used (defaults to whatever is determined by
     166             :     `CMake/Modules/CMakeDetermineFortranCompiler.cmake`)
     167             : - CMAKE_C_FLAGS
     168             :   - Additional flags passed to the `C` compiler.
     169             : - CMAKE_CXX_FLAGS
     170             :   - Additional flags passed to the `C++` compiler.
     171             : - CMAKE_Fortran_FLAGS
     172             :   - Additional flags passed to the `Fortran` compiler.
     173             : - CMAKE_RUNTIME_OUTPUT_DIRECTORY
     174             :   - Sets the directory where the library and executables are placed.
     175             :     By default libraries end up in `<BUILD_DIR>/lib` and executables
     176             :     in `<BUILD_DIR>/bin`.
     177             : - DEBUG_SYMBOLS
     178             :   - Whether or not to use debug symbols (default is `ON`)
     179             :   - Disabling debug symbols will reduce compile time and total size of the build
     180             :     directory.
     181             : - ENABLE_WARNINGS
     182             :   - Whether or not warning flags are enabled (default is `ON`)
     183             : - MEMORY_ALLOCATOR
     184             :   - Set which memory allocator to use. If there are unexplained segfaults or
     185             :     other memory issues, it would be worth setting `MEMORY_ALLOCATOR=SYSTEM` to
     186             :     see if that resolves the issue. It could be the case that different
     187             :     third-party libraries accidentally end up using different allocators, which
     188             :     is undefined behavior and will result in complete chaos.
     189             :     (default is `JEMALLOC`)
     190             : - SPECTRE_UNIT_TEST_TIMEOUT_FACTOR, SPECTRE_INPUT_FILE_TEST_TIMEOUT_FACTOR and
     191             :   SPECTRE_PYTHON_TEST_TIMEOUT_FACTOR
     192             :   - Multiply the timeout for the respective set of tests by this factor (default
     193             :     is `1`).
     194             :   - This is useful to run tests on slower machines.
     195             : - STRIP_SYMBOLS
     196             :   - Whether or not to strip all symbols (default is `OFF`)
     197             :   - If enabled strips all extraneous symbols from libraries and executables,
     198             :     further reducing the size of them.
     199             : - STUB_EXECUTABLE_OBJECT_FILES
     200             :   - Replace object files from executables after linking with empty stubs
     201             :     (default is `OFF`)
     202             :   - This is useful for drastically reducing the build size in CI, but since the
     203             :     object files are replaced with empty stubs will generally cause linking
     204             :     problems if used during development.
     205             : - STUB_LIBRARY_OBJECT_FILES
     206             :   - Replace object files from libraries after linking with empty stubs
     207             :     (default is `OFF`)
     208             :   - This is useful for drastically reducing the build size in CI, but since the
     209             :     object files are replaced with empty stubs will generally cause linking
     210             :     problems if used during development.
     211             : - UBSAN_INTEGER
     212             :   - Whether or not to turn on the undefined behavior sanitizer
     213             :     [unsigned integer
     214             :     overflow](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html) flag
     215             :     (`-fsanitize=integer`) (default is `OFF`)
     216             : - UBSAN_UNDEFINED
     217             :   - Whether or not to turn on the undefined behavior sanitizer
     218             :     [undefined
     219             :     behavior](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html)
     220             :     compile flags (`-fsanitize=undefined`) (default is `OFF`)
     221             : - USE_CCACHE
     222             :   - Use ccache to cache build output so that rebuilding parts of the source tree
     223             :     is faster. The cache will use up space on disk, with the default being
     224             :     around 2-5GB. If you are performing a one time build to test something
     225             :     specific you should consider disabling ccache in order to avoid removing
     226             :     cached files that may be useful in other builds.
     227             :     (default is `ON`)
     228             : - USE_FORMALINE
     229             :   - Write the source tree into HDF5 files written to disk in order to increase
     230             :     reproducibility of results.
     231             :     (default is `ON`)
     232             : - USE_LD
     233             :   - Override the automatically chosen linker. The options are `ld`, `gold`, and
     234             :     `lld`.
     235             :     (default is `OFF`)
     236             : - USE_PCH
     237             :   - Whether or not to use pre-compiled headers (default is `ON`)
     238             :   - This needs to be turned `OFF` in order to use
     239             :     [include-what-you-use
     240             :     (IWYU)](https://github.com/include-what-you-use/include-what-you-use)
     241             : 
     242             : ## Checking Dependencies
     243             : 
     244             : Getting dependencies of libraries correct is quite difficult. SpECTRE offers the
     245             : CMake function `check_spectre_libs_dependencies`, defined in
     246             : `cmake/SpectreCheckDependencies.cmake`, to check the dependencies for all
     247             : libraries in the `libs` target. Individual target dependencies can be checked
     248             : using the `check_target_dependencies` CMake function defined in
     249             : `cmake/SpectreCheckTargetDependencies.cmake`. Please see those functions in the
     250             : source tree for more details on how to use them.
     251             : 
     252             : ## Formaline
     253             : 
     254             : SpECTRE's implementation of Formaline is based on, but distinct in
     255             : implementation from, the original design by
     256             : [Erik Schnetter and Christian Ott](https://github.com/hypercott/formaline),
     257             : which embeds an archive of the source tree into the executable. The original
     258             : design creates a C/C++ file with a function that returns an array/vector of
     259             : `char`s (a byte stream). However, this results in a very large source file (50MB
     260             : or more), which is very slow to compile and ends up more than doubling the link
     261             : time. Instead, SpECTRE's Formaline implementation uses the linker `ld` to
     262             : encode a file into an object, which means
     263             : rather than creating a large source file, we can directly encode the source tree
     264             : archive into the binary at the linking stage.
     265             : 
     266             : Most of SpECTRE's Formaline is implemented
     267             : inside the `tools/WrapExecutableLinker.sh` script. Function declarations are
     268             : provided in `Utilities/Formaline.hpp` and a small function that writes the
     269             : source file to disk is defined in `Utilities/Formaline.cpp`. The first
     270             : Formaline-related thing done in `WrapExecutableLinker.sh` is to archive
     271             : everything in the source directory tracked by git. Once the archive is created
     272             : we run `ld -r -b binary -o object.o src.tar.gz` (with unique names for
     273             : `object.o` and `src.tar.gz` for each executable that is built to avoid name
     274             : collisions) to generate an object file with the source file encoded from
     275             : `_binary_src_tar_gz_start` to `_binary_src_tar_gz_end`. Next we write a C++
     276             : source file that defines a function `get_archive` to convert the byte stream
     277             : into a `std::vector<char>`. We also encode the output of `printenv`, the various
     278             : `PATH` environment variables, and the CMake generated `LibraryVersions.txt` file
     279             : into the source file. Finally, the generated source file is built during the
     280             : linking phase and the object file containing the source archive is linked into
     281             : the executable.
     282             : 
     283             : To further aid in reproducibility, the `printenv` output and
     284             : `LibraryVersions.txt` contents are written to HDF5 files as part of the
     285             : `h5::Header` object. The archive of the source tree is written using the
     286             : `h5::SourceArchive` object and can be extracted by running
     287             : ```
     288             : h5dump -d /src.tar.gz -b LE -o src.tar.gz /path/to/hdf5/file.h5
     289             : ```

Generated by: LCOV version 1.14