Line data    Source code

       1           0 : \cond NEVER
       2             : Distributed under the MIT License.
       3             : See LICENSE.txt for details.
       4             : \endcond
       5             : 
       6             : # GitHub Actions Continuous Integration {#github_actions_guide}
       7             : 
       8             : \tableofcontents
       9             : 
      10             : # Testing SpECTRE with GitHub Actions CI {#github_actions_ci}
      11             : 
      12             : SpECTRE uses
      13             : [GitHub Actions](https://github.com/features/actions) for
      14             : testing the code.  Multiple build jobs (described below) are launched
      15             : each time a pull request is submitted or updated.  GitHub Actions will also
      16             : launch these build jobs each time you push to a branch on your fork of SpECTRE
      17             : if you enable it. GitHub Actions is also used to deploy releases of the code.
      18             : 
      19             : For pull requests, you can view the GitHub Actions CI build by clicking on the
      20             : `Checks` tab. Near the bottom of the `Conversation` tab a summary of the CI
      21             : results are presented. You can view all of the GitHub Actions runs by clicking
      22             : on the `Actions` section.
      23             : 
      24             : ## What is tested {#what-is-tested}
      25             : 
      26             : The GitHub Actions report lists the build jobs which will each have either a
      27             : green check mark if it passes, a red `X` if it has failed, or a yellow
      28             : dot with a circle if the build is in progress.  Clicking on a build job will
      29             : display the log of the build.
      30             : 
      31             : The following build jobs are launched:
      32             : * CHECK_COMMITS runs the script `tools/CheckCommits.sh` and fails the build if
      33             :   any casing of the words in the list below is the first word of the commit
      34             :   message.  This allows developers to flag their commits with these keywords to
      35             :   indicate that a pull request should not be merged in its current state.
      36             :   - fixup
      37             :   - wip (for work in progress)
      38             :   - fixme
      39             :   - deleteme
      40             :   - rebaseme
      41             :   - testing
      42             :   - rebase
      43             : * CHECK_FILES runs the script `tools/CheckFiles.sh` (which also runs the script
      44             :   `tools/FileTestDefs.sh`). The checks fail if any of the following are true:
      45             :   - Any file,
      46             :     * contains a line over 80 characters (We allow exceptions for certain file
      47             :       types and inherently long strings like URLs and include lines.
      48             :       See `tools/FileTestDefs.sh` for the full list of exceptions.)
      49             :     * is missing the license line
      50             :     * does not end with a newline
      51             :     * contains a tab character
      52             :     * contains white space at the end of a line
      53             :     * contains a carriage return character
      54             :   - A `c++` header file (i.e., `*.hpp` or `*.tpp`) is missing `#%pragma once`
      55             :   - A `c++` file (i.e., `*.hpp`, `*.tpp`, or `*.cpp`) file,
      56             :     * includes `<iostream>` (useless when running in parallel)
      57             :     * includes `<lrtslock.h>` (use `<converse.h>` instead)
      58             :     * includes `"Utilities/TmplDebugging.hpp"` (used only for debugging)
      59             :     * includes any non-header `*.cpp` file
      60             :     * contains a `namespace` ending in `_details` (use `_detail`)
      61             :     * contains a `struct TD` or `class TD` (used only for debugging)
      62             :     * contains `std::enable_if` (use `Requires` instead)
      63             :     * contains `Ls` (use `List` instead)
      64             :     * contains additional text after `/*!` (does not render correctly in
      65             :       Doxygen)
      66             :     * contains the string `return Py_None;` (bug prone, use `Py_RETURN_NONE`
      67             :       instead)
      68             :     * contains `.ckLocal()` or `.ckLocalBranch()` (use `Parallel::local` or
      69             :       `Parallel::local_branch` instead)
      70             :   - A `c++` test,
      71             :     * uses `TEST_CASE` (use `SPECTRE_TEST_CASE` instead)
      72             :     * uses `Approx` (use `approx` instead)
      73             :   - A `CMakeLists.txt` file in `src`, but not in an Executables or
      74             :     Python-binding directory,
      75             :     * does not list a `C++` file that is present in the directory
      76             :     * lists a `C++` file that is not present in the directory
      77             :   - A `c++` or `python` file contains a `TODO` (case-insensitive) comment
      78             :   In addition, the CHECK_FILES job tests Python formatting, the release
      79             :   workflow, and other tools in `tools/`.
      80             : * "Check Python formatting" runs the `black` and `isort` formatters over the
      81             :   source code.
      82             : * RUN_CLANG_TIDY runs clang-tidy on the source code. This is done for both
      83             :   `Release` and `Debug` builds.
      84             : * TEST_CHECK_FILES runs `tools/CheckFiles.sh --test` which tests the checks
      85             :   performed in the CHECK_FILES build.
      86             : * The other builds compile the code and run the tests for both
      87             :   `Release` and `Debug` builds, for the `gcc` and `clang` compilers
      88             :   using a Linux OS, and the `AppleClang` compiler for `OS X`.
      89             : * Verify the documentation builds successfully. Builds of `develop` deploy the
      90             :   documentation to GitHub pages.
      91             : 
      92             : ## How to perform the checks locally {#perform-checks-locally}
      93             : 
      94             : Before pushing to GitHub and waiting for GitHub Actions to perform the checks it
      95             : is useful to perform at least the following tests locally:
      96             : - **Unit tests:** Perform a `make unit-tests` and then execute `ctest -L Unit`
      97             :   to run all unit tests. As for `make` you can append a `-jN` flag to `ctest` to
      98             :   run in parallel on `N` cores. To run only a subset of the tests you can use
      99             :   one of the other keywords that the tests are labeled with, such as `ctest -L
     100             :   datastructures`. To run only particular tests you can also execute `ctest -R
     101             :   TEST_NAME` instead, where `TEST_NAME` is a regular expression matching the
     102             :   test identifiers such as `Unit.DataStructures.Mesh`. Pass the flag
     103             :   `--output-on-failure` to get output from failed tests. Consult `ctest -h` for
     104             :   further options.
     105             : 
     106             :   To run the input file tests you must build the executables using
     107             :   `make test-executables`. You can then run `ctest -LE unit` to run everything
     108             :   except for the unit tests, or `ctest` to run all tests.
     109             : - **clang-tidy:** In a clang build directory, run `make clang-tidy
     110             :   FILE=SOURCE_FILE` where `SOURCE_FILE` is a relative or absolute path to a
     111             :   `.cpp` file. To perform this check for all source files that changed in your
     112             :   pull request, `make clang-tidy-hash HASH=UPSTREAM_HEAD` where `UPSTREAM_HEAD`
     113             :   is the hash of the commit that your pull request is based on, usually the
     114             :   `HEAD` of the `upstream/develop` branch.
     115             : - **Python formatting:** Run `black --check .` and `isort --check-only .` over
     116             :   the repository. You can install these tools with `pip3 install -r
     117             :   support/Python/dev_requirements.txt`
     118             : - **Documentation:** To render the documentation for the current state
     119             :   of the source tree the command `make doc` (or `make doc-check` to
     120             :   highlight warnings) can be used, placing its result in the `docs`
     121             :   directory in the build tree.  Once code has been made into a pull
     122             :   request to GitHub, the documentation can be rendered locally using
     123             :   the `tools/pr-docs` script.  To view the documentation, simply open the
     124             :   `index.html` file in the `html` subdirectory in a browser. Some functionality
     125             :   requires a web server (e.g. citation popovers), so just run a
     126             :   `python3 -m http.server` in the `html` directory to enable this.
     127             : - **IWYU:** We experimented for a time using IWYU (include what you
     128             :   use) as an automated check for correct includes and forward
     129             :   declarations.  Unfortunately it gave many incorrect suggestions.  We
     130             :   have decided to no longer have a IWYU check with GitHub Actions, but have
     131             :   left support for IWYU so that it can be used locally.  To do so just
     132             :   for the changed files in a pull request run `make iwyu-hash
     133             :   HASH=UPSTREAM_HEAD`. Since IWYU requires `USE_PCH=OFF` you can
     134             :   create a separate build directory and append `-D USE_PCH=OFF` to the
     135             :   usual `cmake` call. Note that it is very easy to incorrectly install
     136             :   IWYU (if not using the Docker container) and generate nonsense
     137             :   errors.  Note that we have left IWYU pragmas in the code, but no
     138             :   longer require they be added so that IWYU gives no errors when run.
     139             :   As IWYU is still under development, we plan to investigate using it
     140             :   again in the future.
     141             : - The `gcc Debug` build runs code coverage for each GitHub Actions build.
     142             : 
     143             : ## Troubleshooting {#github-actions-troubleshooting}
     144             : 
     145             : * Occasionally, a build job will fail because of a problem with GitHub Actions
     146             :   (e.g. it times out).  On the `Checks` tab you can restart all or only the
     147             :   failed jobs. In the top right corner there's a `Re-run jobs` menu, which also
     148             :   has `Re-run failed jobs`. This button is `Cancel workflow` during the build
     149             :   process. Note that these buttons are only available if you have write access
     150             :   to the repository (core developer status).
     151             : * GitHub Actions caches some things between builds.  Occasionally this may
     152             :   cause a problem leading to strange build failures.  For example, inexplicable
     153             :   segfaults on seemingly random tests or `Illegal instruction` failures. We have
     154             :   to be fairly lax with our caching policies, and so the cache can become stale
     155             :   and outdated when a new container is pushed, among other difficult to
     156             :   understand situations. You can rebuild the ccache by going to `Actions`, then
     157             :   select the `Tests` workflow on the left, click the `Run workflow` drop-down
     158             :   menu, and enter `yes` in the input field below the ccache discussion.
     159             : 
     160             :   If clearing the ccache doesn't help, it could be that a Docker image layer is
     161             :   not being updated. GitHub doesn't (yet) have a way to clear the cache, so
     162             :   instead we clobber it to force GitHub to eject all old caches, both ccache and
     163             :   Docker images, along with anything else. To do this go to `Actions`, select
     164             :   the `Clobber Cache` workflow, then run it on develop. This will dump 9.9GB of
     165             :   random data into the cache. The amount is specified in the `ClobberCache.yaml`
     166             :   workflow file and needs to be updated if GitHub increases their cache
     167             :   size. The current cache size limit is 10GB per repository.
     168             : 
     169             :   Note that starting these workflows is only possible if you have write access
     170             :   to the repository (core developer status).
     171             : 
     172             : ## Precompiled Headers and ccache {#precompiled-headers-ccache}
     173             : 
     174             : Getting ccache to work with precompiled headers on GitHub Actions is a little
     175             : challenging. The header to be precompiled is
     176             : `${SPECTRE_SOURCE_DIR}/tools/SpectrePch.hpp` and is symbolically linked to
     177             : `${SPECTRE_BUILD_DIR}/SpectrePch.hpp`. The configuration that seems to work is
     178             : specifying the environment variables:
     179             : 
     180             : \code{.sh}
     181             : CCACHE_COMPILERCHECK=content
     182             : CCACHE_EXTRAFILES="${SPECTRE_SOURCE_DIR}/tools/SpectrePch.hpp"
     183             : CCACHE_IGNOREHEADERS=\
     184             :   "${SPECTRE_BUILD_DIR}/SpectrePch.hpp:${SPECTRE_BUILD_DIR}/SpectrePch.hpp.gch"
     185             : \endcode
     186             : 
     187             : ## Caching Dependencies on macOS Builds {#caching-mac-os}
     188             : 
     189             : On macOS builds we cache all of our dependencies, like LIBXSMM and
     190             : Charm++. These are cached in `$HOME/mac_cache`. Ultimately this saves about
     191             : 10-12 minutes even when compared to using ccache to cache the object files from
     192             : building the dependencies. We also cache `$HOME/Library/Caches/Homebrew`, which
     193             : is where Homebrew keeps the downloaded formulas. By caching the Homebrew bottles
     194             : we are able to avoid brew formulas building from source because a tarball of the
     195             : package was not available at the time.