SpECTRE Documentation Coverage Report
Current view: top level - __w/spectre/spectre/docs/DevGuide - Travis.md Hit Total Coverage
Commit: f1ddee3e40d81480e49140855d2b0e66fafaa908 Lines: 0 1 0.0 %
Date: 2020-12-02 17:35:08
Legend: Lines: hit not hit

          Line data    Source code
       1           0 : \cond NEVER
       2             : Distributed under the MIT License.
       3             : See LICENSE.txt for details.
       4             : \endcond
       5             : 
       6             : # Travis CI {#travis_guide}
       7             : 
       8             : \tableofcontents
       9             : 
      10             : # Testing SpECTRE with Travis CI {#travis_ci}
      11             : 
      12             : SpECTRE uses
      13             : [TravisCI](https://travis-ci.org/sxs-collaboration/spectre) for
      14             : testing the code.  Multiple build jobs (described below) are launched
      15             : each time a pull request is submitted or updated.  In addition, a
      16             : daily cron job is run.  Travis will also launch these build jobs each
      17             : time you push to a branch on your fork of SpECTRE if you enable
      18             : it. (Go to Travis, click on your name in the upper right corner, and
      19             : then towards the bottom, enable testing on your fork of SpECTRE.)
      20             : 
      21             : For pull requests, you can view the Travis CI build by clicking on
      22             : `Details` next to the `Travis build failed` or `Travis build
      23             : succeeded` in the section for `All checks have failed` or `All checks
      24             : have passed` towards the bottom of the pull request.  You can login to
      25             : Travis using your GitHub credentials in order to cancel your builds,
      26             : and you should do so if you update your pull request while it is
      27             : building.
      28             : 
      29             : ## What is tested {#what-is-tested}
      30             : 
      31             : The Travis report lists the build jobs which will each have either a
      32             : green check mark if it passes, a red `X` if it has failed, or a yellow
      33             : binary if the build is in progress.  Clicking on a build job will
      34             : display the log of the build.
      35             : 
      36             : The following build jobs are launched:
      37             : * CHECK_COMMITS runs the script `tools/CheckCommits.sh` and fails the build if
      38             : any casing of the words in the list below is the first word of the commit
      39             : message.  This allows developers to flag their commits with these keywords to
      40             : indicate that a pull request should not be merged in its current state.
      41             :   - fixup
      42             :   - wip (for work in progress)
      43             :   - fixme
      44             :   - deleteme
      45             :   - rebaseme
      46             :   - testing
      47             :   - rebase
      48             : * CHECK_FILES runs the script `tools/CheckFiles.sh` (which also runs the script
      49             : `tools/FileTestDefs.sh`), and `tools/CheckPythonFormatting.sh`. The checks fail
      50             : if any of the following are true:
      51             :   - Any file,
      52             :     * contains a line over 80 characters (We allow exceptions for certain file
      53             :       types and inherently long strings like URLs and include lines.
      54             :       See `tools/FileTestDefs.sh` for the full list of exceptions.)
      55             :     * is missing the license line
      56             :     * does not end with a newline
      57             :     * contains a tab character
      58             :     * contains white space at the end of a line
      59             :     * contains a carriage return character
      60             :   - A `c++` header file (i.e., `*.hpp` or `*.tpp`) is missing `#%pragma once`
      61             :   - A `c++` file (i.e., `*.hpp`, `*.tpp`, or `*.cpp`) file,
      62             :     * includes `<iostream>` (useless when running in parallel)
      63             :     * includes `<lrtslock.h>` (use `<converse.h>` instead)
      64             :     * includes `"Utilities/TmplDebugging.hpp"` (used only for debugging)
      65             :     * includes any non-header `*.cpp` file
      66             :     * contains a `namespace` ending in `_details` (use `_detail`)
      67             :     * contains a `struct TD` or `class TD` (used only for debugging)
      68             :     * contains `std::enable_if` (use `Requires` instead)
      69             :     * contains `Ls` (use `List` instead)
      70             :     * contains additional text after `/*!` (does not render correctly in
      71             :       Doxygen)
      72             :   - A `c++` test,
      73             :     * uses `TEST_CASE` (use `SPECTRE_TEST_CASE` instead)
      74             :     * uses `Approx` (use `approx` instead)
      75             :   - A `CMakeLists.txt` file in `src`, but not in an Executables or
      76             :     Python-binding directory,
      77             :     * does not list a `C++` file that is present in the directory
      78             :     * lists a `C++` file that is not present in the directory
      79             :   - A python file is not formatted according to the `.style.yapf` file in the
      80             :     root of the repository.
      81             : * RUN_CLANG_TIDY runs the script `.travis/RunClangTidy.sh` which runs
      82             : `clang-tidy` on all files which were modified.  This is done for both `Release`
      83             : and `Debug` builds.
      84             : * TEST_CHECK_FILES runs `tools/CheckFiles.sh --test` which tests the checks
      85             : performed in the CHECK_FILES build.
      86             : * The other builds compile the code and run the tests for both
      87             : `Release` and `Debug` builds, for the `gcc` and `clang` compilers
      88             : using a Linux OS, and the `AppleClang` compiler for `OS X`.
      89             : * The `gcc Debug` build will fail if there are `doxygen` warnings.
      90             : 
      91             : ## How to perform the checks locally {#perform-checks-locally}
      92             : 
      93             : Before pushing to GitHub and waiting for Travis to perform the checks it is
      94             : useful to perform at least the following tests locally:
      95             : - **Unit tests:** Perform a `make test-executables` and then execute `ctest` to
      96             :   run all unit tests. As for `make` you can append a `-jN` flag to `ctest` to
      97             :   run in parallel on `N` cores. To run only a subset of the tests you can use
      98             :   one of the other keywords that the tests are labeled with, such as `ctest -L
      99             :   datastructures`. To run only particular tests you can also execute `ctest -R
     100             :   TEST_NAME` instead, where `TEST_NAME` is a regular expression matching the
     101             :   test identifiers such as `Unit.DataStructures.Mesh`. Pass the flag
     102             :   `--output-on-failure` to get output from failed tests. Consult `ctest -h` for
     103             :   further options.
     104             : - **clang-tidy:** In a clang build directory, run `make clang-tidy
     105             :   FILE=SOURCE_FILE` where `SOURCE_FILE` is a relative or absolute path to a
     106             :   `.cpp` file. To perform this check for all source files that changed in your
     107             :   pull request, `make clang-tidy-hash HASH=UPSTREAM_HEAD` where `UPSTREAM_HEAD`
     108             :   is the hash of the commit that your pull request is based on, usually the
     109             :   `HEAD` of the `upstream/develop` branch.
     110             : - **Documentation:** To render the documentation for the current state
     111             :   of the source tree the command `make doc` (or `make doc-check` to
     112             :   highlight warnings) can be used, placing its result in the `docs`
     113             :   directory in the build tree.  Once code has been made into a pull
     114             :   request to GitHub, the documentation can be rendered locally using
     115             :   the `tools/pr-docs` script.  To view the documentation, simply open the
     116             :   `index.html` file in the `html` subdirectory in a browser. Some functionality
     117             :   requires a web server (e.g. citation popovers), so just run a
     118             :   `python3 -m http.server` in the `html` directory to enable this.
     119             : - **IWYU:** We experimented for a time using IWYU (include what you
     120             :   use) as an automated check for correct includes and forward
     121             :   declarations.  Unfortunately it gave many incorrect suggestions.  We
     122             :   have decided to no longer have a IWYU check for Travis, but have
     123             :   left support for IWYU so that it can be used locally.  To do so just
     124             :   for the changed files in a pull request run `make iwyu-hash
     125             :   HASH=UPSTREAM_HEAD`. Since IWYU requires `USE_PCH=OFF` you can
     126             :   create a separate build directory and append `-D USE_PCH=OFF` to the
     127             :   usual `cmake` call. Note that it is very easy to incorrectly install
     128             :   IWYU (if not using the Docker container) and generate nonsense
     129             :   errors.  Note that we have left IWYU pragmas in the code, but no
     130             :   longer require they be added so that IWYU gives no errors when run.
     131             :   As IWYU is still under development, we plan to investigate using it
     132             :   again in the future.
     133             : 
     134             : ## Travis setup {#travis-setup}
     135             : 
     136             : * The `gcc Debug` build runs code coverage for each Travis build.
     137             : 
     138             : ## Troubleshooting {#travis-troubleshooting}
     139             : 
     140             : * Occasionally, a build job will fail because of a problem with Travis
     141             : (e.g. it times out).  Clicking on the circular arrow icon on the far
     142             : right of the build job will restart it.  All build jobs can be
     143             : restarted by clicking on the `Restart build` button to the right of
     144             : the pull request description.
     145             : * Travis caches some things between builds.  Occasionally this may
     146             : cause a problem leading to strange build failures.  If you suspect
     147             : this may be the case, get a SpECTRE owner to click on the `More
     148             : options` button in the top right, choose `Caches` and delete the cache
     149             : for your pull request.
     150             : 
     151             : ## Precompiled Headers and ccache {#precompiled-headers-ccache}
     152             : 
     153             : Getting ccache to work with precompiled headers on TravisCI is a little
     154             : challenging. The header to be precompiled is
     155             : `${SPECTRE_SOURCE_DIR}/tools/SpectrePch.hpp` and is symbolically linked to
     156             : `${SPECTRE_BUILD_DIR}/SpectrePch.hpp`. The configuration that seems to work is
     157             : specifying the environment variables:
     158             : 
     159             : \code{.sh}
     160             : CCACHE_COMPILERCHECK=content
     161             : CCACHE_EXTRAFILES="${SPECTRE_SOURCE_DIR}/tools/SpectrePch.hpp"
     162             : CCACHE_IGNOREHEADERS="${SPECTRE_BUILD_DIR}/SpectrePch.hpp:${SPECTRE_BUILD_DIR}/SpectrePch.hpp.gch"
     163             : \endcode
     164             : 
     165             : On macOS builds we haven't yet had success with using `ccache` with a
     166             : precompiled header. We disable the precompiled header and build in debug mode
     167             : only to have reasonable build times.
     168             : 
     169             : ## Build Stages {#travis-build-stages}
     170             : 
     171             : In order to avoid timeouts we build SpECTRE in various stages, carrying over the
     172             : ccache from one stage to the next. This allows us to avoid recompiling the code
     173             : in the next stage (there is some small overhead from running ccache instead of
     174             : not doing anything at all). The first stage builds all the SpECTRE libraries but
     175             : none of the executables or testing libraries. The second stage builds builds the
     176             : test executables, runs the tests, and also runs ClangTidy, include-what-you-use,
     177             : and various other checks. Another stage could be added that builds some of the
     178             : test libraries if necessary.
     179             : 
     180             : ## Caching Dependencies on macOS Builds {#caching-mac-os}
     181             : 
     182             : On macOS builds we cache all of our dependencies, like LIBXSMM and
     183             : Charm++. These are cached in `$HOME/mac_cache`. Ultimately this saves about
     184             : 10-12 minutes even when compared to using ccache to cache the object files from
     185             : building the dependencies. We also cache `$HOME/Library/Caches/Homebrew`, which
     186             : is where Homebrew keeps the downloaded formulas. By caching the Homebrew bottles
     187             : we are able to avoid brew formulas building from source because a tarball of the
     188             : package was not available at the time.

Generated by: LCOV version 1.14