Namespaces | Classes | Macros | Typedefs | Enumerations | Functions
Testing Framework

Classes, functions, macros, and instructions for developing tests. More...

Namespaces

 ActionTesting
 Structures used for mocking the parallel components framework in order to test actions.
 
 pypp
 Contains all functions for calling python from C++.
 

Classes

class  UniformCustomDistribution< T >
 A uniform distribution function object which redirects appropriately to either the std::uniform_int_distribution or the std::uniform_real_distribution. This also provides a convenience constructor which takes a 2-element array for the bounds for either floating point or int distributions. More...
 
class  ActionTesting::MockRuntimeSystem< Metavariables >
 A class that mocks the infrastructure needed to run actions. It simulates message passing using the inbox infrastructure and handles most of the arguments to the apply and is_ready action methods. More...
 
class  OrientationMapIterator< VolumeDim >
 An iterator for looping through all possible orientations of the n-dim cube. More...
 

Macros

#define INVOKE_TEST_FUNCTION(FUNCTION_NAME, TUPLE_ARGS, ...)
 Macro used to invoke a test function of multiple template arguments. More...
 
#define CHECK_FOR_DOUBLES_AND_DATAVECTORS(FUNCTION_NAME, ...)
 Macro used to test functions whose parameter can be a double or a DataVector. More...
 
#define CHECK_OP(a, op, b, c)
 Check a op b == c and also the op= version. More...
 
#define MAKE_GENERATOR(...)
 MAKE_GENERATOR(NAME [, SEED]) declares a variable of name NAME containing a generator of type std::mt19937. More...
 
#define CHECK_VARIABLES_APPROX(a, b)
 A wrapper around Catch's CHECK macro that checks approximate equality of each entry in each tag within a variables. More...
 
#define CHECK_VARIABLES_CUSTOM_APPROX(a, b, appx)
 Same as CHECK_VARIABLES_APPROX, but with a user-defined Approx. The third argument should be of type Approx. More...
 
#define CHECK_ELEMENT_WISE_FUNCTION_APPROX(function, arguments)
 A test utility for verifying that an element-wise function, function acts identically to the same operation applied to each element of a container separately. This macro invokes test_element_wise_function() (which gives a more complete documentation of the element-wise checking operations and arguments). More...
 
#define CHECK_CUSTOM_ELEMENT_WISE_FUNCTION_APPROX( function, arguments, at_operator, size_of_operator)
 Same as CHECK_ELEMENT_WISE_FUNCTION_APPROX, but with a user-defined function at_operator and size_of_operator, each of which correspond to arguments of test_element_wise_function() (which gives a more complete documentation of the element-wise checking operations and arguments). More...
 
#define CAPTURE_PRECISE(variable)
 Alternative to Catch's CAPTURE that prints more digits. More...
 
#define SPECTRE_PARALLEL_REQUIRE(expr)
 A similar to Catch's REQUIRE statement, but can be used in tests that spawn several chares with possibly complex interaction between the chares. More...
 
#define SPECTRE_PARALLEL_REQUIRE_FALSE(expr)
 A similar to Catch's REQUIRE_FALSE statement, but can be used in tests that spawn several chares with possibly complex interaction between the chares. More...
 
#define CHECK_ITERABLE_APPROX(a, b)
 A wrapper around Catch's CHECK macro that checks approximate equality of entries in iterable containers. For maplike containers, keys are checked for strict equality and values are checked for approximate equality. More...
 
#define CHECK_ITERABLE_CUSTOM_APPROX(a, b, appx)
 Same as CHECK_ITERABLE_APPROX with user-defined Approx. The third argument should be of type Approx. More...
 
#define ERROR_TEST()
 Mark a test as checking a call to ERROR. More...
 
#define ASSERTION_TEST()
 Mark a test to be checking an ASSERT. More...
 

Typedefs

using TestHelpers::VectorImpl::Bound = std::array< double, 2 >
 Type alias to be more expressive with distribution bounds in vector tests which call the generic math test below.
 

Enumerations

enum  TestHelpers::VectorImpl::TestKind { Normal, Strict, Inplace, GivenOrderOfArgumentsOnly }
 the set of test types that may be used for the math operations More...
 

Functions

template<typename T , typename UniformRandomBitGenerator , typename RandomNumberDistribution >
void fill_with_random_values (const gsl::not_null< T *> data, const gsl::not_null< UniformRandomBitGenerator *> generator, const gsl::not_null< RandomNumberDistribution *> distribution) noexcept
 Fill an existing data structure with random values.
 
template<typename T , typename UniformRandomBitGenerator , typename RandomNumberDistribution >
make_with_random_values (const gsl::not_null< UniformRandomBitGenerator *> generator, const gsl::not_null< RandomNumberDistribution *> distribution) noexcept
 Make a fixed-size data structure and fill with random values. More...
 
template<typename DataType >
tnsr::I< DataType, 1 > random_unit_normal (gsl::not_null< std::mt19937 *> generator, const tnsr::ii< DataType, 1 > &spatial_metric) noexcept
 Make a random unit normal vector at each element of DataType.
 
template<typename VectorType , typename ValueType >
void TestHelpers::VectorImpl::vector_test_construct_and_assign (tt::get_fundamental_type_t< ValueType > low=tt::get_fundamental_type_t< ValueType >{-100.0}, tt::get_fundamental_type_t< ValueType > high=tt::get_fundamental_type_t< ValueType >{100.0}) noexcept
 test construction and assignment of a VectorType with a ValueType
 
template<typename VectorType , typename ValueType >
void TestHelpers::VectorImpl::vector_test_serialize (tt::get_fundamental_type_t< ValueType > low=tt::get_fundamental_type_t< ValueType >{-100.0}, tt::get_fundamental_type_t< ValueType > high=tt::get_fundamental_type_t< ValueType >{ 100.0}) noexcept
 test the serialization of a VectorType constructed with a ValueType
 
template<typename VectorType , typename ValueType >
void TestHelpers::VectorImpl::vector_test_ref (tt::get_fundamental_type_t< ValueType > low=tt::get_fundamental_type_t< ValueType >{-100.0}, tt::get_fundamental_type_t< ValueType > high=tt::get_fundamental_type_t< ValueType >{ 100.0}) noexcept
 test the construction and move of a reference VectorType constructed with a ValueType
 
template<typename VectorType , typename ValueType = typename VectorType::ElementType>
void TestHelpers::VectorImpl::vector_ref_test_size_error (RefSizeErrorTestKind test_kind, tt::get_fundamental_type_t< ValueType > low=tt::get_fundamental_type_t< ValueType >{-100.0}, tt::get_fundamental_type_t< ValueType > high=tt::get_fundamental_type_t< ValueType >{100.0}) noexcept
 Test that assigning to a non-owning VectorType of the wrong size appropriately generates an error. More...
 
template<typename VectorType , typename ValueType >
void TestHelpers::VectorImpl::vector_test_math_after_move (tt::get_fundamental_type_t< ValueType > low=tt::get_fundamental_type_t< ValueType >{-100.0}, tt::get_fundamental_type_t< ValueType > high=tt::get_fundamental_type_t< ValueType >{100.0}) noexcept
 tests a small sample of math functions after a move of a VectorType initialized with ValueType
 
template<TestKind Test, typename VectorType0 , typename... VectorTypes, typename... FunctionsAndArgumentBounds>
void TestHelpers::VectorImpl::test_functions_with_vector_arguments (const std::tuple< FunctionsAndArgumentBounds... > &tuple_of_functions_and_argument_bounds) noexcept
 General entry function for testing arbitrary math functions on vector types. More...
 
template<typename Map >
bool are_maps_equal (const Map &map, const CoordinateMapBase< Frame::Logical, Frame::Inertial, Map::dim > &map_base)
 Given a Map and a CoordinateMapBase, checks that the maps are equal by downcasting map_base and then comparing to map. Returns false if the downcast fails.
 
template<typename SourceFrame , typename TargetFrame , size_t VolumeDim>
void check_if_maps_are_equal (const CoordinateMapBase< SourceFrame, TargetFrame, VolumeDim > &map_one, const CoordinateMapBase< SourceFrame, TargetFrame, VolumeDim > &map_two)
 Given two coordinate maps (but not their types), check that the maps are equal by evaluating them at a random set of points.
 
template<typename Map >
void check_if_map_is_identity (const Map &map)
 Given a coordinate map, check that this map is equal to the identity by evaluating the map at a random set of points.
 
template<typename Map >
void test_jacobian (const Map &map, const std::array< double, Map::dim > &test_point)
 Given a Map map, checks that the jacobian gives expected results when compared to the numerical derivative in each direction.
 
template<typename Map >
void test_inv_jacobian (const Map &map, const std::array< double, Map::dim > &test_point)
 Given a Map map, checks that the inverse jacobian and jacobian multiply together to produce the identity matrix.
 
template<typename Map , typename... Args>
void test_coordinate_map_implementation (const Map &map)
 Checks that the CoordinateMap map functions as expected when used as the template parameter to the CoordinateMap type.
 
template<typename Map , typename... Args>
void test_coordinate_map_argument_types (const Map &map, const std::array< double, Map::dim > &test_point, const Args &... args)
 Checks that the CoordinateMap map functions as expected when used with different argument types.
 
template<typename Map , typename T >
void test_inverse_map (const Map &map, const std::array< T, Map::dim > &test_point)
 Given a Map map, checks that the inverse map gives expected results.
 
template<typename Map >
void test_suite_for_map_on_unit_cube (const Map &map)
 Given a Map map, tests the map functions, including map inverse, jacobian, and inverse jacobian, for a series of points. These points are chosen in a dim-dimensonal cube of side 2 centered at the origin. The map is expected to be valid on the boundaries of the cube.
 
template<typename Map >
void test_suite_for_map_on_sphere (const Map &map, const bool include_origin=true, const double radius_of_sphere=1.0)
 Given a Map map, tests the map functions, including map inverse, jacobian, and inverse jacobian, for a series of points. These points are chosen in a sphere of radius radius_of_sphere, and the map is expected to be valid on the boundary of that sphere as well as in its interior. The flag include_origin indicates whether to test the map at the origin. This test works only in 3 dimensions.
 
std::array< OrientationMap< 3 >, 6 > all_wedge_directions ()
 Wedge OrientationMap in each of the six directions used in the Shell and Sphere domain creators.
 
template<typename T >
test_creation (const std::string &construction_string) noexcept
 Construct an object from a given string. Each line in the string must be indented.
 
template<typename BaseClass >
std::unique_ptr< BaseClass > test_factory_creation (const std::string &construction_string) noexcept
 Construct a factory object from a given string. Each line in the string must be indented.
 
template<typename T >
serialize_and_deserialize (const T &t)
 Serializes and deserializes an object t of type T
 
template<typename T >
void test_serialization (const T &t)
 Tests the serialization of comparable types. More...
 
template<typename B , typename D , typename... Args>
void test_serialization_via_base (Args &&... args)
 Test the serialization of a derived class via a base class pointer. More...
 
template<typename T , typename U >
void check_cmp (const T &less, const U &greater)
 Function to test comparison operators. Pass values with less < greater.
 
template<typename Invocable , size_t VolumeDim>
std::array< double, VolumeDim > numerical_derivative (const Invocable &map, const std::array< double, VolumeDim > &x, const size_t direction, const double delta)
 Calculates the derivative of an Invocable at a point x - represented by an array of doubles - in the domain of map with a sixth-order finite difference method. More...
 
template<typename Exception , typename ThrowingFunctor >
void test_throw_exception (const ThrowingFunctor &func, const Exception &expected)
 Execute func and check that it throws an exception expected. More...
 
template<typename ElementWiseFunction , typename AtFunction = GetContainerElement, typename SizeFunction = GetContainerSize, typename... Arguments>
void test_element_wise_function (ElementWiseFunction element_wise_function, const gsl::not_null< std::tuple< Arguments... > *> arguments, AtFunction at=GetContainerElement{}, SizeFunction size=GetContainerSize{}, Approx custom_approx=approx) noexcept
 Utility function for verifying the action of an element-wise function on containers, or on some combination of containers and compatible non-containers (e.g. DataVectors with doubles). More...
 
template<typename ReturnType , typename T , typename UniformRandomBitGenerator , typename RandomNumberDistribution >
ReturnType make_with_random_values (const gsl::not_null< UniformRandomBitGenerator *> generator, const gsl::not_null< RandomNumberDistribution *> distribution, const T &used_for_size) noexcept
 Make a data structure and fill it with random values. More...
 
template<typename ReturnType , typename T , typename UniformRandomBitGenerator , typename RandomNumberDistribution >
ReturnType make_with_random_values (const gsl::not_null< UniformRandomBitGenerator *> generator, RandomNumberDistribution distribution, const T &used_for_size) noexcept
 Make a data structure and fill it with random values. More...
 
template<class EosType , class T , class... MemberArgs>
void TestHelpers::EquationsOfState::check (std::unique_ptr< EosType > in_eos, const std::string &python_function_prefix, const T &used_for_size, const MemberArgs &... member_args) noexcept
 Test an equation of state by comparing to python functions. More...
 
template<class EosType , class T , class... MemberArgs>
void TestHelpers::EquationsOfState::check (EosType in_eos, const std::string &python_function_prefix, const T &used_for_size, const MemberArgs &... member_args) noexcept
 Test an equation of state by comparing to python functions. More...
 

Detailed Description

Classes, functions, macros, and instructions for developing tests.

Details

SpECTRE uses the testing framework Catch. Catch supports a variety of different styles of tests including BDD and fixture tests. The file cmake/SpectreAddCatchTests.cmake parses the source files and adds the found tests to ctest with the correct properties specified by tags and attributes.

Usage

To run the tests, type ctest in the build directory. You can specify a regex to match the test name using ctest -R Unit.Blah, or run all tests with a certain tag using ctest -L tag.

Comparing double-precision results

To compare two floating-point numbers that may differ by round-off, use the helper object approx. This is an instance of Catch's comparison class Approx in which the relative tolerance for comparisons is set to roughly \(10^{-14}\) (i.e. std::numeric_limits<double>::epsilon()*100). When possible, we recommend using approx for fuzzy comparisons as follows:

Example

CHECK(sin(M_PI / 4.0) == approx(cos(M_PI / 4.0)));

For checks that need more control over the precision (e.g. an algorithm in which round-off errors accumulate to a higher level), we recommend using the approx helper with a one-time tolerance adjustment. A comment should explain the reason for the adjustment:

Example

// This check needs tolerance 1e-12 for X reason.
CHECK(1.0 == approx(1.0 + 5e-13).epsilon(1e-12));

For tests in which the same precision adjustment is re-used many times, a new helper object can be created from Catch's Approx with a custom precision:

Example

// The checks in this test need tolerance 1e-12 for X reason.
Approx my_approx = Approx::custom().epsilon(1e-12);
CHECK(1.0 == my_approx(1.0 + 5e-13));
CHECK(1.0 != my_approx(1.0 + 5e-12));

Note: We provide the approx object because Catch's Approx defaults to a very loose tolerance (std::numeric_limits<float>::epsilon()*100, or roughly \(10^{-5}\) relative error), and so is poorly-suited to checking many numerical algorithms that rely on double-precision accuracy. By providing a tighter tolerance with approx, we avoid having to redefine the tolerance in every test.

Attributes

Attributes allow you to modify properties of the test. Attributes are specified as follows:

// [[TimeOut, 10]]
// [[OutputRegex, The error message expected from the test]]
SPECTRE_TEST_CASE("Unit.Blah", "[Unit]") {

Available attributes are:

Attribute Description
TimeOut override the default timeout and set the timeout to N seconds. This should be set very sparingly since unit tests are designed to be short. If your test is too long you should consider testing smaller portions of the code if possible, or writing an integration test instead.
OutputRegex When testing failure modes the exact error message must be tested, not just that the test failed. Since the string passed is a regular expression you must escape any regex tokens. For example, to match some (word) and you must specify the string some \(word\) and. If your error message contains a newline, you can match it using the dot operator ., which matches any character.

Example

// [[OutputRegex, File './Unit.IO.H5.FileErrorExists.h5' already exists and we
// are not allowed to append. To reduce the risk of accidental deletion you must
// explicitly delete the file first using the file_system library in
// SpECTRE or through your shell.]]
SPECTRE_TEST_CASE("Unit.IO.H5.FileErrorExists", "[Unit][IO][H5]") {

Testing static assert

You are able to test that a static_assert is being triggered using the compilation failure test framework. When creating a new static_assert test you must be sure to not have it in the same file as the runtime tests since the file will not compile. The new file, say Test_StaticAssertDataBox.cpp must be added to the SPECTRE_COMPILATION_TESTS CMake variable, not SPECTRE_TESTS. Here is an example of how to write a compilation failure test:

#ifdef COMPILATION_TEST_TEST_FRAMEWORK_WORKS
// [[TAGS: unit, CompilationTest]]
// [[COMPILER: all REGEX: Testing compilation failure tests]]
static_assert(false, "Testing compilation failure tests");
int main() {}
#endif
FILE_IS_COMPILATION_TEST

Each individual test must be inside an #ifdef COMPILATION_TEST_.* block and each compilation test cpp file must contain FILE_IS_COMPILATION_TEST outside of any #ifdefs and at the end of the file.

Specific compiler versions can be specified for which the regex changes. That is, the compiler version specified and all versions newer than that will use the regex, until a newer compiler version is specified. For example, see the below code prints a different static_assert for pre-GCC 6 and GCC 6 and newer.

#ifdef COMPILATION_TEST_TEST_DIFFERENT_COMPILERS
// [[TAGS: unit, CompilationTest]]
// [[COMPILER: GNU:0.0.0 REGEX: assert with GCC 5]]
// [[COMPILER: GNU:6.0.0 REGEX: assert with GCC 6 or newer]]
// [[COMPILER: Clang REGEX: assert with Clang]]
// [[COMPILER: AppleClang REGEX: assert with AppleClang]]
#ifdef __APPLE__
static_assert(false, "assert with AppleClang");
#else // __APPLE__
#ifdef __clang__
static_assert(false, "assert with Clang");
#else // __clang__
#if __GNUC__ < 6
static_assert(false, "assert with GCC 5");
#else // __GNUC__ < 6
static_assert(false, "assert with GCC 6 or newer");
#endif // __GNUC__ < 6
#endif // __clang__
#endif // __APPLE__
int main() {}
#endif

Debugging Tests in GDB or LLDB

Several tests fail intentionally at the executable level to test error handling like ASSERT statements in the code. CTest is aware of which should fail and passes them. If you want to debug an individual test in a debugger you need to run a single test using the RunTests executable (in dg-charm-build/bin/RunTests) you must specify the name of the test as the first argument. For example, if you want to run just the "Unit.Gradient" test you can run ./bin/RunTests Unit.Gradient. If you are using a debugger launch the debugger, for example if you're using LLDB then run lldb ./bin/RunTests and then to run the executable inside the debugger use run Unit.Gradient inside the debugger.

Macro Definition Documentation

◆ ASSERTION_TEST

#define ASSERTION_TEST ( )
Value:
do { \
ERROR_TEST(); \
Parallel::abort("### No ASSERT tests in release mode ###"); \
} while (false)
void abort(const std::string &message)
Abort the program with an error message.
Definition: Abort.hpp:17

Mark a test to be checking an ASSERT.

Details

Testing error handling is just as important as testing functionality. Tests that are supposed to exit with an error must be annotated with the attribute

// [[OutputRegex, The regex that should be found in the output]]

Note that the regex only needs to be a sub-expression of the error message, that is, there are implicit wildcards before and after the string.

In order to test ASSERT's properly the test must also fail for release builds. This is done by adding this macro at the beginning for the test.

Example

// [[OutputRegex, Out of range slab fraction]]
[[noreturn]] SPECTRE_TEST_CASE("Unit.Time.Time.Init.0", "[Unit][Time]") {
#ifdef SPECTRE_DEBUG
Time(Slab(0., 1.), -1);
ERROR("Failed to trigger ASSERT in an assertion test");
#endif
}

◆ CAPTURE_PRECISE

#define CAPTURE_PRECISE (   variable)
Value:
INFO(#variable << ": " \
<< TestHelpers_detail::format_capture_precise(variable))

Alternative to Catch's CAPTURE that prints more digits.

◆ CHECK_CUSTOM_ELEMENT_WISE_FUNCTION_APPROX

#define CHECK_CUSTOM_ELEMENT_WISE_FUNCTION_APPROX (   function,
  arguments,
  at_operator,
  size_of_operator 
)
Value:
do { \
INFO(__FILE__ ":" + std::to_string(__LINE__) + \
": " #function ", " #arguments); \
test_element_wise_function(function, arguments, at_operator, \
size_of_operator); \
} while (false)

Same as CHECK_ELEMENT_WISE_FUNCTION_APPROX, but with a user-defined function at_operator and size_of_operator, each of which correspond to arguments of test_element_wise_function() (which gives a more complete documentation of the element-wise checking operations and arguments).

◆ CHECK_ELEMENT_WISE_FUNCTION_APPROX

#define CHECK_ELEMENT_WISE_FUNCTION_APPROX (   function,
  arguments 
)
Value:
do { \
INFO(__FILE__ ":" + std::to_string(__LINE__) + \
": " #function ", " #arguments); \
test_element_wise_function(function, arguments); \
} while (false)

A test utility for verifying that an element-wise function, function acts identically to the same operation applied to each element of a container separately. This macro invokes test_element_wise_function() (which gives a more complete documentation of the element-wise checking operations and arguments).

◆ CHECK_FOR_DOUBLES_AND_DATAVECTORS

#define CHECK_FOR_DOUBLES_AND_DATAVECTORS (   FUNCTION_NAME,
  ... 
)
Value:
CHECK_FOR_DOUBLES(FUNCTION_NAME, __VA_ARGS__) \
CHECK_FOR_DATAVECTORS(FUNCTION_NAME, __VA_ARGS__)

Macro used to test functions whose parameter can be a double or a DataVector.

In testing multiple instances of a function template using random values, it often proves useful to write a wrapper around pypp::check_with_random_values. This way, one can easily loop over several values of one or multiple template parameters (e.g. when testing a function templated in the number of spacetime dimensions.) The template parameters of the wrapper will then correspond to the template parameters of the function, which will be used by pypp::check_with_random_values to invoke and test each instance. Each of these wrappers will generally need only one parameter, namely a variable used_for_size passed to pypp::check_with_random_values that can be a double, a DataVector, or both (provided that the function being tested is templated in the type of used_for_size.) Since this is applied in multiple test files, all of these files will share the same way to generate the required calls to the wrapper.

This macro, along with

CHECK_FOR_DOUBLES(FUNCTION_NAME, ...)
CHECK_FOR_DATAVECTORS(FUNCTION_NAME, ...)

allow to generate calls to multiple instances of a test function template in the same way as done by INVOKE_TEST_FUNCTION(FUNCTION_NAME, ARGS_TUPLE, ...) (to which these macros call), except that the tuple of arguments is not passed, as these macros will assume that a double d and/or a DataVector dv will be previously defined. Although any ds and dvs will work, one can (and it is recommended to) generate signaling NaN values for d and dv. This can be done by invoking one of the three provided macros: GENERATE_UNINIATILIZED_DOUBLE, GENERATE_UNINITIALIZED_DATAVECTOR, or GENERATE_UNINITIALIZED_DOUBLE_AND_DATAVECTOR. For example,

GENERATE_UNINITIALIZED_DATAVECTOR;
CHECK_FOR_DATAVECTORS(test_fluxes, (1, 2, 3))

will generate a test case for 1, 2 and 3 dimensions:

const DataVector dv(5);
test_fluxes<1>(dv);
test_fluxes<2>(dv);
test_fluxes<3>(dv);

Analogously, the wrapper

template <size_t Dim, IndexType TypeOfIndex, typename DataType>
test_ricci(const DataType& used_for_size) noexcept { ... }

can be invoked by writing

GENERATE_UNINITIALIZED_DOUBLE_AND_DATAVECTOR;
CHECK_FOR_DOUBLES_AND_DATAVECTORS(test_ricci, (1, 2, 3),

which will generate

const DataVector dv(5);
test_ricci<1, IndexType::Spatial>(d);
test_ricci<1, IndexType::Spacetime>(d);
test_ricci<2, IndexType::Spatial>(d);
test_ricci<2, IndexType::Spacetime>(d);
test_ricci<3, IndexType::Spatial>(d);
test_ricci<3, IndexType::Spacetime>(d);
test_ricci<1, IndexType::Spatial>(dv);
test_ricci<1, IndexType::Spacetime>(dv);
test_ricci<2, IndexType::Spatial>(dv);
test_ricci<2, IndexType::Spacetime>(dv);
test_ricci<3, IndexType::Spatial>(dv);
test_ricci<3, IndexType::Spacetime>(dv);

Note that it is not necessary to pass values for DataType, as they are deduced from used_for_size.

Note
The order of the macro-tuples of values must match the order of the template parameters of the function.
The function to be called must at least have one template argument, so passing an empty set of template parameters will not work.

◆ CHECK_ITERABLE_APPROX

#define CHECK_ITERABLE_APPROX (   a,
 
)
Value:
do { \
INFO(__FILE__ ":" + std::to_string(__LINE__) + ": " #a " == " #b); \
check_iterable_approx<std::common_type_t< \
std::decay_t<decltype(a)>, std::decay_t<decltype(b)>>>::apply(a, b); \
} while (false)
constexpr auto apply(F &&f, const DataBox< BoxTags > &box, Args &&... args)
Apply the function f with argument Tags TagsList from DataBox box
Definition: DataBox.hpp:1595

A wrapper around Catch's CHECK macro that checks approximate equality of entries in iterable containers. For maplike containers, keys are checked for strict equality and values are checked for approximate equality.

◆ CHECK_ITERABLE_CUSTOM_APPROX

#define CHECK_ITERABLE_CUSTOM_APPROX (   a,
  b,
  appx 
)
Value:
do { \
INFO(__FILE__ ":" + std::to_string(__LINE__) + ": " #a " == " #b); \
check_iterable_approx<std::common_type_t< \
std::decay_t<decltype(a)>, std::decay_t<decltype(b)>>>::apply(a, b, \
appx); \
} while (false)
constexpr auto apply(F &&f, const DataBox< BoxTags > &box, Args &&... args)
Apply the function f with argument Tags TagsList from DataBox box
Definition: DataBox.hpp:1595

Same as CHECK_ITERABLE_APPROX with user-defined Approx. The third argument should be of type Approx.

◆ CHECK_OP

#define CHECK_OP (   a,
  op,
  b,
 
)
Value:
do { \
const auto& a_ = a; \
const auto& b_ = b; \
const auto& c_ = c; \
CHECK(a_ op b_ == c_); \
auto f = a_; \
CHECK((f op## = b_) == c_); \
CHECK(f == c_); \
} while (false)

Check a op b == c and also the op= version.

◆ CHECK_VARIABLES_APPROX

#define CHECK_VARIABLES_APPROX (   a,
 
)
Value:
do { \
INFO(__FILE__ ":" + std::to_string(__LINE__) + ": " #a " == " #b); \
check_variables_approx<std::common_type_t< \
std::decay_t<decltype(a)>, std::decay_t<decltype(b)>>>::apply(a, b); \
} while (false)
constexpr auto apply(F &&f, const DataBox< BoxTags > &box, Args &&... args)
Apply the function f with argument Tags TagsList from DataBox box
Definition: DataBox.hpp:1595

A wrapper around Catch's CHECK macro that checks approximate equality of each entry in each tag within a variables.

◆ CHECK_VARIABLES_CUSTOM_APPROX

#define CHECK_VARIABLES_CUSTOM_APPROX (   a,
  b,
  appx 
)
Value:
do { \
INFO(__FILE__ ":" + std::to_string(__LINE__) + ": " #a " == " #b); \
check_variables_approx<std::common_type_t< \
std::decay_t<decltype(a)>, std::decay_t<decltype(b)>>>::apply(a, b, \
appx); \
} while (false)
constexpr auto apply(F &&f, const DataBox< BoxTags > &box, Args &&... args)
Apply the function f with argument Tags TagsList from DataBox box
Definition: DataBox.hpp:1595

Same as CHECK_VARIABLES_APPROX, but with a user-defined Approx. The third argument should be of type Approx.

◆ ERROR_TEST

#define ERROR_TEST ( )
Value:
do { \
std::signal(SIGABRT, spectre_testing_signal_handler); \
} while (false)

Mark a test as checking a call to ERROR.

Details

In order to properly handle aborting with Catch versions newer than 1.6.1 we must install a signal handler after Catch does, which means inside the SPECTRE_TEST_CASE itself. The ERROR_TEST() macro should be the first line in the SPECTRE_TEST_CASE.

Example

// [[OutputRegex, I failed]]
[[noreturn]] SPECTRE_TEST_CASE("Unit.TestingFramework.Abort", "[Unit]") {

◆ INVOKE_TEST_FUNCTION

#define INVOKE_TEST_FUNCTION (   FUNCTION_NAME,
  TUPLE_ARGS,
  ... 
)
Value:
BOOST_PP_ASSERT_MSG(BOOST_PP_NOT(BOOST_VMD_IS_EMPTY(__VA_ARGS__)), \
"You cannot pass an empty set of template parameters " \
"to INVOKE_TEST_FUNCTION") \
BOOST_PP_TUPLE_ENUM( \
0, \
BOOST_PP_IF( \
BOOST_PP_EQUAL( \
BOOST_PP_TUPLE_SIZE(BOOST_PP_VARIADIC_TO_TUPLE(__VA_ARGS__)), \
1), \
(BOOST_PP_LIST_FOR_EACH( \
INVOKE_FUNCTION_WITH_SINGLE_TEMPLATE_PARAM, \
(FUNCTION_NAME, TUPLE_ARGS), \
BOOST_PP_TUPLE_TO_LIST( \
BOOST_PP_VARIADIC_ELEM(0, __VA_ARGS__)))), \
(INVOKE_FUNCTION_WITH_MANY_TEMPLATE_PARAMS( \
BOOST_PP_TUPLE_PUSH_FRONT( \
BOOST_PP_LIST_TO_TUPLE(BOOST_PP_LIST_TRANSFORM( \
INVOKE_FUNCTION_WITH_MANY_TEMPLATE_PARAMS_TUPLE_TO_LIST, \
_, \
BOOST_PP_LIST_REST( \
BOOST_PP_VARIADIC_TO_LIST(__VA_ARGS__)))), \
BOOST_PP_LIST_TRANSFORM( \
INVOKE_FUNCTION_TUPLE_PUSH_BACK, \
(FUNCTION_NAME, TUPLE_ARGS), \
BOOST_PP_TUPLE_TO_LIST( \
BOOST_PP_VARIADIC_ELEM(0, __VA_ARGS__))))))))

Macro used to invoke a test function of multiple template arguments.

This macro allows to generate calls to multiple instances of a test function template, all of which will receive the same parameters. The first argument to this macro is the name of the function. The second argument is a macro-tuple containing the parameters passed to each instance, e.g. (x, y). The remaining arguments are macro-tuples of the values for each template parameter one wants to loop over, e.g. (1, 2, 3), (Frame::Inertial, Frame::Grid). For example, a function template

template <class Arg1, size_t Arg2, class Arg3>
my_function(const double& var_1, const int& var_2) noexcept { ... }

can be invoked by writing

INVOKE_TEST_FUNCTION(my_function, (d, i), (a, b, c), (1, 2, 3), (A, B, C))

which will generate

my_function<a, 1, A>(d, i);
my_function<a, 1, B>(d, i);
my_function<a, 1, C>(d, i);
my_function<a, 2, A>(d, i);
my_function<a, 2, B>(d, i);
my_function<a, 2, C>(d, i);
my_function<a, 3, A>(d, i);
my_function<a, 3, B>(d, i);
my_function<a, 3, C>(d, i);
my_function<b, 1, A>(d, i);
my_function<b, 1, B>(d, i);
my_function<b, 1, C>(d, i);
my_function<b, 2, A>(d, i);
my_function<b, 2, B>(d, i);
my_function<b, 2, C>(d, i);
my_function<b, 3, A>(d, i);
my_function<b, 3, B>(d, i);
my_function<b, 3, C>(d, i);
my_function<c, 1, A>(d, i);
my_function<c, 1, B>(d, i);
my_function<c, 1, C>(d, i);
my_function<c, 2, A>(d, i);
my_function<c, 2, B>(d, i);
my_function<c, 2, C>(d, i);
my_function<c, 3, A>(d, i);
my_function<c, 3, B>(d, i);
my_function<c, 3, C>(d, i);
Note
The order of the macro-tuples of values must match the order of the template parameters of the function.
The function to be called must at least have one template argument, so passing an empty set of template parameters will not work.

◆ MAKE_GENERATOR

#define MAKE_GENERATOR (   ...)
Value:
std::mt19937 MAKE_GENERATOR_IMPL_FIRST_ARG(__VA_ARGS__, DUMMY_TOKEN); \
INFO("Seed is: " << [&MAKE_GENERATOR_IMPL_FIRST_ARG( \
__VA_ARGS__, DUMMY_TOKEN)]() noexcept { \
const auto seed = (MAKE_GENERATOR_IMPL_SECOND_ARG( \
__VA_ARGS__, std::random_device{}(), DUMMY_TOKEN)); \
MAKE_GENERATOR_IMPL_FIRST_ARG(__VA_ARGS__, DUMMY_TOKEN).seed(seed); \
return seed; \
}())

MAKE_GENERATOR(NAME [, SEED]) declares a variable of name NAME containing a generator of type std::mt19937.

Details

As the generator is made, INFO is called to make sure failed tests provide seed information. SEED is chosen randomly if not supplied, otherwise it must be a constant expression.

◆ SPECTRE_PARALLEL_REQUIRE

#define SPECTRE_PARALLEL_REQUIRE (   expr)
Value:
do { \
if (not(expr)) { \
ERROR("\nFailed comparison: " << #expr << "\nLine: " << __LINE__ \
<< "\nFile: " << __FILE__ << "\n"); \
} \
} while (false)

A similar to Catch's REQUIRE statement, but can be used in tests that spawn several chares with possibly complex interaction between the chares.

◆ SPECTRE_PARALLEL_REQUIRE_FALSE

#define SPECTRE_PARALLEL_REQUIRE_FALSE (   expr)
Value:
do { \
if ((expr)) { \
ERROR("\nFailed comparison: " << #expr << "\nLine: " << __LINE__ \
<< "\nFile: " << __FILE__ << "\n"); \
} \
} while (false)

A similar to Catch's REQUIRE_FALSE statement, but can be used in tests that spawn several chares with possibly complex interaction between the chares.

Enumeration Type Documentation

◆ TestKind

the set of test types that may be used for the math operations

Details

Three types of test are provided:

  • Normal is used to indicate those tests which should be performed over all combinations of the supplied vector type(s) and their value types. This is useful for e.g. +.
  • Strict is used to indicate those tests which should be performed over only sets of the vector type and compared to the same operation of the set of its value type. This is useful for e.g. atan2, which cannot take a DataVector and a double as arguments.
  • Inplace is used to indicate those tests which should be performed maintaining the type of the left-hand side of the operator and not including it in the combinations. Inplace operators such as += have a more restrictive condition on the type of the left hand side than do simply +. (e.g. double + complex<double> compiles, but double += complex<double> does not)
  • GivenOrderOfArgumentsOnly is used to indicate that the arguments given should not be taken in any combination apart from the given combination. This should be used for highly restrictive operations which are only supported for certain type combinations.

Function Documentation

◆ check() [1/2]

template<class EosType , class T , class... MemberArgs>
void TestHelpers::EquationsOfState::check ( std::unique_ptr< EosType >  in_eos,
const std::string python_function_prefix,
const T &  used_for_size,
const MemberArgs &...  member_args 
)
noexcept

Test an equation of state by comparing to python functions.

The python functions must be added to tests/Unit/PointwiseFunctions/Hydro/EquationsOfState/TestFunctions.py. The prefix for each class of equation of state is arbitrary, but should generally be something like "polytropic" for polytropic fluids.

The python_function_prefix argument passed to check must be PREFIX. If an EoS class has member variables (these must be doubles currently) that are used to compute the quantities, such as the polytropic constant and polytropic exponent for a fluid, then they must be passed in as the last arguments to the check function`. Each python function must take these same arguments as the trailing arguments.

◆ check() [2/2]

template<class EosType , class T , class... MemberArgs>
void TestHelpers::EquationsOfState::check ( EosType  in_eos,
const std::string python_function_prefix,
const T &  used_for_size,
const MemberArgs &...  member_args 
)
noexcept

Test an equation of state by comparing to python functions.

The python functions must be added to tests/Unit/PointwiseFunctions/Hydro/EquationsOfState/TestFunctions.py. The prefix for each class of equation of state is arbitrary, but should generally be something like "polytropic" for polytropic fluids.

The python_function_prefix argument passed to check must be PREFIX. If an EoS class has member variables (these must be doubles currently) that are used to compute the quantities, such as the polytropic constant and polytropic exponent for a fluid, then they must be passed in as the last arguments to the check function`. Each python function must take these same arguments as the trailing arguments.

◆ make_with_random_values() [1/3]

template<typename ReturnType , typename T , typename UniformRandomBitGenerator , typename RandomNumberDistribution >
ReturnType make_with_random_values ( const gsl::not_null< UniformRandomBitGenerator *>  generator,
const gsl::not_null< RandomNumberDistribution *>  distribution,
const T &  used_for_size 
)
noexcept

Make a data structure and fill it with random values.

Details

Given an object of type T, create an object of type ReturnType whose elements are initialized to random values using the given random number generator and random number distribution.

Requires: the type ReturnType to be creatable using make_with_value<ReturnType>(T)

◆ make_with_random_values() [2/3]

template<typename ReturnType , typename T , typename UniformRandomBitGenerator , typename RandomNumberDistribution >
ReturnType make_with_random_values ( const gsl::not_null< UniformRandomBitGenerator *>  generator,
RandomNumberDistribution  distribution,
const T &  used_for_size 
)
noexcept

Make a data structure and fill it with random values.

Details

Given an object of type T, create an object of type ReturnType whose elements are initialized to random values using the given random number generator and random number distribution.

Requires: the type ReturnType to be creatable using make_with_value<ReturnType>(T)

◆ make_with_random_values() [3/3]

template<typename T , typename UniformRandomBitGenerator , typename RandomNumberDistribution >
T make_with_random_values ( const gsl::not_null< UniformRandomBitGenerator *>  generator,
const gsl::not_null< RandomNumberDistribution *>  distribution 
)
noexcept

Make a fixed-size data structure and fill with random values.

Details

Given a template argument type T, create an object of the same type, fills it with random values, and returns the result. Acts as a convenience function to avoid users needing to put in constructors with signaling_NaN()s or max()s themselves when making with random values. Used as make_with_random_values<Type>(make_not_null(&gen),make_not_null(&dist))

◆ numerical_derivative()

template<typename Invocable , size_t VolumeDim>
std::array<double, VolumeDim> numerical_derivative ( const Invocable &  map,
const std::array< double, VolumeDim > &  x,
const size_t  direction,
const double  delta 
)

Calculates the derivative of an Invocable at a point x - represented by an array of doubles - in the domain of map with a sixth-order finite difference method.

Details

Intended for use with CoordinateMaps taking the domain {xi,eta,zeta} to the range {x,y,z}. This function calculates the derivative along the direction given by direction with a step size of h.

Requires: direction be between 0 and VolumeDim

◆ test_element_wise_function()

template<typename ElementWiseFunction , typename AtFunction = GetContainerElement, typename SizeFunction = GetContainerSize, typename... Arguments>
void test_element_wise_function ( ElementWiseFunction  element_wise_function,
const gsl::not_null< std::tuple< Arguments... > *>  arguments,
AtFunction  at = GetContainerElement{},
SizeFunction  size = GetContainerSize{},
Approx  custom_approx = approx 
)
noexcept

Utility function for verifying the action of an element-wise function on containers, or on some combination of containers and compatible non-containers (e.g. DataVectors with doubles).

Details

The ability to specify custom functions for at and size is useful for more intricate containers. For instance, multidimensional types can be used with this function with a size function that returns the full number of elements, and an at function which indexes the multidimensional type in a flattened fashion.

parameters:

Parameters
element_wise_functionA callable which is expected to act in an element-wise fashion, must be compatible both with the container and its individual elements.
argumentsA tuple of arguments to be tested
atA function to override the container access function. Defaults to an object which simply calls container.at(i). A custom callable must take as arguments the container(s) used in arguments and a size_t index (in that order), and return an element compatible with element_wise_function. This function signature follows the convention of gsl::at.
sizeA function to override the container size function. Defaults to an object which simply calls container.size(). A custom callable must take as argument the container(s) used in arguments, and return a size_t. This function signature follows the convention of cpp17::size.
custom_approxAn object of type Approx specifying an alternative precision with which to test the element-wise function

◆ test_functions_with_vector_arguments()

template<TestKind Test, typename VectorType0 , typename... VectorTypes, typename... FunctionsAndArgumentBounds>
void TestHelpers::VectorImpl::test_functions_with_vector_arguments ( const std::tuple< FunctionsAndArgumentBounds... > &  tuple_of_functions_and_argument_bounds)
noexcept

General entry function for testing arbitrary math functions on vector types.

Details

This utility tests all combinations of the operator on the type arguments, and all combinations of reference or constant reference wrappers on all arguments. In certain test cases (see below), it also tests using the vector type's value_types in the operators as well (e.g. DataVector + double). This is very useful for quickly generating a lot of tests, but the number of tests scales exponentially in the number of arguments. Therefore, functions with many arguments can be time-consuming to run. 4-or-more-argument functions should be used only if completely necessary and with caution. Any number of vector types may be specified, and tests are run on all unique combinations of the provided. For instance, if only one type is provided, the tests will be run only on combinations of that single type and its value_type.

Parameters
tuple_of_functions_and_argument_boundsA tuple of tuples, in which the inner tuple contains first a function object followed by a tuple of 2-element arrays equal to the number of arguments, which represent the bounds for the random generation of the respective arguments. This system is provided for robust testing of operators like /, where the left-hand side has a different valid set of values than the right-hand-side.
Template Parameters
Testfrom the TestKind enum, determines whether the tests will be:
  • TestKind::Normal: executed on all combinations of arguments and value types
  • TestKind::Strict: executed on all combinations of arguments, for only the vector types
  • TestKind::Inplace: executed on all combinations of arguments after the first, so first is always the 'left hand side' of the operator. In this case, at least two VectorTypes must be specified, where the first is used only for the left-hand side.
  • TestKind::GivenOrderOfArgumentsOnly: executed on only the combination of arguments provided, in the order provided. In this case, the number of provided types in typename VectorType0, typename... VectorTypes must precisely match the number of arguments taken by the function.
VectorType0The first vector type for which combinations are tested. The first is accepted as a separate template argument for appropriately handling Inplace tests.
VectorTypesThe remaining types for which combinations are tested. Any number of types may be passed in, and the test will check the appropriate combinations of the vector types and (depending on the Test) the respective value_types.

◆ test_serialization()

template<typename T >
void test_serialization ( const T &  t)

Tests the serialization of comparable types.

Example

SPECTRE_TEST_CASE("Unit.Serialization.tuple", "[Serialization][Unit]") {
um["aaa"] = 1.589;
um["bbb"] = -10.7392;
auto test_tuple = std::make_tuple<int, double, std::string,
2, 0.57, "blah", std::move(um));
test_serialization(test_tuple);
}

◆ test_serialization_via_base()

template<typename B , typename D , typename... Args>
void test_serialization_via_base ( Args &&...  args)

Test the serialization of a derived class via a base class pointer.

Example

SPECTRE_TEST_CASE("Unit.Serialization.unique_ptr.abstract_base",
"[Serialization][Unit]") {
test_serialization_via_base<Test_Classes::Base,
Test_Classes::DerivedInPupStlCpp11>(
std::vector<double>{-1, 12.3, -7, 8});
}
Template Parameters
Bthe base class
Dthe derived class
Argsdeduced from args
Parameters
argsarguments passed to a constructor of the derived class

◆ test_throw_exception()

template<typename Exception , typename ThrowingFunctor >
void test_throw_exception ( const ThrowingFunctor &  func,
const Exception &  expected 
)

Execute func and check that it throws an exception expected.

Note
The .what() strings of the thrown and expected exceptions are compared for a partial match only: the expected.what() string must be contained in (or equal to) the .what() string of the thrown exception.

◆ vector_ref_test_size_error()

template<typename VectorType , typename ValueType = typename VectorType::ElementType>
void TestHelpers::VectorImpl::vector_ref_test_size_error ( RefSizeErrorTestKind  test_kind,
tt::get_fundamental_type_t< ValueType >  low = tt::get_fundamental_type_t<ValueType>{-100.0},
tt::get_fundamental_type_t< ValueType >  high = tt::get_fundamental_type_t<ValueType>{100.0} 
)
noexcept

Test that assigning to a non-owning VectorType of the wrong size appropriately generates an error.

Details

a calling function should be an ASSERTION_TEST() and check for the string "Must copy into same size". Three types of tests are provided and one must be provided as the first function argument:

  • RefSizeErrorTestKind::Copy: Checks that copy-assigning to a non-owning VectorType from a VectorType with the wrong size generates an error.
  • RefSizeErrorTestKind::ExpressionAssign: Checks that assigning to a non-owning VectorType from an expression with alias ResultType of VectorType with the wrong size generates an error
  • RefSizeErrorTestKind::Move: Checks that move-assigning to a non-owning VectorType from a VectorType with the wrong size generates an error.