SpECTRE  v2024.09.29
pypp Namespace Reference

Contains all functions for calling python from C++. More...

Classes

struct  SetupLocalPythonEnvironment
 Enable calling of python in the local scope, and add directory(ies) to the front of the search path for modules. The directory which is appended to the path is relative to the tests/Unit directory. More...
 

Functions

template<size_t NumberOfBounds, class F , class T , Requires< not std::is_same_v< typename tt::function_info< cpp20::remove_cvref_t< F > >::return_type, void > > = nullptr>
void check_with_random_values (F &&f, const std::string &module_name, const std::string &function_name, const std::array< std::pair< double, double >, NumberOfBounds > &lower_and_upper_bounds, const T &used_for_size, const double epsilon=1.0e-12, const typename std::random_device::result_type seed=std::random_device{}())
 Tests a C++ function returning by value by comparing the result to a python function. More...
 
template<size_t NumberOfBounds, class F , class T >
void check_with_random_values (F &&f, const std::string &module_name, const std::vector< std::string > &function_names, const std::array< std::pair< double, double >, NumberOfBounds > &lower_and_upper_bounds, const T &used_for_size, const double epsilon=1.0e-12, const typename std::random_device::result_type seed=std::random_device{}(), const std::optional< double > &initial_result_values=std::nullopt)
 Tests a C++ function returning by gsl::not_null by comparing the result to a python function. More...
 
template<size_t NumberOfBounds, class F , class T , class... MemberArgs, Requires< not std::is_same_v< typename tt::function_info< cpp20::remove_cvref_t< F > >::return_type, void > and not tt::is_a_v< tuples::TaggedTuple, typename tt::function_info< cpp20::remove_cvref_t< F > >::return_type > > = nullptr>
void check_with_random_values (F &&f, const typename tt::function_info< cpp20::remove_cvref_t< F > >::class_type &klass, const std::string &module_name, const std::string &function_name, const std::array< std::pair< double, double >, NumberOfBounds > &lower_and_upper_bounds, const std::tuple< MemberArgs... > &member_args, const T &used_for_size, const double epsilon=1.0e-12, const typename std::random_device::result_type seed=std::random_device{}())
 Tests a member function of a class returning by value by comparing the result to a python function. More...
 
template<size_t NumberOfBounds, class F , class T , class... MemberArgs, Requires< std::is_same_v< typename tt::function_info< cpp20::remove_cvref_t< F > >::return_type, void > or tt::is_a_v< tuples::TaggedTuple, typename tt::function_info< cpp20::remove_cvref_t< F > >::return_type > > = nullptr>
void check_with_random_values (F &&f, const typename tt::function_info< cpp20::remove_cvref_t< F > >::class_type &klass, const std::string &module_name, const std::vector< std::string > &function_names, const std::array< std::pair< double, double >, NumberOfBounds > &lower_and_upper_bounds, const std::tuple< MemberArgs... > &member_args, const T &used_for_size, const double epsilon=1.0e-12, const typename std::random_device::result_type seed=std::random_device{}())
 Tests a member function of a class returning by either gsl::not_null or TaggedTuple by comparing the result to a python function. More...
 
template<typename ReturnType , typename ConversionClassList = tmpl::list<>, typename... Args>
ReturnType call (const std::string &module_name, const std::string &function_name, const Args &... t)
 Calls a Python function from a module/file with given parameters. More...
 
template<typename... Args>
PyObject * make_py_tuple (const Args &... t)
 Create a python tuple from Args. More...
 

Detailed Description

Contains all functions for calling python from C++.

Contains all functions for pypp.

Function Documentation

◆ call()

template<typename ReturnType , typename ConversionClassList = tmpl::list<>, typename... Args>
ReturnType pypp::call ( const std::string module_name,
const std::string function_name,
const Args &...  t 
)

Calls a Python function from a module/file with given parameters.

Parameters
module_namename of module the function is in
function_namename of Python function in module
tthe arguments to be passed to the Python function

Returns: the object returned by the Python function converted to a C++ type

Custom conversion from containers to basic types that can be converted to python objects via the pypp::ToPyObject class can be added by passing a typelist of conversion class as the second template parameter to pypp::call. This is generally used for converting classes like Tensor<DataVector, ...> to a Tensor<double, ...> to support using numpy.einsum in the python code. However, this can also be used to convert a complicated class such as an equation of state to a bunch of numbers that the python code for the particular test can use to compute the required data without having to fully implement python bindings. The conversion classes must have the following:

  • a type alias unpacked_container that is the type at a single grid point.
  • a type alias packed_container the type used when converting the result from the python call back into the C++ code.
  • a type alias packed_type which corresponds to the packed type that the high-level container holds. For example, a Scalar<DataVector> would have packed_type being DataVector, a std::vector<Scalar<DataVector>> would have packed_type being DataVector, and a std::vector<Scalar<double>> would have packed_type being double.
  • an unpack function that takes as its arguments the packed_container and the grid_point_index, and returns an unpacked_container object
  • a pack function that takes as its arguments a gsl::not_null<packed_container*>, an const unpacked_container&, and a size_t corresponding to which grid point to pack
  • a get_size function that takes as its only argument an object of packed_container and returns the number of elements in the packed_type

Examples of conversion classes can be found in the specializations of ContainerPackAndUnpack in the Pypp.hpp file. Below is an example of a conversion class that takes a class (ClassForConversionTest) and returns the a_ member variable.

struct ClassForConversionTest {
double a_;
double b_;
};
struct ConvertClassForConservionTestA {
using unpacked_container = double;
using packed_container = ClassForConversionTest;
using packed_type = double;
static inline unpacked_container unpack(const packed_container t,
const size_t /*grid_point_index*/) {
return t.a_;
}
static inline void pack(const gsl::not_null<packed_container*> packed_t,
const unpacked_container t,
const size_t /*grid_point_index*/) {
packed_t->a_ = t;
}
static inline size_t get_size(const packed_container& /*t*/) { return 1; }
};
Require a pointer to not be a nullptr
Definition: Gsl.hpp:198
decltype(auto) get_size(const T &t, SizeFunction size=GetContainerSize{})
Retrieve the size of t if t.size() is a valid expression, otherwise if T is fundamental or a std::com...
Definition: ContainerHelpers.hpp:117

Here is the call to pypp::call:

const auto result = pypp::call<Scalar<DataVector>,
tmpl::list<ConvertClassForConservionTestA>>(
"PyppPyTests", "custom_conversion", t,
ClassForConversionTest{2.0, 3.0});

A conversion class for retrieving the member variables b_ can also be written as follows:

struct ConvertClassForConservionTestB {
using unpacked_container = double;
using packed_container = ClassForConversionTest;
using packed_type = double;
static inline unpacked_container unpack(const packed_container t,
const size_t /*grid_point_index*/) {
return t.b_;
}
static inline void pack(const gsl::not_null<packed_container*> packed_t,
const unpacked_container t,
const size_t /*grid_point_index*/) {
packed_t->b_ = t;
}
static inline size_t get_size(const packed_container& /*t*/) { return 1; }
};
Note
In order to setup the python interpreter and add the local directories to the path, a SetupLocalPythonEnvironment object needs to be constructed in the local scope.

Example

The following example calls the function test_numeric from the module pypp_py_tests which multiplies two integers.

const auto ret = pypp::call<long>("PyppPyTests", "test_numeric", 3, 4);
CHECK(ret == 3 * 4);

Alternatively, this examples calls test_vector from pypp_py_tests which converts two vectors to python lists and multiplies them pointwise.

const auto ret = pypp::call<std::vector<double>>(
"PyppPyTests", "test_vector", std::vector<double>{1.3, 4.9},
std::vector<double>{4.2, 6.8});
CHECK(approx(ret[0]) == 1.3 * 4.2);
CHECK(approx(ret[1]) == 4.9 * 6.8);

Pypp can also be used to take a function that performs manipulations of NumPy arrays and apply it to either a Tensor of doubles or a Tensor of DataVectors. This is useful for testing functions which act on Tensors pointwise. For example, let's say we wanted to call the NumPy function which performs \( v_i = A B^a C_{ia} + D^{ab} E_{iab} \), which is implemented in python as

def test_einsum(scalar, t_A, t_ia, t_AA, t_iaa):
return scalar * np.einsum("a,ia->i", t_A, t_ia) +
np.einsum("ab, iab->i", t_AA, t_iaa)

where \( v_i \) is the return tensor and \( A, B^a, C_{ia},D^{ab}, E_{iab} \) are the input tensors respectively. We call this function through C++ as:

const auto tensor_from_python = pypp::call<tnsr::i<T, 3>>(
"PyppPyTests", "test_einsum", scalar, vector, tnsr_ia, tnsr_AA, tnsr_iaa);

for type T either a double or DataVector.

Pypp will also support testing of functions which return and operate on std::arrays of DataVectorss. To return a std::array of DataVectors, the python function should return a python list of doubles.

Note
In order to return a Tensor<DataVector...> from pypp::call, at least one Tensor<DataVector...> must be taken as an argument, as the size of the returned tensor needs to be deduced.

◆ check_with_random_values() [1/4]

template<size_t NumberOfBounds, class F , class T , Requires< not std::is_same_v< typename tt::function_info< cpp20::remove_cvref_t< F > >::return_type, void > > = nullptr>
void pypp::check_with_random_values ( F &&  f,
const std::string module_name,
const std::string function_name,
const std::array< std::pair< double, double >, NumberOfBounds > &  lower_and_upper_bounds,
const T &  used_for_size,
const double  epsilon = 1.0e-12,
const typename std::random_device::result_type  seed = std::random_device{}() 
)

Tests a C++ function returning by value by comparing the result to a python function.

Tests the function f by comparing the result to that of the python function function_name in the file module_name. The function is tested by generated random values in the half-open range [lower_bound, upper_bound). The argument used_for_size is used for constructing the arguments of f by calling make_with_value<ArgumentType>(used_for_size, 0.0).

Note
You must explicitly pass the number of bounds you will be passing as the first template parameter, the rest will be inferred.
If you have a test fail you can replay the scenario by feeding in the seed that was printed out in the failed test as the last argument.
Parameters
fThe C++ function to test
module_nameThe python file relative to the directory used in SetupLocalPythonEnvironment
function_nameThe name of the python function inside module_name
lower_and_upper_boundsThe lower and upper bounds for the randomly generated numbers. Must be either an array of a single pair, or of as many pairs as there are arguments to f that are not a gsl::not_null
used_for_sizeThe type X for the arguments of f of type Tensor<X>
epsilonA double specifying the comparison tolerance (default 1.0e-12)
seedThe seed for the random number generator. This should only be specified when debugging a failure with a particular set of random numbers, in general it should be left to the default value.

◆ check_with_random_values() [2/4]

template<size_t NumberOfBounds, class F , class T >
void pypp::check_with_random_values ( F &&  f,
const std::string module_name,
const std::vector< std::string > &  function_names,
const std::array< std::pair< double, double >, NumberOfBounds > &  lower_and_upper_bounds,
const T &  used_for_size,
const double  epsilon = 1.0e-12,
const typename std::random_device::result_type  seed = std::random_device{}(),
const std::optional< double > &  initial_result_values = std::nullopt 
)

Tests a C++ function returning by gsl::not_null by comparing the result to a python function.

Tests the function f by comparing the result to that of the python function function_name in the file module_name. The function is tested by generated random values in the half-open range [lower_bound, upper_bound) for each argument. The argument used_for_size is used for constructing the arguments of f by calling make_with_value<ArgumentType>(used_for_size, 0.0). For functions that return by gsl::not_null, the result will be initialized with random values rather than to signaling NaNs. This means functions do not need to support receiving a signaling NaN in their return argument to be tested using this function. The optional argument initial_result_values allows initializing the result buffers with a given value instead of random data to test functions that mutate the result buffers instead of assigning to it.

Note
You must explicitly pass the number of bounds you will be passing as the first template parameter, the rest will be inferred.
If you have a test fail you can replay the scenario by feeding in the seed that was printed out in the failed test as the last argument.
Parameters
fThe C++ function to test
module_nameThe python file relative to the directory used in SetupLocalPythonEnvironment
function_namesThe names of the python functions inside module_name in the order that they return the gsl::not_null results
lower_and_upper_boundsThe lower and upper bounds for the randomly generated numbers. Must be either an array of a single pair, or of as many pairs as there are arguments to f that are not a gsl::not_null
used_for_sizeThe type X for the arguments of f of type Tensor<X>
epsilonA double specifying the comparison tolerance (default 1.0e-12)
seedThe seed for the random number generator. This should only be specified when debugging a failure with a particular set of random numbers, in general it should be left to the default value.
initial_result_valuesFill the result buffers with this value instead of random data before calling the function f.

◆ check_with_random_values() [3/4]

template<size_t NumberOfBounds, class F , class T , class... MemberArgs, Requires< not std::is_same_v< typename tt::function_info< cpp20::remove_cvref_t< F > >::return_type, void > and not tt::is_a_v< tuples::TaggedTuple, typename tt::function_info< cpp20::remove_cvref_t< F > >::return_type > > = nullptr>
void pypp::check_with_random_values ( F &&  f,
const typename tt::function_info< cpp20::remove_cvref_t< F > >::class_type &  klass,
const std::string module_name,
const std::string function_name,
const std::array< std::pair< double, double >, NumberOfBounds > &  lower_and_upper_bounds,
const std::tuple< MemberArgs... > &  member_args,
const T &  used_for_size,
const double  epsilon = 1.0e-12,
const typename std::random_device::result_type  seed = std::random_device{}() 
)

Tests a member function of a class returning by value by comparing the result to a python function.

Tests the function f by comparing the result to that of the python function function_name in the file module_name. An instance of the class is passed in as the second argument and is the object on which the member function f will be invoked. The member function is invoked as klass.function, so passing in pointers is not supported. The function is tested by generated random values in the half-open range [lower_bound, upper_bound). The argument used_for_size is used for constructing the arguments of f by calling make_with_value<ArgumentType>(used_for_size, 0.0).

Note
You must explicitly pass the number of bounds you will be passing as the first template parameter, the rest will be inferred.
If you have a test fail you can replay the scenario by feeding in the seed that was printed out in the failed test as the last argument.
Parameters
fThe member function to test
klassthe object on which to invoke f
module_nameThe python file relative to the directory used in SetupLocalPythonEnvironment
function_nameThe name of the python function inside module_name
lower_and_upper_boundsThe lower and upper bounds for the randomly generated numbers. Must be either an array of a single pair, or of as many pairs as there are arguments to f that are not a gsl::not_null
member_argsa tuple of the member variables of the object klass that the python function will need in order to perform the computation. These should have the same types as the normal arguments passed to the member function, e.g. Tensor<X>.
used_for_sizeThe type X for the arguments of f of type Tensor<X>
epsilonA double specifying the comparison tolerance (default 1.0e-12)
seedThe seed for the random number generator. This should only be specified when debugging a failure with a particular set of random numbers, in general it should be left to the default value.

◆ check_with_random_values() [4/4]

template<size_t NumberOfBounds, class F , class T , class... MemberArgs, Requires< std::is_same_v< typename tt::function_info< cpp20::remove_cvref_t< F > >::return_type, void > or tt::is_a_v< tuples::TaggedTuple, typename tt::function_info< cpp20::remove_cvref_t< F > >::return_type > > = nullptr>
void pypp::check_with_random_values ( F &&  f,
const typename tt::function_info< cpp20::remove_cvref_t< F > >::class_type &  klass,
const std::string module_name,
const std::vector< std::string > &  function_names,
const std::array< std::pair< double, double >, NumberOfBounds > &  lower_and_upper_bounds,
const std::tuple< MemberArgs... > &  member_args,
const T &  used_for_size,
const double  epsilon = 1.0e-12,
const typename std::random_device::result_type  seed = std::random_device{}() 
)

Tests a member function of a class returning by either gsl::not_null or TaggedTuple by comparing the result to a python function.

Tests the function f by comparing the result to that of the python functions function_names in the file module_name. An instance of the class is passed in as the second argument and is the object on which the member function f will be invoked. The member function is invoked as klass.function, so passing in pointers is not supported. The function is tested by generated random values in the half-open range [lower_bound, upper_bound). The argument used_for_size is used for constructing the arguments of f by calling make_with_value<ArgumentType>(used_for_size, 0.0). For functions that return by gsl::not_null, the result will be initialized with random values rather than to signaling NaNs. This means functions do not need to support receiving a signaling NaN in their return argument to be tested using this function.

Note
You must explicitly pass the number of bounds you will be passing as the first template parameter, the rest will be inferred.
If you have a test fail you can replay the scenario by feeding in the seed that was printed out in the failed test as the last argument.
Parameters
fThe member function to test
klassthe object on which to invoke f
module_nameThe python file relative to the directory used in SetupLocalPythonEnvironment
function_namesThe names of the python functions inside module_name in the order that they return the gsl::not_null results
lower_and_upper_boundsThe lower and upper bounds for the randomly generated numbers. Must be either an array of a single pair, or of as many pairs as there are arguments to f that are not a gsl::not_null
member_argsa tuple of the member variables of the object klass that the python function will need in order to perform the computation. These should have the same types as the normal arguments passed to the member function, e.g. Tensor<X>.
used_for_sizeThe type X for the arguments of f of type Tensor<X>
epsilonA double specifying the comparison tolerance (default 1.0e-12)
seedThe seed for the random number generator. This should only be specified when debugging a failure with a particular set of random numbers, in general it should be left to the default value.

◆ make_py_tuple()

template<typename... Args>
PyObject * pypp::make_py_tuple ( const Args &...  t)

Create a python tuple from Args.

Template Parameters
Argsthe types of the arguments to be put in the tuple (deducible)
Parameters
tthe arguments to put into the tuple

Returns: PyObject* containing a Python tuple