Line data    Source code

       1           0 : \cond NEVER
       2             : Distributed under the MIT License.
       3             : See LICENSE.txt for details.
       4             : \endcond
       5             : # The minimal SpECTRE executable {#tutorial_minimal_parallel_executable}
       6             : 
       7             : \tableofcontents
       8             : 
       9             : This tutorial will illustrate what is needed to compile the minimal
      10             : SpECTRE executable that will simply print some useful information
      11             : about the executable and then exit.
      12             : 
      13             : Specifically, this tutorial will introduce:
      14             : - A user-provided `Metavariables`, a C++ struct that is used to
      15             :   specify the metaprogram that is converted into a C++ executable.
      16             : - how to build a SpECTRE executable
      17             : - how to run a SpECTRE executable
      18             : - useful information that can be extracted from all SpECTRE executables
      19             : - `Main`, the main parallel component that acts as the main function
      20             :   of a C++ executable.
      21             : 
      22             : In this tutorial, `SPECTRE_ROOT` refers to the directory in which
      23             : SpECTRE was cloned, and `SPECTRE_BUILD_DIR` refers to the directory in
      24             : which you built SpECTRE.
      25             : 
      26             : ### Creating a build target for a parallel executable
      27             : 
      28             : The first step is to select (or create) a directory (within
      29             : `SPECTRE_ROOT/src`) that will hold the files from which the executable
      30             : will be created.  For this tutorial we created the directory
      31             : `ParallelTutorial` in `src/Executables/Examples`.  If you create a new
      32             : directory for the executable, you will need to edit the
      33             : `CMakeLists.txt` file in its parent directory and add (replacing
      34             : `ParallelTutorial` with the appropriate directory name)
      35             : 
      36             : \snippet Examples/CMakeLists.txt add_subdirectory
      37             : 
      38             : The second step is to edit (or create) the `CMakeLists.txt` file in
      39             : the executable directory and add a call to `add_spectre_executable` such as:
      40             : 
      41             : \snippet ParallelTutorial/CMakeLists.txt add_spectre_executable
      42             : 
      43             : The `add_spectre_executable` just forwards its arguments to CMake's
      44             : `add_executable`. It works just like `add_executable` in that you pass it a
      45             : source file that defines your executable.
      46             : 
      47             : Your source file needs a `CkRegisterMainModule()` function. This is the "main"
      48             : function of the Charm++ program:
      49             : 
      50             : \snippet ParallelTutorial/MinimalExecutable.cpp main_function
      51             : 
      52             : It just calls `Parallel::charmxx::register_main_module<Metavariables>()`. This
      53             : is where SpECTRE code takes over. SpECTRE programs are defined by the
      54             : `Metavariables` class that is passed to this function.
      55             : 
      56             : ### Writing the metavariables files
      57             : 
      58             : The metavariables can be thought of as a compile-time input file that defines
      59             : what the executable will do:
      60             : 
      61             : \snippet MinimalExecutable.cpp metavariables_definition
      62             : 
      63             : The metavariables struct must define a `static constexpr
      64             : std::array<Parallel::Phase, N> default_phase_order` that is used to
      65             : define the default order in which phases are executed.  In this
      66             : example the executable will execute the `Initialization` phase
      67             : followed by the `Exit` phase.
      68             : 
      69             : The metavariables struct must define a type alias `component_list`
      70             : that is a `tmpl::list` (a typelist defined in `Utilities/TMPL.hpp`) of
      71             : the parallel components used by the executable.  In this example no
      72             : parallel components are used.
      73             : 
      74             : The metavariables struct must define `help`, a `static constexpr
      75             : Options::String` that will be printed as part of the help message of the
      76             : executable. (`Options::String` is defined in `Options/Options.hpp`.)
      77             : 
      78             : ### Building a SpECTRE executable
      79             : 
      80             : Let `$EXECUTABLE` be the name of the executable (the first argument
      81             : passed to the `add_spectre_executable` `CMake` function, so in this
      82             : example `MinimalExample`).  Then the executable can be built by
      83             : running the following command in `$SPECTRE_BUILD_DIR`:
      84             : 
      85             : ```
      86             : make $EXECUTABLE
      87             : ```
      88             : 
      89             : which will produce the executable of the same name in
      90             : `$SPECTRE_BUILD_DIR/bin`.
      91             : 
      92             : ### Running a SpECTRE executable
      93             : 
      94             : To run a SpECTRE executable, run the command:
      95             : 
      96             : ```
      97             : ./$SPECTRE_BUILD_DIR/bin/$EXECUTABLE  <options>
      98             : ```
      99             : 
     100             : where `<options>` must include any required command-line options.  In
     101             : the simple example for this tutorial, no command-line options are
     102             : required.
     103             : 
     104             : On a laptop, we get the following output:
     105             : 
     106             : ```
     107             : Charm++: standalone mode (not using charmrun)
     108             : Charm++> Running in Multicore mode:  1 threads
     109             : Converse/Charm++ Commit ID: v6.8.0-0-ga36028edb
     110             : CharmLB> Load balancer assumes all CPUs are same.
     111             : Charm++> Running on 1 unique compute nodes (4-way SMP).
     112             : Charm++> cpu topology info is gathered in 0.000 seconds.
     113             : 
     114             : Executing 'some_path/MinimalExample' using 1 processors.
     115             : Date and time at startup: Fri Oct 25 15:03:05 2019
     116             : 
     117             : SpECTRE Build Information:
     118             : Version:                      0.0.0
     119             : Compiled on host:             kosh-3.local
     120             : Compiled in directory:        some_path/build
     121             : Source directory is:          some_path/parallel_tutorial
     122             : Compiled on git branch:       feature/parallel_tutorial
     123             : Compiled with git hash:       1f4ab20cbda9ae8072a17df69de9731c43687468
     124             : Linked on:                    Fri Oct 25 13:26:06 2019
     125             : 
     126             : 
     127             : Done!
     128             : Wall time in seconds: 0.004185
     129             : Date and time at completion: Fri Oct 25 15:03:05 2019
     130             : 
     131             : [Partition 0][Node 0] End of program
     132             : ```
     133             : 
     134             : which includes information that will be printed by every SpECTRE
     135             : executable on startup and exit.  First, there is information provided
     136             : by `Charm++` that will depend upon how `Charm++` was built.  Next you
     137             : will see information provided by SpECTRE which includes:
     138             : 
     139             : - the name of the executable
     140             : - how many processes the executable was run on
     141             : - the date and time at startup
     142             : - the version of SpECTRE that was used to compile the executable
     143             : - where the executable was compiled
     144             : - which git branch and hash was used to compile the executable
     145             : - when the executable was linked
     146             : 
     147             : On exit, the executable will print that it is `Done!`, followed by how
     148             : the long the executable took to run as timed by the `Charm++`
     149             : wall-clock timer, and the date and time at completion, followed by any
     150             : information the `Charm++` provides upon exiting the program.
     151             : 
     152             : ### Extracting useful information from a SpECTRE executable
     153             : 
     154             : Every SpECTRE executable comes with a set of command-line options that
     155             : can be used to obtain useful information about the executable (and for
     156             : executables expecting an input file, whether or not the input file can
     157             : be parsed successfully).
     158             : 
     159             : #### Getting a list of available options
     160             : 
     161             : To get a list of available options for a SpECTRE executable, run
     162             : either of the following commands:
     163             : 
     164             : ```
     165             : ./$SPECTRE_BUILD_DIR/bin/$EXECUTABLE  --help
     166             : ./$SPECTRE_BUILD_DIR/bin/$EXECUTABLE  -h
     167             : ```
     168             : 
     169             : In the middle of the `Charm++` startup information will now appear a
     170             : long list of `Charm++` related command-line options which we will
     171             : ignore for this tutorial.  After the SpECTRE startup information,
     172             : there is now a list of available command-line options:
     173             : 
     174             : ```
     175             :   -h [ --help ]             Describe program options
     176             :   --check-options           Check input file options
     177             :   --dump-source-tree-as arg If specified, then a gzip archive of the source
     178             :                             tree is dumped with the specified name. The archive
     179             :                             can be expanded using 'tar -xzf ARCHIVE.tar.gz'
     180             :   --dump-paths              Dump the PATH, CPATH, LD_LIBRARY_PATH,
     181             :                             LIBRARY_PATH, and CMAKE_PREFIX_PATH at compile
     182             :                             time.
     183             :   --dump-environment        Dump the result of printenv at compile time.
     184             :   --dump-build-info         Dump the contents of SpECTRE's BuildInfo.txt
     185             :   --dump-only               Exit after dumping requested information.
     186             : ```
     187             : 
     188             : which we will describe in detail below. This is followed by a
     189             : description of the expected input-file options, which in this example
     190             : will simply print the `help` string from the metavariables struct, and
     191             : the statement that there are no options expected:
     192             : 
     193             : ```
     194             : ==== Description of expected options:
     195             : A minimal executable
     196             : 
     197             : <No options>
     198             : ```
     199             : 
     200             : We will cover input file options in a future tutorial.
     201             : 
     202             : #### Checking options
     203             : 
     204             : As the minimal executable in this tutorial expects no input-file
     205             : options, running the command:
     206             : 
     207             : ```
     208             : ./$SPECTRE_BUILD_DIR/bin/$EXECUTABLE  --check-options
     209             : ```
     210             : 
     211             : will print out `No options to check!` and exit the program.  See a
     212             : future tutorial for an example of checking input-file options.
     213             : 
     214             : #### Dumping the source tree
     215             : 
     216             : In order to aid in reproducibility, all SpECTRE executables contain a
     217             : copy of `$SPECTRE_ROOT`.  To obtain the source tree as an archive
     218             : file, run the command:
     219             : 
     220             : ```
     221             : ./$SPECTRE_BUILD_DIR/bin/$EXECUTABLE  --dump-source-tree-as SpECTRE
     222             : ```
     223             : 
     224             : which will produce the archive file `SpECTRE.tar.gz` which can be
     225             : expanded with the command:
     226             : 
     227             : ```
     228             : tar -xzf SpECTRE.tar.gz
     229             : ```
     230             : 
     231             : #### Dumping other information about a SpECTRE executable
     232             : 
     233             : In addition to dumping the entire source tree there are the following options:
     234             : 
     235             : - `--dump-paths` will print out various paths from when the
     236             :   executable was compiled
     237             : - `--dump-environment` will print the results of `printenv`
     238             :   (i.e. all of the environment variables) from when the executable was
     239             :   compiled
     240             : - `--dump-build-info` will print the contents of SpECTRE's
     241             :   `BuildInfo.txt` which will contain the versions of libraries
     242             :   that SpECTRE linked against, as well as various `CMake` variables
     243             : 
     244             : Also note that the option `--dump-only` can be used to have the
     245             : SpECTRE executable terminate immediately after dumping the
     246             : information.
     247             : 
     248             : ### Behind the scenes:  the main parallel component
     249             : 
     250             : Every Charm++ executable needs a mainchare.  All SpECTRE executables
     251             : use `Main` (found in `src/Parallel/Main.hpp`) as the mainchare.  When
     252             : a SpECTRE executable is run, the constructor of `Main` that is executed
     253             : takes the command line options as an argument (as type
     254             : `CkArgMsg*`).  This constructor performs the following operations:
     255             : 
     256             : - Prints useful startup information
     257             : - Parses the command line options, performing any requested operations
     258             : - Parses the input file options, populating a tagged tuple with their
     259             :   values.  (This is discussed in more detail in a future tutorial on
     260             :   input file options.)
     261             : - Creates the `GlobalCache` (a nodegroup chare) that holds
     262             :   objects created from input file options that are stored
     263             :   once per Charm++ node, as well as proxies to all other parallel
     264             :   components.
     265             : - Creates user-requested non-array parallel components, passing them
     266             :   the proxy to the `GlobalCache` as well as any items they
     267             :   request that can be created from input file options.
     268             : - Creates empty array user-requested parallel components
     269             : - Sends the complete list of parallel components to the
     270             :   `GlobalCache`.
     271             : 
     272             : Once the list of parallel components is sent to the `GlobalCache`
     273             : on each node, the `Main` member function
     274             : `allocate_array_components_and_execute_initial_phase` will be
     275             : executed.  This will allocate the elements of the array parallel
     276             : components by calling the `allocate_array` member function of each
     277             : component.  Then the `start_phase` member function is called on each
     278             : component, which will execute the phase action list for the
     279             : `Initialization` phase for each component.  Charm++ will execute each
     280             : phase until it detects that nothing is happening (quiescence
     281             : detection).  As this represents a global synchronization point, the
     282             : number of phases should be minimized in order to exploit the power of
     283             : SpECTRE.  After each phase, the `execute_next_phase` member function
     284             : of `Main` will be called.  This member function first determines what
     285             : the next phase is.  If the next phase is `Exit`, then some useful
     286             : information is printed and the program exits gracefully.  Otherwise the
     287             : `execute_next_phase` member function of each parallel component is
     288             : called.
     289             :