SpECTRE  v2024.09.29
Public Types | Public Member Functions | Static Public Member Functions | List of all members
evolution::dg::Tags::BoundaryCorrectionAndGhostCellsInbox< Dim, UseNodegroupDgElements > Struct Template Reference

The inbox tag for boundary correction communication and DG-subcell ghost zone cells. More...

#include <InboxTags.hpp>

Public Types

using stored_type = evolution::dg::BoundaryData< Dim >
 
using temporal_id = TimeStepId
 
using type_map = std::map< TimeStepId, DirectionalIdMap< Dim, stored_type > >
 
using type_spsc = evolution::dg::AtomicInboxBoundaryData< Dim >
 
using type = tmpl::conditional_t< UseNodegroupDgElements, type_spsc, type_map >
 
using value_type = type
 

Public Member Functions

void pup (PUP::er &)
 

Static Public Member Functions

template<typename ReceiveDataType >
static size_t insert_into_inbox (const gsl::not_null< type_spsc * > inbox, const temporal_id &time_step_id, ReceiveDataType &&data)
 
template<typename ReceiveDataType >
static size_t insert_into_inbox (const gsl::not_null< type_map * > inbox, const temporal_id &time_step_id, ReceiveDataType &&data)
 
static std::string output_inbox (const type_spsc &inbox, const size_t padding_size)
 
static std::string output_inbox (const type_map &inbox, const size_t padding_size)
 

Detailed Description

template<size_t Dim, bool UseNodegroupDgElements>
struct evolution::dg::Tags::BoundaryCorrectionAndGhostCellsInbox< Dim, UseNodegroupDgElements >

The inbox tag for boundary correction communication and DG-subcell ghost zone cells.

The stored data consists of the following:

  1. the volume mesh of the element.
  2. the volume mesh corresponding to the ghost cell data. This allows eliding projection when all neighboring elements are doing DG.
  3. the mortar mesh of the data on the mortar
  4. the variables at the ghost zone cells for finite difference/volume reconstruction
  5. the data on the mortar needed for computing the boundary corrections (e.g. fluxes, characteristic speeds, conserved variables)
  6. the TimeStepId beyond which the boundary terms are no longer valid, when using local time stepping.
  7. the troublade cell indicator status using for determining halos around troubled cells.
  8. the integration order of the time-stepper.

The TimeStepId is the neighboring element's next time step. When using local time stepping, the neighbor's boundary data is valid up until this time, which may include multiple local time steps. By receiving and storing the neighbor time step, the local element knows whether or not it should remove boundary data and expect new data to be sent from the neighbor.

The ghost cell data will be valid whenever a DG-subcell scheme is being used. Whenever a DG-subcell scheme is being used, elements using DG and not FD/FV always send both the ghost cells and boundary correction data together. Elements using FD/FV send the ghost cells first followed by the boundary correction data once the element has received all neighbor ghost cell data. Note that the second send/receive only modifies the flux and the TimeStepId used for the flux validity range.

When only a DG scheme (not a DG-subcell scheme) is used the ghost cell data will never be valid.

In the DG-subcell scheme this tag is used both for communicating the ghost cell data needed for the FD/FV reconstruction step and the data needed for the boundary corrections.

  • For an element using DG, both ghost cells and boundary corrections are sent using a single communication. After receiving all neighbor boundary corrections the element can finish computing the time step. The ghost cell data from neighbors is unused.
  • For an element using FD/FV, first the ghost cells are sent. Once all neighboring ghost cell data is received, reconstruction is done and the boundary terms are computed and sent to the neighbors. After receiving all neighbor boundary corrections the element can finish computing the time step.
  • Whether or not an extra communication is needed when an element switches from DG to FD/FV depends on how exactly the decision to switch is implemented. If the volume terms are integrated and verified to be valid before a DG element sends ghost cell and boundary data then no additional communication is needed when switching from DG to FD/FV. In this case a second check of the data that includes the boundary correction needs to be done. If the second check determines a switch from DG to FD/FV is needed, we can continue to use the DG fluxes since the evolution in the small was valid, thus avoiding an additional communication. However, to fully guarantee physical realizability a second communication or evolving the neighboring ghost cells needs to be done. We have not yet decided how to deal with the possible need for an additional communication since it also depends on whether or not we decide to couple to Voronoi instead of just Neumann neighbors.
  • The data for the inbox tags is erased after the boundary correction is complete and the solution has been verified to be valid at the new time step. The ghost cells could be invalidated immediately after reconstruction, thus using the ghost cell data after reconstruction is complete is considered undefined behavior. That is, we make no guarantee as to whether or not it will work.
  • The reason for minimizing the number of communications rather than having a more uniform implementation between DG and FD/FV is that it is the number of communications that adds the most overhead, not the size of each communication. Thus, one large communication is cheaper than several small communications.

Return Values:

  • In the case that the type is type_map the insert_into_inbox function returns the size of the inbox.
  • In the case that the type is type_spsc the insert_into_inbox function returns the number of neighbor data contributions made that allow the element to take its next time step/needs a message called on it. When this number is equal to the number of neighbors, a Charm++ message must be sent to the element to have it continue the algorithm. This is so as to minimize the number of communications made through Charm++ and instead to move data atomically between neighbors whenever possible.

DG Element Nodegroup Support

If you are using the DgElementCollection then you must set UseNodegroupDgElements to true. The actions that use this tag check that the parallel component and the UseNodegroupDgElements is consistent.

The documentation for this struct was generated from the following file: