Line data Source code
1 0 : \cond NEVER
2 : Distributed under the MIT License.
3 : See LICENSE.txt for details.
4 : \endcond
5 :
6 : # Adaptive mesh refinement (AMR) {#dev_guide_amr}
7 :
8 : \tableofcontents
9 :
10 : ### Introduction
11 :
12 : SpECTRE implements a block-based anisotropic h-p adaptive mesh
13 : refinement (AMR) algorithm. Currently AMR can only occur at global
14 : synchronization points, but it is planned to remove this restriction
15 : in the future.
16 :
17 : \warning AMR is still under active development. The elliptic
18 : executables fully support AMR. The evolution executables currently
19 : only support p-adaptivity, and only when using a global time stepper,
20 : and only when using a DG method. The executable will abort with an
21 : ERROR if local time stepping or h-refinement is done.
22 :
23 : Here is an overview of what is described in detail in the sections below:
24 :
25 : - \ref dev_guide_amr_algorithm "AMR Algorithm": An overview of the
26 : parallel phases, actions, and components that are used to do
27 : adaptive mesh refinement.
28 : - \ref dev_guide_amr_criteria "AMR Criteria": Describes the
29 : expectations and restrictions on concrete classes implementing
30 : adaptive mesh refinement criteria.
31 : - \ref dev_guide_amr_policies "AMR Policies": Describes restrictions
32 : and assumptions of the current algorithm, and what would be
33 : required to relax them.
34 : - \ref dev_guide_amr_projectors "AMR Projectors": Describes the
35 : requirements on concrete classes which project data to the new or
36 : modified elements after mesh refinement.
37 : - \ref dev_guide_amr_metavariables "AMR Metavariables": Describes the
38 : code that needs to be added to the metavariables of an executable in
39 : order to enable adaptive mesh refinement.
40 : - \ref dev_guide_amr_input_options "Controlling AMR": Describes the
41 : options in the input file that control the behavior of adaptive mesh
42 : refinement.
43 :
44 : ### AMR Algorithm {#dev_guide_amr_algorithm}
45 :
46 : The AMR algorithm requires communication via simple actions between
47 : the elements of the array component that are responsible for the
48 : Element%s of the Domain and a singleton amr::Component. See \ref
49 : dev_guide_parallelization_foundations
50 : "Parallelization and Core Concepts" for an overview of phases,
51 : actions, metavariables, and other SpECTRE parallelization concepts.
52 :
53 : Currently the AMR algorithm is run over several phases when a phase
54 : change is triggered by an executable. See \ref
55 : dev_guide_amr_input_options "Controlling AMR" for how to control when
56 : the AMR phases are triggered.
57 :
58 : #### Evaluating refinement criteria
59 :
60 : When AMR is triggered, the executable should enter
61 : Parallel::Phase::EvaluateAmrCriteria. In this phase, the
62 : amr::Component triggers the simple action
63 : amr::Actions::EvaluateRefinementCriteria on each element of the array
64 : component (specified by `metavariables::amr::element_array`). Each
65 : element evaluates (in order) the user-specified refinement criteria,
66 : each of which returns a refinement decision in each logical dimension
67 : of the element represented as an amr::Flag. The overall refinement
68 : decision in each logical dimension is obtained by taking the highest
69 : priority decision in the dimension over all criteria, where
70 : amr::Flag::Split has the highest priority and amr::Flag::Join has the
71 : lowest. See \ref dev_guide_amr_criteria "AMR Criteria" for more
72 : details on implementing criteria. Once the overall refinement
73 : decision based on the refinement criteria is made, the decision may be
74 : modified to satisfy the \ref dev_guide_amr_policies "AMR Policies".
75 : Finally, the element sends its decision (and other information in
76 : amr::Info) to its neighbors by calling the simple action
77 : amr::Actions::UpdateAmrDecision.
78 :
79 : In amr::Actions::UpdateAmrDecision, an element decides whether or not it
80 : needs to update its own refinement decision (or other information in
81 : amr::Info) based on decisions (or other information in amr::Info) sent
82 : by its neighbor. Possible reasons to update the refinement decision
83 : are, e.g., if a sibling neighbor does not want to join an element that
84 : had initially decided to join, or if any of the amr::Policies would
85 : be violated. If the refinement decision (or other information in
86 : amr::Info) of the element changes, the element will send its
87 : updated amr::Info to its neighbors.
88 :
89 : Note that simple actions are executed asynchronusly, so both
90 : amr::Actions::EvaluateRefinementCriteria and
91 : amr::Actions::UpdateAmrDecision have to handle the possibility that
92 : they are executed out of order. For example,
93 : amr::Actions::UpdateAmrDecision could be executed before
94 : amr::Actions::EvaluateRefinementCriteria. Eventually all elements
95 : will reach a state where they do not need to send new information to
96 : their neighbors, thus reaching a state of quiescence. When this
97 : happens Parallel::Phase::EvaluateAmrCriteria ends. At this point all
98 : elements have reached a consensus on how the domain should be adapted,
99 : but no changes have been made to the domain. In order to adjust the
100 : domain, the executable should next start
101 : Parallel::Phase::AdjustDomain.
102 :
103 : #### Adjusting the domain
104 :
105 : At the beginning of Parallel::Phase::AdjustDomain, the amr::Component
106 : calls amr::Actions::AdjustDomain on each element of the array
107 : component (specified by `metavariables::amr::element_array`). If the
108 : element wants to split in any dimension, it will call the simple
109 : action amr::Actions::CreateChild on the amr::Component in order to
110 : create new refined elements that will replace the existing element.
111 : If the element wants to join in any dimension, it will either call
112 : amr::Actions::CreateParent on the amr::Component (if it is the joining
113 : sibling with the lowest SegmentId in each dimension being joined) or
114 : do nothing in order to create a single new coarsened element that will
115 : replace multiple existing elements. If the element neither wants to
116 : split nor join in any dimension, then the element mutates items in its
117 : db::DataBox. First the Element, Mesh and neighbor Mesh are updated.
118 : Then the element calls the
119 : \ref dev_guide_amr_projectors "AMR Projectors" that will update all
120 : the mutable items in the db::DataBox.
121 : Finally the amr::Info of the element and its neighbors are reset and
122 : cleared so they are ready for a future invocation of AMR.
123 :
124 : When amr::Actions::CreateChild is executed by the amr::Component, it
125 : creates a new element in the `metavariables::amr::element_array`
126 : passing a Parallel::SimpleActionCallback that will be executed after
127 : the new element is created. The callback will be either to call
128 : amr::Actions::CreateChild again for the next child to be created, or
129 : amr::Actions::SendDataToChildren which will be invoked on the element
130 : that is being split. amr::Actions::SendDataToChildren will then
131 : invoke amr::Actions::InitializeChild on each new child element,
132 : passing along all of the mutable items in its
133 : db::DataBox. amr::Actions::SendDataToChildren will then deregister and
134 : delete the element being split. amr::Actions::InitializeChild will
135 : create its Element, Mesh, and neighbor Mesh, and then call the \ref
136 : dev_guide_amr_projectors "AMR Projectors" that will update all the
137 : mutable items in its db::DataBox.
138 :
139 : When amr::Actions::CreateParent is executed by the amr::Component, it
140 : creates a new element in the `metavariables::amr::element_array`
141 : passing a Parallel::SimpleActionCallback that will be executed after
142 : the new element is created. The callback will be
143 : amr::Actions::CollectDataFromChildren which will be invoked on one
144 : of the elements that are joining.
145 : amr::Actions::CollectDataFromChildren will then invoke either itself
146 : on another element that is joining or amr::Actions::InitializeParent
147 : on the new parent element, collecting all of the mutable items in each
148 : joining element's db::DataBox. amr::Actions::CollectDataFromChildren
149 : will then deregister and delete the element that is
150 : joining. amr::Actions::InitializeParent will create its Element, Mesh,
151 : and neighbor Mesh, and then call the \ref dev_guide_amr_projectors
152 : "AMR Projectors" that will update all the mutable items in its
153 : db::DataBox.
154 :
155 : When all of the actions finish, a state of quiescence is reached.
156 : When this happens Parallel::Phase::AdjustDomain ends. The executable
157 : should now either return to the phase which triggered AMR, or enter
158 : Parallel::Phase::CheckDomain.
159 :
160 : ### AMR Criteria {#dev_guide_amr_criteria}
161 :
162 : When amr::Actions::EvaluateRefinementCriteria is executed, for each
163 : element, it loops over a list of criteria that is specified in the
164 : input file options
165 : (See \ref dev_guide_amr_input_options "Controling AMR").
166 : Each criterion must be an option-creatable class derived from
167 : amr::Criterion (see its documentation for a list of requirements for
168 : the derived classes). In particular, each criterion should compute an
169 : array of amr::Flag%s that represent the recommended refinemenent
170 : choice in each logical dimension. These choices can be computed from
171 : any items in the db::DataBox of the element or additional compute
172 : items specified by the criterion. It is expected that a criterion
173 : will compute something and then either choose to recommend refinement,
174 : no change, or coarsening the element either in size (h-refinement) or
175 : polynomial order (p-refinement). A criterion does not need to handle
176 : anything that is enforced by the amr::Policies such as worry about
177 : bounds on the refinement level or number of grid points.
178 :
179 : ### AMR Policies {#dev_guide_amr_policies}
180 :
181 : The current AMR algorithm has a set of rules that are enforced after
182 : the overall refinement choice is determined from the AMR criteria.
183 :
184 : The following rules simplify the code:
185 :
186 : - An Element cannot join if it is splitting in any dimension. Instead
187 : any amr::Flag::Join is changed to amr::Flag::DoNothing. Relaxing
188 : this restriction would be a non-trivial change that would require
189 : more complicated logic in determining if neighboring elements can
190 : join, as well as an additional code path to handle the creation and
191 : initialization of the new elements. This should not be a significant
192 : restriction as in most cases when this would occur it is unlikely
193 : that the siblings of the Element would agree to a valid join.
194 :
195 : - In two or three spatial dimensions, there must be a 2:1 balance
196 : (i.e. within one refinement level) between neighbors in the
197 : dimension(s) parallel to the face of the element. Relaxing this
198 : restriction would be a significant change to the code as the
199 : code for communicating boundary corrections assumes 2:1 balance. In
200 : order to maintain 2:1 balance with neighboring elements, a choice of
201 : amr::Flag::Join may be changed to amr::Flag::DoNothing, and
202 : amr::Flag::DoNothing may be changed to amr::Flag::Split.
203 :
204 : The following policies are among those controlled by input file
205 : options (see \ref dev_guide_amr_input_options "Controlling AMR" and
206 : the documentation for amr::Policies):
207 :
208 : - Whether to do isotropic or anisotropic refinement. If isotropic
209 : refinement is done, the AMR decision in each dimension are changed
210 : to the decision of the dimension with the highest priority amr::Flag
211 : (i.e. amr::Flag::Split has the highest priority).
212 :
213 : - Whether or not to enforce 2:1 balance for the refinement level in
214 : the direction perpendicular to the face.
215 :
216 : - The minimum and maximum of the refinement level and the number of
217 : grid points. The actual bounds on these will be the stricter of
218 : those specified by the input file and those set by the code. For
219 : the number of grid points, the code bounds are set by
220 : Spectral::minimum_number_of_points and
221 : Spectral::maximum_number_of_points for the Spectral::Basis and
222 : Spectral::Quadrature of the Mesh. For the refinement level, the
223 : minimum is zero, and the maximum is set by
224 : `ElementId::max_refinement_level`.
225 :
226 : ### AMR Projectors {#dev_guide_amr_projectors}
227 :
228 : After the grid adapts, the mutable items in the DataBox of new or
229 : existing elements must be updated. The Element, Mesh, neighbor Mesh,
230 : amr::Info, and neighbor amr::Info along with the items in
231 : `Parallel::Tags::distributed_object_tags` are automatically updated by
232 : the AMR actions, but all other items must be explicitly updated via
233 : projectors that conform to amr::protocols::Projector. If any mutable
234 : item has not been explicitly handled by any of the projectors, a
235 : `static_assert` will be triggered listing the tags for the items that
236 : have not been projected.
237 :
238 : When a new element is created, the items in its DataBox are default
239 : constructed. Their Element, Mesh, and neighbor Mesh are mutated to
240 : their desired state. Then the AMR projectors are called, passing
241 : along the items in the DataBox(es) from their splitting parent element
242 : or joining children elements. Existing elements first mutate their
243 : Element, Mesh, and neighbor Mesh, and then call the AMR projectors,
244 : passing along the old Element and Mesh.
245 :
246 : The `return_tags` of each projector lists the tags for the items that
247 : the projector mutates. From the pre-AMR state, the projector must
248 : compute the post-AMR state of these items. Typically during
249 : Parallel::Phase::Initialization, a group of items will be initialized
250 : by a specific action or mutator in the action
251 : `Initialization::Actions::InitializeItems`. Therefore it makes sense
252 : to create a projector that will handle the same group of items.
253 : However some items are initialized from input file options, while
254 : others are left default initialized and not mutated by any
255 : initialization action. These items will still need to be handled by
256 : some projector. Two convenience projectors are provided:
257 : amr::projectors::DefaultInitialize which value initializes the items
258 : corresponding to the listed tags; and
259 : amr::projectors::CopyFromCreatorOrLeaveAsIs that will either copy the
260 : item from the parent (or children, asserting that all children agree)
261 : of a newly created element, or leave the item unmodified for an
262 : existing element for the items corresponding to the listed tags.
263 :
264 : The list of projectors is specified in
265 : `metavariables::amr::projectors` in the executable, and the projectors
266 : are evaluated in the order in which they are specified.
267 :
268 : ### Enabling AMR for an executable {#dev_guide_amr_metavariables}
269 :
270 : In order to enable AMR for an executable, the following changes need
271 : to be made in the metavariables:
272 :
273 : - The addition of a struct named `amr` such as the following example:
274 : ```
275 : struct amr : tt::ConformsTo<::amr::protocols::AmrMetavariables> {
276 : using element_array = dg_element_array;
277 : using projectors = tmpl::list<::amr::projectors::DefaultInitialize<
278 : domain::Tags::InitialExtents<Dim>,
279 : domain::Tags::InitialRefinementLevels<Dim>,
280 : evolution::dg::Tags::Quadrature>>;
281 : static constexpr bool keep_coarse_grids = false;
282 : };
283 : ```
284 :
285 : where `element_array` specifies the array component on which AMR
286 : will operate and `projectors` is a type list of amr::projectors
287 : that govern how the items in the DataBox for the `element_array` are
288 : updated after refinement.
289 :
290 : - The addition of amr::Component to the `component_list` of the metavariables
291 :
292 : - The addition (or modification) of the following `tmpl::pair`s in
293 : `factory_creation::factory_classes`:
294 : ```
295 : tmpl::pair<amr::Criterion,
296 : tmpl::list<LIST_OF_CRITERIA>,
297 : tmpl::pair<
298 : PhaseChange,
299 : tmpl::list<
300 : PhaseControl::VisitAndReturn<
301 : Parallel::Phase::EvaluateAmrCriteria>,
302 : PhaseControl::VisitAndReturn<Parallel::Phase::AdjustDomain>,
303 : PhaseControl::VisitAndReturn<Parallel::Phase::CheckDomain>>,
304 : ```
305 : where `LIST_OF_CRITERIA` should be a list of amr::Criteria.
306 :
307 : - The addition of the following item in the list of PhaseActions for
308 : the component specified in `amr::element_array`:
309 :
310 : ```
311 : Parallel::PhaseActions<Parallel::Phase::CheckDomain,
312 : tmpl::list<::amr::Actions::SendAmrDiagnostics,
313 : Parallel::Actions::TerminatePhase>>,
314 : ```
315 :
316 : - The possible addition of `PhaseControl::Actions::ExecutePhaseChange`
317 : to the action list for the appropriate Phase in the PhaseAction of
318 : the `amr::element_array`
319 :
320 : - The addition of the mutator `amr::Initialization::Initialize` to the
321 : list of mutators in `Initialization::Actions::InitializeItems` in
322 : the Parallel::PhaseActions list for Parallel::Phase::Initialization.
323 :
324 : - The appropriate includes for all of the above.
325 :
326 : ### Controlling AMR {#dev_guide_amr_input_options}
327 :
328 : There are two places in the input file that control how and when AMR happens.
329 :
330 : The "how" is controlled by the option group `Amr`. Here you list the
331 : amr::Criteria being used, the available options for the amr::Policies, and
332 : the `Verbosity` of diagnostic messages that are printed. Here is an
333 : example:
334 : ```
335 : Amr:
336 : Criteria:
337 : - TruncationError:
338 : VariablesToMonitor: [Psi]
339 : AbsoluteTarget: 1.e-6
340 : RelativeTarget: 1.0
341 : Policies:
342 : EnforceTwoToOneBalanceInNormalDirection: true
343 : Isotropy: Anisotropic
344 : Limits:
345 : RefinementLevel: Auto
346 : NumGridPoints: Auto
347 : ErrorBeyondLimits: False
348 : AllowCoarsening: True
349 : Verbosity: Verbose
350 : ```
351 :
352 : Note that the values `Auto` for the Limits options choose the default
353 : limits set by the code. Also note that if a criterion tries to refine beyond the
354 : limits, whether or not the code should error is controlled by
355 : `ErrorBeyondLimits`.
356 :
357 : "When" AMR happens is controlled by specifying a Trigger and a list of
358 : `PhaseChanges` in the top-level option `PhaseChangesAndTriggers`. For
359 : example:
360 : ```
361 : PhaseChangeAndTriggers:
362 : - Trigger:
363 : Slabs:
364 : EvenlySpaced:
365 : Interval: 10
366 : Offset: 0
367 : PhaseChanges:
368 : - VisitAndReturn(EvaluateAmrCriteria)
369 : - VisitAndReturn(AdjustDomain)
370 : - VisitAndReturn(CheckDomain)
371 : ```
372 :
373 : Both `EvaluateAmrCriteria` and `AdjustDomain` are required in order
374 : for AMR to work. `VisitAndReturn(CheckDomain)` performs diagnostics
375 : and can be omitted. To turn off AMR, omit the three phase changes above.
|
