spectre.support.DirectoryStructure¶
Directories that simulations are organized in
Three kinds of directory appear throughout the code, and the name of a variable says which kind it holds:
‘run_dir’: one invocation of an executable runs here. It holds the input file, the submit script, output files, and a ‘Checkpoints’ directory.
‘segments_dir’: holds a numbered sequence of runs, such as ‘0000_Inspiral’ through ‘0013_Ringdown’. See ‘Segment’ below.
‘pipeline_dir’: the directory of a simulation, in which a pipeline creates all its runs. Each pipeline defines how it groups runs in there, e.g. ‘spectre.Pipelines.EccentricityControl.DirectoryStructure’.
A name that qualifies one of these, such as ‘id_run_dir’ or ‘inspiral_run_dir’, refers to that directory of a specific step.
Functions
|
All checkpoints in the 'checkpoints_dir' |
|
All segments in the 'segments_dir' |
Classes
|
State of a simulation saved to disk |
|
Part of a simulation that ran as one executable invocation |
- class spectre.support.DirectoryStructure.Checkpoint(path: Path, id: int)¶
State of a simulation saved to disk
An executable can write multiple checkpoints during its execution, and can be restarted from those checkpoints. The ‘id’ enumerates the checkpoints.
We currently write checkpoints in a directory structure like this:
``` RUN_DIR/
- Checkpoints/
Checkpoint_0000/ Checkpoint_0001/ …
WARNING: Don’t assume checkpoints always exist in the above directory structure. You don’t want your code to break when checkpoints are copied, moved around, or renamed.
- NAME_PATTERN = re.compile('Checkpoint_(\\d+)')¶
- NUM_DIGITS = 4¶
- id: int¶
- classmethod match(path: str | Path) Checkpoint | None¶
Checks if the ‘path’ is a checkpoint
- path: Path¶
- class spectre.support.DirectoryStructure.Segment(path: Path, id: int, label: str)¶
Part of a simulation that ran as one executable invocation
We have to split simulations into segments because supercomputers don’t typically support unlimited run times. Therefore, we terminate the job, write the simulation state to disk as a checkpoint, and submit a new job that restarts from the last checkpoint. Segments are also how a pipeline proceeds from one executable to the next, e.g. from an inspiral to a ringdown. Therefore, each segment carries a ‘label’ that says what ran in it, in addition to the ‘id’ that enumerates the segments.
We currently write segments in a directory structure like this:
``` SEGMENTS_DIR/
- 0000_Inspiral/
Inspiral.yaml Submit.sh Output.h5 Checkpoints/
- 0001_Inspiral/
…
- 0002_Ringdown/
…
Note: “Inspiral” and “Ringdown” are examples, any label can be used. The id goes _before_ the label so that a plain ‘ls’ of the segments directory lists the segments in the order they ran, no matter which executable ran in them.
WARNING: Don’t assume that simulations always have the above directory structure. You don’t want your code to break when files are copied, moved around, or renamed. Instead of relying on some directory structure, have your code take the files it needs as input. This is quite easy using globs.
- NAME_PATTERN = re.compile('(\\d+)_(.+)')¶
- NUM_DIGITS = 4¶
- property checkpoints: List[Checkpoint]¶
- property checkpoints_dir: Path¶
- id: int¶
- property input_file: Path¶
The input file for the segment (has the same name as the label)
- label: str¶
- classmethod last(segments_dir: str | Path) Segment | None¶
The last segment in the ‘segments_dir’, or ‘None’ if it has none
- classmethod next(segments_dir: str | Path, label: str) Segment¶
The next segment to create in the ‘segments_dir’
Continues the numbering of the segments in the ‘segments_dir’, or starts at zero if it has none.
- path: Path¶
- spectre.support.DirectoryStructure.list_checkpoints(checkpoints_dir: str | Path) List[Checkpoint]¶
All checkpoints in the ‘checkpoints_dir’