Cleaning run artifacts

A PyRETIS run leaves a number of generated files and directories behind – log files, the regenerated out.toml / out.rst, restart files, the NNN ensemble directories (each holding its generate scratch, the accepted operational trajectory store and the archive long-term store), the report directory, the multi-run runs directory, Python byte-code caches, and so on. (A flat top-level load directory only exists when it is staged by hand as legacy input, and is no longer a built-in default.) The pyretis tools clean sub-command removes them so a run (or example) directory can be reset to its committed inputs:

pyretis tools clean              # clean the current directory
pyretis tools clean some/dir     # clean another directory
pyretis tools clean --dry-run    # list what would be removed, delete nothing

pyretis tools clean deletes immediately; use --dry-run (or -n) first if you want to preview the list.

Note

pyretis clean (without tools) still works as a deprecated alias: it prints a one-line notice and then behaves identically. Use pyretis tools clean in new scripts.

What is removed

The removed set is the union of a comprehensive built-in default and an optional per-directory configuration file. The built-in defaults cover the full set of artifacts a routed run generates, so a plain run folder resets with no clean.toml at all:

  • files – byte-code / compiled order parameters (*.pyc, *.pyo, *.so), the run file and restart store (output.toml*, out.toml*, out.rst*, infswap.toml, restart.toml*, pyretis.restart, ensemble.restart), the per-run text output (energy.txt*, order.txt*, pathensemble.txt*, trials.txt*, cross.txt*, thermo.txt*, traj.txt*), the logs of both pyretis run and pyretis analyse including their backup copies (pyretis.log*, pyretisanalyse.log*, sim.log, worker[0-9]*.log), the RNG state, the engine scratch of every engine (*.tpr, *.edr, *.cpt, *.lammpstrj, …), and the run-done marker / plots;

  • directories – __pycache__, the NNN ensemble directories ([0-9][0-9][0-9], which subsume their generate / accepted / archive / engine_logs sub-stores), the accepted / archive / engine_logs stores and the previous store names (paths / trajs), the retired worker* scratch, the report and runs directories, and the private directories a run removes once it is done with them, left behind by a run that stopped first: pyretis-gromacs-*, the work directory of a GROMACS engine, removed with the engine, and pyretis-load-*, where a load prepares its files, removed when the load ends; and pyretis-cp2k-wfn, where the streaming CP2K engine keeps the wavefunctions its MD segments start from for the whole run;

  • the store of the paths of a scheduler run, <ensemble>/<load_dir>, under the [simulation] load_dir a TOML input names (by default accepted), in each ensemble directory of that run: the directories named by ensemble number (000, 001, …, 1000, …) in the directory of the file, or in its [output] data_dir. The TOML files are read as for the load input of the runs (below), before any file is removed, so the output.toml of a run names its store too. The store is removed also where its ensemble directory is left, because it holds a kept path or has a name of four digits. A store reached through a symbolic link, or a load_dir that points out of the ensemble directory, is left, and use_defaults = false leaves every store.

Three mechanisms keep these otherwise-aggressive defaults safe (the clean-safety sweep asserts no git-tracked file is ever removed):

  • Excluded reference/input directories. Anything below results, output_data, load, load_copy or an engine input directory (lammps_input, gromacs_input, cp2k_input, openmm_input) is never cleaned. These hold committed reference (golden) output, committed load/NNN initial-path inputs and committed engine input files.

  • The load input of the runs. A TOML input file names the paths the run in its directory reads: the [initial-path] load_folder of a classic load, and the store <load_dir> a fresh scheduler run reads its initial paths from ([simulation] load_dir, by default accepted), both relative to the directory of the file. The TOML files in the cleaned directory and in every directory below it are read, so a clean started above a run directory keeps them too. They are left with all their files, whatever the default patterns, find_files, find_dirs or rm_paths match; remove one by hand once you are done with it. A TOML file that cannot be parsed stops the clean before it removes anything. A load folder that only a legacy .rst input names is protected with keep.

  • The ``keep`` list (below), for committed inputs that live outside those excluded names – for example a non-standard committed load directory, or the load/0 .. load/7 initial paths of the validation suite.

Per-directory configuration

Drop a clean.toml next to the files to extend (or replace) the defaults:

[clean]
use_defaults = true        # apply the built-in defaults (default: true)
find_files = ["out.toml*", "*.log"]   # extra file-name globs (recursive)
find_dirs  = ["report"]               # extra directory names (recursive)
rm_paths   = ["0*", "lammps/system.data"]  # root-relative rm -rf globs
keep       = ["load/0", "load/1"]     # never delete these (or ancestors)

  • find_files – file-name globs deleted wherever they occur in the tree (like find . -name PATTERN -delete).

  • find_dirs – directory names deleted recursively wherever they occur.

  • rm_paths – root-relative paths or globs removed with rm -rf semantics.

  • keep – paths that must survive; a kept path, anything inside it, and – since keep declares a committed input – the declaration is honoured even when the clean was started from an ANCESTOR directory (its clean.toml is read during the walk), and its ancestor directories are never removed. Use this to protect committed inputs that would otherwise match (for example the committed load/0 .. load/7 initial paths of the validation suite).

  • use_defaults = false disables the built-in defaults so the file is the complete, explicit recipe.

This replaces the per-example Makefile clean targets used in earlier PyRETIS versions.