pyretis.testing package¶
This package defines common methods which are used for testing.
Package structure¶
Modules¶
- simulation_comparison.py (
pyretis.testing.simulation_comparison) Common methods for comparing results.
- helpers.py (
pyretis.testing.helpers) Common methods for tests.
- example_compare.py (
pyretis.testing.example_compare) The comparison of the results of an example under
examples/tests/with its committed reference.- devices.py (
pyretis.testing.devices) The devices the engine reference suites run their cases on: the CPU, and a CUDA GPU when CUDA is available and the CUDA build of an engine is installed.
- reference_manifest.py (
pyretis.testing.reference_manifest) The validation of the provenance and the content digests of the numerical references.
- systemhelp.py (
pyretis.testing.systemhelp) Methods for creating systems in tests.
List of submodules¶
pyretis.testing.devices module¶
The devices the engine reference suites run their cases on.
The suites of examples/tests run every case of GROMACS, OpenMM and
LAMMPS on the CPU. A case with a CUDA pass also runs on a CUDA GPU when
CUDA is available and the CUDA build of its engine is installed. CUDA
is available when nvidia-smi lists a GPU that
CUDA_VISIBLE_DEVICES leaves to the CUDA programs; this is checked
first, before any CUDA build is looked for. The environment variable
PYRETIS_TEST_DEVICE selects the passes:
auto(the default)The CPU passes, and the CUDA passes of each engine whose CUDA build is installed when CUDA is available. When CUDA is not available (
nvidia-sminot found, failing to run or to end, or listing no GPU;CUDA_VISIBLE_DEVICESthat hides every GPU, or names an index beyond the GPUsnvidia-smilists), the CPU passes only, whatever CUDA builds are installed; the reason of the CPU passes says why CUDA is not available.cpuThe CPU passes only.
cudaAs
auto; CUDA not available, or an engine without a CUDA build, fails the suite of the engine with a message that starts withCUDA REQUIRED.
The CUDA build of an engine is installed when:
GROMACS: the command that
PYRETIS_GMX_CUDAnames (defaultgmx_cuda) is found and its-versionreportsGPU support: CUDA;LAMMPS: the command that
PYRETIS_LMP_CUDAnames (defaultlmp_cuda) is found and its-hreportsGPU package API: CUDA;OpenMM: the
CUDAplatform is loaded, or a CUDA plugin of OpenMM failed to load.
A command that is found without CUDA support, a PYRETIS_GMX_CUDA or
PYRETIS_LMP_CUDA that is set and names neither an executable file
nor a command on PATH, and a CUDA plugin of OpenMM that failed to
load, are broken builds when CUDA is available, also under auto;
without CUDA they are not looked at. When CUDA is available, an
installed build is checked before the CUDA passes: the engine runs a
small input on the GPU, with the options that make it stop when no GPU
is usable. A broken or failing build fails the suite with a message that
starts with CUDA BROKEN; the CPU is never used in its place: a CUDA
pass that cannot run is not run, and is never counted as passed. The
message of a broken build, and the CUDA REQUIRED messages of
cuda, end with the way on: what to set or fix to give the engine a
CUDA build that runs, or to make CUDA available, and the value of
PYRETIS_TEST_DEVICE that runs the CPU passes only.
Each pass of a case writes one line to the device ledger of its suite,
the file that PYRETIS_DEVICE_LEDGER names:
<engine> | <case> | CUDA | <GPU model> | driver <version> | <build>
<engine> | <case> | CPU | <reason>
A CUDA pass that fails writes its line with the GPU model of the
preflight run followed by (pass failed).
The GPU model of a LAMMPS or an OpenMM pass is read from the output of
that pass: the Device lines of LAMMPS and the platform line of the
OpenMM engine in pyretis.log. A GROMACS pass keeps no md.log;
its GPU model is read from the md.log of the preflight run, and
mdrun runs its cases with the options that stop it when no GPU is
usable.
The shell side of the suites (examples/tests/_devices.sh) uses the
command line of this module:
python -m pyretis.testing.devices select ENGINE PREFLIGHT_DIR
python -m pyretis.testing.devices record-cpu ENGINE CASE REASON
python -m pyretis.testing.devices record-cuda ENGINE CASE OUTPUT_DIR
--gpu MODEL --driver VERSION --build BUILD [--failed]
python -m pyretis.testing.devices case-device
python -m pyretis.testing.devices watch-mdrun --cap SECONDS CASE LOG
-- COMMAND...
python -m pyretis.testing.devices stage-openmm MODULE CUDA_MODULE
INPUT CUDA_INPUT
python -m pyretis.testing.devices counts LEDGER [--device DEVICE]
python -m pyretis.testing.devices record-grade GRADES LEDGER DEVICE
GRADE
python -m pyretis.testing.devices summary LEDGER [--grades GRADES]
python -m pyretis.testing.devices recap LEDGER [--grades GRADES]
The grade file of a run (examples/run-tests.sh) holds the grade of
the CPU and of the CUDA passes of each engine:
<engine> | <device> | <grade>
- pyretis.testing.devices.CASE_DEVICES = ('cpu', 'cuda')¶
The values of
CASE_DEVICE_ENV; the first is the default.
- pyretis.testing.devices.CASE_DEVICE_ENV = 'PYRETIS_CASE_DEVICE'¶
The environment variable that tells a case which pass it runs:
cpu(the default) orcuda.
- pyretis.testing.devices.CUDA_BUILD_NAMES = {'gromacs': 'CUDA build', 'lammps': 'CUDA build', 'openmm': 'CUDA platform'}¶
What the
CUDA REQUIREDmessage says an engine has none of.
- pyretis.testing.devices.CUDA_COMMANDS = {'gromacs': ('PYRETIS_GMX_CUDA', 'gmx_cuda'), 'lammps': ('PYRETIS_LMP_CUDA', 'lmp_cuda')}¶
The environment variable that names the CUDA build of an engine run as a command, and the command it names by default.
- pyretis.testing.devices.CUDA_REPORTS = {'gromacs': ('-version', 'GPU support'), 'lammps': ('-h', 'GPU package API')}¶
The option that makes the CUDA build of an engine run as a command report its GPU support, and the key of the line that reports it.
- class pyretis.testing.devices.CudaAvailability(gpus: list, driver: str, reason: str)¶
Bases:
objectWhether CUDA is available on the machine, from
nvidia-smi.- Variables:
gpus (list of strings) – The models of the GPUs
nvidia-smilists; empty when CUDA is not available.driver (string) – The version of the NVIDIA driver; empty when CUDA is not available.
reason (string) – Why CUDA is not available:
CUDA_VISIBLE_DEVICESthat hides every GPU or names none thatnvidia-smilists;nvidia-sminot found, failing to run or to end, ending with an error, or listing no GPU; empty when it is available.
- property available¶
Return True when
nvidia-smilists a GPU.
- driver: str¶
- gpus: list¶
- reason: str¶
- class pyretis.testing.devices.CudaBuild(engine: str, command: str, version: str, description: str)¶
Bases:
objectThe CUDA build of an engine.
- Variables:
engine (string) – The engine, a key of
ENGINE_TITLES.command (string) – The path of the program; empty for OpenMM, a Python package.
version (string) – The version the build reports (the full banner of LAMMPS).
description (string) – The build as the reports name it.
- command: str¶
- description: str¶
- engine: str¶
- version: str¶
Bases:
DeviceFailureCUDA not available when
PYRETIS_TEST_DEVICEiscuda.- Parameters:
engine (string) – The engine, a key of
ENGINE_TITLES.reason (string) – Why CUDA is not available (
CudaAvailability).
Notes
The kind is
CUDA REQUIRED, and the message readsCUDA REQUIRED: <engine>: CUDA is not available (<reason>). <way on>, with the way on ofVISIBLE_CLAUSEwhenCUDA_VISIBLE_DEVICESis the reason, else ofDRIVER_CLAUSE.Store the failure and its message.
Return the message without its way on.
- Returns:
out (string) –
CUDA REQUIRED: <engine>: CUDA is not available (<reason>).
- pyretis.testing.devices.DEVICE_ENV = 'PYRETIS_TEST_DEVICE'¶
The environment variable that selects the passes of the suites.
- pyretis.testing.devices.DEVICE_MODES = ('auto', 'cpu', 'cuda')¶
The values of
DEVICE_ENV; the first is the default.
- pyretis.testing.devices.DRIVER_CLAUSE = 'install or repair the NVIDIA driver so that nvidia-smi lists the GPU'¶
The way on when CUDA is not available (
way_on_sentence()).
- exception pyretis.testing.devices.DeviceFailure(kind, engine, detail, way_on)¶
Bases:
ExceptionA CUDA build that cannot run the CUDA passes of a suite.
- Parameters:
kind (string) –
CUDA BROKENfor an installed build that does not run on the GPU,CUDA REQUIREDfor an engine without a CUDA build whenPYRETIS_TEST_DEVICEiscuda.engine (string) – The engine, a key of
ENGINE_TITLES.detail (string) – What failed, in one line; for
CUDA REQUIRED, what was looked for and not found (GROMACS, LAMMPS) or the platforms OpenMM has.way_on (string) – The way on, one sentence (
way_on_sentence()).
Notes
The message reads
CUDA BROKEN: <engine>: <detail>. <way on>,CUDA REQUIRED: <engine> has no CUDA build (<detail>). <way on>for GROMACS and LAMMPS, andCUDA REQUIRED: OpenMM has no CUDA platform (<detail>). <way on>. A detail that ends with.,!or?is a finished sentence and takes no full stop. CUDA not available whenPYRETIS_TEST_DEVICEiscudaisCudaUnavailable.- __init__(kind, engine, detail, way_on)¶
Store the failure and its message.
- failure_text()¶
Return the message without its way on.
- Returns:
out (string) – The failure, one sentence (see the notes of the class).
- pyretis.testing.devices.ENGINE_TITLES = {'gromacs': 'GROMACS', 'lammps': 'LAMMPS', 'openmm': 'OpenMM'}¶
The engines with a CUDA tier, in the order of the reports, with the name the reports give them.
- pyretis.testing.devices.GRADES = ('VALIDATED', 'SMOKE', 'SKIPPED', 'FAILED')¶
The grades of the passes of a suite on one device.
- pyretis.testing.devices.GROMACS_GPU_OPTIONS = ('-nb', 'gpu', '-pme', 'gpu', '-bonded', 'cpu', '-update', 'cpu')¶
the nonbonded interactions and PME on the GPU; mdrun stops when no GPU is usable.
- Type:
The mdrun options of a CUDA pass
- pyretis.testing.devices.GROMACS_PREFLIGHT_FILES = ('preflight.mdp', 'conf.gro', 'topol.top')¶
The files of the GROMACS preflight input, in its folder.
- pyretis.testing.devices.LAMMPS_DEVICE_LINE = re.compile('^Device (\\d+): ([^,]+),', re.MULTILINE)¶
Device 0: Quadro T2000, 16 CUs, ....- Type:
A GPU of a LAMMPS run
- pyretis.testing.devices.LAMMPS_GPU_OPTIONS = ('-sf', 'gpu', '-pk', 'gpu', '1')¶
the styles of the GPU package on one GPU; LAMMPS stops when no GPU is usable.
- Type:
The LAMMPS options of a CUDA pass
- pyretis.testing.devices.LAMMPS_PREFLIGHT_INPUT = 'in.preflight'¶
The LAMMPS preflight input, in its folder.
- pyretis.testing.devices.LEDGER_DEVICES = ('CPU', 'CUDA')¶
The devices of a pass, as the ledger names them.
- pyretis.testing.devices.LEDGER_ENV = 'PYRETIS_DEVICE_LEDGER'¶
The environment variable that names the device ledger of a suite.
- pyretis.testing.devices.LEDGER_SEPARATOR = ' | '¶
The separator of the fields of a ledger line.
- pyretis.testing.devices.NVIDIA_SMI_QUERY = ('--query-gpu=name,driver_version', '--format=csv,noheader')¶
The arguments of
nvidia-smithat list the GPUs and the driver.
- pyretis.testing.devices.OPENMM_CUDA_PROPERTIES = {'DeterministicForces': 'true', 'Precision': 'double'}¶
The platform properties of an OpenMM CUDA pass.
- pyretis.testing.devices.OPENMM_PLATFORM_LINE = re.compile('OpenMM platform: (\\S+) \\((?:.*, )?DeviceName ([^,)]+)[,)]')¶
The platform line of the OpenMM engine, with the name of its device.
- pyretis.testing.devices.OPENMM_PLATFORM_LINES = (("PLATFORM = 'Reference'", "PLATFORM = 'CUDA'"), ('PLATFORM_PROPERTIES = {}', "PLATFORM_PROPERTIES = {'Precision': 'double', 'DeterministicForces': 'true'}"))¶
The lines of the module of an OpenMM case that name its platform, and the lines of its CUDA pass (
stage_openmm_cuda()).
- pyretis.testing.devices.OPENMM_PLUGIN_ENV = 'OPENMM_PLUGIN_DIR'¶
The environment variable that names the folder OpenMM loads its plugins from, in place of the plugins folder of its installation.
- pyretis.testing.devices.PREFLIGHT_CLAUSE = 'fix what stops the preflight on the GPU'¶
The way on of a failing preflight (
way_on_sentence()).
- pyretis.testing.devices.PREFLIGHT_TIMEOUT = 300¶
The longest time, in seconds,
nvidia-smior one program of a preflight may take.
- pyretis.testing.devices.STOP_TIMEOUT = 60¶
The longest time, in seconds, a stopped program may take to end before it is killed.
- class pyretis.testing.devices.Selection(engine: str, cuda: bool, reason: str = '', build: CudaBuild = None, gpu: str = '', driver: str = '')¶
Bases:
objectThe device of the CUDA passes of an engine.
- Variables:
engine (string) – The engine, a key of
ENGINE_TITLES.cuda (boolean) – True when the CUDA passes run.
reason (string) – Why the CUDA passes do not run; empty when they run.
build (object like
CudaBuildor None) – The CUDA build the passes run, None when they do not run.gpu (string) – The model of the GPU of the preflight run.
driver (string) – The version of the NVIDIA driver.
- cuda: bool¶
- driver: str = ''¶
- engine: str¶
- gpu: str = ''¶
- reason: str = ''¶
- pyretis.testing.devices.UNAVAILABLE_REASON = 'CUDA not available'¶
The start of the reason of the CPU passes when CUDA is not available.
- pyretis.testing.devices.VISIBLE_CLAUSE = 'unset CUDA_VISIBLE_DEVICES or set it to a device index or a GPU UUID that nvidia-smi lists'¶
The way on when
CUDA_VISIBLE_DEVICESleaves no GPU to the CUDA programs (cuda_availability()).
- pyretis.testing.devices.WATCH_INTERVAL = 0.2¶
The time, in seconds, between two looks at the logs of mdrun (
watch_mdrun()).
- pyretis.testing.devices.WATCH_STOP_SIGNALS = (Signals.SIGTERM, Signals.SIGHUP)¶
The signals that end
watch_mdrun()with the program it runs (SIGINTends it throughKeyboardInterrupt).
- pyretis.testing.devices._COMMANDS = {'case-device': <function _command_case_device>, 'counts': <function _command_ledger>, 'recap': <function _command_ledger>, 'record-cpu': <function _command_record_cpu>, 'record-cuda': <function _record_cuda>, 'record-grade': <function _command_record_grade>, 'select': <function _command_select>, 'stage-openmm': <function _command_stage_openmm>, 'summary': <function _command_ledger>, 'watch-mdrun': <function _command_watch_mdrun>}¶
The function of each subcommand.
- pyretis.testing.devices._command_case_device(_)¶
Print the pass a case runs.
- Returns:
out (int) – 0, or 2 for a wrong value of
PYRETIS_CASE_DEVICE.
- pyretis.testing.devices._command_ledger(args)¶
Print the counts, the summary or the recap of a ledger.
- Parameters:
args (object like
argparse.Namespace) – The arguments ofcounts,summaryorrecap.- Returns:
out (int) – 0, or 1 when the ledger or the grade file cannot be read, or the grade file lacks the grade of a device with a pass.
- pyretis.testing.devices._command_record_cpu(args)¶
Record a CPU pass.
- Parameters:
args (object like
argparse.Namespace) – The arguments ofrecord-cpu.- Returns:
out (int) –
- pyretis.testing.devices._command_record_grade(args)¶
Append the grade of the passes of a suite on a device.
- Parameters:
args (object like
argparse.Namespace) – The arguments ofrecord-grade.- Returns:
out (int) – 0, or 1 when the ledger holds no pass on the device, or the passes of more than one engine.
- pyretis.testing.devices._command_select(args)¶
Print the selection of
selectas shell assignments.- Parameters:
args (object like
argparse.Namespace) – The arguments ofselect.- Returns:
out (int) – 0, or 1 for a
CUDA BROKENor aCUDA REQUIREDfailure, or 2 for a wrong value ofPYRETIS_TEST_DEVICE.
- pyretis.testing.devices._command_stage_openmm(args)¶
Write the module and the input of an OpenMM CUDA pass.
- Parameters:
args (object like
argparse.Namespace) – The arguments ofstage-openmm.- Returns:
out (int) – 0, or 1 when a line to replace is missing or repeated.
- pyretis.testing.devices._command_watch_mdrun(args)¶
Run a program until one of its MD runs of mdrun has ended.
- Parameters:
args (object like
argparse.Namespace) – The arguments ofwatch-mdrun.- Returns:
out (int) – The status of
watch_mdrun().
- pyretis.testing.devices._cpu_reasons(passes)¶
Return the reasons of the CPU passes of an engine, in words.
- Parameters:
passes (list of lists of strings) – The passes of one engine (
read_ledger()).- Returns:
out (string) – Each distinct reason of the CPU passes, joined with
'; '.
- pyretis.testing.devices._cuda_command(engine, environ)¶
Return the path of the CUDA build of an engine, and its name.
- Parameters:
engine (string) –
gromacsorlammps.environ (dict) – The environment.
- Returns:
path (string or None) – The absolute path of the command that the variable of
CUDA_COMMANDSnames, or of its default when the variable is not set or empty; None when the default is not found.named (string) – How a message names the build:
<variable> names <path>, or<default> (the default of <variable>) is <path>; for a default that is not found,<default> not found.
- Raises:
DeviceFailure – If the variable is set and its value is neither an executable file nor a command on
PATH: a build the user named that cannot run is a broken build.
- pyretis.testing.devices._device_grade(grades, title, device)¶
Return the grade of the passes of an engine on a device.
- Parameters:
grades (dict) – The grades (
read_grades()).title (string) – The engine title.
device (string) –
CPUorCUDA.
- Returns:
out (string) – The grade.
- Raises:
ValueError – If the grades hold none for the engine and the device.
- pyretis.testing.devices._engine_entries(entries)¶
Return the passes of a ledger per engine, in the report order.
- Parameters:
entries (list of lists of strings) – The passes (
read_ledger()).- Returns:
out (list of tuples) – The engine title and its passes, for each engine with a pass.
- pyretis.testing.devices._exit_on_signal(number, _frame)¶
End the program with the status of a signal it received.
- Parameters:
number (int) – The signal.
_frame (object) – The frame the signal interrupted.
- Raises:
SystemExit – Always, with the status
128 + number.
- pyretis.testing.devices._finished_md_logs(log)¶
Return the logs of ended mdrun runs that integrated MD steps.
- Parameters:
log (string) – The log path that mdrun is given (
-g). GROMACS moves an earlier log of the path to a backup,#<name>.<n>#, in the same folder.- Returns:
out (list of strings) – The text of the log and of its backups whose run ended (
Finished mdrun) and had MD steps to run (nstepsnot 0; a run withnsteps = 0only evaluates the forces of its configuration).
- pyretis.testing.devices._paragraph_after(lines, index)¶
Return the lines that follow a line, up to the next blank one.
- Parameters:
lines (list of strings) – The lines of an output, stripped.
index (int) – The index of the line.
- Returns:
out (list of strings) – The lines after
lines[index]before the first blank line.
- pyretis.testing.devices._parser()¶
Return the parser of the command line.
- Returns:
out (object like
argparse.ArgumentParser) – The parser, with one subcommand per action.
- pyretis.testing.devices._record_cuda(args)¶
Record a CUDA pass after reading its GPU from its output.
The GPU of a LAMMPS or an OpenMM pass is read from its output, that of a GROMACS pass is the GPU of the preflight run. A failed pass (
--failed) is recorded with the GPU of the preflight run and(pass failed), and its output is not read.- Parameters:
args (object like
argparse.Namespace) – The arguments ofrecord-cuda.- Returns:
out (int) – 0 when the pass is recorded, 1 when the output of a LAMMPS or an OpenMM pass names no GPU or more than one.
- pyretis.testing.devices._replace_lines(text, replacements, source)¶
Replace whole lines of a text, each found exactly once.
- Parameters:
text (string) – The text.
replacements (sequence of (string, string)) – Each line to replace and the line that replaces it.
source (string) – The file the text comes from, for the message of a failure.
- Returns:
out (string) – The text with the lines replaced.
- Raises:
ValueError – If a line to replace is not in the text exactly once.
- pyretis.testing.devices._report_value(output, key)¶
Return the value of a
key: valueline of a program’s report.- Parameters:
output (string) – The report.
key (string) – The key, at the start of its line.
- Returns:
out (string) – The value, stripped; empty when no line starts with the key.
- pyretis.testing.devices._stop_session(process)¶
Stop a program started in a session of its own, and its children.
- Parameters:
process (object like
subprocess.Popen) – The program, the leader of its session.
- pyretis.testing.devices._text_files(folder, suffixes)¶
Yield the text of the files of a folder tree with given suffixes.
- Parameters:
folder (string) – The top of the tree.
suffixes (tuple of strings) – The ends of the names of the files to read.
- Yields:
out (string) – The text of one file.
- pyretis.testing.devices._watch_process(process, log, cap, case)¶
Watch a program until one of its MD runs of mdrun has ended.
- Parameters:
process (object like
subprocess.Popen) – The program, the leader of its session.log (string) – The log path given to mdrun.
cap (float) – The longest time, in seconds, to wait for an mdrun run to end.
case (string) – The name of the run, for the device line.
- Returns:
out (int) – The status of
watch_mdrun().
- pyretis.testing.devices.append_ledger(line, environ=None)¶
Append a line to the device ledger, when there is one.
- Parameters:
line (string) – The line (
ledger_line()).environ (dict, optional) – The environment, with
PYRETIS_DEVICE_LEDGER;os.environwhen None. A run without the variable keeps no ledger.
- pyretis.testing.devices.case_device(environ=None)¶
Return the pass a case runs, from
PYRETIS_CASE_DEVICE.- Parameters:
environ (dict, optional) – The environment;
os.environwhen None.- Returns:
out (string) –
cpuwhen the variable is not set or empty, orcuda.- Raises:
ValueError – If the variable holds another value.
- pyretis.testing.devices.cuda_availability()¶
Return whether CUDA is available: the GPUs
nvidia-smilists.nvidia-smiis looked up onPATHand run once, withNVIDIA_SMI_QUERY, for at mostPREFLIGHT_TIMEOUTseconds.The GPUs are the lines of its standard output that the query gives,
<name>, <driver version>; any other line (a message such asNo devices were found, or a warning) is not a GPU.CUDA_VISIBLE_DEVICESthat is empty, or whose first entry is neither a device index nor a GPU or MIG UUID (-1, for example), leaves no GPU to the CUDA programs, whatevernvidia-smilists: CUDA is then not available, andnvidia-smiis not run. A first entry that is a GPU or MIG UUID is not checked against the GPUsnvidia-smilists: CUDA is available with one that names none, and the CUDA programs then find no GPU.- Returns:
out (object like
CudaAvailability) – The GPUs and the driver version, or why CUDA is not available:nvidia-smi not found,nvidia-smi could not be run: <the error>,nvidia-smi did not end within <t> s,nvidia-smi ended with status <n>: <its error line>(first_error_line()),nvidia-smi lists no GPU, orCUDA_VISIBLE_DEVICES=<value> hides every GPU, orCUDA_VISIBLE_DEVICES=<value> names no GPU that nvidia-smi lists (<n> listed).
- pyretis.testing.devices.cuda_build_clause(engine)¶
Return how to give an engine its CUDA build, as part of a way on.
- Parameters:
engine (string) – The engine, a key of
ENGINE_TITLES.- Returns:
out (string) – For GROMACS and LAMMPS, the variable of
CUDA_COMMANDSand the report ofCUDA_REPORTSthat a CUDA build gives; for OpenMM,OPENMM_PLUGIN_ENVand the version of the plugins, the version of the OpenMM that runs. A CUDA plugin loads when the libraries it needs are found.
- pyretis.testing.devices.device_passes(entries, device)¶
Return the number of passes of a ledger on one device.
- Parameters:
entries (list of lists of strings) – The passes (
read_ledger()).device (string) –
CPUorCUDA.
- Returns:
out (int) – The number of passes on the device.
- pyretis.testing.devices.engine_device(passes)¶
Return the device an engine ran its CUDA passes on, in words.
- Parameters:
passes (list of lists of strings) – The passes of one engine (
read_ledger()).- Returns:
out (string) –
CUDA, <GPU model>, driver <version>, <build>for an engine with CUDA passes (each distinct GPU, driver and build, joined with/), orCPU (<reason>)with the reasons of its passes.
- pyretis.testing.devices.find_gromacs_cuda(environ)¶
Return the CUDA build of GROMACS, or why there is none.
- Parameters:
environ (dict) – The environment, with
PYRETIS_GMX_CUDA.- Returns:
build (object like
CudaBuildor None) – The build, None whenPYRETIS_GMX_CUDAis not set and its default is not found.missing (string) – What was looked for and not found; empty when there is a build.
- Raises:
DeviceFailure – If
PYRETIS_GMX_CUDAis set and names no executable file or command onPATH, or the command reports no CUDA support, or its-versionfails.
- pyretis.testing.devices.find_lammps_cuda(environ)¶
Return the CUDA build of LAMMPS, or why there is none.
- Parameters:
environ (dict) – The environment, with
PYRETIS_LMP_CUDA.- Returns:
build (object like
CudaBuildor None) – The build, None whenPYRETIS_LMP_CUDAis not set and its default is not found. Its version is the full banner: the release date and the update number (- Update N).missing (string) – What was looked for and not found; empty when there is a build.
- Raises:
DeviceFailure – If
PYRETIS_LMP_CUDAis set and names no executable file or command onPATH, or the GPU package of the command is not a CUDA one, or its-hfails.
- pyretis.testing.devices.find_openmm_cuda()¶
Return the CUDA platform of OpenMM, or why there is none.
- Returns:
build (object like
CudaBuildor None) – The build, None when OpenMM has no CUDA platform.missing (string) – What was looked for and not found; empty when there is a build.
- Raises:
DeviceFailure – If a CUDA plugin of OpenMM failed to load.
- pyretis.testing.devices.first_error_line(text)¶
Return the line of a program’s output that says what failed.
- Parameters:
text (string) – The output of the program.
- Returns:
out (string) – The paragraph after the first
Fatal errorline (GROMACS), its lines joined; or the first line that holdserror(in any letter case), joined with the rest of its paragraph when it ends with:(a header such asError in user input:orERROR 1 [file grompp.mdp, line 3]:), or without its:when no line follows; or the last line;(no output)for an output without text.
- pyretis.testing.devices.grade_line(title, device, grade)¶
Return one line of the grade file.
- Parameters:
title (string) – The engine title, a value of
ENGINE_TITLES.device (string) –
CPUorCUDA.grade (string) – One of
GRADES.
- Returns:
out (string) – The line, without its end of line.
- Raises:
ValueError – If a field is not one of its values.
- pyretis.testing.devices.gromacs_log_gpu(md_log)¶
Return the model of the GPU an mdrun run used, from its log.
- Parameters:
md_log (string) – The text of the
md.logof the run.- Returns:
out (string or None) – The model of the GPU of the particle-particle tasks, None when the log maps no task to a GPU or names no model for its GPU.
- pyretis.testing.devices.lammps_output_gpus(text)¶
Return the GPU models of the
Devicelines of LAMMPS output.- Parameters:
text (string) – The screen output of LAMMPS.
- Returns:
out (set of strings) – The models of the GPUs; empty when the output has no
Deviceline.
- pyretis.testing.devices.ledger_counts(entries)¶
Return the passes of a ledger per device, as a report says them.
- Parameters:
entries (list of lists of strings) – The passes (
read_ledger()).- Returns:
out (string) –
CUDA: <n>, CPU: <m>, orCPU: <m>when no pass ran on CUDA.
- pyretis.testing.devices.ledger_engine(entries)¶
Return the engine of the passes of the ledger of one suite.
- Parameters:
entries (list of lists of strings) – The passes (
read_ledger()).- Returns:
out (string) – The engine title of the passes.
- Raises:
ValueError – If the ledger holds no pass, or passes of more than one engine.
- pyretis.testing.devices.ledger_line(engine, case, device, fields)¶
Return one line of the device ledger.
- Parameters:
engine (string) – The engine, a key of
ENGINE_TITLES.case (string) – The case, as its suite names it.
device (string) –
CPUorCUDA.fields (list of strings) – The reason of a CPU pass; the GPU model,
driver <version>and the build of a CUDA pass.
- Returns:
out (string) – The line, without its end of line.
- pyretis.testing.devices.ledger_recap(entries, grades=None)¶
Return the devices of the engines in one line.
- Parameters:
entries (list of lists of strings) – The passes (
read_ledger()).grades (dict, optional) – The grades of the passes per engine and device (
read_grades()); None leaves them out.
- Returns:
out (string) – Without grades:
GROMACS CUDA, OpenMM CUDA, LAMMPS CPU (<reason>)for the engines with a pass. With grades:GROMACS CUDA VALIDATED, CPU VALIDATED; OpenMM CUDA SMOKE, CPU VALIDATED; LAMMPS CPU VALIDATED (<reason>).none recordedfor a ledger without a pass.- Raises:
ValueError – If
gradesis given and lacks the grade of a device with a pass.
- pyretis.testing.devices.ledger_summary(entries, grades=None)¶
Return the device summary of a ledger, one line per engine.
- Parameters:
entries (list of lists of strings) – The passes (
read_ledger()).grades (dict, optional) – The grades of the passes per engine and device (
read_grades()); None leaves them out.
- Returns:
out (list of strings) – For each engine with a pass:
<engine>: <device>; <n> cases on CUDA, <m> on CPU(engine_device()), with the grade after the count of each device that has a pass whengradesis given:<n> cases on CUDA (SMOKE), <m> on CPU (VALIDATED).- Raises:
ValueError – If
gradesis given and lacks the grade of a device with a pass.
- pyretis.testing.devices.main(argv=None)¶
Run the command line of the device tier.
- Parameters:
argv (list of strings, optional) – The arguments;
sys.argv[1:]when None.- Returns:
out (int) – The exit status: 0 on success, 1 for a
CUDA BROKENor aCUDA REQUIREDfailure, a CUDA pass whose output names no GPU, an OpenMM module without its platform lines, or a ledger or a grade file that cannot be read, 2 for a wrong value ofPYRETIS_TEST_DEVICEorPYRETIS_CASE_DEVICE.
- pyretis.testing.devices.openmm_platforms()¶
Return the names of the platforms OpenMM has loaded.
- Returns:
out (list of strings) – The names, in the order of OpenMM.
- pyretis.testing.devices.pass_gpus(engine, output_dir)¶
Return the GPU models that the output of a CUDA pass names.
- Parameters:
engine (string) –
lammpsoropenmm.output_dir (string) – The folder the pass ran in.
- Returns:
out (set of strings) – The models of the
Devicelines of the.screenfiles of LAMMPS, or of theDeviceNameof the CUDA platform lines of thepyretis.logfiles of OpenMM.
- pyretis.testing.devices.preflight_gromacs(build, input_dir)¶
Run mdrun on the GPU on a small input and return the GPU model.
- Parameters:
build (object like
CudaBuild) – The CUDA build of GROMACS.input_dir (string) – The folder with
GROMACS_PREFLIGHT_FILES.
- Returns:
out (string) – The model of the GPU the run used.
- Raises:
DeviceFailure – If grompp or mdrun fails, or the log of mdrun names no GPU.
- pyretis.testing.devices.preflight_lammps(build, input_dir)¶
Run LAMMPS on the GPU on a small input and return the GPU model.
- Parameters:
build (object like
CudaBuild) – The CUDA build of LAMMPS.input_dir (string) – The folder with
LAMMPS_PREFLIGHT_INPUT.
- Returns:
out (string) – The model of the GPU the run used.
- Raises:
DeviceFailure – If LAMMPS fails, or its output names no GPU.
- pyretis.testing.devices.preflight_openmm()¶
Build a context on the CUDA platform and return the GPU model.
- Returns:
out (string) – The
DeviceNameof the context.- Raises:
DeviceFailure – If OpenMM cannot build the context.
- pyretis.testing.devices.read_grades(path)¶
Return the grades of a grade file.
- Parameters:
path (string) – The grade file.
- Returns:
out (dict) – The grade of each engine title and device.
- Raises:
ValueError – If a line is not a grade line (
grade_line()), or gives a second grade to an engine and a device.
- pyretis.testing.devices.read_ledger(path)¶
Return the passes a device ledger records.
- Parameters:
path (string) – The ledger.
- Returns:
out (list of lists of strings) – The fields of each line: the engine title, the case, the device and the fields of
ledger_line().- Raises:
ValueError – If a line has fewer than four fields, an engine that is not in
ENGINE_TITLES, or a device that is notCPUorCUDA.
- pyretis.testing.devices.required_failure(engine, missing)¶
Return the failure of an engine without a CUDA build in
cuda.- Parameters:
engine (string) – The engine, a key of
ENGINE_TITLES.missing (string) – What was looked for and not found.
- Returns:
out (object like
DeviceFailure) – TheCUDA REQUIREDfailure; its detail ismissingfor GROMACS and LAMMPS, and the platforms OpenMM has for OpenMM.
- pyretis.testing.devices.run_program(command, engine, clauses, cwd=None)¶
Run one program of a preflight and return its output.
- Parameters:
command (list of strings) – The program and its arguments.
engine (string) – The engine the program belongs to, for the message of a failure.
clauses (tuple of strings) – What to set or fix when the program fails, the first clauses of the way on of its message (
way_on_sentence()).cwd (string, optional) – The folder to run the program in.
- Returns:
out (string) – The standard output and the standard error of the program.
- Raises:
DeviceFailure – If the program cannot be started, takes longer than
PREFLIGHT_TIMEOUTseconds, or ends with a nonzero exit status.
- pyretis.testing.devices.select_device(engine, preflight_dir, environ=None)¶
Select the device of the CUDA passes of an engine.
- Parameters:
engine (string) – The engine, a key of
ENGINE_TITLES.preflight_dir (string) – The folder of the preflight input of the engine (GROMACS and LAMMPS).
environ (dict, optional) – The environment;
os.environwhen None.
- Returns:
out (object like
Selection) – The CUDA build and its GPU, or why the CUDA passes do not run:PYRETIS_TEST_DEVICE=cpu,CUDA not available: <why>(cuda_availability(), checked first), orno CUDA build: <what was not found>.- Raises:
ValueError – If
PYRETIS_TEST_DEVICEholds a value that is not one ofDEVICE_MODES.DeviceFailure – If CUDA is available and the CUDA build is broken or fails its preflight; or, when
PYRETIS_TEST_DEVICEiscuda, if the engine has no CUDA build, or if CUDA is not available (CudaUnavailable).
- pyretis.testing.devices.selection_shell(selection)¶
Return the shell assignments that hold a selection.
- Parameters:
selection (object like
Selection) – The selection.- Returns:
out (string) – One
NAME=valueline, quoted for the shell, for each ofDEVICE_CUDA(1or0),DEVICE_REASON,DEVICE_COMMAND,DEVICE_OPTIONS(the options that run the program on the GPU,GROMACS_GPU_OPTIONSorLAMMPS_GPU_OPTIONS; empty for OpenMM),DEVICE_VERSION,DEVICE_BUILD,DEVICE_GPUandDEVICE_DRIVER.
- pyretis.testing.devices.stage_openmm_cuda(module, cuda_module, settings, cuda_settings)¶
Write the module and the input of the CUDA pass of an OpenMM case.
The module of an OpenMM case names its platform in the lines of
OPENMM_PLATFORM_LINES(the Reference platform, without properties); the CUDA pass runs a copy that names the CUDA platform withOPENMM_CUDA_PROPERTIES, from a copy of the input whoseopenmm_modulenames that copy.- Parameters:
module (string) – The module of the case.
cuda_module (string) – The module to write, in the folder of
module.settings (string) – The input (
.toml) of the case, with the lineopenmm_module = "<name of module>".cuda_settings (string) – The input to write.
- Raises:
ValueError – If a line to replace is not in its file exactly once, or
cuda_moduleis not in the folder ofmodule.
- pyretis.testing.devices.test_device_mode(environ=None)¶
Return the value of
PYRETIS_TEST_DEVICE.- Parameters:
environ (dict, optional) – The environment;
os.environwhen None.- Returns:
out (string) – One of
DEVICE_MODES;autowhen the variable is not set or empty.- Raises:
ValueError – If the variable holds another value.
- pyretis.testing.devices.watch_mdrun(command, log, cap, case)¶
Run a program until one of its MD runs of mdrun has ended.
The program (a
pyretis runof GROMACS on the GPU, with mdrun given-g <log>) runs in a session of its own. When the log of an mdrun run that integrated MD steps and ended appears atlog, or as its backup (_finished_md_logs()), the program and its children are stopped, and the log must map the particle-particle tasks of the run to a GPU (gromacs_log_gpu()). A signal ofWATCH_STOP_SIGNALSends the watch with the status128 + <signal>(SystemExit); on that, onSIGINTand on an error the program and its children are stopped as well.- Parameters:
command (list of strings) – The program and its arguments.
log (string) – The log path given to mdrun.
cap (float) – The longest time, in seconds, to wait for an mdrun run to end.
case (string) – The name of the run, for the device line.
- Returns:
out (int) – 0 when an MD run of mdrun ended on a GPU; 1 when the program ended or the cap passed before an MD run ended, or the log of the run names no GPU; the exit status of the program when it failed first.
- pyretis.testing.devices.way_on_sentence(kind, *clauses)¶
Return the way on of a failure: its clauses and the CPU passes.
- Parameters:
kind (string) –
CUDA BROKENorCUDA REQUIRED.clauses (strings) – What to set or fix to give the engine a CUDA build that runs, each one an alternative.
- Returns:
out (string) – One sentence: the clauses and the value of
PYRETIS_TEST_DEVICEthat runs the CPU passes,autoorcpuforCUDA REQUIRED(autoruns the CPU passes of an engine without a CUDA build, and of every engine when CUDA is not available),cpuforCUDA BROKEN(CUDA is available, andautofails on a broken build ascudadoes).
pyretis.testing.example_compare module¶
Shared result-comparison for the example run.sh scripts.
One implementation of the checks against a folder’s committed
results/ reference, shared by every example under
examples/tests/. A folder’s run.sh reaches it as python -m
pyretis.testing.example_compare compare <config.toml> through its
-C/-cc flag.
The checks run for a given config are selected from its simulation
task:
golden – every committed per-ensemble reference file (
pathensemble.txtalways;moves.txt/order.txt/energy.txt/traj.txtwhen the folder commits them) is compared against the run’s matching file with the file-type-appropriate primitive (compare_ensemble_files_golden()). Always run; a folder selects which files to check simply by which it commits underresults/.--energy-skip/--path-skip/--rel-toltune this for a folder whose reference legitimately differs in a named term (e.g. GROMACS’s restart-dependent long-range dispersion correction) or needs a looser tolerance (e.g. a build-sensitive external engine).archive – the per-trial trajectory archive (
<ensemble>/paths/<pn>/plus the per-ensemble long-term<ensemble>/archive/<pn>/) is byte-compared (tolerating--energy-skipthe same way as the golden check;compare_archive_check()). Opt-in (not part of the per-task default): the two-run restart/load/two-engine equivalence checks that need it request it explicitly via--checks.paths – every accepted path is validated (start/middle/end interface labels, length, crossings;
check_accepted_paths()). Run for the path-sampling tasks.relaxed-paths – like paths, but the
middle/crossing invariants are relaxed for sparse-loaded (ld-move) initial paths (check_accepted_paths_relaxed()).zero_leftis read from the config automatically, so a[0^-]path may legitimately end'L'under that geometry without being flagged.swaps – the accepted zero swaps (
s+in[0^-],s-in[0^+]) are checked for order-parameter continuity with the paths the two ensembles held when the swap started (check_retis_swaps()). Opt-in (not a per-task default): the scheduler of a RETIS run folds the swaps between the other ensembles into weighted bookkeeping, so the zero swap is its only swap move, and a run need not make one; the check fails if it examines none, and for a run with more than one worker.
--run-dir/--ref-dir need not point at a committed results/
golden – pointing --ref-dir at a second, independently-run directory
of the same config turns the golden/archive checks into a two-run
equivalence check (e.g. a restart or two-engine comparison that has no
committed reference of its own).
A few folders instead check that a restarted (stopped-and-resumed) run
reproduces the uninterrupted one, with no committed golden. Their
run.sh selects a dedicated subcommand: restart-whole (the restart
already holds the whole history; compare_restart_whole()),
restart-concat (part 1 + part 2 must equal the full run;
compare_restart_concat()) and restart-md (the same, with a
trajectory compared by per-snapshot MSE; compare_restart_md()).
Exit status is the number of failing checks (0 == success), so it drops
straight into a run.sh && chain.
- pyretis.testing.example_compare.MIN_ATTEMPTS_FOR_ACCEPTANCE_CLAUSE = 10¶
How many attempted moves an ensemble needs before the
zero_leftacceptance clause can conclude anything. Below this the clause abstains with a warning: a verdict drawn from a handful of attempts would be scatter dressed up as a contract violation.
- pyretis.testing.example_compare._ALL_CHECKS = ('golden', 'paths', 'relaxed-paths', 'swaps', 'structural', 'reports', 'archive')¶
The check names a folder may request (see
compare_run()).
- pyretis.testing.example_compare._GOLDEN_COMPARATORS = {'energy.txt': <function compare_energy_data>, 'moves.txt': <function compare_moves_data>, 'order.txt': <function compare_order_data>, 'pathensemble.txt': <function compare_path_ensemble_data>, 'traj.txt': <function compare_simulation_files>}¶
Map a committed per-ensemble golden file to the primitive that compares it. The comparison a folder wants is inferred from which of these files it actually commits under
results/– seecompare_ensemble_files_golden(). Insertion order fixes the report order (path ensemble first, then the numerical trajectories).
- pyretis.testing.example_compare._MISSING = <object object>¶
Unique sentinel used as the
zip_longestfill value when pairing two iterators of trajectory blocks/snapshots, so a length mismatch on either side is detected (a real snapshot is never this object).
- pyretis.testing.example_compare._PATH_SAMPLING_TASKS = frozenset({'explore', 'pptis', 'repptis', 'retis', 'tis'})¶
Tasks that sample paths (so the accepted-path validity check applies).
- pyretis.testing.example_compare._ROW_PATH_NUMBER = 17¶
The column of the path number in a
pathensemble.txtrow.
- pyretis.testing.example_compare._STRUCTURAL_PROFILES = {'explore': {'has_zero_minus': False, 'min_length': 3, 'require_finite_lockstep': True, 'require_maxo_ge_mino': True}, 'pptis': {'has_zero_minus': True, 'min_length': 2, 'require_finite_lockstep': False, 'require_maxo_ge_mino': False}, 'repptis': {'has_zero_minus': True, 'min_length': 2, 'require_finite_lockstep': False, 'require_maxo_ge_mino': False}}¶
Structural-check profiles for the golden-less path-sampling folders, keyed by task.
exploreproducesN - 1positive ensembles and no[0^-];repptis/pptiskeep[0^-]. The accepted-length threshold and the extra order-parameter checks also differ.
- pyretis.testing.example_compare._TRAJ_READER_SPECS = {<class 'pyretis.inout.formats.path.PathIntFile'>: {'nested': True, 'snapshot_diff': <function _snapshot_diff_pathint>}, <class 'pyretis.inout.formats.snapshot.SnapshotFile'>: {'nested': False, 'snapshot_diff': <function _snapshot_diff_snapshot>}}¶
Per-reader trajectory-MSE behaviour.
PathIntFileyields path blocks (nested; each block carries adatalist of snapshots) and is used for the whole-file restart;SnapshotFileyields snapshots directly (flat) and is used for the concatenated restart. The matching per-snapshot diff function reflects the file format’s snapshot keys.
- pyretis.testing.example_compare._accepted_zero_swaps(run_dir, ens_number, move)¶
Return the accepted zero swaps of
moves.txtof an ensemble.- Parameters:
run_dir (str) – The directory the simulation ran in.
ens_number (int) – The number of the ensemble’s directory.
move (str) –
s+for[0^-],s-for[0^+].
- Returns:
dict – The number of the path each accepted swap produced, by cycle.
- pyretis.testing.example_compare._archive_subdir_name(base_dir, ens_dir)¶
Return the ensemble’s operational-archive subdirectory name.
A directory holding a run file,
output.toml, stores the paths of each ensemble in<ensemble>/<load_dir>, named by the[simulation] load_dirthe file records (see_recorded_load_dir()), and the store is looked for under that name only. In a directory without it, such as a committed reference, the store is the first of the default (ARCHIVE_SUBDIR,accepted) and the alternativepaths/trajsspellings that is present, so a run can be compared against a reference recorded under any of them.Nonewhen the ensemble has no archive.
- pyretis.testing.example_compare._assess_mse(error, what, tol)¶
Log and grade one averaged error term against a tolerance.
- Parameters:
error (float) – The mean per-snapshot error term.
what (str) – A label for the term (e.g.
positions) used in the log message.tol (float) – The tolerance the error must stay below.
- Returns:
int – 0 when
abs(error) < tol, else 1.
- pyretis.testing.example_compare._chained_restart_snaps(part_files, reader)¶
Chain the restart parts’ snapshots, dropping each overlap frame.
- Parameters:
part_files (list of str) – The restart parts, in order.
reader (callable) – The trajectory reader class (
SnapshotFile).
- Returns:
iterator – The chained snapshots, with every part after the first missing its leading frame (which repeats the previous part’s last frame).
- pyretis.testing.example_compare._check_zero_swap(cycle, new_paths, old_paths, order)¶
Check the order parameters of one accepted zero swap.
- Parameters:
cycle (int) – The cycle of the swap.
new_paths (tuple of int) – The new
[0^-]and the new[0^+]path.old_paths (tuple of (set of int or None)) – The paths
[0^-]and[0^+]held when the swap started (_held_before()).order (dict) – The order parameters of the paths (
_path_order_data()).
- Returns:
failures (int) – The number of failing comparisons, or 1 when the swap cannot be examined.
examined (bool) – True when the order parameters of the swap were compared.
- pyretis.testing.example_compare._configure_console_logging()¶
Route this module’s log records to the console.
Every check reports what it compared and, on a mismatch, why (which ensemble, which file, which line) through
logger. The module attaches only alogging.NullHandlerat import time, so importing it as a library stays silent. Attach a single stdoutlogging.StreamHandleratINFOso the CLI entry point (main()) prints what it checked and why it failed. Idempotent: a second call does not stack a duplicate handler.
- pyretis.testing.example_compare._count_attempted_moves(pathensemble_file)¶
Count every move attempted in an ensemble, whatever its outcome.
_parse_ensemble_status_counts()tracks only the six statuses the contract clauses name, so summing it under-counts: a run whose rejections are allBWIsums to zero attempts, which would be a fabricated number of exactly the kind this suite exists to catch.- Parameters:
pathensemble_file (str) – Path to an ensemble’s
pathensemble.txt. Its directory is used to find themoves.txtbeside it.- Returns:
int – The number of attempted moves.
- pyretis.testing.example_compare._cycle_index(traj)¶
Extract the cycle index from an order.txt block comment.
- pyretis.testing.example_compare._default_checks(task)¶
Return the checks a task runs when a folder does not name its own.
- Parameters:
task (str) – The (lower-cased) simulation task.
- Returns:
list of str – The default check names:
goldenalways andpathsfor the path-sampling tasks. Swap validation must be requested explicitly because infinite swapping does not emit discrete swap moves.
- pyretis.testing.example_compare._generated_report_path(run_dir, name)¶
Resolve a committed report name to the run’s generated report file.
Reports are written under
<run_dir>/report/and may carry a_cycles-NNNsuffix (the cycle count at write time). Given the plain reference name, return the matching generated path, preferring an exact match and otherwise the highest-numbered_cycles-variant.- Parameters:
run_dir (str) – The directory the simulation ran in.
name (str) – The reference report file name (e.g.
foo_report.html).
- Returns:
str – The path to the generated report file.
- pyretis.testing.example_compare._golden_ensemble_dirs(ref_dir)¶
Return the ensemble subdirectories that carry golden files.
The reference directory holds one zero-padded, three-digit subdirectory per compared ensemble (
000,001…). Some folders start at001(e.g. themake-tis-filesexamples), so the set is discovered fromref_dir.- Parameters:
ref_dir (str) – The committed reference directory (
results/).- Returns:
list of str – The ensemble directory names present, sorted ascending.
- pyretis.testing.example_compare._held_before(held, initial, cycle)¶
Return the paths an ensemble held when a move of a cycle started.
- Parameters:
held (dict) – The paths the ensemble held in each cycle (
_held_paths()).initial (set of int) – The initial paths that the ensemble may hold before cycle 1.
cycle (int) – The cycle of the move.
- Returns:
set of int or None – The paths of the cycle before, the initial paths for cycle 1, or None when the output records neither.
- pyretis.testing.example_compare._held_paths(run_dir, ens_number)¶
Return the paths an ensemble held in each cycle.
- Parameters:
run_dir (str) – The directory the simulation ran in.
ens_number (int) – The number of the ensemble’s directory.
- Returns:
dict – The path numbers of the rows of
pathensemble.txt, by cycle.- Raises:
ValueError – If a row has no path number.
- pyretis.testing.example_compare._is_minus_window(path_ensemble)¶
Say whether a path ensemble is the
[0^-]of its layout.A path ensemble is named by its role in the layout its number counts in (
pyretis.core.pathensemble.ensemble_label()). In apptislayout without a minus window, ensemble 0 is a body window or the[0^+].- Parameters:
path_ensemble (object like
PathEnsemble) – The path ensemble of an ensemble the accepted-path checks examine.- Returns:
out (boolean) – True for the
[0^-].
- pyretis.testing.example_compare._parse_ensemble_energy_terms(specifications, parser)¶
Parse repeated
ENSEMBLE:TERM,TERMCLI specifications.
- pyretis.testing.example_compare._parse_ensemble_status_counts(pathensemble_file)¶
Count the per-status attempted moves of an ensemble.
The counts must come from the ATTEMPTED moves, which live in
moves.txt.pathensemble.txtrecords which path the ensemble HELD, and a held path is accepted by construction – counting its rows would report zero rejections of every kind and pass this check vacuously, which is worse than failing it.The output of an earlier PyRETIS holds both row kinds in
pathensemble.txt, told apart by a trailingTrialcolumn; such a file is counted from its trial rows.- Parameters:
pathensemble_file (str) – Path to an ensemble’s
pathensemble.txt. Its directory is used to find themoves.txtbeside it.- Returns:
dict – A count for each of
ACC/FTX/BTX/FTL/BTL/0-L.
- pyretis.testing.example_compare._path_order_data(run_dir, ensembles)¶
Return the order parameters of the paths a run keeps on disk.
- Parameters:
run_dir (str) – The directory the simulation ran in.
ensembles (list) – The ensemble dictionaries from
create_ensembles.
- Returns:
order (dict) – The order parameters of each path (a row per frame, the time column left out), by path number, from the
order.txtof its folder (<ensemble>/accepted/<pn>or<ensemble>/archive/<pn>).initial (tuple of (set of int)) – The paths that hold a cycle-0 block (the initial paths): those of the first ensemble of
ensembles([0^-]), and those of the others.
- Raises:
ValueError – If two folders hold the same path number with other order parameters.
- pyretis.testing.example_compare._pathensemble_rows(pathensemble_file)¶
Return the split occupancy rows of a
pathensemble.txt.Rows whose
Trialcolumn is set record an ATTEMPTED move rather than a path the ensemble held. They have no path-data block, so counting them here would break the order.txt lockstep check.
- pyretis.testing.example_compare._read_pathensemble_rows(run_dir, ens_number)¶
Yield the occupancy rows of an ensemble’s
pathensemble.txt.Rows whose trailing
Trialcolumn is set record an ATTEMPTED move rather than a path the ensemble held. A rejected trial never became a path, so it has no trajectory to check and no interface flags to be consistent about; the checks here are about sampled paths.
- pyretis.testing.example_compare._recorded_load_dir(base_dir)¶
Return the
[simulation] load_dira run directory records.- Parameters:
base_dir (str) – A run directory, or a reference directory.
- Returns:
str or None – The
[simulation] load_dirof<base_dir>/output.toml, the run file the scheduler writes in the run directory. None whenbase_dirholds nooutput.toml, or the file sets noload_dir.
- pyretis.testing.example_compare._report_file_pair(name, equal, msg)¶
Log a single file comparison result and return its failure count.
- Parameters:
name (str) – The file name being compared (for the log message).
equal (bool) – Whether the comparison judged the files equal.
msg (str) – The descriptive message returned by the comparison primitive.
- Returns:
int – 0 when
equalis True, else 1.
- pyretis.testing.example_compare._resolved_ref_dir(ref_dir)¶
Yield a directory-shaped reference, extracting a tarball first.
Every check that reads
ref_direxpects a plain directory laid out as<ensemble>/<file>(a committedresults/, or a second live run directory). Some folders instead commit a single gzipped tarball with that same layout stored under aresults/prefix inside the archive – e.g. LAMMPS, where keeping the reference to one small binary-diffable file is preferable to a committed tree (generate_golden_tarball()writes it). Transparently extract it to a temporary directory so every check downstream stays unaware of which form the reference took.- Parameters:
ref_dir (str) – Either a plain reference directory, or the path to a
.tgz/.tar.gzarchive holding the same layout under aresults/prefix.- Yields:
str – A directory laid out as
<ensemble>/<file>–ref_diritself, or the archive’s extractedresults/subdirectory.
- pyretis.testing.example_compare._run_compare_subcommand(args, parser)¶
Handle the
comparesubcommand (single run vs golden/structural).
- pyretis.testing.example_compare._snapshot_diff_pathint(snap1, snap2)¶
Return the summed squared position/velocity differences of a snapshot.
For snapshots loaded by
pyretis.inout.formats.path.PathIntFile, whose positions and velocities are(n_atoms, dim)arrays under thepos/velkeys.- Parameters:
snap1 (dict) – The first snapshot (keys
posandvel).snap2 (dict) – The second snapshot, compared against
snap1.
- Returns:
diff_pos (float) – The summed squared position difference over all atoms.
diff_vel (float) – The summed squared velocity difference over all atoms.
- pyretis.testing.example_compare._snapshot_diff_snapshot(snap1, snap2)¶
Return the summed squared position/velocity differences of a snapshot.
For snapshots loaded by
pyretis.inout.formats.snapshot.SnapshotFile, whose coordinates live in the per-axisx/y/zandvx/vy/vzkeys.- Parameters:
snap1 (dict) – The first snapshot (keys
x/y/zandvx/vy/vz).snap2 (dict) – The second snapshot, compared against
snap1.
- Returns:
diff_pos (float) – The summed squared position difference over all atoms.
diff_vel (float) – The summed squared velocity difference over all atoms.
- pyretis.testing.example_compare._store_is_empty(directory)¶
Return True if
directoryexists but contains no files at all.
- pyretis.testing.example_compare._struct_check_layout(run_dir, ens_dirs, zero_dir_expected)¶
Structural check 1: the expected ensemble dirs and files exist.
zero_dir_expectedis whether a000directory belongs to this layout at all. It is not the same question as whether the layout has a minus window: a PPTIS run without one still numbers its windows from zero, so000holds a body ensemble there. Only explore numbers its positive ensembles from001.
- pyretis.testing.example_compare._struct_check_lockstep(run_dir, ens_dirs, require_finite)¶
Structural check 3: one order.txt block per pathensemble.txt row.
- pyretis.testing.example_compare._struct_check_moves(run_dir, ens_dirs)¶
Structural check 4: real shooting/swap moves (not only loaded).
- pyretis.testing.example_compare._struct_check_paths(run_dir, ens_dirs, min_length, require_maxo_ge_mino)¶
Structural check 2: every accepted path is a valid sampling path.
- pyretis.testing.example_compare._structural_ensemble_dirs(n_interfaces, has_zero_minus)¶
Return the ensemble directory names for a golden-less run.
- Parameters:
n_interfaces (int) – The number of interfaces in the config.
has_zero_minus (bool) – Whether the topology includes the
[0^-](000) ensemble.
- Returns:
list of str – Ensemble directory names –
000 .. 0(N-1)when[0^-]is present, else001 .. 0(N-1)(the explore positive ensembles).
- pyretis.testing.example_compare._traj_mse_flat(part_files, full_file, reader, snapshot_diff)¶
Accumulate per-snapshot squared errors for a concatenated restart.
The restart is split into
part_fileswritten to separate files. They are chained (dropping each overlapping restart frame; see_chained_restart_snaps()) and compared snapshot-for-snapshot against the full run. The residual flag reports a differing total snapshot count.- Parameters:
part_files (list of str) – The restart parts, in order.
full_file (str) – The uninterrupted run’s trajectory file.
reader (callable) – The trajectory reader class (
SnapshotFile).snapshot_diff (callable) – The per-snapshot difference function.
- Returns:
error_pos (list of float) – The per-snapshot squared position errors.
error_vel (list of float) – The per-snapshot squared velocity errors.
residual (bool) – True if the chained restart and the full run differ in length.
- pyretis.testing.example_compare._traj_mse_nested(part_files, full_file, reader, snapshot_diff)¶
Accumulate per-snapshot squared errors for a whole-file restart.
The restart already holds the whole history, so it is compared block-for-block (each block a path) and, within a block, snapshot-for-snapshot against the full run. The residual flag reports a differing number of path blocks (the trajectory-count check).
- Parameters:
part_files (list of str) – The restart trajectory file(s); the whole-file restart passes one.
full_file (str) – The uninterrupted run’s trajectory file.
reader (callable) – The trajectory reader class (
PathIntFile).snapshot_diff (callable) – The per-snapshot difference function.
- Returns:
error_pos (list of float) – The per-snapshot squared position errors.
error_vel (list of float) – The per-snapshot squared velocity errors.
residual (bool) – True if the two runs hold a different number of path blocks.
- pyretis.testing.example_compare.check_accepted_paths(run_dir, ensembles)¶
Validate every accepted path in every ensemble.
Each accepted path must have length >= 3, a start/middle/end interface label consistent with the ensemble, and cross its two bounding interfaces.
- Parameters:
run_dir (str) – The directory the simulation ran in.
ensembles (list) – The ensemble dictionaries from
pyretis.setup.createsimulation.create_ensembles().
- Returns:
int – The number of ensembles with a suspicious accepted path.
- pyretis.testing.example_compare.check_accepted_paths_relaxed(run_dir, ensembles, zero_left=False)¶
Validate accepted paths with the relaxed (sparse-load) rules.
Like
check_accepted_paths(), but themiddle == 'M'marker and the interface-crossing invariants are relaxed for the sparse-LOADED initial paths ONLY – the rows whose move isld. A short loaded[0^-]minus path need not reach the middle interface, so those stricter invariants do not apply to it; every genuinely sampled path (any other move) must still satisfy them, exactly as in the strict check. Length, start label and end label are always enforced.- Parameters:
run_dir (str) – The directory the simulation ran in.
ensembles (list) – The ensemble dictionaries from
create_ensembles.zero_left (bool, optional) – Whether the config sets
[simulation] zero_left. Underzero_leftan accepted[0^-](ensemble 0) path can end'L'(not only'R'): the[0^-']of a permeability run holds paths that end at either interface, and the kick initiation starts the[0^-]of a rate run from a path that starts on the right, whichever end it reaches. Without this flag such a path is indistinguishable from a genuine wrong-end bug and would be flagged as one.
- Returns:
int – The number of ensembles with a suspicious accepted path.
- pyretis.testing.example_compare.check_retis_swaps(run_dir, ensembles, workers=1)¶
Check the order parameters of the accepted zero swaps of a run.
The zero swap is the one swap move of the scheduler: from the path
[0^-]holds and a path[0^+]holds it makes a new path for each (pyretis.core.moves.retis_swap_zero()). The new[0^-]path (s+in itsmoves.txt) ends with the first two frames of the[0^+]path, and the new[0^+]path (s-) starts with the last two frames of the[0^-]path. For each accepted swap the check finds the paths the two ensembles held when it started (theirpathensemble.txtrows of the cycle before, or the initial paths for cycle 1) and compares those frames: the[0^-]path is the one[0^-]held, the[0^+]path one of those[0^+]held, and their order parameters must equal the ones of the new paths exactly. The order parameters of every path are read from its folder (_path_order_data()).- Parameters:
run_dir (str) – The directory the simulation ran in.
ensembles (list) – The ensemble dictionaries from
create_ensembles; the first two are[0^-]and[0^+].workers (int, optional) – The workers of the run. With more than one, a move can start before the cycle before it ends, so the paths held when it started are not recorded and the check fails.
- Returns:
int – The number of failing swaps, plus one when the check examined no accepted swap or cannot be made.
- pyretis.testing.example_compare.check_structural_suite(run_dir, n_interfaces, task, settings=None)¶
Run the structural validity checks for a golden-less run.
The suite (
--checks structural) checks the validity of a run for which noresults/reference is committed: the expected per-ensemble output exists, every accepted path obeys the (task-specific) length and order-parameter rules,order.txtis in lockstep withpathensemble.txt, and real (non-loaded) moves were committed.- Parameters:
run_dir (str) – The directory the simulation ran in.
n_interfaces (int) – The number of interfaces in the config.
task (str) – The (lower-cased) simulation task; selects the profile from
_STRUCTURAL_PROFILES.settings (dict, optional) – The parsed input settings. A
pptisrun builds its minus window and its[0^+]only whenzero_left/fluxandzero_ensembleask for them, so the expected ensemble directories cannot be derived fromn_interfacesalone.
- Returns:
int – The number of failing structural checks (0 == valid run).
- pyretis.testing.example_compare.check_zero_left_contract(nozero_dir, withzero_dir, acc_ratio_max=3.0)¶
Check the two-run behavioural contract of the
zero_leftfeature.no-zeroandwith-zerorun the same short RETIS simulation and differ only in the[0^-]configuration.zero_leftmay reject paths with0-Lin[0^-](and propagate one to[0^+]via a failed swap-zero), but must never (a) introduce0-Lin the inner ensembles[1^+],[2^+]…, nor (b) introduce maxlength (FTX/BTX) rejections in[0^-]. The two runs’ accepted-path counts should also stay within a generous factor.- Parameters:
nozero_dir (str) – The directory holding the plain (no
zero_left) run’s ensembles.withzero_dir (str) – The directory holding the
zero_leftrun’s ensembles.acc_ratio_max (float, optional) – The largest tolerated ratio between the two runs’ accepted-path counts in an ensemble (default 3.0; the run is short, fixed-seed).
- Returns:
int – The number of violated contract clauses (0 == contract holds).
- pyretis.testing.example_compare.compare_archive_check(run_dir, ref_dir, energy_skip=None, frames='byte')¶
Compare the two-tier trajectory stores of two runs.
For each ensemble directory present under
ref_dir, byte-compares the per-ensemble OPERATIONAL archive (<ensemble>/<load_dir>/<pn>/, written bypyretis.inout.scheduler_archive. SchedulerPathStorage; each side’s name is found by_archive_subdir_name()) between the two runs, and likewise the per-ensemble LONG-TERM store (<ensemble>/archive/<pn>/, the replaced paths) when either side has one – viacompare_traj_archive_soft()in both cases. This check is independent of thegoldencheck (which compares only the golden files at the top of each ensemble directory, such aspathensemble.txt): a restart or continuation can reproduce those exactly while still silently diverging in the raw archived frames, which is exactly what this check exists to catch.- Parameters:
run_dir (str) – The directory the simulation ran in.
ref_dir (str) – The committed reference directory, or a second live run directory.
energy_skip (list of str, optional) – Named energy terms to exclude from an archived
energy.txtfragment’s comparison (seecompare_traj_archive_soft()).frames (str, optional) – How the engine frame payloads are compared,
'byte'(default) or'names'(seecompare_traj_archive_soft()).
- Returns:
int – The number of stores that differ (0 == all equal). An ensemble whose archive exists on only ONE side is a loud failure; one absent on BOTH sides is skipped (no path was born there on either run) – but a check that ends up comparing nothing at all fails loudly.
- pyretis.testing.example_compare.compare_ensemble_files_golden(run_dir, ref_dir, energy_skip=None, energy_unavailable=None, energy_unavailable_by_ensemble=None, path_skip=None, rel_tol=1e-05, abs_tol=0.0)¶
Compare every committed per-ensemble golden file against the run.
For each ensemble directory present under
ref_dir, every golden file it commits (pathensemble.txt/moves.txt/order.txt/energy.txt/traj.txt) is compared against the matching file the run produced, using the file-type-appropriate primitive. A folder therefore selects its golden comparison simply by which files it commits – a RETIS folder can commit justpathensemble.txtandmoves.txt; a permeability or sparse-load folder also commitsorder.txt(andenergy.txt); a committedtraj.txtis compared byte-for-byte.- Parameters:
run_dir (str) – The directory the simulation ran in (holds
000/,001/…).ref_dir (str) – The committed reference directory (
results/– or a second live run directory, e.g. for a restart/continuation-equivalence check that has no committed golden of its own).energy_skip (list of str, optional) – Named energy terms (as read by
EnergyPathFile, e.g.'vpot') to exclude fromenergy.txt. When given, the comparison switches from the default NaN-tolerantcompare_energy_data()to a by-column comparison that additionally excludes these named terms – for a case where a term is legitimately expected to differ (e.g. GROMACS applying the long-range dispersion correction differently across a restart), not merely absent (all-NaN).energy_unavailable (list of str, optional) – Named energy terms expected to be entirely NaN in every ensemble. The comparator verifies that they remain unavailable on both sides.
energy_unavailable_by_ensemble (dict, optional) – Per-ensemble unavailable terms, keyed by the three-digit ensemble directory name. This preserves finite coverage in other ensembles.
path_skip (list of int, optional) – Column indices to exclude from
pathensemble.txt.rel_tol (float, optional) – Relative tolerance forwarded to every numerical comparator (default matches each comparator’s own 1e-5 default).
abs_tol (float, optional) – Absolute tolerance forwarded to the comparators of
pathensemble.txt,moves.txt,order.txtandenergy.txt(default 0, each comparator’s own default). Anenergy.txtcompared withenergy_skiptakes neither: it compares with the default tolerances ofnumpy.allclose()(rtol1e-5,atol1e-8), and atraj.txtcompares byte for byte. Where both tolerances apply, they add up; in every file the integer columns, the step of each frame among them, compare exactly.
- Returns:
int – The number of mismatching golden files (0 == all equal). A missing or empty reference, or a reference that holds none of the recognized golden files, is a loud failure (>= 1): a comparison that inspected nothing has validated nothing.
- pyretis.testing.example_compare.compare_reports_golden(run_dir, ref_dir)¶
Compare the committed top-level report / generated-config goldens.
For every non-directory file committed at the top of
ref_dir:*.toml(a generated per-interface config) is compared line by line ignoring the run-dependentexe_path;*.html/*.rst/*.tex(a report) is compared with version/timestamp normalisation, resolving the generated report’s_cycles-NNNsuffix.- Parameters:
run_dir (str) – The directory the simulation ran in.
ref_dir (str) – The committed reference directory (
results/).
- Returns:
int – The number of mismatching golden files (0 == all equal). A committed top-level reference file with no comparator (an unexpected extension) is a loud failure, and a
reportscheck that compared nothing (no report/config golden underref_dir) also fails.
- pyretis.testing.example_compare.compare_restart_concat(part1_dir, part2_dir, full_dir)¶
Compare a two-part (concatenated) restart against the full run.
Part 1 plus part 2 (dropping the single overlapping record) must equal the full run for each output file. The concatenation-aware primitives from
pyretis.testing.simulation_comparisonhandle the overlap internally:cross.txt–compare_restarted_cross_files();energy.txtandorder.txt–compare_restarted_text_files().
- Parameters:
part1_dir (str) – The first part’s run directory.
part2_dir (str) – The second (restarted) part’s run directory.
full_dir (str) – The uninterrupted (full) run directory.
- Returns:
int – The total number of failing checks (0 == the restart reproduces the full run).
- pyretis.testing.example_compare.compare_restart_md(traj_files, text_triples, tol=1e-12)¶
Compare a concatenated MD restart against the full run.
The MD restart is split into two parts written with per-run file-name prefixes. The trajectory is compared by per-snapshot MSE (
compare_restart_traj_mse()withSnapshotFile, which drops the overlapping restart frame); the remaining text files (thermo, energy) are compared withcompare_restarted_text_files().- Parameters:
traj_files (list of str) – The trajectory files
[part1, ..., full]; the last is the full run, the rest are the ordered restart parts.text_triples (list of (list or tuple) of str) – Each entry is a
(part1, part2, full)triple of text-file paths (e.g. the thermo and energy files) checked with the concatenation-aware text comparison.tol (float, optional) – The trajectory MSE tolerance (1e-12).
- Returns:
int – The total number of failing checks (0 == the restart reproduces the full run).
- pyretis.testing.example_compare.compare_restart_traj_mse(part_files, full_file, reader, tol=1e-12)¶
Compare a restart trajectory against the full run by snapshot MSE.
The restart and full trajectories are read snapshot-by-snapshot with
readerand their per-snapshot squared position/velocity differences are averaged; each average must stay belowtol. The two must also hold the same number of snapshots (or path blocks, for the whole-file case). The reader selects the whole-file vs concatenated behaviour and the matching snapshot-difference function (see_TRAJ_READER_SPECS):PathIntFilereads a whole-file restart block-for-block,SnapshotFilechains the concatenated parts (dropping each later part’s overlapping first frame).- Parameters:
part_files (list of str) – The restart trajectory file(s). A whole-file restart passes a single file; a concatenated restart passes the ordered parts.
full_file (str) – The uninterrupted (full) run’s trajectory file.
reader (callable) – The trajectory reader class (
PathIntFileorSnapshotFile).tol (float, optional) – The tolerance each averaged error term must stay below (1e-12).
- Returns:
int – The number of failing checks: a length mismatch counts as 1, and each out-of-tolerance term (positions, velocities) adds 1.
- pyretis.testing.example_compare.compare_restart_whole(config_path, restart_dir, full_dir, tol=1e-12)¶
Compare a whole-file restart against the uninterrupted run.
The restart leg appends its steps onto the existing history, so its per-ensemble output already holds the WHOLE trajectory; it is compared file-for-file against the full run for the single ensemble named by the config’s
[tis] ensemble_number:the two-tier trajectory stores (
<ens>/paths/<pn>+ the per-ensemblearchive/) – byte-compared viacompare_archive_check(): every sampled path’s full trajectory, not just the initial one.energy.txt– column comparison skipping theetot/tempterms the internal engine leaves entirely NaN (an all-NaN term tests nothing;ekin/vpotare still compared);order.txt– numerical comparison.
- Parameters:
config_path (str) – The run config (
.toml); its[tis] ensemble_numbernames the single ensemble to compare.restart_dir (str) – The restarted run directory (holds the whole history).
full_dir (str) – The uninterrupted (full) run directory.
tol (float, optional) – Unused; the archives are byte-compared.
- Returns:
int – The total number of failing checks (0 == the restart reproduces the full run).
- pyretis.testing.example_compare.compare_run(config_path, run_dir=None, ref_dir=None, checks=None, energy_skip=None, energy_unavailable=None, energy_unavailable_by_ensemble=None, path_skip=None, rel_tol=1e-05, archive_frames='byte', abs_tol=0.0)¶
Run the reference comparison for a completed example simulation.
- Parameters:
config_path (str) – The simulation config (
.toml) the example ran with.run_dir (str, optional) – Where the run produced its per-ensemble output (default: the config’s directory).
ref_dir (str, optional) – The committed reference directory (default:
results/next to the config). This may instead be a second, independently-run directory of the SAME config – e.g. a restart/load/two-engine equivalence check that has no committed golden of its own; the golden and archive checks only require a matching per-ensemble layout, not thatref_dirbe a committed reference.checks (list of str, optional) – The checks to run (any of
_ALL_CHECKS). WhenNonethe set is inferred from the task (_default_checks()). A folder overrides this when its faithful verdict is a subset – e.g. the permeability example istask = retisbut compares only the golden files (itszero_leftgeometry does not satisfy the generic accepted-path invariants), so it passeschecks=['golden'].energy_skip (list of str, optional) – Named energy terms to exclude from
energy.txt(golden check) and any archivedenergy.txtfragment (archive check) – seecompare_ensemble_files_golden()/compare_archive_check().energy_unavailable (list of str, optional) – Terms expected to be all-NaN in every golden
energy.txtfile.energy_unavailable_by_ensemble (dict, optional) – Per-ensemble expected unavailable terms for the golden check.
path_skip (list of int, optional) – Column indices to exclude from
pathensemble.txt(golden check only).rel_tol (float, optional) – Relative tolerance forwarded to the golden check’s numerical comparators (default 1e-5, matching each comparator’s own default).
archive_frames (str, optional) – The
framesmode of the archive check,'byte'(default) or'names'(seecompare_archive_check()).abs_tol (float, optional) – Absolute tolerance forwarded to the golden check’s numerical comparators (default 0, see
compare_ensemble_files_golden()).
- Returns:
int – The total number of failing checks (0 == success).
- pyretis.testing.example_compare.generate_golden_tarball(config_path, output_tgz, run_dir=None, files=None)¶
Pack a run’s own per-ensemble files into a golden reference tarball.
The “re-bless the reference” counterpart to the golden check’s tarball form (
_resolved_ref_dir()): for a folder whose engine is only bit-reproducible at a pinned, reference-grade version (e.g. LAMMPS), the committed reference is one small binary-diffable.tgzrather than a committed tree, generated by running the example once on that reference build and packing its own output. Every ensemble’straj.txtis reconstructed first (a no-op for an ensemble withouttraj.txtfragments) viareconstruct_traj_for_run().- Parameters:
config_path (str) – The simulation config (
.toml) the run used.output_tgz (str) – Where to write the
.tgzarchive.run_dir (str, optional) – Where the run produced its per-ensemble output (default: the config’s directory).
files (list of str) – The per-ensemble file names to pack (e.g.
['pathensemble.txt', 'order.txt', 'traj.txt']) – deliberately explicit and required, since this writes the reference every future run of this folder is graded against.
- Returns:
int – The number of requested files missing from the run (and therefore not packed); 0 means every requested file, for every ensemble, was found and written.
- pyretis.testing.example_compare.main(argv=None)¶
Command-line entry point for the shared example comparison.
Subcommands¶
compare <config.toml>Compare one completed run against its committed
results/reference (golden files + task-appropriate structural checks; override with--checks/--ref-dir).zero-left <nozero_dir> <withzero_dir>Check the two-run
zero_leftbehavioural contract.restart-whole <config> <restart_dir> <full_dir>Check a whole-file restart reproduces the uninterrupted run.
restart-concat <part1_dir> <part2_dir> <full_dir>Check a two-part concatenated restart equals the full run.
restart-md --traj P1 P2 FULL --text P1 P2 FULL ...Check a concatenated MD restart (trajectory + text files).
reconstruct-traj <config.toml>Reconstruct fragmented
traj.txtfiles beforecompare.generate-golden-tarball <config.toml> <output.tgz> --files ...Re-bless a tarball-form golden reference from the run’s own output.
- pyretis.testing.example_compare.reconstruct_traj_for_run(config_path, run_dir=None)¶
Reconstruct every ensemble’s fragmented
traj.txtfor a run.A thin per-run driver over
reconstruct_trajectory_text(), which joins the per-trialtraj.txtfragments under<ensemble>/traj/into<ensemble>/traj.txt. A folder that commitstraj.txtas a golden file (seecompare_ensemble_files_golden()) runs this once, right after the simulation and beforecompare.- Parameters:
config_path (str) – The simulation config (
.toml) the run used.run_dir (str, optional) – Where the run produced its per-ensemble output (default: the config’s directory).
- Returns:
int – The number of ensembles whose
traj.txtwas (re)written (0 is not an error by itself – an ensemble withouttraj.txtfragments has nothing to reconstruct).
- pyretis.testing.example_compare.tolerance_option(text)¶
Return the value of a tolerance option of a command line.
It is the
typeof the tolerance options of the command lines of the comparisons (--rel-tol,--abs-tol,--rel-tol-per-frame).- Parameters:
text (string) – The value as given.
- Returns:
out (float) – The tolerance, a finite number of 0 or more.
- Raises:
argparse.ArgumentTypeError – If the value is not a number, or is negative, NaN or infinite: a NaN tolerance makes no difference fail a
>test, and an infinite one accepts every finite difference.
pyretis.testing.reference_manifest module¶
Validate provenance and content digests for numerical references.
- pyretis.testing.reference_manifest._artifact_path(root, relative)¶
Resolve one safe repository-relative artifact path.
- pyretis.testing.reference_manifest._content_sha256(path)¶
Hash a file without loading the complete artifact into memory.
- pyretis.testing.reference_manifest._required_text(mapping, key, context)¶
Return a required non-empty string from a manifest object.
- pyretis.testing.reference_manifest._validate_artifact_digests(artifact_set, name, root)¶
Compare one set’s declared digests against the on-disk artifacts.
- pyretis.testing.reference_manifest._validate_artifact_set(artifact_set, name, context, root)¶
Validate one artifact set’s provenance fields and content digests.
- pyretis.testing.reference_manifest._validate_energy_term_list(terms, context)¶
Validate one explicit list of unavailable physical quantities.
Validate global or per-ensemble unavailable-energy declarations.
- pyretis.testing.reference_manifest.calculate_artifact_set(root, paths)¶
Calculate a deterministic digest and inventory for artifact paths.
Directory inputs are expanded recursively. The tree digest includes each repository-relative path, byte count, and file-content SHA-256 digest, in lexicographic path order. Overlapping path declarations are rejected.
- Parameters:
root (path-like) – Repository root for resolving the relative paths.
paths (sequence of str) – Files or directories included in one reference artifact set.
- Returns:
dict –
tree_sha256,file_count, andbyte_countvalues suitable for the manifest’sartifactsobject.
- pyretis.testing.reference_manifest.main(argv=None)¶
Run reference-manifest validation from the command line.
- pyretis.testing.reference_manifest.validate_manifest(manifest_path, root=None)¶
Validate manifest schema, provenance fields, and artifact digests.
- Parameters:
manifest_path (path-like) – JSON provenance manifest to validate.
root (path-like, optional) – Repository root. By default this is inferred for the canonical
examples/tests/reference_provenance.jsonlocation.
- Returns:
list of str – Names of the artifact sets successfully validated.
pyretis.testing.simulation_comparison module¶
Methods for comparing simulation results.
This module defines methods that can be used for comparing results from different simulations, such as output files, reports, and path ensembles.
- pyretis.testing.simulation_comparison.NUMERICAL_ABS_TOL_FLOOR = 1e-08¶
The smallest absolute tolerance of
compare_numerical_data(), the default absolute tolerance ofnumpy.allclose.
- pyretis.testing.simulation_comparison.OPTIONAL_COLUMNS = frozenset({16, 17, 18, 19})¶
Trailing
pathensemble.txtcolumns that only the infinite-swapping coordinator writes. They are compared when both files carry them and passed over when either does not; columns 0-15 are mandatory.
- pyretis.testing.simulation_comparison.STEP_COLUMN = 0¶
The column of
order.txtandenergy.txtthat holds the step of each frame, an integer counter that compares exactly.
- pyretis.testing.simulation_comparison.STEP_TERM = 'time'¶
The name the reader of
energy.txtgives to the step of each frame.
- pyretis.testing.simulation_comparison._archived_path_dirs(top)¶
Return the directories under
topthat hold files, relative to it.- Parameters:
top (str) – An archive directory.
- Returns:
set of str – The relative directories that hold at least one file.
- pyretis.testing.simulation_comparison._block_terms_agree(key, values1, values2)¶
Tell whether one term of two data blocks agrees.
- Parameters:
key (str) – The name of the term.
values1 (numpy.ndarray) – The values of the term in the first block.
values2 (numpy.ndarray) – The values of the term in the second block.
- Returns:
bool – True if the values agree: exactly for the step of each frame (
STEP_TERM), a counter; with the default tolerances ofnumpy.allclosefor every other term.
- pyretis.testing.simulation_comparison._compare_block_comments(comment1, comment2)¶
Compare two block comment lists, tolerating 1-ULP float differences.
- Parameters:
comment1 (list of str) – Comment lines from the first file block.
comment2 (list of str) – Comment lines from the second file block.
- Returns:
bool – True if the comments are considered equal.
- pyretis.testing.simulation_comparison._files_agree_over_common_prefix(file1, file2)¶
Check two binary files agree byte-for-byte over their common prefix.
Engine frame payloads (e.g. GROMACS
.trrsegments) may carry unreferenced TRAILING surplus frames: the propagation dump interval does not stop exactly at the frame the path keeps, and how much surplus is written is not reproducible between two runs. The frames a path actually references are pinned by itstraj.txt, which is compared exactly and separately. A rewrite, corruption, or genuine divergence of trajectory data shows up inside the common prefix and still fails.- Parameters:
file1 (str) – The path to the first file to compare.
file2 (str) – The path to the second file to compare.
- Returns:
equal (bool) – True if the files agree over their whole common prefix.
surplus (bool) – True if the files agree but one carries trailing surplus bytes.
- pyretis.testing.simulation_comparison._files_hold_same_bytes(file1, file2)¶
Return True when two files hold the same bytes.
The files are compared as they are on disk at the call. Two regular files of the same size are read side by side, in blocks of 1 MiB, until a pair of blocks differs or both files end; they hold the same bytes when every pair of blocks is equal. The result is False for files of different sizes and for a path that is not a regular file; both cases are decided from the status of the paths, without reading them.
- Parameters:
file1 (str) – The path to the first file to compare.
file2 (str) – The path to the second file to compare.
- Returns:
equal (bool) – True if both paths are regular files that hold the same bytes, False otherwise.
- Raises:
OSError – If the status of either path cannot be read, e.g. FileNotFoundError for a path that does not exist, or if two regular files of the same size cannot be opened or read.
- pyretis.testing.simulation_comparison._read_file_lines(filepath)¶
Read all lines from a file.
- pyretis.testing.simulation_comparison._traj_frame_files(traj_file)¶
Return the frame files a
traj.txtnames, in their first order.- Parameters:
traj_file (str) – The
traj.txtof an archived path.- Returns:
list of str – The file names of the second column, each once, in the order of the row that first names it.
- pyretis.testing.simulation_comparison._traj_text_by_place(traj_file, names)¶
Return the rows of a
traj.txtwith each file named by its place.- Parameters:
traj_file (str) – The
traj.txtof an archived path.names (list of str) – Its frame files, from
_traj_frame_files().
- Returns:
list of str – The lines of the file; in a data row the file name is replaced by its place in
names.
- pyretis.testing.simulation_comparison._without_part_lines(lines)¶
Return the lines that do not mark the start or end of a part.
- Parameters:
lines (list of strings) – The lines of an output file of an md-flux run.
- Returns:
out (list of strings) – The lines without those that mark the start or the end of a part (
pyretis.core.flux_state.parse_part_line()).
- pyretis.testing.simulation_comparison.compare_data_by_columns(file1, file2, file_type, skip=None)¶
Compare two output PyRETIS data files by columns.
This method compares files where numbers are stored in columns and the columns have specific labels. Here, we also compare labels and comments.
- Parameters:
file1 (str) – The path to the first file to compare.
file2 (str) – The path to the second file to compare.
file_type (str) – A string for determining the file type (e.g., ‘energy’).
skip (list of str, optional) – A list of items from the loaded data we are to skip. This can, for instance, be certain energy terms that are not absolute and can’t easily be compared.
Notes
The step of each frame (the term
STEP_TERM) compares exactly; every other term compares with the default tolerances ofnumpy.allclose.- Returns:
equal (bool) – True if the files are deemed to be equal.
msg (str) – A descriptive message of the result of the comparison.
- pyretis.testing.simulation_comparison.compare_energy_columns_mse(energy, reference, pairs, tol=1e-05)¶
Per-term MSE check of a pyretis
energy.txtarray vs an engine’s.The
test-integrateexample comparators (cp2k, gromacs) share this: each checks that pyretis reproduces the engine’s own energies term by term, differing only in whichenergy.txtcolumn maps to which engine-output key.- Parameters:
energy (numpy.ndarray) – The pyretis
energy.txtdata (asnumpy.loadtxt()returns; columns cycle, vpot, ekin, etot, temp, …).reference (mapping of str to numpy.ndarray) – The engine’s energies keyed by term name (the keys used in
pairs).pairs (sequence of (int, str)) –
(column_index_in_energy, reference_key)pairs to compare, e.g.((1, 'vpot'), (2, 'ekin'), (3, 'etot'), (4, 'temp')).tol (float, optional) – Per-term MSE tolerance; a falsy
tolreports the MSEs without judging them.
- Returns:
equal (bool) – True when every compared term matches in length and (if
tol) is within tolerance.msg (str) – The per-term MSEs, plus the failing term / tolerance on failure.
- pyretis.testing.simulation_comparison.compare_energy_columns_relative(energy, reference, pairs, rel_tol)¶
Frame-by-frame check of a pyretis
energy.txtarray vs an engine’s.Each compared term agrees when, at every frame, it differs from the engine’s value by at most
rel_toltimes the engine’s value (max_relative_difference()). It is the check of runs that are not repeatable to the last digit, such as GROMACS on a GPU, where a mean squared error over the frames falls below the run-to-run noise of large energies.- Parameters:
energy (numpy.ndarray) – The pyretis
energy.txtdata (asnumpy.loadtxt()returns; columns cycle, vpot, ekin, etot, temp, …).reference (mapping of str to numpy.ndarray) – The engine’s energies keyed by term name (the keys used in
pairs).pairs (sequence of (int, str)) –
(column_index_in_energy, reference_key)pairs to compare.rel_tol (float) – The relative tolerance per frame.
- Returns:
equal (bool) – True when every compared term agrees at every frame.
msg (str) – The largest relative difference of each term and its frame, plus the failing terms and the tolerance on failure.
- pyretis.testing.simulation_comparison.compare_energy_data(file1, file2, rel_tol=1e-05, unavailable_terms=None, abs_tol=0.0)¶
Compare two
energy.txtfiles with explicit unavailable terms.Energy files always carry a fixed set of columns (time, potential, kinetic, total, temperature), but not every engine or path populates all of them: paths loaded from disk carry no computed energy, and an engine may not report e.g. the total energy or the temperature. Such a term is written as an entirely-NaN column.
This wraps
compare_numerical_data()with energy-specific handling of those columns. Unavailable physical quantities must be named by the caller; matching all-NaN columns are not inferred as a success:A term named in
unavailable_termsmust be entirely NaN in both files and is then excluded from the numerical comparison.An all-NaN term that was not declared is an unexpected coverage gap and fails.
Every other column is compared normally. A column that is all-NaN in only one file (a run that suddenly computes – or stops computing – a term) therefore fails, via the NaN-position check.
The step of each frame compares exactly, whatever the tolerances (
compare_steps()).
- Parameters:
file1 (str) – The path to the first file to compare.
file2 (str) – The path to the second file to compare.
rel_tol (float, optional) – Relative tolerance forwarded to
compare_numerical_data().unavailable_terms (sequence of str, optional) – Energy terms that this reference intentionally cannot provide. Valid names are
vpot,ekin,etot, andtemp.abs_tol (float, optional) – Absolute tolerance forwarded to
compare_numerical_data().
- Returns:
equal (bool) – True if the files are deemed to be equal.
msg (str) – A descriptive message of the result of the comparison.
- pyretis.testing.simulation_comparison.compare_moves_data(file1, file2, rel_tol=1e-05, skip=None, abs_tol=0.0)¶
Compare two
moves.txtfiles.One line per attempted Monte Carlo move. Columns that do not apply to a move are written
-rather than as a number, so a column is compared numerically only when BOTH files carry a value there; a-matches only another-. That asymmetry is deliberate: a run that started writing a real shooting point where the reference has none is a change worth failing on, not a rounding difference.- Parameters:
file1 (str) – The path to the first file to consider in the comparison.
file2 (str) – The path to the second file to consider in the comparison.
rel_tol (float, optional) – A relative tolerance for determining if numbers are equal.
skip (list of int, optional) – These are columns we are to skip in the comparison.
abs_tol (float, optional) – An absolute tolerance for determining if numbers are equal: two numbers are equal when they differ by at most
abs_tolplusrel_toltimes the larger magnitude (numbers_agree()).
- Returns:
equal (bool) – True if the files are equal, False otherwise.
msg (str) – A message describing the result of the comparison.
- pyretis.testing.simulation_comparison.compare_numerical_data(file1, file2, rel_tol=1e-05, skip_cols=None, abs_tol=0.0)¶
Compare two files containing numerical data.
Here, we compare files that contain numerical data. We don’t care about comments here, we just compare the actual numerical data.
A meaningful comparison must not pass on degenerate input. This rejects empty data, shape mismatches, NaNs that sit at different positions in the two files (a divergence that turned a finite value into NaN or vice versa), and data/columns that are entirely NaN (which test nothing). Partial NaN at matching positions – e.g. loaded (
ld) frames that legitimately have no computed energy – is allowed; the finite values around them are still compared. Columns that are unavailable for an engine (e.g. a potential energy that engine never reports) must be excluded explicitly viaskip_cols, which names the gap in the call.- Parameters:
file1 (str) – The path to the first file to compare.
file2 (str) – The path to the second file to compare.
rel_tol (float, optional) – Relative tolerance for the comparison.
skip_cols (list of int, optional) – Column indices to exclude from the comparison (e.g. a column an engine does not populate).
abs_tol (float, optional) – Absolute tolerance for the comparison: two values agree when they differ by at most
abs_tol + rel_tol * |value of file2|. The absolute tolerance is at leastNUMERICAL_ABS_TOL_FLOOR.
- Returns:
equal (bool) – True if the files are deemed to be equal.
msg (str) – A descriptive message of the result of the comparison.
- pyretis.testing.simulation_comparison.compare_numerical_mse(file1, file2, tol=1e-12)¶
Compare two numerical files using mean squared error.
- Parameters:
file1 (str) – The path to the first file to compare.
file2 (str) – The path to the second file to compare.
tol (float, optional) – Tolerance for the mean squared error.
- Returns:
equal (bool) – True if the MSE is below the tolerance.
msg (str) – A descriptive message with the MSE value.
- pyretis.testing.simulation_comparison.compare_order_data(file1, file2, rel_tol=1e-05, abs_tol=0.0)¶
Compare two
order.txtfiles.The order parameters compare with the tolerances (
compare_numerical_data()), and the step of each frame compares exactly (compare_steps()).- Parameters:
file1 (str) – The path to the first file to compare.
file2 (str) – The path to the second file to compare.
rel_tol (float, optional) – Relative tolerance forwarded to
compare_numerical_data().abs_tol (float, optional) – Absolute tolerance forwarded to
compare_numerical_data().
- Returns:
equal (bool) – True if the files are deemed to be equal.
msg (str) – A descriptive message of the result of the comparison.
- pyretis.testing.simulation_comparison.compare_path_ensemble_data(file1, file2, rel_tol=1e-05, skip=None, abs_tol=0.0)¶
Compare two path ensemble files.
We compare line-by-line, but skip comments and we check that numbers are close, as judged by the given relative tolerance.
- Parameters:
file1 (str) – The path to the first file to consider in the comparison.
file2 (str) – The path to the second file to consider in the comparison.
rel_tol (float, optional) – A relative tolerance for determining if numbers are equal.
skip (list of int, optional) – These are columns we are to skip in the comparison.
abs_tol (float, optional) – An absolute tolerance for determining if numbers are equal: two numbers are equal when they differ by at most
abs_tolplusrel_toltimes the larger magnitude (numbers_agree()).
- Returns:
equal (bool) – True if the files are equal, False otherwise.
msg (str) – A message describing the result of the comparison.
- pyretis.testing.simulation_comparison.compare_reports_normalized(fil1, fil2)¶
Compare two reports, normalizing common version/time differences.
This function ignores Docutils version meta-data, timestamps, and common spelling variations (grey/gray) in CSS to remain robust against environment differences.
- Parameters:
fil1 (str) – The path to the first report to compare.
fil2 (str) – The path to the second report to compare.
- Returns:
equal (bool) – True if reports are essentially equal.
msg (str) – Description of mismatch if found.
- pyretis.testing.simulation_comparison.compare_restarted_cross_files(file11, file12, file2)¶
Compare CrossFile data from a restarted simulation.
The data rows of the files are compared; their comment lines (the header, and the lines that mark the start and the end of a part of an md-flux run) are not.
- Parameters:
file11 (str) – Path to the first part of the crossing data.
file12 (str) – Path to the second part of the crossing data.
file2 (str) – Path to the full (continuous) crossing data.
- Returns:
equal (bool) – True if the crossing data matches.
msg (str) – A descriptive message.
- pyretis.testing.simulation_comparison.compare_restarted_text_files(file11, file12, file2)¶
Check if file2 is equal to file11 + file12 minus one overlapping line.
We handle headers (lines starting with ‘#’) by skipping them in the second file part. The lines that mark the start and the end of a part of an md-flux run (
pyretis.core.flux_state.part_start_line(),pyretis.core.flux_state.part_end_line()) are left out of the three files, so the parts of a run compare equal to the run made in one part.- Parameters:
file11 (str) – Path to the first part of the restarted simulation output.
file12 (str) – Path to the second part of the restarted simulation output.
file2 (str) – Path to the full (continuous) simulation output.
- Returns:
equal (bool) – True if the files match the pattern.
msg (str) – A descriptive message of the result.
- pyretis.testing.simulation_comparison.compare_simulation_files(file1, file2, skip=None, mode='line')¶
Top-level function to compare two simulation output files.
- Parameters:
file1 (str) – The path to the first file to compare.
file2 (str) – The path to the second file to compare.
skip (list of str or list of int, optional) – A list of items that are to be skipped in the comparison.
mode (str, optional) – A string for determining how we do the comparison: ‘numerical’ will select a comparison of numerical blocks; ‘line’ will select a line-by-line text comparison; anything else will perform a literal file comparison, in which the files are equal when they hold the same bytes (
_files_hold_same_bytes()).
- Returns:
equal (bool) – True if the files were found to be equal, False otherwise.
msg (str) – A string with information about the comparison result.
- pyretis.testing.simulation_comparison.compare_steps(file1, file2)¶
Compare the step of each frame of two numerical files exactly.
The step (column
STEP_COLUMNoforder.txtandenergy.txt) is a counter, so no tolerance applies to it. Each file is read as rows and columns, so that a file of one row keeps its columns apart.- Parameters:
file1 (str) – The path to the first file to compare.
file2 (str) – The path to the second file to compare.
- Returns:
equal (bool) – True if the two files hold the same steps.
msg (str) – A descriptive message of the result of the comparison.
- pyretis.testing.simulation_comparison.compare_text_line_by_line(file1, file2, skip=None, skip_keys=None)¶
Compare two files, line by line.
- Parameters:
file1 (str) – The path to the first file to compare.
file2 (str) – The path to the second file to compare.
skip (list of int, optional) – These are 0-indexed line numbers we are to skip.
skip_keys (list of str, optional) – Lines whose first token matches any key in this list are filtered out from both files before comparison. Useful for ignoring settings like
exe_paththat differ by run directory.
- Returns:
equal (bool) – True if the files are deemed to be equal.
msg (str) – A descriptive message of the result of the comparison.
- pyretis.testing.simulation_comparison.compare_traj_archive(dir1, dir2)¶
Compare archived trajectories between two directories.
These archives consist of trajectory information such as energies, order parameters and positions. Here, we verify that the output written by PyRETIS is identical in the two cases.
- Parameters:
dir1 (str) – The path to the first directory to use in the comparison.
dir2 (str) – The path to the second directory to use in the comparison.
- Returns:
errors (list of tuple) – This list contains the files which differed, if any. A directory that is absent on either side, and a pair of directories that both hold no files, are reported as a difference.
- pyretis.testing.simulation_comparison.compare_traj_archive_by_place(dir1, dir2, energy_skip=None)¶
Compare two archives, pairing the frame files of a path by place.
A run that is resumed names the frame files of its new paths from a counter that starts again in each process, so the same path can be stored as
6_trajB.trrby a run that went on and as0_trajB.trrby a run that was resumed. In each archived path the frame files are paired by their place in itstraj.txt(the order of the rows that first name them): each pair agrees byte for byte over the common prefix (_files_agree_over_common_prefix()), and the twotraj.txtagree once each file name is replaced by its place. The other files are paired by name and compared ascompare_traj_archive_soft()compares them.- Parameters:
dir1 (str) – The first archive directory.
dir2 (str) – The second archive directory.
energy_skip (list of str, optional) – Energy terms an archived
energy.txtis compared without.
- Returns:
errors (list of tuple) – The files (or directories) which differed, if any; a directory absent on either side, or two archives without files, is a difference.
- pyretis.testing.simulation_comparison.compare_traj_archive_soft(dir1, dir2, energy_skip=None, frames='byte')¶
Compare an archived trajectory, tolerating named energy terms.
Like
compare_traj_archive()(byte-exact for every archived file), with engine-facing relaxations for the run-vs-run self-consistency suites:an archived
energy.txtfragment is compared by column instead of byte-for-byte, skipping the named terms inenergy_skip. Some external engines recompute a term (e.g. GROMACS’s long-range dispersion correction to the potential energy) slightly differently when continuing a run;with
frames='byte'(the default), engine frame payloads (any file that is not one of the PyRETIS-writtenenergy.txt/order.txt/traj.txt) must agree byte-for-byte over their COMMON PREFIX – trailing unreferenced surplus frames are logged and tolerated (see_files_agree_over_common_prefix());with
frames='names', frame payloads are checked by name and count only. This is for comparisons across DIFFERENT engine implementations of the same MD (e.g. the relaunch vs streaming GROMACS engines), whose physics text output is exact-equal while their archived payload framing (frame counts, dump cadence) is implementation-specific and never byte-comparable.
- Parameters:
dir1 (str) – The path to the first directory to use in the comparison.
dir2 (str) – The path to the second directory to use in the comparison.
energy_skip (list of str, optional) – Energy terms (as read by
EnergyPathFile, e.g.'vpot') to exclude from the comparison of any archivedenergy.txtfragment.Nonecompares every term.frames (str, optional) –
'byte'(default) or'names'; see above. The PyRETIS-written text files are compared exactly in BOTH modes.'place'pairs the frame files of each archived path by their place in itstraj.txt(compare_traj_archive_by_place()).
- Returns:
errors (list of tuple) – The files which differed, if any (see
compare_traj_archive()for the missing-directory behaviour).
- pyretis.testing.simulation_comparison.max_relative_difference(values, reference)¶
Return the largest difference of two series relative to a reference.
- Parameters:
values (array_like) – The values, one per frame.
reference (array_like) – The reference values, one per frame.
- Returns:
largest (float) – The largest
|value - reference| / |reference|over the frames.frame (int) – The index of the frame of the largest difference.
- Raises:
ValueError – If the two series differ in length, hold no frame, hold a value that is not finite (a NaN or an infinity), or the reference holds a 0, which a relative difference cannot judge.
- pyretis.testing.simulation_comparison.numbers_agree(value, reference, rel_tol=1e-05, abs_tol=0.0)¶
Tell whether two numbers agree within the two tolerances.
Two numbers agree when they are equal, or when both are finite and differ by at most
abs_tol + rel_tol * max(|value|, |reference|): the two tolerances add up, as innumpy.allclose. Withabs_tol0 the rule is that ofmath.isclosewith the same relative tolerance. A NaN agrees with nothing, and an infinity only with the same infinity.- Parameters:
value (float) – The number to judge.
reference (float) – The number it is compared against.
rel_tol (float, optional) – The relative tolerance, applied to the larger magnitude of the two numbers.
abs_tol (float, optional) – The absolute tolerance.
- Returns:
bool – True if the two numbers agree.
- Raises:
ValueError – If a tolerance is negative.
- pyretis.testing.simulation_comparison.read_files(*files, read_comments=True)¶
Read files into memory.
Here, we assume that we are given small files and that we can read these into memory.
- Parameters:
files (tuple of str) – These are the paths to the files we are to read.
read_comments (bool, optional) – If False, we skip lines starting with a “#”.
- Returns:
all_data (list of list of str) – The data read from the different files.
- pyretis.testing.simulation_comparison.reconstruct_trajectory_text(ensemble_dir)¶
Reconstruct a consolidated
traj.txtfrom per-trial fragments.The fragments are the
traj.txtfiles found anywhere under<ensemble_dir>/traj/, one per trial (e.g.<ensemble_dir>/traj/<trial>/traj.txt). Each fragment’s first line carries a# Cycle:<n>, ...header; the fragments are concatenated in ascending cycle order into<ensemble_dir>/traj.txt, the file the golden check compares.- Parameters:
ensemble_dir (str) – Path to a single ensemble’s output directory (e.g.
.../000).- Returns:
written (bool) – True if at least one fragment was found and
traj.txtwas (re)written; False if<ensemble_dir>/trajdoes not exist or holds notraj.txtfragment, in which case nothing is written (the ensemble has no fragmented trajectory to reconstruct – not a failure by itself).
pyretis.testing.helpers module¶
Methods that might be useful for testing.
This module defines generic methods for testing.
- pyretis.testing.helpers.VALIDATE_ENV = 'PYRETIS_VALIDATE'¶
Environment variable that turns the numerical reference comparison on (
"1", the default) or off ("0"). It is set to"0"by the engine test runners when only a non-reference-grade external MD engine is available (for example a single-precision GROMACS build when the reference data was generated in double precision). External MD engines are not bit-for-bit reproducible across build, precision and hardware, so a reference comparison is only meaningful against the exact reference-grade engine.
- pyretis.testing.helpers.clean_dir(dirname)¶
Remove ALL files in the given directory.
- pyretis.testing.helpers.reference_validation_enabled()¶
Return whether numerical reference comparison should be performed.
- Returns:
out (boolean) –
Trueunless thePYRETIS_VALIDATEenvironment variable is set to"0". The default (variable unset) isTrue, so a run with a reference-grade engine is validated.
- pyretis.testing.helpers.search_for_files(rootdir, match=None)¶
Find files by walking the given directory.
- Parameters:
rootdir (string) – The path where we will search from.
match (string, optional) – If given, the method will only return files that are equal to the given match.
- Returns:
out (list of strings) – The paths of the found files.
- pyretis.testing.helpers.skip_reference_validation(reason)¶
Report that the run was executed but not numerically validated.
This prints a clearly labelled
NOT VALIDATEDmessage and exits with status0. It is used by the engine test scripts as the “execution smoke” outcome: the simulation ran without error, but the numerical comparison against the committed reference was skipped because no reference-grade engine was available. Following the project rule “better a fail than a wrong pass”, a skipped comparison is never reported as a pass.- Parameters:
reason (string) – Why the numerical comparison was skipped (for example
"single-precision GROMACS; reference is double precision").
pyretis.testing.systemhelp module¶
Methods that might be useful for testing.
This module defines methods that are useful in connection with systems.
- pyretis.testing.systemhelp.create_system_ext(pos=None, vel=False)¶
Create an external system with given positions and velocities.