The Analysis section

The analysis section defines parameters for the analysis.

Example Analysis section:
[analysis]
method = "matched"
skip_initial_cycles = 0
maxblock = 1000
blockskip = 1
bins = 100
ngrid = 1001
report = [
    "latex",
    "rst",
    "html",
]
maxordermsd = 100
txt-output = "txt.gz"

[analysis.plot]
plotter = "mpl"
output = "png"
style = "pyretis"

A shorter example, a WHAM analysis that leaves out the first 100 cycles of the run:

[analysis]
method = "wham"
skip_initial_cycles = 100
ngrid = 1001
bins = 100

Choosing the rate-estimation method

PyRETIS can estimate the crossing probability and the rate constant with two complementary methods, selected with the method keyword. Both read the sampled paths from the same simulation; they differ in how those paths are combined:

  • matched (the default; crossing is accepted as the historical spelling) – the matched / point-matching analysis. It reads the per-ensemble pathensemble.txt files and matches the per-ensemble crossing-probability curves at the interfaces. This is the classic PyRETIS analysis and is available for every path-sampling task (tis, retis, pptis, repptis); the PPTIS/REPPTIS memory-kernel analyses build on its per-ensemble local probabilities. A biretis run is analysed this way only, for the two directions (see the biretis task): it refuses wham and both.

  • wham – the weighted-histogram (WHAM) analysis. It reweights the shared paths across all ensembles in a single histogram, built from the same per-ensemble output every scheduler run writes (a literal infswap_data.txt, if one is present – from an older run, or from merging several runs’ path data – is read directly instead). Use it when you have an infinite-swapping run.

  • both – run the two analyses and report each. When the run holds no path data for the WHAM analysis (an md-flux run), the matched analysis is run and the WHAM analysis is skipped with a warning (method = "wham" instead fails loudly if no usable path data is found). When the matched analysis fails, the analysis stops with an error and the WHAM analysis is not run.

The two methods are different estimators of the same physical rate, so on a converged simulation they agree within their statistical error. The standard RETIS report now includes a WHAM comparison automatically, using the same retained production occupancy rows and initial flux as the matched estimate. method = "both" additionally writes the standalone wham_analysis.txt.

The report comparison lists absolute standard errors for the crossing probabilities and rates. WHAM probability errors use a delete-one-block jackknife over contiguous cycle blocks, removed from every ensemble together. Blocks shorter than the correlation time of the run give too small an error, so the block length doubles from the shortest (one cycle or, for more than 512 retained cycles, their number divided by 512 and rounded up) while at least eight blocks remain, and the error is taken at the shortest length from which at least two and every longer one meet the reblocking criterion \(L^3 > 2N(\sigma_L/\sigma_1)^4\) of Lee et al., Phys. Rev. E 83, 066706 (2011), with the block \(L\) and the run \(N\) counted in shortest blocks and \(\sigma_1\) the error of the shortest blocks; maxblock does not limit these blocks. When the longest length and the next one down do not both meet it, the error is n/a with the reason in the report: the run is too short for its correlation time, or too short to confirm an error at the longest blocks, or it has fewer than 32 retained cycles (one-cycle blocks never meet the criterion, so that an error needs blocks of two and of four cycles at least). The error is n/a as well for a run of fewer than eight retained cycles, when deleting a block leaves an ensemble without weight, and when no path crossed the last interface. Rate errors combine the probability and flux errors in quadrature, neglecting their covariance. These are estimated standard errors, not 95% confidence intervals. Unavailable errors are shown as n/a, including runs with no observed reactive crossing.

The RETIS report also includes four sampling diagnostics:

  • Local crossing probabilities with absolute block standard errors and a shaded target band of 0.2–0.5. The last positive ensemble measures the probability of reaching the product boundary.

  • Unmatched crossing histograms overlaid for all positive ensembles, each normalised at its own interface. Dashed, labelled vertical lines mark all interfaces and the product boundary.

  • Path length autocorrelation functions (ACF) against lag in scheduler cycles (moves). Rejections repeat the held path; fractional occupants contribute their occupancy-weighted mean length per cycle. Missing cycles remain gaps, and constant series have undefined ACF. The normalisation is the centred lagged-product sum divided by the total centred sum of squares, following Hall et al., J. Chem. Phys. 156, 200901 (2022), Eq. (6).

  • Path mobility across ensembles, highlighting up to eight paths with the largest recorded residence, with the remaining occupancy in grey. Symbol area represents fractional occupancy and dotted lines connect the occupancy-weighted ensemble positions. This tracks global path identities, not replica lineages through shooting. Older output without global path numbers shows an unavailable panel.

The WHAM row carries an asterisk referring to a final appendix that defines the recorded inputs and gives the HA correction, the Q-factor probability recursion and the block-jackknife standard-error formula:

  • \(c_{nj}\) comes from the FracInc column of ensemble \(j\)’s pathensemble.txt. It is the infinite-swap assignment probability for the path in retained occupancy record \(n\). The sampler calculates it by summing weighted one-to-one assignments of the current free paths to free ensembles, using its configured permanent calculator. A path held over several cycles contributes repeated occupancy records.

  • \(h_{nj}\) is the recorded HA-weight. It is calculated from the path geometry and the ensemble’s sampling rule: one for ordinary shooting and non-HA stone skipping, a crossing count for HA stone skipping, and an eligible-interior-frame count for wire fencing. HA moves apply their endpoint/orientation correction, including the factor of two for opposite-side endpoints. This factor is distinct from acceptance probability.

  • \(u_{nj}=c_{nj}/h_{nj}\) is the HA-corrected histogram contribution for positive HA weight. It removes HA sampling multiplicity from the recorded occupancy; zero-membership records contribute zero. For example, \(c_{nj}=1/2\) and \(h_{nj}=2\) give \(u_{nj}=1/4\).

The crossing indicator \(\chi_n(\lambda)\) is one when \(\lambda_{\max,n}>\lambda\) and zero otherwise. Thus

\[\widehat p_j(\lambda)= \frac{\sum_n u_{nj}\chi_n(\lambda)}{\sum_n u_{nj}}\]

counts only paths that exceed the tested level in the numerator. This uses the same strict crossing convention as the analysis code. The report also defines \(N_j=\sum_n c_{nj}\) as total occupancy, rather than the number of independent paths.

The main WHAM section contains the figures and brief captions:

  • The pooled distribution of positive-ensemble path maxima before HA correction. Shared and repeated paths are counted once when global path IDs are present; otherwise the axis states that it shows occupancy counts. Maxima at or beyond the product boundary enter the final bin.

  • The relative contribution of each ensemble to the WHAM numerator at each order-parameter value after HA correction. These fractions describe the estimator’s composition, rather than an independent sample count.

  • The reconstructed WHAM crossing curve, its one-standard-error band and the matching curve. The band is clipped to the displayed probability range on the logarithmic axis.

  • Endpoint probabilities recomputed after omitting each cycle block. These values determine the jackknife error; histogram counts alone do not capture the temporal correlations that affect that error. When the error is n/a, the panel gives the reason instead.

The PDF places the four Combined results plots on its first page, followed by the WHAM figures. Definitions, formulas and the uncertainty-method explanation are collected in the WHAM methods and notation appendix at the very end, after all ensemble results. HTML and text reports use the same separation of results and methods.

The underlying local probabilities, ACFs and occupancy records are also written as text files beside the figures. The existing per-ensemble crossing-probability files contain the unmatched histogram data. These diagnostics require no new settings and respect the report’s equilibration and trailing-cycle cuts. The files wham-path-maxima.txt.gz, wham-ensemble-contributions.txt.gz, wham-reconstruction.txt.gz and wham-block-estimates.txt.gz contain the numerical data behind the WHAM panels (using the configured text-output extension).

The skip_initial_cycles and skip_final_cycles keywords leave the start and the end of the sampling out of the statistics; on the command line, pyretis analyse -skipb N -skipe M sets them and overrides the input file. Both methods leave out the rows of the first N and of the last M cycles of the run, the cycles that the pathensemble.txt files of the ensembles record, the same cycles in every ensemble, so method = "both" computes the two estimates over the same cycles. The WHAM matrix also leaves out the paths held at the end of the run, their rows in the kept cycles included, as without a skip, which makes the two estimates differ. With neither the keywords nor the options, both methods use the whole run. The ensembles of a make-tis-files input are separate single TIS runs: the analysis of that input counts the cycles of each ensemble’s own pathensemble.txt.

Keywords for the Analysis section

For the analysis section the following keywords can be set:

Table 45 Supported keywords for the analysis section.

Keyword

Description

method

Select the rate-estimation method.

skip_initial_cycles

Initial cycles to skip.

skip_final_cycles

Final cycles to skip.

bins

Set the number of bins to use when creating histograms.

blockskip

Selects certain block lengths to skip in the in the block error analysis

maxblock

Set maximum length of blocks to consider in the block error analysis.

maxordermsd

Set the lags of the mean square displacement of the order parameter, in rows of the order file.

ngrid

Number of points used for calculating the crossing probability.

plot

Settings related to plotting of results.

report

Define output format(s) for the report.

report-dir

Refused: the report goes to the folder report where pyretis analyse is run.

skipcross

Set time window for initial flux calculation.

txt-output

Select format for text-based output.

skip

Initial cycles to skip, in an older input; set skip_initial_cycles instead.

tau_ref_bin

Set the reference region of the permeability analysis.

Keyword method

method = string

The method keyword selects which rate-estimation method pyretis analyse runs. See Choosing the rate-estimation method for guidance. Valid choices are:

  • matched – the matched (point-matching) analysis of the per-ensemble pathensemble.txt files (the classic PyRETIS analysis; crossing is accepted as the historical spelling).

  • wham – the weighted-histogram (WHAM) analysis of an infinite-swapping run’s shared-path data (built from the per-ensemble output, or a literal infswap_data.txt if one is present).

  • both – run both analyses and report each.

Default

The default value is: method = matched

Keyword skip_initial_cycles

skip_initial_cycles = integer

The skip_initial_cycles keyword sets the number of initial (equilibration) cycles to discard before the statistics are accumulated. The matched and the wham methods leave out the rows of the first skip_initial_cycles cycles of the run (the cycles that any ensemble records), the same cycles in every ensemble. A set value is used as it is: a count of 0 or less (skip_initial_cycles = 0) leaves out no cycle. When it is not set, both methods use the skip keyword of an older input instead. An input that sets skip_initial_cycles = 0 next to a skip above 0 (a run file of an earlier version records both) is refused by pyretis analyse unless -skipb N gives the number.

Default

Not set: the skip keyword applies (default 0).

Note

pyretis analyse -skipb N sets this on the command line, overriding the input file. That is usually what you want: how much equilibration to drop is a question you answer after looking at the run, without editing (and re-saving) the input.

Keyword skip_final_cycles

skip_final_cycles = integer

The counterpart of skip_initial_cycles: the number of cycles to discard at the end of the sampling. This is what you reach for when a run was stopped part-way – a wall-time kill, a crashed node – and its final cycles are not to be trusted, or when you want to compare two runs over the same number of cycles.

pyretis analyse -skipe N sets it on the command line, overriding the input file.

The matched and the wham methods leave out the rows of the last skip_final_cycles cycles of the run, the same cycles in every ensemble; a count of 0 or less leaves out no cycle. The WHAM analysis of a literal infswap_data.txt, which records no cycle, stops with an error when a cycle skip above 0 is set.

Default

Nothing is discarded.

Keyword bins

bins = integer

The bins keyword defines the number of bins to use when histograms are created.

Default

The default value is: bins = 100

Keyword blockskip

blockskip = integer

The blockskip keyword can be used to skip certain block lengths in the block error analysis. That is:

  • blockskip = 1 will consider all blocks up to the value given in the keyword maxblock

  • blockskip = n will consider every n’th block up to the value given in the keyword maxblock, That is, it will use block lengths equal to 1, 1 + n, 1 + 2*n and so on.

Default

The default value is: blockskip = 1

Keyword maxblock

maxblock = integer

The maxblock keyword defines the maximum length of the blocks to consider for the block error analysis. If a negative number is set, PyRETIS will set maxblock equal to (1/2) * length of the data we are analysing. This is also the maximum length that PyRETIS will consider, even if the input maxblock is set higher.

The block error analysis of an md-flux run takes blocks of up to half the rows, or of up to maxblock rows when that is less, and averages the errors of the blocks longer than half that length. When there is none – a file of fewer than 2 rows, or a blockskip that skips every such length – pyretis analyse gives no block error of the energies or of the order parameter (Rel.err and Ncor are nan in the header of their text files), and the report and the log say why and what to set. The averages and the distributions of the rows are given. A block holds the rows of one part of a run in parts, so the longest block is the longest that gives two blocks: half the rows of a part, or the rows of a part when two parts hold that many (see the parts of the run).

Default

The default value is: maxblock = 1000

Keyword maxordermsd

maxordermsd = integer

The maxordermsd keyword sets the lags of the mean square displacement of the order parameter that pyretis analyse of an md-flux run computes: maxordermsd = n gives lags of 1 to n - 1 rows of order.txt, and a value less than 1 sets n to one fifth of its rows, rounded down, so an order file of fewer than 10 rows gives no lag. A lag of k rows needs more than k rows of one part of the run, as the displacements are taken within each part and pooled over the parts (see the parts of the run). The analysis computes the lags that the rows give: the report and the log say which lags it does not compute, and why, and the other results of the order parameter analysis are given. The report gives a lag of k rows as the time of k times order-file steps. The rows of a part must be equally spaced in steps, with the same spacing in every part: an order file whose rows are at two intervals (a continuation wrote it with another order-file) gives no mean square displacement, and the report and the log say why.

Default

The default value is: maxordermsd = -1

Keyword ngrid

ngrid = integer

The ngrid keyword defines the number of points used for the order parameter when PyRETIS is calculating the crossing probability as a function of the order parameter.

Default

The default value is: ngrid = 1001

Keyword plot

plot = dictionary

The plot keyword defines how plotting should be carried out. It is given as a dictionary, where the following keys can be used:

  • 'plotter': Which selects the kind of plotter to use. Currently, PyRETIS only implements a plotter using matplotlib which is selected by setting plot = {'plotter': 'mpl'}.

  • 'output': Which selects the output format for plots. This can be set to any of the supported output formats by the plotter. Often this will depend on your specific system, but common choices are 'png', 'pdf' and 'svg'.

  • 'style': Selects a style for creating plots. Here you can, for instance, create your own matplotlib style or use one of the built-in styles. Please see http://matplotlib.org/users/style_sheets.html for more information.

Default

The default value is: plot = {'plotter': 'mpl', 'style': 'pyretis', 'output': 'png'}

Keyword report

report = list of strings

The report keyword defines the output format for the report. This is provided as a list of strings, where each item in the list correspond to a specific format:

  • 'latex' selects latex output.

  • 'rst' selects output in reStructuredText.

  • 'html' selects html output.

Default

The default value is: report = ['latex', 'rst', 'html']

Keyword report-dir

report-dir = string

pyretis analyse writes the report, its figures and its text files to the folder report in the directory it is run from, and the settings parse refuses a report-dir in the input, of every task. To keep a report apart from the next analysis, copy or rename the report folder.

Default

The folder report in the directory pyretis analyse is run from.

Keyword skipcross

skipcross = integer

The length, in simulation steps, of the time window the initial flux is averaged over in an md-flux analysis.

Note

The name is historical and misleading: nothing is skipped. The keyword is a window length.

The flux is the number of effective crossings of an interface per unit time. Measuring it once over the whole run would give a single number with no way to put an error on it, so the time the system spends in the overall \(A\) state is instead cut into windows of skipcross steps, and the flux is computed in each. When that time is not a whole number of windows, the last window holds the steps that remain and is shorter. That series, each window weighted by its length, is what the running-average plot and the block-error estimate are built from. The time of each part of a run in parts is cut into windows on its own: a window never holds the time of two parts (see the parts of the run).

It does not change the flux itself. The mean over the windows, each weighted by its length, is the total crossing count divided by the total time however the windows are drawn; only the spread of the series changes, and with it the error estimate:

skipcross

windows

mean flux

spread

500

200

0.50000

0.313

1000

100

0.50000

0.215

5000

20

0.50000

0.067

20000

5

0.50000

0.039

(the same 500 synthetic crossings over 100000 steps, binned four ways.)

So choose it for the statistics, not for the answer. Too small and each window holds a handful of crossings, so the series is noisy; too large and there are too few windows for a meaningful block analysis. A useful target is enough windows to block over – a few tens – with several crossings in each.

If the time in the overall \(A\) state is shorter than one window, the flux analysis stops with a message: reduce skipcross or run longer. The energy and order parameter analyses of the run complete and their reports are written, the report says why it holds no flux, and pyretis analyse exits with status 1.

Only the analysis of an md-flux run reads skipcross. A tis or retis run takes its flux from the [0^-] and [0^+] path lengths instead (see Where each factor comes from in a run), and the input of a path-sampling run refuses skipcross.

Default

The default value is: skipcross = 1000

Keyword txt-output

txt-output = string

The txt-output keyword defines the format for how text-based output should be written. Valid choices are:

  • txt for writing plain text files.

  • txt.gz for creating gzipped files.

Default

The default value is: txt-output = txt.gz

Keyword skip

skip = integer

The number of initial cycles left out of the analysis, as an input of PyRETIS 3 sets it; set skip_initial_cycles instead. The matched and the wham methods use skip when skip_initial_cycles is not set. A set skip_initial_cycles, 0 included, and pyretis analyse -skipb N, -skipb 0 included, take its place. An input that sets skip_initial_cycles = 0 next to a skip above 0 is refused by pyretis analyse unless -skipb N gives the number.

Default

The default value is 0.

Keyword tau_ref_bin

tau_ref_bin = list of floats

The order-parameter bin the permeability analysis takes as its reference region when it computes \(\tau\), given as the two edges of the bin. See Studying permeability with PyRETIS for a worked example.

Default

The default value is [].