Analysis configuration

The analysis workflow reads saved predictions from Inferencer. It selects samples, computes per-sample values, reduces them to summaries, and writes reports and figures. The pipeline is independent of model training and inference.

An analysis configuration requires five lists:

• selectors choose samples or sample groups.

• collectors compute values for each selected sample.

• aggregators reduce collected values to summaries.

• reporters write structured results.

• plotters write figures and table outputs.

Analysis pipeline

Analysis starts with one prediction reader for each inference run. The readers provide prediction records and, when the inference datasource can be loaded, matching structures.

1. Selectors create the sample streams to analyse. Each configured selector is applied independently to each prediction reader. Use a chain selector when several filters must be applied in sequence.

2. Collectors process every sample in each selected stream. Their per-sample values are written to JSONL files under the analysis run’s aux/ directory.

3. Aggregators read the selector streams and collector values and reduce them to summary results such as errors, distributions, rankings, or per-channel statistics.

4. Reporters and plotters consume the combined analysis results. Reporters write structured files, while plotters create figures and tables.

With this pipeline it is easily possible to analyze and compare multiple inference runs.

Available implementations

Selectors

• all and none (IdentitySelector) keep all samples.

• index_list (IndexSelector) selects explicit zero-based indices.

• index_range (IndexRangeSelector) selects an inclusive start to exclusive end range.

• random (BernoulliSelector) keeps samples with probability p.

• element (ElementSelector) selects target sites by chemical element.

• structure_cluster (StructureClusterSelector) partitions structures using a descriptor.

• chain (ChainSelector) applies an ordered sequence of selectors.

Collectors

• loss (LossCollector) computes a configured loss for each sample and can collect an energy-resolved vector.

• descriptor (DescriptorCollector) computes a descriptor for each matched structure.

Aggregators

• scalar (ScalarAggregator) computes scalar summary statistics.

• vector (VectorAggregator) summarises vector values such as energy-resolved losses.

• spectrum (SpectrumAggregator) computes per-channel prediction and target statistics.

• ranking (RankingAggregator) selects best and worst samples using a collected value.

• bias_variance (BiasVarianceAggregator) computes a per-channel bias and variance decomposition.

Reporters

• scalar (ScalarReporter) writes per-sample scalar values as CSV files.

• statistics (StatisticsReporter) writes aggregated values as YAML or JSON.

Plotters

• scalar (ScalarPlotter) plots scalar-value distributions.

• spectra_all (AllSpectraPlotter) plots selected prediction and target spectra.

• spectra_comparison (SpectraComparisonPlotter) compares best and worst samples using sort_key.

• stat_table (StatTablePlotter) writes PDF tables of aggregated statistics.

• stat_table_latex (StatTableLatexPlotter) writes LaTeX sources and optional PDFs.

• energy_resolved_loss (EnergyResolvedLossPlotter) plots energy-resolved loss curves.

• mean_spectrum (MeanSpectrumPlotter) plots mean spectra and best or worst tail means.

• parity (ParityPlotter) plots predicted-versus-target intensity parity.

• bias_variance (BiasVariancePlotter) plots per-channel bias and variance.

• error_correlation (ErrorCorrelationPlotter) plots pairwise error correlations using sort_key.

• pca (PcaPlotter) plots structure groups in descriptor PCA space.

• selector_overview (SelectorOverviewPlotter) compares selector groups using err_key.

Common configuration

The optional top-level fields are seed and preload. preload loads prediction records and matched structures during setup instead of reading them on demand. This can speed up analysis significantly for small test datasets.

Custom modules

Custom modules add analysis outputs beyond the standard selectors, collectors, aggregators, reporters, and plotters. Implementations derive from CustomModule. Each entry selects a registered implementation with custom_module_type.

The tensorboard module (TensorBoardModule) plots training loss, validation loss, and learning rate from TensorBoard event files. Provide one directory for each inference run passed to --inference-runs, in the same order; the module writes individual learning-curve PDFs and a combined comparison PDF.

custom_modules:
  - custom_module_type: tensorboard
    tensorboard_dirs:
      - runs/train_mlp_001/tensorboard
      - runs/train_schnet_001/tensorboard
    y_scale: log

Examples

Independent selector streams:

selectors:
  - selector_type: element
    elements: [Fe]
  - selector_type: random
    p: 0.1

Sequential filtering:

selectors:
  - selector_type: chain
    chain:
      - selector_type: element
        elements: [Fe]
      - selector_type: random
        p: 0.1

A complete combination of analysis components is shown in configs/analyze_example.yaml.

API reference

See also xanesnet.analysis and the schemas under xanesnet/schemas/analysis/ for complete fields and defaults.