pyfli.simulator.calibration#

Calibrate the pyfli simulator (priors, IRF, detector noise) to experimental data.

class CalibrationInputs(data_path, detector_type, model_type, irf_path=None, irf_mode='auto', bg_path=None, mask_path=None, reference_path=None, reference_source='auto', reference_alpha='auto', laser_freq_mhz=80.0, bin_radius=0, bit_depth=None, jitter='auto', mono_fraction=None, prior_inflation=1.0, optimize_irf=True, optimize_noise=True, optimize_priors=True, reference_weight=0.05, n_exp_pixels=20000, n_sim_pixels=2000, n_fit_pixels=1500, n_validation_pixels=6000, n_bank_pixels=20000, bank_rounds=3, max_evals_stage1=120, max_evals_stage2=1500, target_coverage=0.99, n_coverage_pixels=20000, max_coverage_iter=10, seed=0, output_dir=None, selection=<factory>)[source]#

Bases: object

Everything the calibration needs.

Parameters:

  • data_path (str) – Raw 3D decay data (.sdt, .ptu, .tif, .npy, …), loaded via pyfli.io.DataOperations.

  • detector_type (str) – “continuous” (gated/ICCD-like) or “discrete” (TCSPC photon counting).

  • model_type (str) – “bi_exponential” or “mono_exponential”.

  • irf_path (str | None) – Experimental IRF. When omitted the IRF is simulated with pyfli.simulator.IRFGenerator.gaussianExpIRF and its parameters are optimised.

  • mask_path (str | None) – Boolean mask of sample pixels. When omitted, one is derived automatically.

  • reference_path (str | None) – Optional parameter maps (AlliGator folder, SPCImage folder, or a pyfli session .npy file / folder) used to initialise and regularise the priors.

  • target_coverage (float) – Required fraction of experimental pixels inside the simulated distribution for every coverage statistic (raw phasors h=1/2, photons, rising edge, background); see coverage. After the best-fit calibration the config is widened until this is reached, so that the experiment is a subset of what the simulator produces. 0 keeps the best-fit config.

  • n_coverage_pixels (int) – Simulated pixels per coverage check.

  • max_coverage_iter (int) – Maximum number of widening steps.

  • irf_mode (str)

  • bg_path (str | None)

  • reference_source (str)

  • reference_alpha (str)

  • laser_freq_mhz (float)

  • bin_radius (int)

  • bit_depth (int | None)

  • jitter (bool | str)

  • mono_fraction (float | None)

  • prior_inflation (float)

  • optimize_irf (bool)

  • optimize_noise (bool)

  • optimize_priors (bool)

  • reference_weight (float)

  • n_exp_pixels (int)

  • n_sim_pixels (int)

  • n_fit_pixels (int)

  • n_validation_pixels (int)

  • n_bank_pixels (int)

  • bank_rounds (int)

  • max_evals_stage1 (int)

  • max_evals_stage2 (int)

  • seed (int)

  • output_dir (str | None)

  • selection (PixelSelectionCriteria)

data_path: str#
detector_type: str#
model_type: str#
irf_path: str | None = None#
irf_mode: str = 'auto'#
bg_path: str | None = None#
mask_path: str | None = None#
reference_path: str | None = None#
reference_source: str = 'auto'#
reference_alpha: str = 'auto'#
laser_freq_mhz: float = 80.0#
bin_radius: int = 0#
bit_depth: int | None = None#
jitter: bool | str = 'auto'#
mono_fraction: float | None = None#
prior_inflation: float = 1.0#
optimize_irf: bool = True#
optimize_noise: bool = True#
optimize_priors: bool = True#
reference_weight: float = 0.05#
n_exp_pixels: int = 20000#
n_sim_pixels: int = 2000#
n_fit_pixels: int = 1500#
n_validation_pixels: int = 6000#
n_bank_pixels: int = 20000#
bank_rounds: int = 3#
max_evals_stage1: int = 120#
max_evals_stage2: int = 1500#
target_coverage: float = 0.99#
n_coverage_pixels: int = 20000#
max_coverage_iter: int = 10#
seed: int = 0#
output_dir: str | None = None#
selection: PixelSelectionCriteria#
property laser_period_ns: float#
property is_bi: bool#
class CalibrationResult(config: dict, sim_generator_kwargs: dict, state: CalibrationState, mask: ndarray, selection_report: dict, validation: dict, history: DataFrame, output_dir: str, files: dict = <factory>, best_fit: dict = <factory>)[source]#

Bases: object

Parameters:
config: dict#
sim_generator_kwargs: dict#
state: CalibrationState#
mask: ndarray#
selection_report: dict#
validation: dict#
history: DataFrame#
output_dir: str#
files: dict#
best_fit: dict#
make_generator()[source]#
Return type:

SimGenerator

class DetectorSimCalibrator(inputs)[source]#

Bases: object

Parameters:

inputs (CalibrationInputs)

run()[source]#
Return type:

CalibrationResult

exception MaskShapeError[source]#

Bases: ValueError

The mask’s spatial shape differs from the decay cube’s.

class PixelSelectionCriteria(min_signal_photons=50.0, snr_min=5.0, flatness_sigma=5.0, peak_window_bins=None, use_otsu='auto', otsu_min_separability=0.6, hot_pixel_factor=10.0, min_component_size=9, refine_user_mask=False)[source]#

Bases: object

Thresholds that decide whether a pixel carries sample fluorescence or only detector/background noise. See pixel_selection for the tests.

Parameters:

  • min_signal_photons (float)

  • snr_min (float)

  • flatness_sigma (float)

  • peak_window_bins (int | None)

  • use_otsu (str)

  • otsu_min_separability (float)

  • hot_pixel_factor (float)

  • min_component_size (int)

  • refine_user_mask (bool)

min_signal_photons: float = 50.0#
snr_min: float = 5.0#
flatness_sigma: float = 5.0#
peak_window_bins: int | None = None#
use_otsu: str = 'auto'#
otsu_min_separability: float = 0.6#
hot_pixel_factor: float = 10.0#
min_component_size: int = 9#
refine_user_mask: bool = False#
class PooledCalibrator(session_jsons, output_dir, weighting='equal', n_per_exp=4000, n_validation=20000, target_coverage=0.90, max_flatten=8, flatten_factor=0.8, seed=0, folders=None)[source]#

Bases: object

Build one covering simulator config from per-experiment calibrations.

Parameters:

  • session_jsons (dict[str, str]) – experiment name -> calibrated_sim_config.json of that experiment.

  • weighting (str) – “equal” (each experiment counts the same) or “pixels” (by number of sample pixels).

  • target_coverage (float) – Required fraction of every experiment’s pixels inside the simulated phasor clouds (harmonics 1 and 2). Priors are flattened up to max_flatten times to reach it.

  • output_dir (str)

  • n_per_exp (int)

  • n_validation (int)

  • max_flatten (int)

  • flatten_factor (float)

  • seed (int)

  • folders (dict[str, str] | None)

run()[source]#
Return type:

PooledResult

calibrate_folder(folder, detector_type, model_type, output_root=None, mask_folder=None, irf_path=None, extensions=DATA_EXTENSIONS, pattern='*', force=False, combine=True, weighting='equal', verify=True, n_verify_pixels=10000, **calibration_kwargs)[source]#

Calibrate each data file in folder separately, then combine the calibrations.

mask_folder: folder of masks matched to the data files by name (see the module docstring); files without one get an automatic mask. irf_path: one IRF file for all experiments, or None to simulate each experiment’s IRF. calibration_kwargs go to CalibrationInputs for every file (e.g. bin_radius, irf_mode, target_coverage, budgets); mask_path and reference_path are per-experiment and not accepted here. A file whose calibration fails is reported in failed and left out of the combination. verify checks, per experiment, that its pixels are covered by its own values and by the whole combined setup (verify_combined()), simulating n_verify_pixels pixels per check.

Parameters:

  • folder (str)

  • detector_type (str)

  • model_type (str)

  • output_root (str | None)

  • mask_folder (str | None)

  • irf_path (str | None)

  • pattern (str)

  • force (bool)

  • combine (bool)

  • weighting (str)

  • verify (bool)

  • n_verify_pixels (int)

Return type:

dict

combination_generator(setup, sensor_type, mono_fraction=None)[source]#

pyfli.data_cc.ConfigCombinationGenerator for the combined setup (enumerates every combination).

Parameters:

  • setup (dict)

  • sensor_type (str)

  • mono_fraction (float | None)

Return type:

ConfigCombinationGenerator

combine_calibrations(json_paths, output_dir, weighting='equal')[source]#

Build the combined sweep setup from per-experiment calibrated_sim_config.json files (name -> path) and write it to output_dir as combined_sim_setup.json, combined_sim_setup.py (same constants, importable) and irfs/<name>.txt.

weighting: “equal” (every experiment counts the same) or “pixels” (by number of sample pixels). Returns the setup dict.

Parameters:
Return type:

dict

coverage_fractions(exp_stats, sim_stats, q=1e-3)[source]#

Fraction of experimental pixels inside the region populated by the simulation, per statistic.

Parameters:
Return type:

dict[str, float]

covered_pixels(exp_stats, sim_stats, q=1e-3)[source]#

Per experimental pixel and statistic: is it inside the region holding 1 - q of the simulated pixels? Raw phasors use the 2D highest-density region (hdr_covered()); photons, edge and background use the central [q, 1 - q] quantile range.

Parameters:
Return type:

dict[str, ndarray]

detect_clusters(decay, mask, max_components=4, separation=2.0, min_fraction=0.05, use_intensity=False, n_fit=20000, smooth=True, seed=0, max_dip_ratio=0.5)[source]#

Find well-separated pixel populations inside mask; see the module docstring.

Two mixture components stay separate only if Ashman’s D is at least separation and the pixel density projected onto the line joining their means has a real dip between them (_dip_ratio() below max_dip_ratio). A mixture fitted to a few hundred pixels of one skewed population can reach D > 2 by chance, but its projected density has no dip (ratio ~0.7-1), while separate populations give ~0.

Parameters:
Return type:

ClusterResult

ensure_coverage(state, ctx, exp_stats, exp_counts, target=0.99, n_sim=20000, max_iter=10, seed=0)[source]#

Widen state until every statistic in COVERAGE_STATS covers at least target of the experimental pixels (or max_iter widening steps were made).

Returns (state, trace, coverage, sim, sim_stats); trace has one row per iteration with the coverage of each statistic, the distance to the experiment and the widening applied.

Parameters:
Return type:

tuple[CalibrationState, list[dict], dict, dict, dict]

find_data_files(folder, extensions=DATA_EXTENSIONS, pattern='*')[source]#

Raw decay files directly inside folder (not in sub-folders), sorted by name.

Parameters:
Return type:

list[str]

find_session_configs(root, pattern='*')[source]#

name -> calibrated_sim_config.json for every calibrated experiment under root.

Parameters:
Return type:

dict[str, str]

load_calibrated_generator(json_path)[source]#

Rebuild the calibrated SimGenerator from calibrated_sim_config.json (and the calibrated_irf.npy next to it). Use it as:

gen = load_calibrated_generator(".../calibrated_sim_config.json")
sample = gen.simulate_once()          # dict with decay, irf, tau1, tau2, alpha1, ...
Parameters:

json_path (str)

Return type:

SimGenerator

load_combined_setup(path)[source]#

Read combined_sim_setup.json back, restoring the tuples the simulator expects.

Parameters:

path (str)

Return type:

dict

load_experimental_data(data_path, irf_path=None, bg_path=None, mask_path=None, bin_radius=0)[source]#

Load the 3D decay cube (and optional IRF / mask) with pyfli.io.DataOperations and optionally apply pyfli’s spatial binning (FLIBinner). Binning a provided mask is not needed: it stays in the original pixel grid.

Parameters:

  • data_path (str)

  • irf_path (str | None)

  • bg_path (str | None)

  • mask_path (str | None)

  • bin_radius (int)

Return type:

ExperimentalData

load_reference_maps(path, source='auto', alpha_kind='auto')[source]#

Load optional parameter maps from AlliGator (*Map.txt folder), SPCImage (*.asc folder) or pyfli (a session .npy file, or a session folder in which case the first match of PYFLI_REFERENCE_PATTERNS is used).

alpha_kind says whether the source’s fraction is an intensity fraction (pyfli, AlliGator f1_i) or an amplitude fraction (SPCImage a1[%], AlliGator f1_a).

Parameters:
Return type:

ReferenceMaps

load_session_calibration(json_path, name='', folder='')[source]#

Rebuild the calibrated CalibrationState from a calibrated_sim_config.json.

Parameters:
Return type:

SessionCalibration

make_combined_generator(setup_path, run=0, mono_fraction=None, seed=None, exact_ranges=True)[source]#

Simulator for one IRF run of the combined setup: gen.simulate_once() draws a combination of the calibrated values and simulates one pixel with IRF_RUNS[run]. exact_ranges uses that IRF’s own calibrated a_range / b_range and inactive time bins (IRF_RUN_INFO); otherwise the global A_RANGE / B_RANGE.

Parameters:

  • setup_path (str)

  • run (int)

  • mono_fraction (float | None)

  • seed (int | None)

  • exact_ranges (bool)

match_masks(data_files, mask_folder, extensions=MASK_EXTENSIONS, suffixes=MASK_SUFFIXES)[source]#

data file -> mask file for every data file whose name matches a mask in mask_folder: the mask’s name (without extension) is the data file’s name plus one of suffixes (case-insensitive). With several candidates, the exact name wins. Data files without a mask, and masks without a data file, are logged as warnings.

Parameters:
Return type:

dict[str, str]

select_signal_pixels(decay, user_mask=None, criteria=None, adc_ceiling=None)[source]#

Returns (mask, report). report records how many pixels each test removed, the background bins and the per-bin background level, for the calibration log.

Parameters:
Return type:

tuple[ndarray, dict]

simulate_image(json_path, shape=(8, 8), seed=0)[source]#

Simulate an (H, W) image with the calibrated simulator. Returns the decay cube (H, W, T), the per-pixel IRF cube and (H, W) ground-truth maps (tau1, tau2, tau, alpha1, Efficiency, photon_count, h_shift, v_shift_bgp).

Parameters:
Return type:

dict[str, ndarray]

simulate_multicluster_tile(labels_tile, cluster_configs, irf_json, detector_type='discrete', sample_mask_tile=None, seed=0)[source]#

pyfli FLIModelImageGenerator image for a cluster label tile, as in the pyfli multi-ROI example: roi_params follows np.unique(roi_mask). Label 0 (outside the sample) is background with an empty config and is zeroed by the intensity / bool mask.

Parameters:
Return type:

dict

verify_combined(setup_path, n_sim=10000, n_exp=20000, seed=0)[source]#

For every experiment of the combined setup, the coverage (coverage_fractions()) of its sample pixels by

  • own values: the combination made of that experiment’s sweep values, i.e. its own calibration after re-expression on the common supports;

  • combined: the whole combined distribution (all combinations, with weights).

Both use the experiment’s own IRF run. The experimental pixels are re-loaded from the experiment’s data file with its calibration’s bin radius and sample mask. The table is also written next to the setup as coverage_per_experiment.csv.

Parameters:
Return type:

DataFrame

Modules

calibrator

End-to-end calibration: experimental 3D decay cube -> pyfli simulator config.

config

Inputs and constants for calibrating the pyfli simulator against an experimental cube.

coverage

Check that the experimental data lies within what the calibrated simulator produces.

data_io

Load the experimental decay cube, IRF, mask and AlliGator / SPCImage / pyfli maps.

expected_decay

Analytic noise-free expected decay of the pyfli simulator for given lifetimes and IRF.

folder_calibration

Calibrate every experiment in a folder, then combine them.

irf_estimation

IRF template and shift range for the simulator, from a measured IRF or the data.

multicluster

Multi-cluster priors and images, following the pyfli multi-ROI example.

optimizer

Simulation-based calibration of the pyfli simulator config.

pixel_selection

Separate sample pixels from pixels that only record detector noise / background.

pooled_calibration

One simulator config that covers several experiments.

prior_estimation

Beta priors for the pyfli simulator config.

run_calibration

Command-line calibration of the pyfli simulator against an experimental decay cube.

sim_bridge

Build the pyfli simulator config from calibrated quantities and run SimGenerator.

summary_stats

Model-free per-pixel summary statistics and distances between simulated and experimental decays.

sweep_combination

Combine per-experiment calibrations into one sweep-style simulator setup.
On this page
Show Source