longitudinal/REGISTRATION_BENCHMARK_RIGID.md

Registration Pipeline Benchmark — CT → T1c (M6)

Benchmarks six cross-modality registration pipelines — all rigid (6-DOF) — that align a head-and-neck CT to a tightly-cropped T1c MRI so the GTV can be warped onto the T1c (MR) native grid. This is the registration step of scripts/02_build_m6_dataset.py. The pairs are hard: different modality (CT Hounsfield vs MR intensity) and the patient is repositioned between the CT and MR sessions, so the inter-session head tilt can reach ~30°.

• Pipelines: src/registration_pipelines.py
• Benchmark harness: scripts/bench_registration.py
• Results: results/bench_registration/ (per-pipeline JSON, summary.json, summary.md, bench_registration.png)

Unified contract

Every pipeline returns a SimpleITK transform T such that

sitk.Resample(ct, t1c, T, sitk.sitkLinear)              # CT  onto the MR grid
sitk.Resample(tv,  t1c, T, sitk.sitkNearestNeighbor, 0) # GTV onto the MR grid

places the CT / GTV concentric with the MR head. In Resample's destination→source convention this T maps T1c → CT. Shared rules for every pipeline:

• No mask, no crop / pad: registration runs on the full input volumes exactly as given (no FixedMask / MovingMask, no head-box resampling, no padding).
• The T1c (MR) is always the fixed image (moving = CT) for every pipeline. (A CT clipped to a non-negative window reads as MR, so the T1c slot decides; both-CT pairs keep the given order.)
• Every transform is rigid (6-DOF) — Euler / rigid-matrix parameterizations per toolkit. No affine, no deformable.

All pipelines also use each toolkit's native automatic transform initialization (no manual centroid offset): elastix / impact use AutomaticTransformInitialization (CenterOfGravity), antspyx its default center-of-mass alignment, simpleitk starts from identity, niftyreg (reg_aladin) initializes from the image centres, synthmorph needs none. One consequence of the no-mask rule: the IMPACT metric only evaluates samples that map inside the moving buffer. With fixed=T1c the samples are drawn from the tight T1c head crop (fixed, ~220×220×150 mm) and map mostly inside the large CT head-neck FOV (moving, ~300×300×251 mm), so the gate rarely fires; the pipeline still sets RequiredRatioOfValidSamples 0.1 (vs the build default 0.25) as a safety override for the FOV mismatch — see the IMPACT section. The comparison is on each toolkit's native optimizer / metric / global-search capability.

The six pipelines

Listed alphabetically. All register on the full volumes (no head box, no mask, no crop / pad) and use each toolkit's native automatic transform initialization (no manual centroid offset): elastix / impact use elastix's AutomaticTransformInitialization (CenterOfGravity), antspyx uses its default center-of-mass alignment, simpleitk starts from identity, niftyreg initializes from the image centres, and synthmorph needs none (global deep method). All use fixed = T1c, moving = CT (shared rule above). niftyreg and synthmorph clip the CT (the moving image) to [0, 80] HU before registration (per the CT-clipping A/B below; clip_ct=None disables it).

#	pipeline	fixed / moving	init	metric	transform	status in this env
1	antspyx	T1c / CT	auto COM	Mattes MI (multi-res)	ANTs Rigid	✅ ran (main env; stochastic MI subsampling)
2	elastix	T1c / CT	auto CenterOfGravity	Mattes MI (32 bins)	EulerTransform rigid (5.x), ASGD, 3 resolutions	✅ ran (main env, in-process python API — no CLI)
3	impact	T1c / CT	auto CenterOfGravity	IMPACT semantic (MIND R1D2_3D L1 + TotalSegmentator M852/M850 deepest-layer Dice, per-level weighted, Static) + Mattes MI (w 0.4)	EulerTransform rigid, ASGD, 4 scales	✅ ran (Docker vboussot/elastix_impact + GPU — see IMPACT section)
4	niftyreg	T1c / CT	image centres	NCC block matching + trimmed least squares (symmetric)	reg_aladin -rigOnly, 3-level pyramid	✅ ran (Docker niftyreg:2.1.1-ubuntu, GPU -platf 1)
5	simpleitk	T1c / CT	identity	Mattes MI (32 bins)	Euler3D rigid, pyramid [4,2,1]	✅ ran
6	synthmorph	T1c / CT	none (global)	SynthMorph anatomy-aware contrast-invariant (deep, pretrained)	6-DOF rigid matrix	✅ ran (main env, Docker + GPU)

Benchmark budget (best-results settings): elastix and simpleitk run with MaximumNumberOfIterations = 2000 (per resolution / per pyramid level), with the sampler and metric left at their defaults (NumberOfSpatialSamples 5000, NumberOfHistogramBins 32). impact runs the MIND + TotalSegmentator static feature combination of the ImpactReg MR_CT_TS reference parameter map (per VBoussot/ImpactReg) as a rigid EulerTransform: 4 levels, per-level voxel sizes 6/3/2/2 mm (no image-resampling pyramid), per-level MaximumNumberOfIterations 400/300/200/200, NumberOfSpatialSamples 2000,

• Mattes MI weight 0.4, CenterOfGravity auto-init, RequiredRatioOfValidSamples 0.1. antspyx, niftyreg and synthmorph keep their native defaults.

simpleitk has two budgets: --simpleitk-mode benchmark (single pyramid, identity start — the default, comparable to the other tools) and --simpleitk-mode production (the full multi-start tilt grid from 02_build_m6_dataset.py, now also on the full volumes per the no-crop rule, ~20× slower, the most robust but not used for the head-to-head).

Test cases

First 10 labeled M6 cases (--n-cases 10 --sample first), reproducible. 9 have a complete CT; 1 (m6_22GD5VL2_20230724) has an incomplete CT head and is reported as incomplete-CT for every pipeline (a data issue, not a pipeline failure).

m6_22GD5VL2_20230724  (incomplete CT)
m6_22IRRKU6_20240115  m6_23XIMO4M_20230222  m6_24V62QQM_20230223
m6_24V62QQM_20231017  m6_24V62QQM_20240820  m6_26TYQI3C_20221027
m6_2BBN5XA4_20250411  m6_2C4TKI6H_20240506  m6_2CDNN7BH_20240102

Metrics (identical for every pipeline)

Alignment is scored two ways, both as the registration's achieved metric read off as a standalone score: the intensity-based Mattes MI (mi_ct_mr) and the semantic ImpactLoss (impact_loss):

• mi_ct_mr — mean Mattes MI (nats) between the CT inverse-transformed into MR coordinates (the warped CT on the T1c grid) and the original MR, restricted to the MR head and each image percentile-binned independently (SimpleITK Mattes-MI style). Higher = the warped-CT and MR intensities are better in-sync = better aligned; it collapses toward 0 when the CT is rotated/translated away from the MR.
• impact_loss — the IMPACT semantic loss from vboussot/ImpactLoss (the metric family the impact pipeline registers with): a weighted L1 distance between the deep feature maps of the warped CT and the original MR extracted by the pretrained TotalSegmentator TS/M730.pt TorchScript model (two deepest layers, weights 1/1, no resampling), over the full co-registered volumes. The model standardizes intensity internally (per-image z-score), so both volumes go in raw and the score is invariant to affine intensity scaling. Lower = the two volumes carry more in-sync anatomy = better aligned; it rises when the CT is rotated/translated away from the MR.
• reg_wall_s — wall-clock registration time (excludes I/O + scoring).

Tables and plots report mi_ct_mr and impact_loss (plus registration time).

The production geometric QA is still computed per case and kept in the JSON (it drives dataset acceptance in 02_build_m6_dataset.py, but its summary numbers were found to correlate poorly with perceived alignment, so they are no longer reported in the tables/plots):

• skull_ring_gap_mm — march rays from the MR head center; the gap between the MR head silhouette boundary and the warped-CT skull ring (p97 of positive CT) at mid and low levels, plus the angular std. mid measures translational concentricity; std measures rotational concentricity.
• tv_nvox — warped GTV volume.
• accepted — the production acceptance rule of 02_build_m6_dataset.py (warped GTV non-empty and concentric with the head; aligned ⇔ |mid| ≤ 20 mm and std ≤ 12 mm — a small negative mid is fine: on full-volume registration the CT skull ring can land slightly inside the MR silhouette, which is equally concentric).

Results (10 cases, no clip)

Alignment is scored by the achieved Mattes MI (nats, warped-CT ~ MR over the MR head; higher = better) and the ImpactLoss semantic score (warped-CT ~ MR over the full volumes; lower = better), with the unclipped CT, so all rows are comparable on the same score scales. Full volumes, no mask / crop / pad, fixed = T1c / moving = CT (shared rule). niftyreg and synthmorph clip the CT (the moving image) to [0, 80] HU before registration (per the CT-clipping A/B below; clip_ct=None disables it); antspyx, elastix, impact and simpleitk register the CT as-is. niftyreg is the v2.1.1 Docker build on the GPU (-platf 1); impact runs the vboussot/elastix_impact Docker image on the GPU (see the IMPACT section). All six transforms are rigid (6-DOF).

pipeline	toolkit	registered	s/reg (med)	ImpactLoss (med/mean)	MI (med/mean, nats)
antspyx	ANTsPy 0.63	9	16.7	0.157 / 0.155	0.652 / 0.650
elastix	ITK-Elastix 0.25.4	9	11.7	0.157 / 0.156	0.646 / 0.638
impact	Docker vboussot/elastix_impact (Elastix 5.2 + IMPACT, rigid MIND+TS static config)	9	111.2	0.157 / 0.156	0.652 / 0.641
niftyreg	NiftyReg 2.1.1 (reg_aladin, Docker + CUDA)	9	15.4	0.166 / 0.170	0.292 / 0.338
simpleitk	SimpleITK 2.5.6	9	10.8	0.156 / 0.156	0.647 / 0.642
synthmorph	Docker SynthMorph (TensorFlow)	9	22.2	0.167 / 0.170	0.277 / 0.328

Per-case MI (nats, unclipped score; bold = best for the case). All rows are re-runs under the fixed = T1c / moving = CT convention (the shared rule); the CT-fixed-era numbers and the retired konfai-preset run are in the git history / archived JSONs. The impact column is the Docker rigid MIND+TS static config (the CT-fixed-era Docker run is archived in results/bench_registration/impact_docker_legacy.json).

case	antspyx	elastix	impact	niftyreg	simpleitk	synthmorph
m6_22GD5VL2_20230724	incomplete-CT	incomplete-CT	incomplete-CT	incomplete-CT	incomplete-CT	incomplete-CT
m6_22IRRKU6_20240115	0.648	0.644	0.645	0.292	0.647	0.277
m6_23XIMO4M_20230222	0.655	0.652	0.654	0.632	0.654	0.616
m6_24V62QQM_20230223	0.703	0.698	0.700	0.260	0.704	0.256
m6_24V62QQM_20231017	0.602	0.590	0.596	0.493	0.597	0.484
m6_24V62QQM_20240820	0.652	0.646	0.652	0.118	0.638	0.111
m6_26TYQI3C_20221027	0.718	0.650	0.658	0.609	0.670	0.590
m6_2BBN5XA4_20250411	0.577	0.577	0.572	0.069	0.579	0.061
m6_2C4TKI6H_20240506	0.645	0.638	0.639	0.233	0.636	0.231
m6_2CDNN7BH_20240102	0.655	0.650	0.655	0.340	0.652	0.323

Per-case ImpactLoss (semantic, unclipped score; bold = best (lowest) for the case):

case	antspyx	elastix	impact	niftyreg	simpleitk	synthmorph
m6_22GD5VL2_20230724	incomplete-CT	incomplete-CT	incomplete-CT	incomplete-CT	incomplete-CT	incomplete-CT
m6_22IRRKU6_20240115	0.1507	0.1508	0.1510	0.1656	0.1504	0.1664
m6_23XIMO4M_20230222	0.1565	0.1563	0.1564	0.1570	0.1564	0.1575
m6_24V62QQM_20230223	0.1466	0.1470	0.1469	0.1664	0.1467	0.1665
m6_24V62QQM_20231017	0.1520	0.1539	0.1534	0.1577	0.1532	0.1576
m6_24V62QQM_20240820	0.1566	0.1571	0.1567	0.1840	0.1585	0.1842
m6_26TYQI3C_20221027	0.1554	0.1620	0.1622	0.1623	0.1604	0.1620
m6_2BBN5XA4_20250411	0.1617	0.1612	0.1615	0.1847	0.1616	0.1856
m6_2C4TKI6H_20240506	0.1573	0.1579	0.1580	0.1754	0.1547	0.1757
m6_2CDNN7BH_20240102	0.1591	0.1590	0.1591	0.1734	0.1587	0.1741

Interpretation

• The top four are a statistical tie (median MI 0.646–0.652, median ImpactLoss 0.156–0.157): antspyx, impact, simpleitk and elastix all recover the gross offset on every case, including the two ~30° repositioned ones (m6_2BBN5XA4 0.572–0.579, m6_24V62QQM_20240820 0.638–0.652). The two alignment scores — an intensity histogram and a deep semantic one — agree on the ranking.
• IMPACT (Docker, rigid, T1c-fixed) is in the top tier at median MI 0.652 (0.6523) — best or within 0.006 of the best on 8 of 9 cases (m6_24V62QQM_20230223 0.700, m6_26TYQI3C 0.658 where it trails ANTsPy's 0.718). This is a large step up from the legacy CT-fixed Docker config (median 0.467, archived in impact_docker_legacy.json): the T1c-fixed direction fixes the sample-coverage problem the same way it fixed Elastix (the metric draws samples from the fixed T1c crop and evaluates them where they map into the large CT FOV, instead of the other way round; previously only ~14% of the samples were in-buffer at the auto-init). The m6_2BBN5XA4 ~30° tilt case that defeated every earlier IMPACT variant (0.223 CT-fixed Docker, 0.258 konfai preset) now registers at 0.572. Cost: the slowest of the six (~111 s/case, ~10× the intensity tools). Concentricity is comparable to the MI tools (4/9 accepted by the production rule).
• ImpactLoss corroborates the MI leaderboard from a different metric family. The top four cluster within 0.001 of each other on the median (0.1564–0.1571), and on every case the best (lowest) ImpactLoss belongs to one of them (antspyx 4/9, elastix 3/9, simpleitk 2/9, impact 0/9); niftyreg (0.1664) and synthmorph (0.1665) are clearly higher, and the gap widens on the hard cases (0.184–0.186 for the two tail tools vs 0.155–0.162 for the top four). A semantic score computed by pretrained segmentation features (not the registration's own objective) ranking the pipelines the same way the intensity score does is a strong check that the leaderboard is not an artifact of the MI scale.
• The grossly repositioned cases still separate the leaders from the tail. On m6_2BBN5XA4 / m6_24V62QQM_20240820 (the two most repositioned pairs — large inter-session translation; the required rotations are only ~10° per the references, see the SynthMorph investigation below) the top four hold 0.57–0.65, while niftyreg (0.069 / 0.118) and synthmorph (0.061 / 0.111) — the single-start block-matching and pretrained-MRI methods — lock onto local minima. This is exactly why production 02_build_m6_dataset.py uses a multi-start tilt grid.
• NiftyReg (reg_aladin) is second-from-bottom: single-start symmetric block matching is strong on moderate cases (m6_23XIMO4M 0.632; m6_26TYQI3C 0.609) but falls into the same tilt local minima (m6_2BBN5XA4 0.069, m6_24V62QQM_20240820 0.118). The v2.1.1 Docker build on the GPU (-platf 1) is ~15 s/case (vs ~4 min CPU for the conda 1.5.69 build). It clips the CT to [0,80] HU by default (per the A/B below) — neutral on the unclipped score scale (0.296→0.292 median).
• SynthMorph is weakest of the six on these cross-modality CT pairs (median 0.277 with its default CT clip; 0.241 unclipped). The [0,80] brain-window clip of the CT is its only genuine A/B win (+0.036 median, 7/9 cases), so it is the default. It needs no centroid init (global deep method). The low score is not a tilt effect: the required rotations are ≤10.6° (the reference tools' own). The investigation (below) shows the rigid matrix model fails gross-pose estimation case-by-case on the repositioned pairs (19–66 mm translation errors; on m6_24V62QQM_20240820 its transform is worse than no registration at all), fails equally on an in-domain MR→MR pair with a ~55° relative pose, and is not fixed by the affine model — while the joint (affine + deformable) model recovers all three tested failure cases to the best ImpactLoss of the six tools.
• Speed (median wall, 3–4 cases in parallel, 24 threads, T1c-fixed): SimpleITK 10.8 s, elastix 11.7 s, NiftyReg 15.4 s (v2.1.1, GPU block matching), ANTs 16.7 s, SynthMorph 22.2 s (GPU forward pass), IMPACT 111.2 s (Docker, static feature mode, no backprop, 400/300/200/200 rigid ASGD iterations).
• Stochasticity: ANTs' Mattes-MI subsamples the image at random, so its per-case numbers carry run-to-run variance (this table is one draw); SimpleITK's MI (and registration) is deterministic at fixed thread count; the in-process Elastix ASGD is deterministic given its fixed seed; SynthMorph is deterministic; NiftyReg is deterministic on the CPU (conda 1.5.69) but shows small run-to-run variation on the GPU (v2.1.1, ~±0.01–0.05 MI on the weak cases). IMPACT in the Static config re-samples spatial coordinates every iteration, but the feature maps are deterministic given the pose. The earlier Jacobian-mode Docker experiments (archived in results/bench_registration/impact_*.json) resampled patches through the network each iteration, so their per-case numbers are one sample from a distribution — e.g. the MRSegmentator-only Jacobian configs reached 0.631 (500/2000) and 0.591 (2000/5000) at ~5×/~35× the Static cost, and the MIND+TS/MRSeg static variants 0.417/0.406 (~72/~37 s).

CT clipping A/B (--clip-ct 0 80)

Question: does clipping the CT to a brain window ([0, 80] HU — skull and air removed) before registration improve the result? The harness clips the CT before registration AND before scoring (percentile-binned MI is scale-free, so the A/B compares like for like). Same 10 cases, same budgets.

Median achieved MI (nats) and wall-clock (9 complete-CT cases):

pipeline	no clip	clip [0,80]	Δ (median MI)	clip wins (cases by MI)
antspyx	0.653 / 36.6 s	0.466 / 34.7 s	−0.186	0/9
elastix	0.517 / 6.0 s	0.425 / 5.9 s	−0.092	1/9
niftyreg	0.296 / 23.8 s	0.320 / 20.2 s	+0.024	5/9
simpleitk	0.651 / 10.6 s	0.436 / 11.1 s	−0.215	0/9
synthmorph	0.241 / 26.0 s	0.318 / 25.6 s	+0.077	5/9
impact	abort	abort	-	- (historical: at the time both aborted on the old 0.25 sample-validity gate; the gate is now 0.1, A/B to be re-done)

Verdict: no — clipping the CT to [0, 80] HU does not improve the result; do not clip.

• The headline alignment score drops for all three leading intensity pipelines: antspyx 0.653→0.466, simpleitk 0.651→0.436, elastix 0.517→0.425 (median nats; simpleitk and antspyx lose on all 9 cases, elastix on 8/9).
• The two weaker pipelines show small median-MI gains (niftyreg +0.024, synthmorph +0.077) but only mixed case-by-case (5/9), and their geometric QA does not support a real improvement.
• Geometric concentricity (skull-ring gap) is a wash: mean mid/std move by ~1–5 mm in either direction (niftyreg mid 23.6→18.1 the only clear gain; antspyx std 16.4→18.0 the clear loss). Production acceptance even falls (no-clip 4/4/2/4/2 → clip 1/0/1/0/2).
• Mechanism. Clipping is a saturating (non-invertible) intensity map: the skull atom (800+ HU → 80) removes the strongest intensity anchor (skull ↔ MR scalp) and compresses the CT rank distribution, so the percentile-binned MI between the warped CT and the MR is inherently lower for the same pose. The pose itself does not systematically degrade — the head is rigid, so skull-ring concentricity is its fingerprint, and that is unchanged. The MI drop is therefore mostly a score-scale effect, but it is exactly the score the benchmark reports, and there is no geometric benefit to pay it for.
• Scoring note: _skull_ring_gap tests the ring with >= p97 (not >) so the orientation gate and the gap QA also work on a clipped CT (where p97 sits exactly on the clipped-skull atom).
• Decision / adopted default. The pipelines were updated: niftyreg and synthmorph clip the CT to [0, 80] HU before registration (clip_ct=(0.0, 80.0) default, clip_ct=None disables it; MR inputs are never clipped); antspyx / elastix / simpleitk register the CT as-is. Re-checked on the unclipped score scale (the protocol used by the table above): synthmorph keeps a genuine gain (0.241→0.277 median, 7/9 cases), niftyreg is neutral (0.296→0.292; its apparent +0.024 above is mostly the clip-scoring effect). All rows of the main table are scored on the unclipped warped CT, so pipelines remain comparable.

SynthMorph low-performance investigation (2026-09-30)

Question: why is synthmorph weakest (median MI 0.277; 0.06–0.11 on the repositioned cases)? The earlier explanation ("trained on brain MRI, defeated by the ~30° tilt") was checked and turned out to be wrong in its specifics. Everything below was scored with the benchmark's own scoring (bench_registration.py::evaluate); results in results/synthmorph_investigation/, reproducible with scripts/investigate_synthmorph.py (parts A–D, F) and scripts/investigate_synthmorph_mm.py (part E).

0. It is not a pipeline bug. The rigid run is deterministic and reproduces the benchmark JSON exactly (e.g. m6_26TYQI3C MI 0.5904, gap 13.1/4.9). The read-back direction is right (8/9 auto_inverted: false, verified by the fix_orientation skull-ring gate), and the matrix is rigid (det 0.999–1.001, ‖RᵀR−I‖ ≤ 0.0015). The lone auto_inverted: true (m6_2BBN5XA4) is the gate choosing the less bad of two wrong directions — a symptom of the failure, not its cause.

1. The failure is a gross-pose estimation failure, not a tilt failure. Each case's synthmorph transform was decomposed against the simpleitk benchmark reference (MI 0.58–0.70 on all 9 cases; its rotations are 1.4–10.6°, not ~30°):

case	MI ref / identity / rigid	head-centre err (mm)	rot err (deg)
m6_22IRRKU6_20240115	0.647 / 0.250 / 0.277	28.2	6.2
m6_23XIMO4M_20230222	0.654 / 0.246 / 0.616	1.9	2.2
m6_24V62QQM_20230223	0.704 / 0.276 / 0.256	25.6	3.7
m6_24V62QQM_20231017	0.597 / 0.331 / 0.484	3.8	17.7
m6_24V62QQM_20240820	0.638 / 0.274 / 0.111	66.3	5.9
m6_26TYQI3C_20221027	0.670 / 0.397 / 0.590	6.5	10.2
m6_2BBN5XA4_20250411	0.579 / 0.046 / 0.061	61.1	22.1
m6_2C4TKI6H_20240506	0.636 / 0.231 / 0.231	40.6	3.4
m6_2CDNN7BH_20240102	0.652 / 0.197 / 0.323	19.4	3.0

• 6/9 cases carry a 19–66 mm head-centre error (translation), and 3/9 a 10–22° rotation error; the two "clean" failures are translation-only.
• On m6_24V62QQM_20240820 the model's transform (MI 0.111) is worse than registering nothing at all (identity MI 0.274); on m6_2C4TKI6H it is exactly identity-level (0.231 = 0.231). The net gain over no-registration is ≤0.03 on 5/9 cases.
• The error vectors are case-specific, not a systematic bias: the fraction of the required network-space translation that goes uncorrected is ≈0 on the successes (23XIMO4M 0.07, 20231017 −0.01), ≈1.0 on m6_24V62QQM_20240820 (the model outputs ≈ its network-space identity — the null that aligns the two volume centres), and −1.7 on m6_2BBN5XA4 (it moves the head 1.7× the required amount, in the wrong direction).

2. Cross-modality is not the (only) cause. An in-domain control — two T1c scans of the same subject 8 months apart (m6_24V62QQM, both tight head crops, no CT, no HU issue) — has a true relative pose of ~55–57° (benchmark simpleitk mode 55.3°, production multi-start tilt grid 56.6°, head centres agreeing to 1.2 mm) and SynthMorph rigid fails it equally: 18.5° rotation error, 18 mm head-centre error, MI 0.361 vs the reference's 0.689 (ImpactLoss 0.140 vs 0.114).

3. The affine matrix model does not help. Same 9 cases, same budget: MI within ±0.025 of rigid on 8/9 cases (m6_23XIMO4M: −0.096, the one case where rigid succeeded); the linear part is a near-pure rotation (scale 0.98–1.03, anisotropy ≤2%) — the extra 6 DOF are essentially unused.

4. The joint (affine + deformable) model recovers the failures. Run directly in Docker (-m joint, outside the rigid contract; the tool's own -o resampling warps the CT onto the T1c grid, scored as-is) on the three worst cases:

case	MI rigid → joint	ImpactLoss joint (vs best intensity tool)
m6_24V62QQM_20230223	0.256 → 0.529	0.133 (simpleitk 0.147)
m6_24V62QQM_20240820	0.111 → 0.481	0.132 (antspyx 0.157)
m6_2BBN5XA4_20250411	0.061 → 0.433	0.129 (antspyx 0.162)

The model family's anatomy is fine — its semantic alignment on the two grossly repositioned cases is the best of the six tools — but the matrix head (6- or 12-DOF) cannot estimate the gross inter-session pose, and the deformable field absorbs what it misses (~40 s/case on the GPU).

5. Pipeline-side suspects were ruled out.

• Network-space extent truncation: minor. Each volume is resampled into its own 256 mm isotropic box (centred on its own volume centre, per the image code); with the head placed in CT space via the reference transform, only 0–6.5% of head voxels fall outside the CT box (0% T1c). Re-centring the CT volume on the head (crop to head bbox + 15 mm) changes nothing (ΔMI ≤ 0.012, Δhead-centre-err ≤ 1.2 mm on all 9 cases) — the model is not anchored to the volume centre.
• Min-max normalization: a real, quantified cross-modality penalty. The network min-max-normalizes each volume; over the raw CT's full HU range (−3024…+3071, air/metal) the brain (0–80 HU) collapses to a near-flat 0.496–0.509 band (air sits at ~0.33, the skull at 0.63–1.0), so the parenchyma carries almost no internal contrast; the T1c brain spans ~0–0.6 (p95 0.26–0.58), so the raw CT reads as a flat-gray blob with a bright ring. The [0,80] clip restores full-range brain contrast (this is the mechanism behind the CT-clip A/B win, +0.036–0.077 MI). Clipping does not make the model domain-typical, which is why it only partially helps.

Conclusion. SynthMorph's matrix models are a local pose prior: on the in-distribution cases (small relative pose) they estimate the M6 alignment well (23XIMO4M 1.9 mm / 2.2°, 20231017 3.8 mm / 17.7°, 26TYQI3C 6.5 mm / 10.2° residual), but the M6 inter-session repositioning (tens of mm translation and, in the same-patient control, a 55° world-frame rotation) is outside the range the pretrained model (trained on same-subject MRI pairs) reliably inverts, and the output is then case-specifically wrong — do-nothing, half-done, or opposite. It is a valid data point for how far a pretrained-MRI deep matrix model generalizes out-of-domain, and it should not be expected to compete with the intensity tools on repositioned CT pairs in rigid mode. If a SynthMorph variant is wanted for these data, the joint model is the one that works (at the cost of leaving the rigid-transform contract).

Reproducing (needs the freesurfer/synthmorph image + GPU; ~15 min):

source /opt/conda/etc/profile.d/conda.sh && conda activate longitudinal-313
python scripts/investigate_synthmorph.py --parts ABCD --workers 3  # A ref+rigid, B affine, C joint, D network-space
python scripts/investigate_synthmorph.py --parts F --workers 3     # re-centred CT probe (needs ABCD first)
python scripts/investigate_synthmorph_mm.py --gpu 0                # in-domain MR->MR control

Reproducing

# main env — SimpleITK + Elastix + ANTsPy + NiftyReg (niftyreg via Docker v2.1.1)
source /opt/conda/etc/profile.d/conda.sh && conda activate longitudinal-313
python scripts/bench_registration.py --pipelines "simpleitk elastix antspyx niftyreg" \
    --n-cases 10 --workers 4 --threads 24 --simpleitk-mode benchmark
# force the conda 1.5.69 CPU build: --niftyreg-backend conda

# impact (Docker vboussot/elastix_impact; GPU; ~110-120 s/case; see the IMPACT
# section). --impact-iters / --impact-samples override the reference map's
# per-level budget (one iters value for all levels)
python scripts/bench_registration.py --pipelines impact --n-cases 10 \
    --workers 3 --threads 24

# synthmorph (Docker, needs the pulled image; see the SynthMorph section)
python scripts/bench_registration.py --pipelines synthmorph --n-cases 10 \
    --workers 4 --threads 24

# A/B: does clipping the CT to a brain window help? (--clip-ct clips the CT
# for ALL pipelines, before registration and scoring; niftyreg/synthmorph
# already clip by default, so this only affects antspyx/elastix/simpleitk)
python scripts/bench_registration.py --n-cases 10 --workers 4 --threads 24 \
    --out-dir results/bench_reg_noclip
python scripts/bench_registration.py --n-cases 10 --workers 4 --threads 24 \
    --clip-ct 0 80 --out-dir results/bench_reg_clip0_80
python scripts/bench_registration.py --merge --out-dir results/bench_reg_noclip
python scripts/bench_registration.py --merge --out-dir results/bench_reg_clip0_80

# combine the per-pipeline JSONs accumulated across envs into one table + plot
python scripts/bench_registration.py --merge

Other flags: --sample random --seed 0 (reproducible random 10), --keys k1,k2,... (explicit cases), --n-cases N, --workers W, --threads T (also the OpenMP thread count for reg_aladin), --simpleitk-mode {benchmark,production}, --synthmorph-model {rigid,affine}, --synthmorph-gpu <int|-1>, --synthmorph-extent <int>, --impact-gpu <int|-1>, --impact-iters N (one MaximumNumberOfIterations value for all levels; default the reference map's 400/300/200/200), --impact-samples N (NumberOfSpatialSamples; default 2000), --out-dir <dir>, --qa-images / --qa-out <dir> (visual QA, below).

Visual QA images (checkerboard + color overlay)

--qa-images writes a checkreg-style PNG per (pipeline, case) to <out-dir>/qa/<pipeline>__<case>.png for eyeballing the alignment. Each image is a 3×2 grid: rows are the three orthogonal mid-slices through the head center (mid-axial, mid-coronal, mid-sagittal), and columns are:

• left — checkerboard blend: alternating blocks of the CT inverse-transformed into MR coordinates and the original MR; a clean registration gives a seamless mosaic, a misregistration shows hard edges at the block boundaries.
• right — additive color overlay: CT in red, MR in green. Structures that coincide blend to yellow; rotation/translation error appears as red/green fringing at the skull and brain edges.

The CT is shown in a brain window (L40/W80 = 0–80 HU) so the brain parenchyma is visible and comparable to the MR (the skull, ~800+ HU, clips to white); the MR uses a 2–98 % head window. The coronal and sagittal views are oriented superior-up to match the axial view.

The title carries the pipeline, case key and the achieved Mattes MI, so a glance tells you whether the visuals agree with the score. Generate for the available pipelines with:

source /opt/conda/etc/profile.d/conda.sh && conda activate longitudinal-313
python scripts/bench_registration.py \
    --pipelines "simpleitk elastix antspyx niftyreg" --n-cases 10 --qa-images
python scripts/bench_registration.py \
    --pipelines "synthmorph impact" --n-cases 10 --qa-images

(9 images each; the incomplete-CT case is skipped.)

SynthMorph pipeline (Docker) — one-time setup

The synthmorph pipeline runs the freesurfer/synthmorph image: SynthMorph (Hoffmann et al., IEEE TMI 2022), an anatomy-aware, acquisition-agnostic deep registration (TensorFlow VxmAffineFeatureDetector with the pretrained synthmorph.rigid.1.h5 weights). No skull-stripping or intensity preprocessing is needed: each volume is normalized to an isotropic 1-mm LIA network space internally. It is the second Docker-based pipeline and the only deep-learning one.

Prerequisites (check with RP.availability(); a missing piece raises PipelineUnavailable with the exact command):

# 1. Docker (with NVIDIA container toolkit so --gpus works)
docker --version

# 2. Pull the image (~11 GB, one-time)
docker pull freesurfer/synthmorph

No extra model assets: the rigid weights ship inside the image (/freesurfer/models).

How the adapter drives the image

1. Rigid matrix model, no init — register -m rigid on GPU when one is visible (auto-spread over the host GPUs by pid; --synthmorph-gpu -1 forces CPU). SynthMorph is a global deep method, so it recovers the gross inter-session offset on its own and needs no initial transform.
2. Read-back — the model estimates first→second, so the pair is passed as (fixed=T1c, moving=CT) and the -T LTA (a LINEAR_RAS_TO_RAS 4×4 in shared world space) is the T1c→CT contract transform directly; it is parsed into a SimpleITK AffineTransform and run through the usual fix_orientation gate.
3. GPU sharing — the container is run as the host uid (clean scratch removal, TF still reaches the GPU) with TF_FORCE_GPU_ALLOW_GROWTH set so concurrent benchmark workers can share a GPU.

NiftyReg pipeline — one-time setup

The niftyreg pipeline runs reg_aladin from the NiftyReg suite (KCL-BMEIS, the actively-maintained NiftyReg): symmetric block matching + trimmed least squares (Modat et al., J. Med. Img. 1(2) 024003, 2014). Instead of a histogram MI objective it matches 4-voxel image blocks by normalised cross-correlation at each level of a 3-level Gaussian pyramid and solves a robust (trimmed) least-squares for the rigid transform — a genuinely different algorithmic family, and cross-modality tolerant because NCC is invariant to intensity scaling.

Default backend: the official v2.1.1 release in Docker. The v2.x release binaries need GLIBC_2.34, which this host (Ubuntu 20.04, glibc 2.31) lacks, so the v2.1.1 Ubuntu zip is packaged in the niftyreg:2.1.1-ubuntu image (nvidia/cuda ubuntu22.04 base for the CUDA plugin). GPU runs use docker run --gpus all + -platf 1 -gpuid N and are ~20× faster than CPU; the v2.1.1 CUDA path supports linear interpolation only (-interp 1), so GPU runs use linear and CPU/conda runs use cubic (-interp 3) — the poses they recover on these cases agree to ~4 decimal places. The conda-forge build (1.5.69, CPU) remains the fallback when the Docker image is absent (--niftyreg-backend conda).

Prerequisites (check with RP.availability(); a missing piece raises PipelineUnavailable with the exact command):

# default: build the v2.1.1 Docker image (one-time)
docker build -t niftyreg:2.1.1-ubuntu niftyreg/   # from NiftyReg-Ubuntu-v2.1.1.zip

# fallback: conda-forge build (provides reg_aladin + reg_f3d, NiftyReg 1.5.69)
conda install -c conda-forge niftyreg

How the adapter drives the tool

1. Rigid 6-DOF, full volumes — reg_aladin -ref T1c -flo CT -rigOnly (the T1c is the reference/fixed image, the CT the floating/moving one; the CT is clipped to [0,80] HU first). Symmetric variant on by default, which matters: the non-symmetric mode drifts grossly on these repositioned pairs. -ln 3 pyramid, -maxit 15 TLS iterations per level (the tool's default of 5 is too coarse), -interp 1 (GPU) / -interp 3 (CPU) internal warping, default initialisation (image centres). GPU: --gpus all, -platf 1, -gpuid <auto by pid>; CPU/conda: -platf 0 (v2.1.1) or no -platf (1.5.69).
2. Read-back — the output affine (-aff) is a 4×4 row-major matrix, documented as Affine * Reference = Floating; the adapter parses it into a SimpleITK AffineTransform and runs it through the usual fix_orientation gate (skull-ring concentricity), which resolves the direction empirically rather than trusting the docs. Under -ref T1c -flo CT the file's affine comes out the other way, so the gate inverts it on the M6 cases (8/9 auto_inverted: True) to the contract transform (T1c→CT).

IMPACT pipeline (Docker vboussot/elastix_impact) — one-time setup

The impact pipeline runs the vboussot/elastix_impact image: Elastix 5.2 with the IMPACT semantic similarity metric (vboussot/ImpactLoss) as a native Elastix metric. The parameter map is the MIND + TotalSegmentator static feature combination of the MR_CT_TS reference map in the VBoussot/ImpactReg repo, with a rigid 6-DOF EulerTransform in place of the reference's RecursiveBSpline (its bending-energy penalty is zero for a rigid transform, so it is dropped) — i.e. the reference metric/budget, rigid-only, on the GPU.

Prerequisites (check with RP.availability(); a missing piece raises PipelineUnavailable with the exact command):

# 1. Docker (with NVIDIA container toolkit so --gpus works)
docker --version

# 2. Pull the image (~38 GB, one-time)
docker pull vboussot/elastix_impact

# 3. Feature models under the cache (the pipeline prints the exact
#    curl commands if any are missing):
#    data/impact_models/MIND/R1D2_3D.pt   (~15 KB)
#    data/impact_models/TS/M852.pt        (~200 MB)
#    data/impact_models/TS/M850.pt        (~200 MB)
#    (from HF VBoussot/impact-torchscript-models, or `sh
#     Data/Models/download_models.sh` from the ImpactLoss repo)

How the adapter drives the image

1. Full volumes, fixed=T1c (MR), moving=CT — the shared fixed-image rule: the metric draws its random spatial samples from the fixed buffer and evaluates them where they map inside the moving buffer. With the tight T1c head crop (~220×220×150 mm) as fixed, the samples map mostly inside the large CT head-neck FOV (~300×300×251 mm), so the RequiredRatioOfValidSamples 0.1 gate (build default 0.25) rarely fires.
2. Parameter map (the rigid MR_CT_TS reference config): (Metric "Impact" "AdvancedMattesMutualInformation") with weights 1.0 / 0.4; per level (coarse → fine) the MIND handcrafted descriptor (MIND/R1D2_3D.pt, L1, subset 32, layer mask "1") combined with TotalSegmentator TS/M852.pt (levels 0-1) / TS/M850.pt (levels 2-3, Dice, subset 64, deepest layers), layer weights 0.2/0.8, 0.3/0.7, 0.6/0.4, 0.7/0.3, at ImpactVoxelSize 6/3/2/2 mm (scaled to the moving spacing); Static mode (whole-image features, no backprop); no image-resampling pyramid (all-1 schedules — the scale is carried by ImpactVoxelSize, per the IMPACT docs); RandomCoordinate sampler; ASGD with per-level MaximumNumberOfIterations 400/300/200/200 and NumberOfSpatialSamples 2000; EulerTransform + AutomaticTransformInitialization (CenterOfGravity) + AutomaticScalesEstimation. A copy of the exact map is dumped to data/m6reg/impact/paramdump/ per run for inspection.
3. GPU auto-spread across the host GPUs by pid (docker run --gpus all, ImpactGPU = the worker's device; --impact-gpu -1 forces CPU and turns mixed precision off).
4. Read-back — the transform comes from Out/TransformParameters.0.itk.txt (an Euler3DTransform in the world physical frame; the full combination transform, with the auto-init folded in by elastix) and is direction-checked by fix_orientation against the skull ring (with fixed=T1c / moving=CT the gate keeps the file's transform, auto_inverted: False).
5. Intensity normalization happens inside the metric / models (no pipeline-side normalization): the MIND descriptor normalizes each 5-voxel neighbourhood to [0, 1] locally (modality-agnostic by construction); the segmentation models are standardized to zero mean / unit variance per the model card. Pre-normalizing the volumes would be an exact no-op for the z-scored models and would only corrupt the HU scale used by MI scoring and the QA brain-window images.

Notes / caveats

• The benchmark compares each toolkit in its native configuration: all register on the full volumes (no head box, no mask, no crop / pad) and use each toolkit's native automatic transform initialization (no manual centroid_offset) — elastix / impact AutomaticTransformInitialization (CenterOfGravity), antspyx's center-of-mass alignment, simpleitk identity, synthmorph none. Production uses --simpleitk-mode production's multi-start tilt grid, which raises SimpleITK's acceptance at ~20× the cost; that is out of scope for the head-to-head but is the config the dataset actually ships with.
• synthmorph is deterministic (one fixed pretrained model, no random subsampling), so its per-case numbers are stable run to run — unlike ANTs / IMPACT. Its low MI (median 0.277 with the default CT clip; 0.06–0.11 on the repositioned cases) is a genuine gross-pose-estimation weakness of the pretrained matrix model (case-by-case, not variance) — see the "SynthMorph low-performance investigation" section above.
• fix_orientation (elastix/ants/impact/niftyreg/synthmorph) chooses between a transform and its inverse by skull-ring concentricity — a correctness gate that makes the result independent of each toolkit's transform-direction convention, so the fixed/moving swap is absorbed by the gate rather than re-derived per toolkit. Under the shared fixed=T1c / moving=CT rule the raw transforms come out in the contract direction for elastix (combination transform = fixed→moving), impact (its TransformParameters.0.itk.txt, the full combination Euler transform) and antspyx (ANTs forward transform) and synthmorph (the -T LTA, model estimates first→second) — completed runs report auto_inverted: False for those; niftyreg's -aff file comes out the other way and the gate inverts it (8/9 auto_inverted: True).
• SynthMorph specifics (Docker freesurfer/synthmorph, TensorFlow, GPU):
  • Gross-pose estimation weakness (measured, not a bug). The rigid matrix model is trained on brain MRI (same-subject pairs, small relative poses). On these repositioned CT↔T1c pairs it estimates the translation wrongly on 6/9 cases (19–66 mm head-centre error; the two worst, m6_2BBN5XA4 / m6_24V62QQM_20240820, at 61 / 66 mm) and the rotation on 3/9 (10–22°) — and the required rotations are only 1.4–10.6°, so it is not a "defeated by tilt" story; on m6_24V62QQM_20240820 its transform scores below the unregistered (identity) one. An in-domain MR→MR control (same subject, ~55° relative pose) fails equally, and the affine model changes nothing, while the joint (affine + deformable) model recovers all three tested failures to the best ImpactLoss of the six tools. Full evidence: the "SynthMorph low-performance investigation" section.
  • No centroid init needed. Being a global deep method it recovers the gross offset on its own; the -i initial-transform path is not used (which also sidesteps this image's float64/float32 bug on the -i code path).
  • GPU sharing. The container runs as the host uid with TF_FORCE_GPU_ALLOW_GROWTH so concurrent benchmark workers share a GPU.
• ~28 s/case (GPU: one pretrained forward pass), between SimpleITK (~11 s) / ANTs (~17 s) and IMPACT (~115 s, rigid docker config).
• NiftyReg specifics (conda-forge niftyreg 1.5.69, reg_aladin, symmetric block matching + TLS, CPU/OpenMP):
  • Different algorithmic family. No histogram MI objective: 4-voxel blocks are matched by normalised cross-correlation and the rigid transform is solved by a trimmed least squares, iterated per pyramid level (-ln 3 -maxit 15). NCC's scale invariance is what makes it cross-modality tolerant.
  • Symmetric mode matters. The default symmetric variant (optimising forward and backward transforms concurrently) is the robust one on these repositioned pairs; -noSym drifts grossly (gap std 45 mm on a clean case every other tool passes).
  • -maxit 5 (the tool default) is too coarse — on the clean case m6_26TYQI3C it reaches MI 0.41 vs 0.61 at 15 TLS iterations/level; 15 is the benchmark setting.
  • GPU is the default now (v2.1.1, -platf 1 via docker run --gpus all): ~20–25 s/case on the full volumes vs ~4 min CPU (120–440 s, -omp thread-bound) for the conda 1.5.69 build. --niftyreg-gpu -1 forces the CPU platform (-platf 0).
  • Version note. The upstream KCL-BMEIS release binaries (v2.x) need glibc ≥ 2.34; this host (Ubuntu 20.04, glibc 2.31) runs them in the niftyreg:2.1.1-ubuntu Docker image (nvidia/cuda ubuntu22.04 base; the CUDA plugin needs libcusolver.so.11, provided by the image). The conda-forge 1.5.69 build remains as the no-Docker fallback and gives essentially identical poses (MI matches to ±0.001 on the 10 cases).
• Elastix specifics (ITK-Elastix 0.25.4, elastix 5.3.1, in-process python API — no CLI binary involved):
  • Elastix 5.x renamed components vs the classic parameter files: the rigid 3D transform is EulerTransform (6-DOF, Rigid3DTransform is gone from the component set), the metric key is Metric (not ImageMetric), the random sampler is Random (not RandomSampler). The adapter uses the 5.x names and is not compatible with elastix ≤ 4.x parameter files.
  • AutomaticTransformInitialization "true" (CenterOfGravity) provides the start (no manual centroid_offset); AutomaticScalesEstimation sets the optimizer scales. An output directory is still set (SetOutputDirectory) for the C++ filter to run; the adapter uses a scratch dir under data/m6reg/elastix/itkelastix/.
  • WriteResultImage must be the string "false" (Python False); the default (absent) makes the filter raise "Could not read result image" after a successful registration.
• With fixed=T1c/moving=CT the combination transform is T1c→CT — the contract direction — so fix_orientation keeps it as-is (auto_inverted: False). The result is recovered by point-mapping (GetCombinationTransform().TransformPoint) into an affine — independent of elastix's Euler-angle parameterization — and wrapped in a SimpleITK AffineTransform(3).
  • Benchmark budget: MaximumNumberOfIterations 2000 per resolution (ASGD 2000/2000/2000 over 3 resolutions) with the sampler/metric at their defaults (NumberOfSpatialSamples 5000, NumberOfHistogramBins 32).
• IMPACT specifics (Docker vboussot/elastix_impact, Elastix 5.2 + IMPACT metric, GPU; rigid MIND+TS static config of the MR_CT_TS reference map):
  • Runs on M6 with the T1c-fixed direction + relaxed sample-validity gate. Full volumes, no mask, no crop / pad, fixed=T1c/moving=CT (the shared rule): the metric's samples (drawn from the tight T1c head crop) map mostly inside the large CT head-neck FOV, so the gate rarely fires. The map still sets RequiredRatioOfValidSamples 0.1 (build default 0.25) as a safety override — a genuinely diverging transform, <10% in-buffer, still aborts. The combination Euler transform comes out in the contract direction (completed runs report auto_inverted: False).
  • Auto init. AutomaticTransformInitialization (CenterOfGravity) aligns the intensity centroids (no manual translation); the reference map's GeometricalCenter start is too crude for the head+neck FOV mismatch.
  • Rigid, like every other pipeline. The reference map's RecursiveBSpline + bending-energy regularizer is replaced by a 6-DOF EulerTransform (the bending penalty is zero for a rigid transform).
  • History: single-init convergence. The semantic landscape has more local minima than Mattes-MI. The T1c-fixed direction fixes most of the hard cases (the repositioned m6_22IRRKU6 / m6_23XIMO4M now register as well as the MI tools do), but the single CenterOfGravity start on the ~30° tilt pair m6_2BBN5XA4 remains a weakness — a multi-start tilt grid (as production SimpleITK uses) would be the next step.
  • Stochastic. The config is Static mode: it re-samples spatial coordinates every iteration, but the feature maps are deterministic given the pose (see Stochasticity). The older Jacobian-mode configs (docker-era experiments, archived in results/bench_registration/impact_*.json) resampled patches through the network each iteration, so their per-case numbers are one sample from a distribution.
  • Models are under the data root, not the repo. The TorchScript feature models live in data/impact_models/ (MIND ~15 KB + TS ~200 MB each, from HF VBoussot/impact-torchscript-models); the pipeline raises with the exact curl commands if one is missing.
  • CPU fallback: --impact-gpu -1 (ImpactGPU -1, mixed precision off — much slower).