833 lines
No EOL
49 KiB
Markdown
833 lines
No EOL
49 KiB
Markdown
# Registration Pipeline Benchmark — CT → T1c (M6)
|
||
|
||
Benchmarks **six cross-modality registration pipelines — all rigid** (6-DOF)
|
||
— that align a head-and-neck CT to a tightly-cropped T1c MRI so the GTV can be
|
||
warped onto the T1c (MR) native grid. This is the registration step of
|
||
`scripts/02_build_m6_dataset.py`. The pairs are hard: different modality (CT
|
||
Hounsfield vs MR intensity) and the patient is repositioned between the CT and
|
||
MR sessions, so the inter-session head tilt can reach ~30°.
|
||
|
||
- Pipelines: [`src/registration_pipelines.py`](src/registration_pipelines.py)
|
||
- Benchmark harness: [`scripts/bench_registration.py`](scripts/bench_registration.py)
|
||
- Results: `results/bench_registration/` (per-pipeline JSON, `summary.json`,
|
||
`summary.md`, `bench_registration.png`)
|
||
|
||
## Unified contract
|
||
|
||
Every pipeline returns a SimpleITK transform `T` such that
|
||
|
||
```python
|
||
sitk.Resample(ct, t1c, T, sitk.sitkLinear) # CT onto the MR grid
|
||
sitk.Resample(tv, t1c, T, sitk.sitkNearestNeighbor, 0) # GTV onto the MR grid
|
||
```
|
||
|
||
places the CT / GTV concentric with the MR head. In `Resample`'s
|
||
destination→source convention this `T` maps **T1c → CT**. Shared rules for
|
||
every pipeline:
|
||
|
||
- **No mask, no crop / pad**: registration runs on the full input volumes
|
||
exactly as given (no `FixedMask` / `MovingMask`, no head-box resampling, no
|
||
padding).
|
||
- **The T1c (MR) is always the fixed image** (moving = CT) for every
|
||
pipeline. (A CT clipped to a non-negative window reads as MR, so the T1c
|
||
slot decides; both-CT pairs keep the given order.)
|
||
- **Every transform is rigid (6-DOF)** — Euler / rigid-matrix parameterizations
|
||
per toolkit. No affine, no deformable.
|
||
|
||
All pipelines also use each toolkit's **native automatic transform
|
||
initialization** (no manual centroid offset): elastix / impact use
|
||
`AutomaticTransformInitialization` (CenterOfGravity), antspyx its default
|
||
center-of-mass alignment, simpleitk starts from identity, niftyreg
|
||
(reg_aladin) initializes from the image centres, synthmorph needs none. One
|
||
consequence of the no-mask rule: the IMPACT metric only evaluates samples that
|
||
map inside the moving buffer. With fixed=T1c the samples are drawn from the
|
||
tight T1c head crop (fixed, ~220×220×150 mm) and map mostly inside the large
|
||
CT head-neck FOV (moving, ~300×300×251 mm), so the gate rarely fires; the
|
||
pipeline still sets `RequiredRatioOfValidSamples 0.1` (vs the build default
|
||
0.25) as a safety override for the FOV mismatch — see the IMPACT section.
|
||
The comparison is on each toolkit's native optimizer / metric / global-search
|
||
capability.
|
||
|
||
## The six pipelines
|
||
|
||
Listed alphabetically. All register on the **full volumes** (no head box, no
|
||
mask, no crop / pad) and use each toolkit's **native automatic transform
|
||
initialization** (no manual centroid offset): elastix / impact use elastix's
|
||
`AutomaticTransformInitialization` (CenterOfGravity), antspyx uses its default
|
||
center-of-mass alignment, simpleitk starts from identity, niftyreg
|
||
initializes from the image centres, and synthmorph needs none (global deep
|
||
method). **All use fixed = T1c, moving = CT** (shared rule above). `niftyreg`
|
||
and `synthmorph` clip the CT (the moving image) to [0, 80] HU before
|
||
registration (per the CT-clipping A/B below; `clip_ct=None` disables it).
|
||
|
||
| # | pipeline | fixed / moving | init | metric | transform | status in this env |
|
||
|---|----------|----------------|------|--------|-----------|--------------------|
|
||
| 1 | `antspyx` | T1c / CT | auto COM | Mattes MI (multi-res) | ANTs `Rigid` | ✅ ran (main env; stochastic MI subsampling) |
|
||
| 2 | `elastix` | T1c / CT | auto CenterOfGravity | Mattes MI (32 bins) | `EulerTransform` rigid (5.x), ASGD, 3 resolutions | ✅ ran (main env, in-process python API — no CLI) |
|
||
| 3 | `impact` | T1c / CT | auto CenterOfGravity | **IMPACT semantic** (MIND `R1D2_3D` L1 + TotalSegmentator `M852`/`M850` deepest-layer Dice, per-level weighted, Static) + Mattes MI (w 0.4) | `EulerTransform` rigid, ASGD, 4 scales | ✅ ran (Docker `vboussot/elastix_impact` + GPU — see IMPACT section) |
|
||
| 4 | `niftyreg` | T1c / CT | image centres | NCC block matching + trimmed least squares (symmetric) | `reg_aladin -rigOnly`, 3-level pyramid | ✅ ran (Docker `niftyreg:2.1.1-ubuntu`, GPU `-platf 1`) |
|
||
| 5 | `simpleitk` | T1c / CT | identity | Mattes MI (32 bins) | `Euler3D` rigid, pyramid `[4,2,1]` | ✅ ran |
|
||
| 6 | `synthmorph` | T1c / CT | none (global) | **SynthMorph** anatomy-aware contrast-invariant (deep, pretrained) | 6-DOF rigid matrix | ✅ ran (main env, Docker + GPU) |
|
||
|
||
Benchmark budget (best-results settings): `elastix` and `simpleitk` run with
|
||
**MaximumNumberOfIterations = 2000** (per resolution / per pyramid level),
|
||
with the sampler and metric left at their defaults (**NumberOfSpatialSamples
|
||
5000**, **NumberOfHistogramBins 32**). `impact` runs the MIND + TotalSegmentator
|
||
**static** feature combination of the ImpactReg `MR_CT_TS` reference parameter
|
||
map (per `VBoussot/ImpactReg`) as a **rigid** `EulerTransform`: 4 levels,
|
||
per-level voxel sizes 6/3/2/2 mm (no image-resampling pyramid), per-level
|
||
**MaximumNumberOfIterations 400/300/200/200**, **NumberOfSpatialSamples 2000**,
|
||
+ Mattes MI weight 0.4, CenterOfGravity auto-init,
|
||
`RequiredRatioOfValidSamples 0.1`. `antspyx`, `niftyreg` and `synthmorph` keep
|
||
their native defaults.
|
||
|
||
`simpleitk` has two budgets: `--simpleitk-mode benchmark` (single pyramid,
|
||
identity start — the default, comparable to the other tools) and
|
||
`--simpleitk-mode production` (the full multi-start tilt grid from
|
||
`02_build_m6_dataset.py`, now also on the full volumes per the no-crop rule,
|
||
~20× slower, the most robust but not used for the head-to-head).
|
||
|
||
## Test cases
|
||
|
||
First 10 labeled M6 cases (`--n-cases 10 --sample first`), reproducible. 9 have
|
||
a complete CT; 1 (`m6_22GD5VL2_20230724`) has an incomplete CT head and is
|
||
reported as `incomplete-CT` for every pipeline (a data issue, not a pipeline
|
||
failure).
|
||
|
||
```
|
||
m6_22GD5VL2_20230724 (incomplete CT)
|
||
m6_22IRRKU6_20240115 m6_23XIMO4M_20230222 m6_24V62QQM_20230223
|
||
m6_24V62QQM_20231017 m6_24V62QQM_20240820 m6_26TYQI3C_20221027
|
||
m6_2BBN5XA4_20250411 m6_2C4TKI6H_20240506 m6_2CDNN7BH_20240102
|
||
```
|
||
|
||
## Metrics (identical for every pipeline)
|
||
|
||
Alignment is scored two ways, both as the registration's *achieved* metric read
|
||
off as a standalone score: the intensity-based **Mattes MI** (`mi_ct_mr`) and
|
||
the semantic **ImpactLoss** (`impact_loss`):
|
||
|
||
- **`mi_ct_mr`** — mean Mattes MI (nats) between the **CT inverse-transformed
|
||
into MR coordinates** (the warped CT on the T1c grid) and the **original MR**,
|
||
restricted to the MR head and each image percentile-binned independently
|
||
(SimpleITK Mattes-MI style). Higher = the warped-CT and MR intensities are
|
||
better in-sync = better aligned; it collapses toward 0 when the CT is
|
||
rotated/translated away from the MR.
|
||
- **`impact_loss`** — the IMPACT semantic loss from
|
||
[vboussot/ImpactLoss](https://github.com/vboussot/ImpactLoss) (the metric
|
||
family the `impact` pipeline registers with): a weighted L1 distance between
|
||
the deep feature maps of the **warped CT** and the **original MR** extracted
|
||
by the pretrained TotalSegmentator `TS/M730.pt` TorchScript model (two
|
||
deepest layers, weights 1/1, no resampling), over the **full co-registered
|
||
volumes**. The model standardizes intensity internally (per-image z-score),
|
||
so both volumes go in raw and the score is invariant to affine intensity
|
||
scaling. Lower = the two volumes carry more in-sync anatomy = better
|
||
aligned; it rises when the CT is rotated/translated away from the MR.
|
||
- **`reg_wall_s`** — wall-clock registration time (excludes I/O + scoring).
|
||
|
||
Tables and plots report `mi_ct_mr` and `impact_loss` (plus registration time).
|
||
|
||
The production geometric QA is still computed per case and kept in the JSON
|
||
(it drives dataset acceptance in `02_build_m6_dataset.py`, but its summary
|
||
numbers were found to correlate poorly with perceived alignment, so they are no
|
||
longer reported in the tables/plots):
|
||
|
||
- **`skull_ring_gap_mm`** — march rays from the MR head center; the gap between
|
||
the MR head silhouette boundary and the warped-CT skull ring (p97 of positive
|
||
CT) at mid and low levels, plus the angular std. `mid` measures
|
||
translational concentricity; `std` measures rotational concentricity.
|
||
- **`tv_nvox`** — warped GTV volume.
|
||
- **`accepted`** — the production acceptance rule of `02_build_m6_dataset.py`
|
||
(warped GTV non-empty and concentric with the head; aligned ⇔ `|mid| ≤ 20 mm`
|
||
and `std ≤ 12 mm` — a small negative `mid` is fine: on full-volume
|
||
registration the CT skull ring can land slightly inside the MR silhouette,
|
||
which is equally concentric).
|
||
|
||
## Results (10 cases, no clip)
|
||
|
||
Alignment is scored by the achieved **Mattes MI** (nats, warped-CT ~ MR over
|
||
the MR head; higher = better) **and** the **ImpactLoss** semantic score
|
||
(warped-CT ~ MR over the full volumes; lower = better), with the
|
||
**unclipped** CT, so all rows are comparable on the same score scales. Full
|
||
volumes, no mask / crop / pad, fixed = T1c / moving = CT (shared rule).
|
||
`niftyreg` and `synthmorph` clip the CT (the moving image) to [0, 80] HU
|
||
*before registration* (per the CT-clipping A/B below; `clip_ct=None`
|
||
disables it); `antspyx`, `elastix`, `impact` and `simpleitk` register the CT
|
||
as-is. `niftyreg` is the v2.1.1 Docker build on the GPU (`-platf 1`);
|
||
`impact` runs the `vboussot/elastix_impact` Docker image on the GPU (see the
|
||
IMPACT section). All six transforms are rigid (6-DOF).
|
||
|
||
| pipeline | toolkit | registered | s/reg (med) | ImpactLoss (med/mean) | MI (med/mean, nats) |
|
||
|---|---|---|---|---|---|
|
||
| **antspyx** | ANTsPy 0.63 | 9 | 16.7 | 0.157 / 0.155 | **0.652 / 0.650** |
|
||
| **elastix** | ITK-Elastix 0.25.4 | 9 | 11.7 | 0.157 / 0.156 | 0.646 / 0.638 |
|
||
| **impact** | Docker `vboussot/elastix_impact` (Elastix 5.2 + IMPACT, rigid MIND+TS static config) | 9 | 111.2 | 0.157 / 0.156 | **0.652 / 0.641** |
|
||
| **niftyreg** | NiftyReg 2.1.1 (reg_aladin, Docker + CUDA) | 9 | 15.4 | 0.166 / 0.170 | 0.292 / 0.338 |
|
||
| **simpleitk** | SimpleITK 2.5.6 | 9 | 10.8 | **0.156 / 0.156** | 0.647 / 0.642 |
|
||
| **synthmorph** | Docker SynthMorph (TensorFlow) | 9 | 22.2 | 0.167 / 0.170 | 0.277 / 0.328 |
|
||
|
||
Per-case MI (nats, unclipped score; **bold** = best for the case). All rows
|
||
are re-runs under the **fixed = T1c / moving = CT** convention (the shared
|
||
rule); the CT-fixed-era numbers and the retired konfai-preset run are in the
|
||
git history / archived JSONs. The `impact` column is the Docker rigid
|
||
MIND+TS static config (the CT-fixed-era Docker run is archived in
|
||
`results/bench_registration/impact_docker_legacy.json`).
|
||
|
||
| case | antspyx | elastix | impact | niftyreg | simpleitk | synthmorph |
|
||
|---|---|---|---|---|---|---|
|
||
| m6_22GD5VL2_20230724 | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT |
|
||
| m6_22IRRKU6_20240115 | **0.648** | 0.644 | 0.645 | 0.292 | 0.647 | 0.277 |
|
||
| m6_23XIMO4M_20230222 | **0.655** | 0.652 | 0.654 | 0.632 | 0.654 | 0.616 |
|
||
| m6_24V62QQM_20230223 | 0.703 | 0.698 | 0.700 | 0.260 | **0.704** | 0.256 |
|
||
| m6_24V62QQM_20231017 | **0.602** | 0.590 | 0.596 | 0.493 | 0.597 | 0.484 |
|
||
| m6_24V62QQM_20240820 | **0.652** | 0.646 | 0.652 | 0.118 | 0.638 | 0.111 |
|
||
| m6_26TYQI3C_20221027 | **0.718** | 0.650 | 0.658 | 0.609 | 0.670 | 0.590 |
|
||
| m6_2BBN5XA4_20250411 | 0.577 | 0.577 | 0.572 | 0.069 | **0.579** | 0.061 |
|
||
| m6_2C4TKI6H_20240506 | **0.645** | 0.638 | 0.639 | 0.233 | 0.636 | 0.231 |
|
||
| m6_2CDNN7BH_20240102 | **0.655** | 0.650 | 0.655 | 0.340 | 0.652 | 0.323 |
|
||
|
||
Per-case ImpactLoss (semantic, unclipped score; **bold** = best (lowest) for
|
||
the case):
|
||
|
||
| case | antspyx | elastix | impact | niftyreg | simpleitk | synthmorph |
|
||
|---|---|---|---|---|---|---|
|
||
| m6_22GD5VL2_20230724 | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT |
|
||
| m6_22IRRKU6_20240115 | 0.1507 | 0.1508 | 0.1510 | 0.1656 | **0.1504** | 0.1664 |
|
||
| m6_23XIMO4M_20230222 | 0.1565 | **0.1563** | 0.1564 | 0.1570 | 0.1564 | 0.1575 |
|
||
| m6_24V62QQM_20230223 | **0.1466** | 0.1470 | 0.1469 | 0.1664 | 0.1467 | 0.1665 |
|
||
| m6_24V62QQM_20231017 | **0.1520** | 0.1539 | 0.1534 | 0.1577 | 0.1532 | 0.1576 |
|
||
| m6_24V62QQM_20240820 | **0.1566** | 0.1571 | 0.1567 | 0.1840 | 0.1585 | 0.1842 |
|
||
| m6_26TYQI3C_20221027 | **0.1554** | 0.1620 | 0.1622 | 0.1623 | 0.1604 | 0.1620 |
|
||
| m6_2BBN5XA4_20250411 | 0.1617 | **0.1612** | 0.1615 | 0.1847 | 0.1616 | 0.1856 |
|
||
| m6_2C4TKI6H_20240506 | 0.1573 | 0.1579 | 0.1580 | 0.1754 | **0.1547** | 0.1757 |
|
||
| m6_2CDNN7BH_20240102 | 0.1591 | **0.1590** | 0.1591 | 0.1734 | 0.1587 | 0.1741 |
|
||
|
||
### Interpretation
|
||
|
||
- **The top four are a statistical tie** (median MI 0.646–0.652, median
|
||
ImpactLoss 0.156–0.157): `antspyx`, `impact`, `simpleitk` and `elastix`
|
||
all recover the gross offset on every case, including the two ~30°
|
||
repositioned ones (`m6_2BBN5XA4` 0.572–0.579, `m6_24V62QQM_20240820`
|
||
0.638–0.652). The two alignment scores — an intensity histogram and a deep
|
||
semantic one — agree on the ranking.
|
||
- **IMPACT (Docker, rigid, T1c-fixed) is in the top tier** at median MI
|
||
0.652 (0.6523) — best or within 0.006 of the best on 8 of 9 cases
|
||
(`m6_24V62QQM_20230223` 0.700, `m6_26TYQI3C` 0.658 where it trails
|
||
ANTsPy's 0.718). This is a large step
|
||
up from the legacy CT-fixed Docker config (median 0.467, archived in
|
||
`impact_docker_legacy.json`): the T1c-fixed direction fixes the
|
||
sample-coverage problem the same way it fixed Elastix (the metric draws
|
||
samples from the fixed T1c crop and evaluates them where they map into the
|
||
large CT FOV, instead of the other way round; previously only ~14% of the
|
||
samples were in-buffer at the auto-init). The `m6_2BBN5XA4` ~30° tilt case
|
||
that defeated every earlier IMPACT variant (0.223 CT-fixed Docker, 0.258
|
||
konfai preset) now registers at 0.572. Cost: the slowest of the six
|
||
(~111 s/case, ~10× the intensity tools). Concentricity is comparable to
|
||
the MI tools (4/9 accepted by the production rule).
|
||
- **ImpactLoss corroborates the MI leaderboard from a different metric
|
||
family.** The top four cluster within 0.001 of each other on the median
|
||
(0.1564–0.1571), and on every case the best (lowest) ImpactLoss belongs to
|
||
one of them (antspyx 4/9, elastix 3/9, simpleitk 2/9, impact 0/9);
|
||
`niftyreg` (0.1664) and `synthmorph` (0.1665) are clearly higher, and the
|
||
gap widens on the hard cases (0.184–0.186 for the two tail tools vs
|
||
0.155–0.162 for the top four). A semantic score computed by pretrained
|
||
segmentation features (not the registration's own objective) ranking the
|
||
pipelines the same way the intensity score does is a strong check that the
|
||
leaderboard is not an artifact of the MI scale.
|
||
- **The grossly repositioned cases still separate the leaders from the
|
||
tail.** On `m6_2BBN5XA4` / `m6_24V62QQM_20240820` (the two most
|
||
repositioned pairs — large inter-session translation; the required
|
||
rotations are only ~10° per the references, see the SynthMorph
|
||
investigation below) the top four hold 0.57–0.65, while niftyreg
|
||
(0.069 / 0.118) and synthmorph (0.061 / 0.111) — the single-start
|
||
block-matching and pretrained-MRI methods — lock onto local minima. This
|
||
is exactly why production `02_build_m6_dataset.py` uses a **multi-start
|
||
tilt grid**.
|
||
- **NiftyReg (reg_aladin) is second-from-bottom**: single-start symmetric
|
||
block matching is strong on moderate cases (`m6_23XIMO4M` 0.632;
|
||
`m6_26TYQI3C` 0.609) but falls into the same tilt local minima
|
||
(`m6_2BBN5XA4` 0.069, `m6_24V62QQM_20240820` 0.118). The v2.1.1 Docker build
|
||
on the GPU (`-platf 1`) is ~15 s/case (vs ~4 min CPU for the conda 1.5.69
|
||
build). It clips the CT to [0,80] HU by default (per the A/B below) —
|
||
neutral on the unclipped score scale (0.296→0.292 median).
|
||
- **SynthMorph is weakest of the six on these cross-modality CT pairs**
|
||
(median 0.277 with its default CT clip; 0.241 unclipped). The [0,80]
|
||
brain-window clip of the CT is its only genuine A/B win (+0.036 median,
|
||
7/9 cases), so it is the default. It needs **no centroid init** (global
|
||
deep method). The low score is *not* a tilt effect: the required rotations
|
||
are ≤10.6° (the reference tools' own). The investigation (below) shows the
|
||
`rigid` matrix model fails gross-pose estimation case-by-case on the
|
||
repositioned pairs (19–66 mm translation errors; on `m6_24V62QQM_20240820`
|
||
its transform is worse than no registration at all), fails equally on an
|
||
in-domain MR→MR pair with a ~55° relative pose, and is not fixed by the
|
||
`affine` model — while the `joint` (affine + deformable) model recovers
|
||
all three tested failure cases to the best ImpactLoss of the six tools.
|
||
- **Speed** (median wall, 3–4 cases in parallel, 24 threads, T1c-fixed):
|
||
SimpleITK 10.8 s, elastix 11.7 s, NiftyReg 15.4 s (v2.1.1, GPU block
|
||
matching), ANTs 16.7 s, SynthMorph 22.2 s (GPU forward pass), **IMPACT
|
||
111.2 s** (Docker, static feature mode, no backprop, 400/300/200/200
|
||
rigid ASGD iterations).
|
||
- **Stochasticity**: ANTs' Mattes-MI subsamples the image at random, so its
|
||
per-case numbers carry run-to-run variance (this table is one draw);
|
||
SimpleITK's MI (and registration) is deterministic at fixed thread count;
|
||
the in-process Elastix ASGD is deterministic given its fixed seed;
|
||
SynthMorph is deterministic; NiftyReg is deterministic on the CPU
|
||
(conda 1.5.69) but shows small run-to-run variation on the GPU (v2.1.1,
|
||
~±0.01–0.05 MI on the weak cases). IMPACT in the **Static** config
|
||
re-samples spatial coordinates every iteration, but the feature maps are
|
||
deterministic given the pose. The earlier Jacobian-mode Docker experiments
|
||
(archived in `results/bench_registration/impact_*.json`) resampled patches
|
||
through the network each iteration, so their per-case numbers are one
|
||
sample from a distribution — e.g. the MRSegmentator-only Jacobian configs
|
||
reached 0.631 (500/2000) and 0.591 (2000/5000) at ~5×/~35× the Static
|
||
cost, and the MIND+TS/MRSeg static variants 0.417/0.406 (~72/~37 s).
|
||
|
||
## CT clipping A/B (`--clip-ct 0 80`)
|
||
|
||
Question: does clipping the CT to a brain window ([0, 80] HU — skull and air
|
||
removed) **before registration** improve the result? The harness clips the CT
|
||
before registration AND before scoring (percentile-binned MI is scale-free,
|
||
so the A/B compares like for like). Same 10 cases, same budgets.
|
||
|
||
Median achieved MI (nats) and wall-clock (9 complete-CT cases):
|
||
|
||
| pipeline | no clip | clip [0,80] | Δ (median MI) | clip wins (cases by MI) |
|
||
|---|---|---|---|---|
|
||
| antspyx | 0.653 / 36.6 s | 0.466 / 34.7 s | **−0.186** | 0/9 |
|
||
| elastix | 0.517 / 6.0 s | 0.425 / 5.9 s | **−0.092** | 1/9 |
|
||
| niftyreg | 0.296 / 23.8 s | 0.320 / 20.2 s | +0.024 | 5/9 |
|
||
| simpleitk | 0.651 / 10.6 s | 0.436 / 11.1 s | **−0.215** | 0/9 |
|
||
| synthmorph | 0.241 / 26.0 s | 0.318 / 25.6 s | +0.077 | 5/9 |
|
||
| impact | abort | abort | - | - (historical: at the time both aborted on the old 0.25 sample-validity gate; the gate is now 0.1, A/B to be re-done) |
|
||
|
||
**Verdict: no — clipping the CT to [0, 80] HU does not improve the result; do
|
||
not clip.**
|
||
|
||
- The headline alignment score drops for **all three leading intensity
|
||
pipelines**: antspyx 0.653→0.466, simpleitk 0.651→0.436, elastix
|
||
0.517→0.425 (median nats; simpleitk and antspyx lose on **all 9** cases,
|
||
elastix on 8/9).
|
||
- The two weaker pipelines show small median-MI gains (niftyreg +0.024,
|
||
synthmorph +0.077) but only mixed case-by-case (5/9), and their geometric
|
||
QA does not support a real improvement.
|
||
- **Geometric concentricity (skull-ring gap) is a wash**: mean mid/std move by
|
||
~1–5 mm in either direction (niftyreg mid 23.6→18.1 the only clear gain;
|
||
antspyx std 16.4→18.0 the clear loss). Production acceptance even falls
|
||
(no-clip 4/4/2/4/2 → clip 1/0/1/0/2).
|
||
- **Mechanism.** Clipping is a saturating (non-invertible) intensity map: the
|
||
skull atom (800+ HU → 80) removes the strongest intensity anchor
|
||
(skull ↔ MR scalp) and compresses the CT rank distribution, so the
|
||
percentile-binned MI between the warped CT and the MR is inherently lower
|
||
*for the same pose*. The pose itself does not systematically degrade — the
|
||
head is rigid, so skull-ring concentricity is its fingerprint, and that is
|
||
unchanged. The MI drop is therefore mostly a score-scale effect, but it is
|
||
exactly the score the benchmark reports, and there is no geometric benefit
|
||
to pay it for.
|
||
- Scoring note: `_skull_ring_gap` tests the ring with `>= p97` (not `>`) so
|
||
the orientation gate and the gap QA also work on a clipped CT (where p97
|
||
sits exactly on the clipped-skull atom).
|
||
- **Decision / adopted default.** The pipelines were updated: `niftyreg` and
|
||
`synthmorph` clip the CT to [0, 80] HU before registration
|
||
(`clip_ct=(0.0, 80.0)` default, `clip_ct=None` disables it; MR inputs are
|
||
never clipped); `antspyx` / `elastix` / `simpleitk` register the CT as-is.
|
||
Re-checked on the **unclipped score scale** (the protocol used by the table
|
||
above): synthmorph keeps a genuine gain (0.241→0.277 median, 7/9 cases),
|
||
niftyreg is neutral (0.296→0.292; its apparent +0.024 above is mostly the
|
||
clip-scoring effect). All rows of the main table are scored on the
|
||
unclipped warped CT, so pipelines remain comparable.
|
||
|
||
## SynthMorph low-performance investigation (2026-09-30)
|
||
|
||
Question: *why* is `synthmorph` weakest (median MI 0.277; 0.06–0.11 on the
|
||
repositioned cases)? The earlier explanation ("trained on brain MRI, defeated
|
||
by the ~30° tilt") was checked and turned out to be wrong in its specifics.
|
||
Everything below was scored with the benchmark's own scoring
|
||
(`bench_registration.py::evaluate`); results in
|
||
`results/synthmorph_investigation/`, reproducible with
|
||
`scripts/investigate_synthmorph.py` (parts A–D, F) and
|
||
`scripts/investigate_synthmorph_mm.py` (part E).
|
||
|
||
**0. It is not a pipeline bug.** The `rigid` run is deterministic and
|
||
reproduces the benchmark JSON exactly (e.g. `m6_26TYQI3C` MI 0.5904, gap
|
||
13.1/4.9). The read-back direction is right (8/9 `auto_inverted: false`,
|
||
verified by the `fix_orientation` skull-ring gate), and the matrix is rigid
|
||
(`det 0.999–1.001`, `‖RᵀR−I‖ ≤ 0.0015`). The lone `auto_inverted: true`
|
||
(`m6_2BBN5XA4`) is the gate choosing the *less bad* of two wrong directions —
|
||
a symptom of the failure, not its cause.
|
||
|
||
**1. The failure is a gross-pose estimation failure, not a tilt failure.**
|
||
Each case's synthmorph transform was decomposed against the `simpleitk`
|
||
benchmark reference (MI 0.58–0.70 on all 9 cases; its rotations are
|
||
**1.4–10.6°**, not ~30°):
|
||
|
||
| case | MI ref / identity / rigid | head-centre err (mm) | rot err (deg) |
|
||
|---|---|---|---|
|
||
| m6_22IRRKU6_20240115 | 0.647 / 0.250 / **0.277** | 28.2 | 6.2 |
|
||
| m6_23XIMO4M_20230222 | 0.654 / 0.246 / **0.616** | 1.9 | 2.2 |
|
||
| m6_24V62QQM_20230223 | 0.704 / 0.276 / **0.256** | 25.6 | 3.7 |
|
||
| m6_24V62QQM_20231017 | 0.597 / 0.331 / **0.484** | 3.8 | 17.7 |
|
||
| m6_24V62QQM_20240820 | 0.638 / 0.274 / **0.111** | 66.3 | 5.9 |
|
||
| m6_26TYQI3C_20221027 | 0.670 / 0.397 / **0.590** | 6.5 | 10.2 |
|
||
| m6_2BBN5XA4_20250411 | 0.579 / 0.046 / **0.061** | 61.1 | 22.1 |
|
||
| m6_2C4TKI6H_20240506 | 0.636 / 0.231 / **0.231** | 40.6 | 3.4 |
|
||
| m6_2CDNN7BH_20240102 | 0.652 / 0.197 / **0.323** | 19.4 | 3.0 |
|
||
|
||
- 6/9 cases carry a 19–66 mm head-centre error (translation), and 3/9 a
|
||
10–22° rotation error; the two "clean" failures are translation-only.
|
||
- On `m6_24V62QQM_20240820` the model's transform (MI 0.111) is **worse than
|
||
registering nothing at all** (identity MI 0.274); on `m6_2C4TKI6H` it is
|
||
exactly identity-level (0.231 = 0.231). The net gain over no-registration
|
||
is ≤0.03 on 5/9 cases.
|
||
- The error *vectors* are case-specific, not a systematic bias: the fraction
|
||
of the required network-space translation that goes uncorrected is ≈0 on
|
||
the successes (23XIMO4M 0.07, 20231017 −0.01), ≈1.0 on `m6_24V62QQM_20240820`
|
||
(the model outputs ≈ its network-space identity — the null that aligns the
|
||
two *volume* centres), and −1.7 on `m6_2BBN5XA4` (it moves the head 1.7× the
|
||
required amount, in the wrong direction).
|
||
|
||
**2. Cross-modality is not the (only) cause.** An in-domain control — two
|
||
T1c scans of the *same* subject 8 months apart (`m6_24V62QQM`, both tight
|
||
head crops, no CT, no HU issue) — has a true relative pose of **~55–57°**
|
||
(benchmark `simpleitk` mode 55.3°, production multi-start tilt grid 56.6°,
|
||
head centres agreeing to 1.2 mm) and SynthMorph `rigid` fails it equally:
|
||
18.5° rotation error, 18 mm head-centre error, MI 0.361 vs the reference's
|
||
0.689 (ImpactLoss 0.140 vs 0.114).
|
||
|
||
**3. The `affine` matrix model does not help.** Same 9 cases, same budget:
|
||
MI within ±0.025 of `rigid` on 8/9 cases (`m6_23XIMO4M`: −0.096, the one case
|
||
where rigid succeeded); the linear part is a near-pure rotation
|
||
(scale 0.98–1.03, anisotropy ≤2%) — the extra 6 DOF are essentially unused.
|
||
|
||
**4. The `joint` (affine + deformable) model recovers the failures.** Run
|
||
directly in Docker (`-m joint`, outside the rigid contract; the tool's own
|
||
`-o` resampling warps the CT onto the T1c grid, scored as-is) on the three
|
||
worst cases:
|
||
|
||
| case | MI rigid → joint | ImpactLoss joint (vs best intensity tool) |
|
||
|---|---|---|
|
||
| m6_24V62QQM_20230223 | 0.256 → **0.529** | **0.133** (simpleitk 0.147) |
|
||
| m6_24V62QQM_20240820 | 0.111 → **0.481** | **0.132** (antspyx 0.157) |
|
||
| m6_2BBN5XA4_20250411 | 0.061 → **0.433** | **0.129** (antspyx 0.162) |
|
||
|
||
The model family's anatomy is fine — its *semantic* alignment on the two
|
||
grossly repositioned cases is the **best of the six tools** — but the matrix
|
||
head (6- or 12-DOF) cannot estimate the gross inter-session pose, and the
|
||
deformable field absorbs what it misses (~40 s/case on the GPU).
|
||
|
||
**5. Pipeline-side suspects were ruled out.**
|
||
|
||
- **Network-space extent truncation: minor.** Each volume is resampled into
|
||
its own 256 mm isotropic box (centred on its own volume centre, per the
|
||
image code); with the head placed in CT space via the reference transform,
|
||
only 0–6.5% of head voxels fall outside the CT box (0% T1c). Re-centring
|
||
the CT volume on the head (crop to head bbox + 15 mm) changes *nothing*
|
||
(ΔMI ≤ 0.012, Δhead-centre-err ≤ 1.2 mm on all 9 cases) — the model is not
|
||
anchored to the volume centre.
|
||
- **Min-max normalization: a real, quantified cross-modality penalty.** The
|
||
network min-max-normalizes each volume; over the raw CT's full HU range
|
||
(−3024…+3071, air/metal) the brain (0–80 HU) collapses to a near-flat
|
||
**0.496–0.509** band (air sits at ~0.33, the skull at 0.63–1.0), so the
|
||
parenchyma carries almost no internal contrast; the T1c brain spans
|
||
~0–0.6 (p95 0.26–0.58), so the raw CT reads as a flat-gray blob with a
|
||
bright ring. The [0,80] clip restores full-range brain contrast (this is
|
||
the mechanism behind the CT-clip A/B win, +0.036–0.077 MI). Clipping does
|
||
not make the model domain-typical, which is why it only partially helps.
|
||
|
||
**Conclusion.** SynthMorph's matrix models are a *local pose prior*: on the
|
||
in-distribution cases (small relative pose) they estimate the M6 alignment
|
||
well (23XIMO4M 1.9 mm / 2.2°, 20231017 3.8 mm / 17.7°, 26TYQI3C 6.5 mm /
|
||
10.2° residual), but the M6 inter-session repositioning (tens of mm
|
||
translation and, in the same-patient control, a 55° world-frame rotation) is
|
||
outside the range the pretrained model (trained on same-subject MRI pairs)
|
||
reliably inverts, and the output is
|
||
then case-specifically wrong — do-nothing, half-done, or opposite. It is a
|
||
valid data point for how far a pretrained-MRI deep matrix model generalizes
|
||
out-of-domain, and it should not be expected to compete with the intensity
|
||
tools on repositioned CT pairs in rigid mode. If a SynthMorph variant is
|
||
wanted for these data, the `joint` model is the one that works (at the cost
|
||
of leaving the rigid-transform contract).
|
||
|
||
Reproducing (needs the `freesurfer/synthmorph` image + GPU; ~15 min):
|
||
|
||
```bash
|
||
source /opt/conda/etc/profile.d/conda.sh && conda activate longitudinal-313
|
||
python scripts/investigate_synthmorph.py --parts ABCD --workers 3 # A ref+rigid, B affine, C joint, D network-space
|
||
python scripts/investigate_synthmorph.py --parts F --workers 3 # re-centred CT probe (needs ABCD first)
|
||
python scripts/investigate_synthmorph_mm.py --gpu 0 # in-domain MR->MR control
|
||
```
|
||
|
||
## Reproducing
|
||
|
||
```bash
|
||
# main env — SimpleITK + Elastix + ANTsPy + NiftyReg (niftyreg via Docker v2.1.1)
|
||
source /opt/conda/etc/profile.d/conda.sh && conda activate longitudinal-313
|
||
python scripts/bench_registration.py --pipelines "simpleitk elastix antspyx niftyreg" \
|
||
--n-cases 10 --workers 4 --threads 24 --simpleitk-mode benchmark
|
||
# force the conda 1.5.69 CPU build: --niftyreg-backend conda
|
||
|
||
# impact (Docker vboussot/elastix_impact; GPU; ~110-120 s/case; see the IMPACT
|
||
# section). --impact-iters / --impact-samples override the reference map's
|
||
# per-level budget (one iters value for all levels)
|
||
python scripts/bench_registration.py --pipelines impact --n-cases 10 \
|
||
--workers 3 --threads 24
|
||
|
||
# synthmorph (Docker, needs the pulled image; see the SynthMorph section)
|
||
python scripts/bench_registration.py --pipelines synthmorph --n-cases 10 \
|
||
--workers 4 --threads 24
|
||
|
||
# A/B: does clipping the CT to a brain window help? (--clip-ct clips the CT
|
||
# for ALL pipelines, before registration and scoring; niftyreg/synthmorph
|
||
# already clip by default, so this only affects antspyx/elastix/simpleitk)
|
||
python scripts/bench_registration.py --n-cases 10 --workers 4 --threads 24 \
|
||
--out-dir results/bench_reg_noclip
|
||
python scripts/bench_registration.py --n-cases 10 --workers 4 --threads 24 \
|
||
--clip-ct 0 80 --out-dir results/bench_reg_clip0_80
|
||
python scripts/bench_registration.py --merge --out-dir results/bench_reg_noclip
|
||
python scripts/bench_registration.py --merge --out-dir results/bench_reg_clip0_80
|
||
|
||
# combine the per-pipeline JSONs accumulated across envs into one table + plot
|
||
python scripts/bench_registration.py --merge
|
||
```
|
||
|
||
Other flags: `--sample random --seed 0` (reproducible random 10),
|
||
`--keys k1,k2,...` (explicit cases), `--n-cases N`, `--workers W`,
|
||
`--threads T` (also the OpenMP thread count for `reg_aladin`),
|
||
`--simpleitk-mode {benchmark,production}`,
|
||
`--synthmorph-model {rigid,affine}`, `--synthmorph-gpu <int|-1>`,
|
||
`--synthmorph-extent <int>`, `--impact-gpu <int|-1>`, `--impact-iters N`
|
||
(one MaximumNumberOfIterations value for all levels; default the reference
|
||
map's 400/300/200/200), `--impact-samples N` (NumberOfSpatialSamples; default
|
||
2000),
|
||
`--out-dir <dir>`, `--qa-images` / `--qa-out <dir>` (visual QA, below).
|
||
|
||
## Visual QA images (checkerboard + color overlay)
|
||
|
||
`--qa-images` writes a `checkreg`-style PNG per (pipeline, case) to
|
||
`<out-dir>/qa/<pipeline>__<case>.png` for eyeballing the alignment. Each image
|
||
is a 3×2 grid: **rows** are the three orthogonal mid-slices through the head
|
||
center (**mid-axial, mid-coronal, mid-sagittal**), and **columns** are:
|
||
|
||
- **left — checkerboard blend:** alternating blocks of the CT inverse-transformed
|
||
into MR coordinates and the original MR; a clean registration gives a seamless
|
||
mosaic, a misregistration shows hard edges at the block boundaries.
|
||
- **right — additive color overlay:** CT in red, MR in green. Structures that
|
||
coincide blend to yellow; rotation/translation error appears as red/green
|
||
fringing at the skull and brain edges.
|
||
|
||
The CT is shown in a **brain window (L40/W80 = 0–80 HU)** so the brain
|
||
parenchyma is visible and comparable to the MR (the skull, ~800+ HU, clips to
|
||
white); the MR uses a 2–98 % head window. The coronal and sagittal views are
|
||
oriented superior-up to match the axial view.
|
||
|
||
The title carries the pipeline, case key and the achieved Mattes MI, so a
|
||
glance tells you whether the visuals agree with the score. Generate for the
|
||
available pipelines with:
|
||
|
||
```bash
|
||
source /opt/conda/etc/profile.d/conda.sh && conda activate longitudinal-313
|
||
python scripts/bench_registration.py \
|
||
--pipelines "simpleitk elastix antspyx niftyreg" --n-cases 10 --qa-images
|
||
python scripts/bench_registration.py \
|
||
--pipelines "synthmorph impact" --n-cases 10 --qa-images
|
||
```
|
||
|
||
(9 images each; the incomplete-CT case is skipped.)
|
||
|
||
## SynthMorph pipeline (Docker) — one-time setup
|
||
|
||
The `synthmorph` pipeline runs the [`freesurfer/synthmorph`](https://hub.docker.com/r/freesurfer/synthmorph)
|
||
image: [SynthMorph](https://synthmorph.io) (Hoffmann et al., IEEE TMI 2022), an
|
||
anatomy-aware, acquisition-agnostic **deep** registration (TensorFlow
|
||
`VxmAffineFeatureDetector` with the pretrained `synthmorph.rigid.1.h5` weights).
|
||
No skull-stripping or intensity preprocessing is needed: each volume is
|
||
normalized to an isotropic 1-mm LIA network space internally. It is the
|
||
second Docker-based pipeline and the only deep-learning one.
|
||
|
||
Prerequisites (check with `RP.availability()`; a missing piece raises
|
||
`PipelineUnavailable` with the exact command):
|
||
|
||
```bash
|
||
# 1. Docker (with NVIDIA container toolkit so --gpus works)
|
||
docker --version
|
||
|
||
# 2. Pull the image (~11 GB, one-time)
|
||
docker pull freesurfer/synthmorph
|
||
```
|
||
|
||
No extra model assets: the rigid weights ship inside the image
|
||
(`/freesurfer/models`).
|
||
|
||
### How the adapter drives the image
|
||
|
||
1. **Rigid matrix model, no init** — `register -m rigid` on GPU when one is
|
||
visible (auto-spread over the host GPUs by pid; `--synthmorph-gpu -1` forces
|
||
CPU). SynthMorph is a global deep method, so it recovers the gross
|
||
inter-session offset on its own and needs **no** initial transform.
|
||
2. **Read-back** — the model estimates first→second, so the pair is passed as
|
||
`(fixed=T1c, moving=CT)` and the `-T` LTA (a `LINEAR_RAS_TO_RAS` 4×4 in
|
||
shared world space) is the T1c→CT contract transform directly; it is
|
||
parsed into a SimpleITK `AffineTransform` and run through the usual
|
||
`fix_orientation` gate.
|
||
3. **GPU sharing** — the container is run as the host uid (clean scratch
|
||
removal, TF still reaches the GPU) with `TF_FORCE_GPU_ALLOW_GROWTH` set so
|
||
concurrent benchmark workers can share a GPU.
|
||
|
||
## NiftyReg pipeline — one-time setup
|
||
|
||
The `niftyreg` pipeline runs `reg_aladin` from the [NiftyReg](https://github.com/KCL-BMEIS/niftyreg)
|
||
suite (KCL-BMEIS, the actively-maintained NiftyReg): **symmetric block
|
||
matching + trimmed least squares** (Modat et al., J. Med. Img. 1(2) 024003,
|
||
2014). Instead of a histogram MI objective it matches 4-voxel image blocks by
|
||
**normalised cross-correlation** at each level of a 3-level Gaussian pyramid
|
||
and solves a robust (trimmed) least-squares for the rigid transform — a
|
||
genuinely different algorithmic family, and cross-modality tolerant because
|
||
NCC is invariant to intensity scaling.
|
||
|
||
**Default backend: the official v2.1.1 release in Docker.** The v2.x release
|
||
binaries need GLIBC_2.34, which this host (Ubuntu 20.04, glibc 2.31) lacks, so
|
||
the v2.1.1 Ubuntu zip is packaged in the `niftyreg:2.1.1-ubuntu` image
|
||
(nvidia/cuda ubuntu22.04 base for the CUDA plugin). GPU runs use
|
||
`docker run --gpus all` + `-platf 1 -gpuid N` and are ~20× faster than CPU;
|
||
the v2.1.1 CUDA path supports **linear interpolation only** (`-interp 1`), so
|
||
GPU runs use linear and CPU/conda runs use cubic (`-interp 3`) — the poses
|
||
they recover on these cases agree to ~4 decimal places. The conda-forge
|
||
build (1.5.69, CPU) remains the fallback when the Docker image is absent
|
||
(`--niftyreg-backend conda`).
|
||
|
||
Prerequisites (check with `RP.availability()`; a missing piece raises
|
||
`PipelineUnavailable` with the exact command):
|
||
|
||
```bash
|
||
# default: build the v2.1.1 Docker image (one-time)
|
||
docker build -t niftyreg:2.1.1-ubuntu niftyreg/ # from NiftyReg-Ubuntu-v2.1.1.zip
|
||
|
||
# fallback: conda-forge build (provides reg_aladin + reg_f3d, NiftyReg 1.5.69)
|
||
conda install -c conda-forge niftyreg
|
||
```
|
||
|
||
### How the adapter drives the tool
|
||
|
||
1. **Rigid 6-DOF, full volumes** — `reg_aladin -ref T1c -flo CT -rigOnly`
|
||
(the T1c is the reference/fixed image, the CT the floating/moving one;
|
||
the CT is clipped to [0,80] HU first). Symmetric variant on by default,
|
||
which matters: the non-symmetric mode drifts grossly on these
|
||
repositioned pairs. `-ln 3` pyramid, `-maxit 15` TLS iterations per level
|
||
(the tool's default of 5 is too coarse), `-interp 1` (GPU) / `-interp 3`
|
||
(CPU) internal warping, default initialisation (image centres). GPU:
|
||
`--gpus all`, `-platf 1`, `-gpuid <auto by pid>`; CPU/conda: `-platf 0`
|
||
(v2.1.1) or no `-platf` (1.5.69).
|
||
2. **Read-back** — the output affine (`-aff`) is a 4×4 row-major matrix,
|
||
documented as `Affine * Reference = Floating`; the adapter parses it into
|
||
a SimpleITK `AffineTransform` and runs it through the usual
|
||
`fix_orientation` gate (skull-ring concentricity), which resolves the
|
||
direction empirically rather than trusting the docs. Under
|
||
`-ref T1c -flo CT` the file's affine comes out the other way, so the gate
|
||
inverts it on the M6 cases (8/9 `auto_inverted: True`) to the contract
|
||
transform (T1c→CT).
|
||
|
||
## IMPACT pipeline (Docker `vboussot/elastix_impact`) — one-time setup
|
||
|
||
The `impact` pipeline runs the
|
||
[`vboussot/elastix_impact`](https://hub.docker.com/r/vboussot/elastix_impact)
|
||
image: Elastix 5.2 with the [IMPACT](https://arxiv.org/abs/2503.24121)
|
||
*semantic* similarity metric ([vboussot/ImpactLoss](https://github.com/vboussot/ImpactLoss))
|
||
as a native Elastix metric. The parameter map is the MIND + TotalSegmentator
|
||
**static** feature combination of the `MR_CT_TS` reference map in the
|
||
[`VBoussot/ImpactReg`](https://huggingface.co/VBoussot/ImpactReg) repo, with a
|
||
**rigid 6-DOF `EulerTransform`** in place of the reference's RecursiveBSpline
|
||
(its bending-energy penalty is zero for a rigid transform, so it is dropped) —
|
||
i.e. the reference metric/budget, rigid-only, on the GPU.
|
||
|
||
Prerequisites (check with `RP.availability()`; a missing piece raises
|
||
`PipelineUnavailable` with the exact command):
|
||
|
||
```bash
|
||
# 1. Docker (with NVIDIA container toolkit so --gpus works)
|
||
docker --version
|
||
|
||
# 2. Pull the image (~38 GB, one-time)
|
||
docker pull vboussot/elastix_impact
|
||
|
||
# 3. Feature models under the cache (the pipeline prints the exact
|
||
# curl commands if any are missing):
|
||
# data/impact_models/MIND/R1D2_3D.pt (~15 KB)
|
||
# data/impact_models/TS/M852.pt (~200 MB)
|
||
# data/impact_models/TS/M850.pt (~200 MB)
|
||
# (from HF VBoussot/impact-torchscript-models, or `sh
|
||
# Data/Models/download_models.sh` from the ImpactLoss repo)
|
||
```
|
||
|
||
### How the adapter drives the image
|
||
|
||
1. **Full volumes, fixed=T1c (MR), moving=CT** — the shared fixed-image rule:
|
||
the metric draws its random spatial samples from the fixed buffer and
|
||
evaluates them where they map inside the moving buffer. With the tight T1c
|
||
head crop (~220×220×150 mm) as fixed, the samples map mostly inside the
|
||
large CT head-neck FOV (~300×300×251 mm), so the
|
||
`RequiredRatioOfValidSamples 0.1` gate (build default 0.25) rarely fires.
|
||
2. **Parameter map** (the rigid `MR_CT_TS` reference config):
|
||
`(Metric "Impact" "AdvancedMattesMutualInformation")` with weights
|
||
1.0 / 0.4; per level (coarse → fine) the **MIND** handcrafted descriptor
|
||
(`MIND/R1D2_3D.pt`, L1, subset 32, layer mask `"1"`) combined with
|
||
**TotalSegmentator** `TS/M852.pt` (levels 0-1) / `TS/M850.pt` (levels
|
||
2-3, Dice, subset 64, deepest layers), layer weights 0.2/0.8, 0.3/0.7,
|
||
0.6/0.4, 0.7/0.3, at ImpactVoxelSize 6/3/2/2 mm (scaled to the moving
|
||
spacing); **Static** mode (whole-image features, no backprop); no
|
||
image-resampling pyramid (all-1 schedules — the scale is carried by
|
||
ImpactVoxelSize, per the IMPACT docs); `RandomCoordinate` sampler; ASGD
|
||
with per-level MaximumNumberOfIterations 400/300/200/200 and
|
||
NumberOfSpatialSamples 2000; `EulerTransform` +
|
||
`AutomaticTransformInitialization` (CenterOfGravity) +
|
||
`AutomaticScalesEstimation`. A copy of the exact map is dumped to
|
||
`data/m6reg/impact/paramdump/` per run for inspection.
|
||
3. **GPU auto-spread** across the host GPUs by pid (`docker run --gpus all`,
|
||
`ImpactGPU` = the worker's device; `--impact-gpu -1` forces CPU and turns
|
||
mixed precision off).
|
||
4. **Read-back** — the transform comes from
|
||
`Out/TransformParameters.0.itk.txt` (an `Euler3DTransform` in the world
|
||
physical frame; the full combination transform, with the auto-init folded
|
||
in by elastix) and is direction-checked by `fix_orientation` against the
|
||
skull ring (with fixed=T1c / moving=CT the gate keeps the file's transform,
|
||
`auto_inverted: False`).
|
||
5. **Intensity normalization happens inside the metric / models** (no
|
||
pipeline-side normalization): the MIND descriptor normalizes each 5-voxel
|
||
neighbourhood to [0, 1] locally (modality-agnostic by construction); the
|
||
segmentation models are standardized to zero mean / unit variance per the
|
||
model card. Pre-normalizing the volumes would be an exact no-op for the
|
||
z-scored models and would only corrupt the HU scale used by MI scoring and
|
||
the QA brain-window images.
|
||
|
||
## Notes / caveats
|
||
|
||
- The benchmark compares each toolkit in its **native configuration**: all
|
||
register on the full volumes (no head box, no mask, no crop / pad) and use
|
||
each toolkit's native automatic transform initialization (no manual
|
||
`centroid_offset`) — elastix / impact `AutomaticTransformInitialization`
|
||
(CenterOfGravity), antspyx's center-of-mass alignment, simpleitk identity,
|
||
synthmorph none. Production uses `--simpleitk-mode production`'s
|
||
multi-start tilt grid, which raises SimpleITK's acceptance at ~20× the
|
||
cost; that is out of scope for the head-to-head but is the config the
|
||
dataset actually ships with.
|
||
- `synthmorph` is deterministic (one fixed pretrained model, no random
|
||
subsampling), so its per-case numbers are stable run to run — unlike ANTs /
|
||
IMPACT. Its low MI (median 0.277 with the default CT clip; 0.06–0.11 on the
|
||
repositioned cases) is a genuine gross-pose-estimation weakness of the
|
||
pretrained matrix model (case-by-case, not variance) — see the
|
||
"SynthMorph low-performance investigation" section above.
|
||
- `fix_orientation` (elastix/ants/impact/niftyreg/synthmorph) chooses between
|
||
a transform and its inverse by skull-ring concentricity — a correctness gate
|
||
that makes the result independent of each toolkit's transform-direction
|
||
convention, so the fixed/moving swap is absorbed by the gate rather than
|
||
re-derived per toolkit. Under the shared fixed=T1c / moving=CT rule the raw
|
||
transforms come out in the contract direction for elastix (combination
|
||
transform = fixed→moving), impact (its `TransformParameters.0.itk.txt`,
|
||
the full combination Euler transform) and antspyx (ANTs forward transform)
|
||
and synthmorph (the `-T` LTA, model estimates first→second) — completed
|
||
runs report `auto_inverted: False` for those; `niftyreg`'s `-aff` file
|
||
comes out the other way and the gate inverts it (8/9 `auto_inverted: True`).
|
||
- **SynthMorph specifics** (Docker `freesurfer/synthmorph`, TensorFlow, GPU):
|
||
- **Gross-pose estimation weakness (measured, not a bug).** The `rigid`
|
||
matrix model is trained on brain *MRI* (same-subject pairs, small
|
||
relative poses). On these repositioned CT↔T1c pairs it estimates the
|
||
translation wrongly on 6/9 cases (19–66 mm head-centre error; the two
|
||
worst, `m6_2BBN5XA4` / `m6_24V62QQM_20240820`, at 61 / 66 mm) and the
|
||
rotation on 3/9 (10–22°) — and the required rotations are only 1.4–10.6°,
|
||
so it is *not* a "defeated by tilt" story; on `m6_24V62QQM_20240820`
|
||
its transform scores below the unregistered (identity) one. An in-domain
|
||
MR→MR control (same subject, ~55° relative pose) fails equally, and the
|
||
`affine` model changes nothing, while the `joint` (affine + deformable)
|
||
model recovers all three tested failures to the best ImpactLoss of the
|
||
six tools. Full evidence: the "SynthMorph low-performance investigation"
|
||
section.
|
||
- **No centroid init needed.** Being a global deep method it recovers the
|
||
gross offset on its own; the `-i` initial-transform path is not used
|
||
(which also sidesteps this image's float64/float32 bug on the `-i` code
|
||
path).
|
||
- **GPU sharing.** The container runs as the host uid with
|
||
`TF_FORCE_GPU_ALLOW_GROWTH` so concurrent benchmark workers share a GPU.
|
||
- **~28 s/case** (GPU: one pretrained forward pass), between SimpleITK
|
||
(~11 s) / ANTs (~17 s) and IMPACT (~115 s, rigid docker config).
|
||
- **NiftyReg specifics** (conda-forge `niftyreg` 1.5.69, `reg_aladin`,
|
||
symmetric block matching + TLS, CPU/OpenMP):
|
||
- **Different algorithmic family.** No histogram MI objective: 4-voxel
|
||
blocks are matched by normalised cross-correlation and the rigid
|
||
transform is solved by a trimmed least squares, iterated per pyramid
|
||
level (`-ln 3 -maxit 15`). NCC's scale invariance is what makes it
|
||
cross-modality tolerant.
|
||
- **Symmetric mode matters.** The default symmetric variant (optimising
|
||
forward and backward transforms concurrently) is the robust one on these
|
||
repositioned pairs; `-noSym` drifts grossly (gap std 45 mm on a clean
|
||
case every other tool passes).
|
||
- **`-maxit 5` (the tool default) is too coarse** — on the clean case
|
||
`m6_26TYQI3C` it reaches MI 0.41 vs 0.61 at 15 TLS iterations/level;
|
||
15 is the benchmark setting.
|
||
- **GPU is the default now** (v2.1.1, `-platf 1` via
|
||
`docker run --gpus all`): ~20–25 s/case on the full volumes vs ~4 min
|
||
CPU (120–440 s, `-omp` thread-bound) for the conda 1.5.69 build.
|
||
`--niftyreg-gpu -1` forces the CPU platform (`-platf 0`).
|
||
- **Version note.** The upstream KCL-BMEIS release binaries (v2.x) need
|
||
glibc ≥ 2.34; this host (Ubuntu 20.04, glibc 2.31) runs them in the
|
||
`niftyreg:2.1.1-ubuntu` Docker image (nvidia/cuda ubuntu22.04 base;
|
||
the CUDA plugin needs libcusolver.so.11, provided by the image). The
|
||
conda-forge 1.5.69 build remains as the no-Docker fallback and gives
|
||
essentially identical poses (MI matches to ±0.001 on the 10 cases).
|
||
- **Elastix specifics** (ITK-Elastix 0.25.4, elastix 5.3.1, in-process python
|
||
API — no CLI binary involved):
|
||
- Elastix 5.x renamed components vs the classic parameter files: the rigid
|
||
3D transform is `EulerTransform` (6-DOF, `Rigid3DTransform` is gone from
|
||
the component set), the metric key is `Metric` (not `ImageMetric`), the
|
||
random sampler is `Random` (not `RandomSampler`). The adapter uses the 5.x
|
||
names and is not compatible with elastix ≤ 4.x parameter files.
|
||
- `AutomaticTransformInitialization "true"` (CenterOfGravity) provides the
|
||
start (no manual `centroid_offset`); `AutomaticScalesEstimation` sets the
|
||
optimizer scales. An **output directory** is still set
|
||
(`SetOutputDirectory`) for the C++ filter to run; the adapter uses a
|
||
scratch dir under `data/m6reg/elastix/itkelastix/`.
|
||
- `WriteResultImage` must be the string `"false"` (Python `False`); the
|
||
default (absent) makes the filter raise "Could not read result image"
|
||
after a *successful* registration.
|
||
- With fixed=T1c/moving=CT the combination transform is T1c→CT — the
|
||
contract direction — so `fix_orientation` keeps it as-is
|
||
(`auto_inverted: False`). The result is recovered by point-mapping
|
||
(`GetCombinationTransform().TransformPoint`) into an affine — independent
|
||
of elastix's Euler-angle parameterization — and wrapped in a SimpleITK
|
||
`AffineTransform(3)`.
|
||
- Benchmark budget: `MaximumNumberOfIterations` 2000 per resolution
|
||
(ASGD 2000/2000/2000 over 3 resolutions) with the sampler/metric at their
|
||
defaults (`NumberOfSpatialSamples` 5000, `NumberOfHistogramBins` 32).
|
||
- **IMPACT specifics** (Docker `vboussot/elastix_impact`, Elastix 5.2 +
|
||
IMPACT metric, GPU; rigid MIND+TS static config of the `MR_CT_TS` reference
|
||
map):
|
||
- **Runs on M6 with the T1c-fixed direction + relaxed sample-validity
|
||
gate.** Full volumes, no mask, no crop / pad, fixed=T1c/moving=CT (the
|
||
shared rule): the metric's samples (drawn from the tight T1c head crop)
|
||
map mostly inside the large CT head-neck FOV, so the gate rarely fires.
|
||
The map still sets `RequiredRatioOfValidSamples 0.1` (build default
|
||
0.25) as a safety override — a genuinely diverging transform, <10%
|
||
in-buffer, still aborts. The combination Euler transform comes out in
|
||
the contract direction (completed runs report `auto_inverted: False`).
|
||
- **Auto init.** `AutomaticTransformInitialization` (CenterOfGravity)
|
||
aligns the intensity centroids (no manual translation); the reference
|
||
map's GeometricalCenter start is too crude for the head+neck FOV
|
||
mismatch.
|
||
- **Rigid, like every other pipeline.** The reference map's
|
||
RecursiveBSpline + bending-energy regularizer is replaced by a 6-DOF
|
||
`EulerTransform` (the bending penalty is zero for a rigid transform).
|
||
- **History: single-init convergence.** The semantic landscape has more
|
||
local minima than Mattes-MI. The T1c-fixed direction fixes most of the
|
||
hard cases (the repositioned `m6_22IRRKU6` / `m6_23XIMO4M` now register
|
||
as well as the MI tools do), but the single CenterOfGravity start on the
|
||
~30° tilt pair `m6_2BBN5XA4` remains a weakness — a multi-start tilt
|
||
grid (as production SimpleITK uses) would be the next step.
|
||
- **Stochastic.** The config is **Static** mode: it re-samples spatial
|
||
coordinates every iteration, but the feature maps are deterministic
|
||
given the pose (see Stochasticity). The older Jacobian-mode configs
|
||
(docker-era experiments, archived in `results/bench_registration/impact_*.json`)
|
||
resampled patches through the network each iteration, so their
|
||
per-case numbers are one sample from a distribution.
|
||
- **Models are under the data root, not the repo.** The TorchScript
|
||
feature models live in `data/impact_models/` (MIND ~15 KB + TS ~200 MB
|
||
each, from HF `VBoussot/impact-torchscript-models`); the pipeline
|
||
raises with the exact `curl` commands if one is missing.
|
||
- **CPU fallback:** `--impact-gpu -1` (`ImpactGPU -1`, mixed precision
|
||
off — much slower). |