# Registration Pipeline Benchmark — CT → T1c (M6)

Benchmarks **four rigid, cross-modality registration pipelines** that align a
head-and-neck CT to a tightly-cropped T1c MRI so the GTV can be warped onto the
T1c (MR) native grid. This is the registration step of
`scripts/02_build_m6_dataset.py`. The pairs are hard: different modality (CT
Hounsfield vs MR intensity) and the patient is repositioned between the CT and
MR sessions, so the inter-session head tilt can reach ~30°.

- Pipelines: [`src/registration_pipelines.py`](src/registration_pipelines.py)
- Benchmark harness: [`scripts/bench_registration.py`](scripts/bench_registration.py)
- Results: `results/bench_registration/` (per-pipeline JSON, `summary.json`,
  `summary.md`, `bench_registration.png`)

## Unified contract

Every pipeline returns a SimpleITK transform `T` such that

```python
sitk.Resample(ct, t1c, T, sitk.sitkLinear)              # CT  onto the MR grid
sitk.Resample(tv,  t1c, T, sitk.sitkNearestNeighbor, 0) # GTV onto the MR grid
```

places the CT / GTV concentric with the MR head. In `Resample`'s
destination→source convention this `T` maps **T1c → CT**. All pipelines register
on the same CT-space head box (MI computed on head tissue, not neck/air) and,
for the single-init tools, from the **same centroid-aligned start**
(`centroid_offset`), so the comparison is on the optimizer / metric / transform
handling — not on who finds the ~30–60 mm gross offset first.

## The four pipelines

| # | pipeline | toolkit | metric | transform | status in this env |
|---|----------|---------|--------|-----------|--------------------|
| 1 | `elastix`   | ITK-Elastix 0.25.4 (pip `itk` + `itk-elastix`, elastix 5.3.1) | Mattes MI (10 bins) | `EulerTransform` rigid (5.x replacement for `Rigid3DTransform`), ASGD, 3 resolutions | ✅ ran (main env, in-process python API — no CLI) |
| 2 | `simpleitk` | SimpleITK 2.5.6 | Mattes MI (32 bins) | `Euler3D` rigid, pyramid `[4,2,1]` | ✅ ran |
| 3 | `antspyx`   | ANTsPy / ANTs 0.6.3 | Mattes MI (multi-res) | ANTs `Rigid` | ✅ ran (isolated py3.12 env) |
| 4 | `deepreg`   | Oxford DeepReg (CycleGAN) | CycleGAN + reg. loss (pretrained) | 3D displacement field | **not installed** (see below) |

`simpleitk` has two budgets: `--simpleitk-mode benchmark` (single pyramid,
centered start — the default, comparable to the other tools) and
`--simpleitk-mode production` (the full multi-start tilt grid from
`02_build_m6_dataset.py`, ~20× slower, the most robust but not used for the
head-to-head).

## Test cases

First 10 labeled M6 cases (`--n-cases 10 --sample first`), reproducible. 9 have
a complete CT; 1 (`m6_22GD5VL2_20230724`) has an incomplete CT head and is
reported as `incomplete-CT` for every pipeline (a data issue, not a pipeline
failure).

```
m6_22GD5VL2_20230724  (incomplete CT)
m6_22IRRKU6_20240115  m6_23XIMO4M_20230222  m6_24V62QQM_20230223
m6_24V62QQM_20231017  m6_24V62QQM_20240820  m6_26TYQI3C_20221027
m6_2BBN5XA4_20250411  m6_2C4TKI6H_20240506  m6_2CDNN7BH_20240102
```

## Metrics (identical for every pipeline)

Production QA, crop-independent:

- **`skull_ring_gap_mm`** — march rays from the MR head center; the gap between
  the MR head silhouette boundary and the warped-CT skull ring (p97 of positive
  CT) at mid and low levels, plus the angular std. **Aligned ⇔
  `0 ≤ mid ≤ 20 mm` and `std ≤ 12 mm`.** `mid` measures translational
  concentricity; `std` measures rotational concentricity.
- **`tv_in_ct_frac`** — fraction of the dilated warped GTV inside the warped-CT
  head tissue (the slab must straddle the lesion). Acceptance needs `≥ 0.5`.
- **`tv_nvox`** — warped GTV volume.
- **`accepted`** — `tv_nvox > 0 and tv_in_ct_frac ≥ 0.5 and aligned` (the
  production acceptance rule).
- **`mi_ct_mr`** — mean Mattes MI (nats) between the **CT inverse-transformed
  into MR coordinates** (the warped CT on the T1c grid) and the **original MR**,
  restricted to the MR head and each image percentile-binned independently
  (SimpleITK Mattes-MI style). This is the registration's *achieved* metric read
  off as a standalone alignment score: higher = the warped-CT and MR intensities
  are better in-sync = better aligned; it collapses toward 0 when the CT is
  rotated/translated away from the MR.
- **`reg_wall_s`** — wall-clock registration time (excludes I/O + scoring).

## Results (10 cases)

| pipeline | toolkit | registered | accepted | gap mid (med/mean) | gap std (med/mean) | s/reg (med) | tv-in-CT | MI (med/mean, nats) |
|---|---|---|---|---|---|---|---|---|
| **simpleitk** | SimpleITK 2.5.6 | 9 | **4/10 (0.40)** | 12.6 / 16.1 | 15.4 / 16.0 | 8.6 | 0.93 | **0.643 / 0.630** |
| **antspyx** | ANTsPy 0.6.3 | 9 | **1/10 (0.10)** | 14.2 / 17.3 | 24.8 / 22.4 | 12.7 | 0.93 | **0.600 / 0.523** |
| **elastix** | ITK-Elastix 0.25.4 | 9 | **5/10 (0.50)** | 15.3 / 16.1 | 9.9 / 13.8 | 3.2 | 0.93 | **0.562 / 0.549** |
| deepreg | — | — | — | — | — | — | — | — |

Per-case (mid / std, `OK` = accepted):

| case | simpleitk | antspyx | elastix |
|---|---|---|---|
| m6_22GD5VL2_20230724 | incomplete-CT | incomplete-CT | incomplete-CT |
| m6_22IRRKU6_20240115 | 25.7 / 29.3 | 26.3 / 29.2 | 26.4 / 30.3 |
| m6_23XIMO4M_20230222 | 22.2 / 23.9 | 23.8 / 25.0 | 21.1 / 23.3 |
| m6_24V62QQM_20230223 | 10.7 / 16.9 | 9.4 / 18.2 | 10.6 / 8.2 **OK** |
| m6_24V62QQM_20231017 | 11.5 / 7.3 **OK** | 12.5 / 12.8 | 11.0 / 6.1 **OK** |
| m6_24V62QQM_20240820 | 11.6 / 9.6 **OK** | 14.8 / 11.7 **OK** | 15.3 / 14.0 |
| m6_26TYQI3C_20221027 | 12.2 / 3.7 **OK** | 6.1 / 37.2 | 11.4 / 3.2 **OK** |
| m6_2BBN5XA4_20250411 | 18.6 / 8.0 **OK** | 14.3 / 12.7 | 18.1 / 9.9 **OK** |
| m6_2C4TKI6H_20240506 | 12.6 / 15.4 | 13.6 / 14.7 | 12.0 / 6.8 **OK** |
| m6_2CDNN7BH_20240102 | 19.4 / 30.2 | 18.2 / 33.6 | 18.9 / 22.7 |

### Interpretation

- **Translation is recovered well by all three** (mid gaps mostly 6–20 mm,
  within the `≤ 20 mm` band). `tv-in-CT` is identical (~0.93): the warped GTV
  lands inside the CT head for every pipeline, so the target is in the right
  place.
- **The binding failure is rotation** — `std > 12 mm` on the repositioned cases
  (the ~30° inter-session tilt produces MI local minima that a single centered
  start can lock onto). This is exactly why production `02_build_m6_dataset.py`
  uses a **multi-start tilt grid**.
- **Elastix is the most accurate of the three standard single-init configs**
  (5/10 accepted, lowest median std 9.9 vs SimpleITK 15.4 and ANTs 24.8) and
  the fastest (3.2 s/case median wall, ~1.8 s pure registration vs 8.6 s
  SimpleITK / 12.7 s ANTs): ASGD's adaptive step sizes climb out of small
  misalignments better than RSGD/ANTs on these boxes. It accepts `m6_24V62QQM_20230223`
  (std 8.2) and `m6_2C4TKI6H` (std 6.8), which the other two tools fail on
  (std 16.9 / 15.4), and misses `m6_24V62QQM_20240820` (std 14.0) that both
  others accept — different optimizers settle in different local minima on the
  hard tilted cases.
- **ANTs' Rigid is the weakest** (1/10 accepted): it frequently locks onto the
  tilted local minimum. Note `m6_26TYQI3C`: ANTs nailed the position (mid 6.1)
  but missed the orientation (std 37.2) — a textbook tilted local minimum.
- **Speed** (median wall, 4 cases in parallel, 24 threads): Elastix ~3.2 s
  (warm workers; a worker's first case pays a ~10 s warm-up), SimpleITK ~8.6 s,
  ANTs ~12.7 s (includes the pre-shift resample + its affine-MI init stages).
  All CPU-bound.
- **Mattes MI (warped-CT ~ MR) is an independent, intensity-based confirmation**:
  medians 0.643 (SimpleITK) / 0.600 (ANTs) / 0.562 (Elastix), and it diverges
  sharply on the tilted cases — ANTs `m6_26TYQI3C` drops to 0.094 nats
  (rotation ~37 mm off) while SimpleITK holds 0.670. A high MI means the CT
  skull/tissue intensities fall on the matching MR structures after the inverse
  transform.
- **Stochasticity**: ANTs' Mattes-MI subsamples the image at random, so its
  per-case numbers carry run-to-run variance (this table is one draw);
  SimpleITK's MI (and registration) is deterministic at fixed thread count
  (RSGD wall time is thread-sensitive, convergence is not); Elastix's ASGD is
  deterministic given its fixed seed.

## Reproducing

```bash
# main env — SimpleITK + Elastix (itk-elastix is pip-installed here;
# antspyx/deepreg report unavailable and are recorded as such)
source /opt/conda/etc/profile.d/conda.sh && conda activate longitudinal
python scripts/bench_registration.py --n-cases 10 \
    --workers 4 --threads 24 --simpleitk-mode benchmark

# isolated py3.12 env with antspyx — ANTs (+ optional SimpleITK cross-check)
/home/xfr/.conda/envs/ants312/bin/python scripts/bench_registration.py \
    --pipelines antspyx --n-cases 10 --workers 4 --threads 24

# combine the per-pipeline JSONs accumulated across envs into one table + plot
python scripts/bench_registration.py --merge
```

Other flags: `--sample random --seed 0` (reproducible random 10),
`--keys k1,k2,...` (explicit cases), `--n-cases N`, `--workers W`,
`--threads T`, `--simpleitk-mode {benchmark,production}`,
`--deepreg-ckpt <.ckpt>` (DeepReg), `--out-dir <dir>`,
`--qa-images` / `--qa-out <dir>` (visual QA, below).

## Visual QA images (checkerboard + color overlay)

`--qa-images` writes a `checkreg`-style PNG per (pipeline, case) to
`<out-dir>/qa/<pipeline>__<case>.png` for eyeballing the alignment.  Each image
is a 3×2 grid: **rows** are the three orthogonal mid-slices through the head
center (**mid-axial, mid-coronal, mid-sagittal**), and **columns** are:

- **left — checkerboard blend:** alternating blocks of the CT inverse-transformed
  into MR coordinates and the original MR; a clean registration gives a seamless
  mosaic, a misregistration shows hard edges at the block boundaries.
- **right — additive color overlay:** CT in red, MR in green. Structures that
  coincide blend to yellow; rotation/translation error appears as red/green
  fringing at the skull and brain edges.

The CT is shown in a **brain window (L40/W80 = 0–80 HU)** so the brain
parenchyma is visible and comparable to the MR (the skull, ~800+ HU, clips to
white); the MR uses a 2–98 % head window. The coronal and sagittal views are
oriented superior-up to match the axial view.

The title carries the pipeline, case key, Mattes MI, and the skull-ring gap
(mid/std), so a glance tells you both the achieved-metric score and whether the
visuals agree. Generate for the available pipelines with:

```bash
/home/xfr/.conda/envs/longitudinal/bin/python scripts/bench_registration.py \
    --pipelines simpleitk elastix --n-cases 10 --qa-images
/home/xfr/.conda/envs/ants312/bin/python scripts/bench_registration.py \
    --pipelines antspyx --n-cases 10 --qa-images
```

(9 images each; the incomplete-CT case is skipped.)

## Enabling the blocked pipeline

**DeepReg (Oxford)** — the PyPI `deepreg` (0.1.2) is an unrelated stub that pins
`numpy==1.18.5`; do not install it. DeepReg is not a one-shot rigid tool: it is a
CycleGAN image-to-image registration that must be **trained or run from a
pretrained checkpoint**. To benchmark it:

```bash
git clone https://github.com/OB-CVHU/DeepReg && cd DeepReg && pip install -e .
python scripts/bench_registration.py --pipelines deepreg --n-cases 10 \
    --deepreg-ckpt /path/to/pretrained_3d.ckpt
```

The adapter runs `deepreg test` to predict a voxel displacement field, converts
it to a SimpleITK `DisplacementFieldTransform`, and scores it like the others.

## Notes / caveats

- The benchmark compares **standard single-init** configurations (centered
  start). Production uses `--simpleitk-mode production`'s multi-start tilt grid,
  which raises SimpleITK's acceptance at ~20× the cost; that is out of scope for
  the head-to-head but is the config the dataset actually ships with.
- `deepreg` could not be executed here, so its row is "not installed", not
  measured; its code path is implemented and unit-verified for import/convention.
- `fix_orientation` (elastix/ants/deepreg) chooses between a transform and its
  inverse by skull-ring concentricity — a correctness gate that makes the result
  independent of each toolkit's transform-direction convention. In practice it
  never fired (all `auto_inverted: False`): elastix/ants returned the T1c→CT
  direction as expected.
- **Elastix specifics** (ITK-Elastix 0.25.4, elastix 5.3.1, in-process python
  API — no CLI binary involved):
  - Elastix 5.x renamed components vs the classic parameter files: the rigid
    3D transform is `EulerTransform` (6-DOF, `Rigid3DTransform` is gone from
    the component set), the metric key is `Metric` (not `ImageMetric`), the
    random sampler is `Random` (not `RandomSampler`). The adapter uses the 5.x
    names and is not compatible with elastix ≤ 4.x parameter files.
  - The initial transform (the shared `centroid_offset` start) is passed as an
    `InitialTransformParameterObject` (a `TranslationTransform` with
    `NumberOfParameters 3`); the C++ filter only wires it up when an
    **output directory is set** (`SetOutputDirectory`), so the adapter uses a
    scratch dir under `data/m6reg/elastix/itkelastix/`.
  - `WriteResultImage` must be the string `"false"` (Python `False`); the
    default (absent) makes the filter raise "Could not read result image"
    after a *successful* registration.
  - The result transform is recovered by point-mapping
    (`GetCombinationTransform().TransformPoint`) into an affine — independent
    of elastix's Euler-angle parameterization — and wrapped in a SimpleITK
    `AffineTransform(3)`.
  - Registration wall time is ~1.8 s/case at full budget (ASGD 500/500/250
    over 3 resolutions); each worker's first case pays a ~10 s warm-up, so the
    median wall is 3.2 s and the mean is inflated by warm-up.