Furen Xiao
e8dfe0fe35
Update M6 manifest building to register T1c to CT with rigid Mattes MI on a CT-space head box, including multi-start initial tilts for large inter-session head motion. The inverse transform warps CT and GTV into T1c native space, with acceptance based on concentricity of the warped CT skull ring and MR head silhouette. Rerun preprocessing when a row has a label but the warped label file is missing. Normalize volume orientation in read_nii_img. Add registration benchmark documentation, benchmark script, and shared registration pipeline helpers.
177 lines
No EOL
8.5 KiB
Markdown
177 lines
No EOL
8.5 KiB
Markdown
# Registration Pipeline Benchmark — CT → T1c (M6)
|
||
|
||
Benchmarks **four rigid, cross-modality registration pipelines** that align a
|
||
head-and-neck CT to a tightly-cropped T1c MRI so the GTV can be warped onto the
|
||
T1c (MR) native grid. This is the registration step of
|
||
`scripts/02_build_m6_dataset.py`. The pairs are hard: different modality (CT
|
||
Hounsfield vs MR intensity) and the patient is repositioned between the CT and
|
||
MR sessions, so the inter-session head tilt can reach ~30°.
|
||
|
||
- Pipelines: [`src/registration_pipelines.py`](src/registration_pipelines.py)
|
||
- Benchmark harness: [`scripts/bench_registration.py`](scripts/bench_registration.py)
|
||
- Results: `results/bench_registration/` (per-pipeline JSON, `summary.json`,
|
||
`summary.md`, `bench_registration.png`)
|
||
|
||
## Unified contract
|
||
|
||
Every pipeline returns a SimpleITK transform `T` such that
|
||
|
||
```python
|
||
sitk.Resample(ct, t1c, T, sitk.sitkLinear) # CT onto the MR grid
|
||
sitk.Resample(tv, t1c, T, sitk.sitkNearestNeighbor, 0) # GTV onto the MR grid
|
||
```
|
||
|
||
places the CT / GTV concentric with the MR head. In `Resample`'s
|
||
destination→source convention this `T` maps **T1c → CT**. All pipelines register
|
||
on the same CT-space head box (MI computed on head tissue, not neck/air) and,
|
||
for the single-init tools, from the **same centroid-aligned start**
|
||
(`centroid_offset`), so the comparison is on the optimizer / metric / transform
|
||
handling — not on who finds the ~30–60 mm gross offset first.
|
||
|
||
## The four pipelines
|
||
|
||
| # | pipeline | toolkit | metric | transform | status in this env |
|
||
|---|----------|---------|--------|-----------|--------------------|
|
||
| 1 | `elastix` | ITK-Elastix | Mattes MI | `Rigid3DTransform` (ASGD) | **not installed** (conda-forge unreachable) |
|
||
| 2 | `simpleitk` | SimpleITK 2.5.6 | Mattes MI (32 bins) | `Euler3D` rigid, pyramid `[4,2,1]` | ✅ ran |
|
||
| 3 | `antspyx` | ANTsPy / ANTs 0.6.3 | Mattes MI (multi-res) | ANTs `Rigid` | ✅ ran (isolated py3.12 env) |
|
||
| 4 | `deepreg` | Oxford DeepReg (CycleGAN) | CycleGAN + reg. loss (pretrained) | 3D displacement field | **not installed** (see below) |
|
||
|
||
`simpleitk` has two budgets: `--simpleitk-mode benchmark` (single pyramid,
|
||
centered start — the default, comparable to the other tools) and
|
||
`--simpleitk-mode production` (the full multi-start tilt grid from
|
||
`02_build_m6_dataset.py`, ~20× slower, the most robust but not used for the
|
||
head-to-head).
|
||
|
||
## Test cases
|
||
|
||
First 10 labeled M6 cases (`--n-cases 10 --sample first`), reproducible. 9 have
|
||
a complete CT; 1 (`m6_22GD5VL2_20230724`) has an incomplete CT head and is
|
||
reported as `incomplete-CT` for every pipeline (a data issue, not a pipeline
|
||
failure).
|
||
|
||
```
|
||
m6_22GD5VL2_20230724 (incomplete CT)
|
||
m6_22IRRKU6_20240115 m6_23XIMO4M_20230222 m6_24V62QQM_20230223
|
||
m6_24V62QQM_20231017 m6_24V62QQM_20240820 m6_26TYQI3C_20221027
|
||
m6_2BBN5XA4_20250411 m6_2C4TKI6H_20240506 m6_2CDNN7BH_20240102
|
||
```
|
||
|
||
## Metrics (identical for every pipeline)
|
||
|
||
Production QA, crop-independent:
|
||
|
||
- **`skull_ring_gap_mm`** — march rays from the MR head center; the gap between
|
||
the MR head silhouette boundary and the warped-CT skull ring (p97 of positive
|
||
CT) at mid and low levels, plus the angular std. **Aligned ⇔
|
||
`0 ≤ mid ≤ 20 mm` and `std ≤ 12 mm`.** `mid` measures translational
|
||
concentricity; `std` measures rotational concentricity.
|
||
- **`tv_in_ct_frac`** — fraction of the dilated warped GTV inside the warped-CT
|
||
head tissue (the slab must straddle the lesion). Acceptance needs `≥ 0.5`.
|
||
- **`tv_nvox`** — warped GTV volume.
|
||
- **`accepted`** — `tv_nvox > 0 and tv_in_ct_frac ≥ 0.5 and aligned` (the
|
||
production acceptance rule).
|
||
- **`reg_wall_s`** — wall-clock registration time (excludes I/O + scoring).
|
||
|
||
## Results (10 cases)
|
||
|
||
| pipeline | toolkit | registered | accepted | gap mid (med/mean) | gap std (med/mean) | s/reg (med) | tv-in-CT |
|
||
|---|---|---|---|---|---|---|---|
|
||
| **simpleitk** | SimpleITK 2.5.6 | 9 | **4/10 (0.40)** | 12.6 / 16.1 | 15.4 / 16.0 | 8.3 | 0.93 |
|
||
| **antspyx** | ANTsPy 0.6.3 | 9 | **1/10 (0.10)** | 14.3 / 15.4 | 18.2 / 21.7 | 13.4 | 0.93 |
|
||
| elastix | — | — | — | — | — | — | — |
|
||
| deepreg | — | — | — | — | — | — | — |
|
||
|
||
Per-case (mid / std, `OK` = accepted):
|
||
|
||
| case | simpleitk | antspyx |
|
||
|---|---|---|
|
||
| m6_22GD5VL2_20230724 | incomplete-CT | incomplete-CT |
|
||
| m6_22IRRKU6_20240115 | 25.7 / 29.3 | 26.3 / 29.2 |
|
||
| m6_23XIMO4M_20230222 | 22.2 / 23.9 | 23.8 / 25.0 |
|
||
| m6_24V62QQM_20230223 | 10.7 / 16.9 | 9.4 / 18.2 |
|
||
| m6_24V62QQM_20231017 | 11.5 / 7.3 **OK** | 12.5 / 12.8 |
|
||
| m6_24V62QQM_20240820 | 11.6 / 9.6 **OK** | 14.8 / 11.7 **OK** |
|
||
| m6_26TYQI3C_20221027 | 12.2 / 3.7 **OK** | 6.1 / 37.2 |
|
||
| m6_2BBN5XA4_20250411 | 18.6 / 8.0 **OK** | 14.3 / 12.7 |
|
||
| m6_2C4TKI6H_20240506 | 12.6 / 15.4 | 13.6 / 14.7 |
|
||
| m6_2CDNN7BH_20240102 | 19.4 / 30.2 | 18.2 / 33.6 |
|
||
|
||
### Interpretation
|
||
|
||
- **Translation is recovered well by both** (mid gaps mostly 6–20 mm, within the
|
||
`≤ 20 mm` band). `tv-in-CT` is identical (0.93): the warped GTV lands inside
|
||
the CT head for both, so the target is in the right place.
|
||
- **The binding failure is rotation** — `std > 12 mm` on the repositioned cases
|
||
(the ~30° inter-session tilt produces MI local minima that a single centered
|
||
start can lock onto). This is exactly why production `02_build_m6_dataset.py`
|
||
uses a **multi-start tilt grid**.
|
||
- **SimpleITK is the more robust of the two** (4 vs 1 accepted, lower median
|
||
std 15.4 vs 18.2): its RSGD + physical-step-scales + 32-bin Mattes-MI
|
||
landscape handles the tilt better than ANTs' Rigid. Note `m6_26TYQI3C`: ANTs
|
||
nailed the position (mid 6.1) but missed the orientation (std 37.2) — a
|
||
textbook tilted local minimum.
|
||
- **Speed**: SimpleITK ~8.3 s/case, ANTs ~13.4 s/case (ANTs includes the
|
||
pre-shift resample + its affine-MI init stages). Both are CPU-bound and ran
|
||
4 cases in parallel.
|
||
|
||
## Reproducing
|
||
|
||
```bash
|
||
# main env — SimpleITK baseline (elastix/deepreg report unavailable here)
|
||
source /opt/conda/etc/profile.d/conda.sh && conda activate longitudinal
|
||
python scripts/bench_registration.py --pipelines simpleitk --n-cases 10 \
|
||
--workers 4 --threads 24 --simpleitk-mode benchmark
|
||
|
||
# isolated py3.12 env with antspyx — ANTs (+ optional SimpleITK cross-check)
|
||
/home/xfr/.conda/envs/ants312/bin/python scripts/bench_registration.py \
|
||
--pipelines antspyx --n-cases 10 --workers 4 --threads 24
|
||
|
||
# combine the per-pipeline JSONs accumulated across envs into one table + plot
|
||
python scripts/bench_registration.py --merge
|
||
```
|
||
|
||
Other flags: `--sample random --seed 0` (reproducible random 10),
|
||
`--keys k1,k2,...` (explicit cases), `--n-cases N`, `--workers W`,
|
||
`--threads T`, `--simpleitk-mode {benchmark,production}`,
|
||
`--deepreg-ckpt <.ckpt>` (DeepReg), `--out-dir <dir>`.
|
||
|
||
## Enabling the two blocked pipelines
|
||
|
||
**ITK-Elastix** — not on PyPI; needs conda-forge (which this host cannot reach).
|
||
Once conda-forge is available:
|
||
|
||
```bash
|
||
conda install -c conda-forge elastix # provides the elastix/transformix CLI
|
||
python scripts/bench_registration.py --pipelines elastix --n-cases 10
|
||
```
|
||
|
||
The adapter registers `fixed=T1c, moving=CT` with `Rigid3DTransform` + Mattes MI
|
||
on the head box and reads the result transform.
|
||
|
||
**DeepReg (Oxford)** — the PyPI `deepreg` (0.1.2) is an unrelated stub that pins
|
||
`numpy==1.18.5`; do not install it. DeepReg is not a one-shot rigid tool: it is a
|
||
CycleGAN image-to-image registration that must be **trained or run from a
|
||
pretrained checkpoint**. To benchmark it:
|
||
|
||
```bash
|
||
git clone https://github.com/OB-CVHU/DeepReg && cd DeepReg && pip install -e .
|
||
python scripts/bench_registration.py --pipelines deepreg --n-cases 10 \
|
||
--deepreg-ckpt /path/to/pretrained_3d.ckpt
|
||
```
|
||
|
||
The adapter runs `deepreg test` to predict a voxel displacement field, converts
|
||
it to a SimpleITK `DisplacementFieldTransform`, and scores it like the others.
|
||
|
||
## Notes / caveats
|
||
|
||
- The benchmark compares **standard single-init** configurations (centered
|
||
start). Production uses `--simpleitk-mode production`'s multi-start tilt grid,
|
||
which raises SimpleITK's acceptance at ~20× the cost; that is out of scope for
|
||
the head-to-head but is the config the dataset actually ships with.
|
||
- `elastix` / `deepreg` could not be executed here, so their rows are
|
||
"not installed", not measured. Their code paths are implemented and
|
||
unit-verified for import/convention; run them where the toolkits are present.
|
||
- `fix_orientation` (elastix/ants/deepreg) chooses between a transform and its
|
||
inverse by skull-ring concentricity — a correctness gate that makes the result
|
||
independent of each toolkit's transform-direction convention. |