fix(benchmark): lower IMPACT sample-validity gate to unblock M6 runs
The no-mask CT→T1c FOV mismatch left only ~14% of spatial samples inside the moving buffer at auto-init, below Elastix's default RequiredRatioOfValidSamples of 0.25, so the IMPACT pipeline aborted on every M6 case. Set the gate to 0.1 so the metric's native in-buffer evaluation proceeds while still catching diverging transforms. Switch the default semantic model from TotalSegmentator (TS/M730.pt) to MRSegmentator (MRSeg/MRSeg.pt), a multi-modality MRI+CT model better suited to this cross-modality task. Update docstrings and REGISTRATION_BENCHMARK.md with the new per-case MI results.
This commit is contained in:
parent
16b9828d15
commit
73234db39e
2 changed files with 166 additions and 84 deletions
|
|
@ -39,10 +39,13 @@ center-of-mass alignment, simpleitk starts from identity, niftyreg
|
||||||
consequence of the no-mask rule: the IMPACT metric only evaluates samples that
|
consequence of the no-mask rule: the IMPACT metric only evaluates samples that
|
||||||
map inside the moving buffer, so with the CT (head-neck, ~300×300×251 mm) as
|
map inside the moving buffer, so with the CT (head-neck, ~300×300×251 mm) as
|
||||||
the fixed image and the tight T1c head crop (~220×220×150 mm) as moving, most
|
the fixed image and the tight T1c head crop (~220×220×150 mm) as moving, most
|
||||||
of the CT buffer maps outside the T1c buffer and `impact` aborts on the M6
|
of the CT buffer maps outside the T1c buffer (~14% of samples in-buffer at the
|
||||||
pairs ("too many samples map outside moving image buffer", ~14% of samples
|
auto-init). Elastix's sample-validity gate (build default
|
||||||
in-buffer at the auto-init) — see the IMPACT section. The comparison is on
|
`RequiredRatioOfValidSamples 0.25`) used to abort the M6 pairs on that ("too
|
||||||
each toolkit's native optimizer / metric / global-search capability.
|
many samples map outside moving image buffer"); the parameter map now sets
|
||||||
|
`RequiredRatioOfValidSamples 0.1` so the FOV-mismatched runs proceed — see the
|
||||||
|
IMPACT section. The comparison is on each toolkit's native optimizer / metric
|
||||||
|
/ global-search capability.
|
||||||
|
|
||||||
## The six pipelines
|
## The six pipelines
|
||||||
|
|
||||||
|
|
@ -61,7 +64,7 @@ below; `clip_ct=None` disables it).
|
||||||
|---|----------|----------------|------|--------|-----------|--------------------|
|
|---|----------|----------------|------|--------|-----------|--------------------|
|
||||||
| 1 | `antspyx` | CT / T1c | auto COM | Mattes MI (multi-res) | ANTs `Rigid` | ✅ ran (main env; stochastic MI subsampling) |
|
| 1 | `antspyx` | CT / T1c | auto COM | Mattes MI (multi-res) | ANTs `Rigid` | ✅ ran (main env; stochastic MI subsampling) |
|
||||||
| 2 | `elastix` | CT / T1c | auto CenterOfGravity | Mattes MI (32 bins) | `EulerTransform` rigid (5.x), ASGD, 3 resolutions | ✅ ran (main env, in-process python API — no CLI) |
|
| 2 | `elastix` | CT / T1c | auto CenterOfGravity | Mattes MI (32 bins) | `EulerTransform` rigid (5.x), ASGD, 3 resolutions | ✅ ran (main env, in-process python API — no CLI) |
|
||||||
| 3 | `impact` | **CT / T1c** | auto CenterOfGravity | **IMPACT semantic** (TotalSegmentator `TS/M730.pt`, L2, Jacobian/GPU) | `EulerTransform` rigid, ASGD, 4 scales | ⛔ aborts on M6 (no-mask rule; see IMPACT section) |
|
| 3 | `impact` | **CT / T1c** | auto CenterOfGravity | **IMPACT semantic** (MRSegmentator `MRSeg/MRSeg.pt`, L2, Jacobian/GPU) | `EulerTransform` rigid, ASGD, 4 scales | ✅ ran (Docker + GPU; sample gate 0.1 — see IMPACT section) |
|
||||||
| 4 | `niftyreg` | CT / T1c | image centres | NCC block matching + trimmed least squares (symmetric) | `reg_aladin -rigOnly`, 3-level pyramid | ✅ ran (Docker `niftyreg:2.1.1-ubuntu`, GPU `-platf 1`) |
|
| 4 | `niftyreg` | CT / T1c | image centres | NCC block matching + trimmed least squares (symmetric) | `reg_aladin -rigOnly`, 3-level pyramid | ✅ ran (Docker `niftyreg:2.1.1-ubuntu`, GPU `-platf 1`) |
|
||||||
| 5 | `simpleitk` | CT / T1c | identity | Mattes MI (32 bins) | `Euler3D` rigid, pyramid `[4,2,1]` | ✅ ran |
|
| 5 | `simpleitk` | CT / T1c | identity | Mattes MI (32 bins) | `Euler3D` rigid, pyramid `[4,2,1]` | ✅ ran |
|
||||||
| 6 | `synthmorph` | CT / T1c | none (global) | **SynthMorph** anatomy-aware contrast-invariant (deep, pretrained) | 6-DOF rigid matrix | ✅ ran (main env, Docker + GPU) |
|
| 6 | `synthmorph` | CT / T1c | none (global) | **SynthMorph** anatomy-aware contrast-invariant (deep, pretrained) | 6-DOF rigid matrix | ✅ ran (main env, Docker + GPU) |
|
||||||
|
|
@ -139,7 +142,7 @@ pairs under the no-mask rule (see the IMPACT section).
|
||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
| **antspyx** | ANTsPy 0.6.3 | 9 | 36.6 | 0.93 | **0.653 / 0.644** |
|
| **antspyx** | ANTsPy 0.6.3 | 9 | 36.6 | 0.93 | **0.653 / 0.644** |
|
||||||
| **elastix** | ITK-Elastix 0.25.4 | 9 | 6.0 | 0.93 | **0.517 / 0.520** |
|
| **elastix** | ITK-Elastix 0.25.4 | 9 | 6.0 | 0.93 | **0.517 / 0.520** |
|
||||||
| **impact** | Docker Elastix 5.2 + IMPACT | 0 | - | - | - (aborts, no mask) |
|
| **impact** | Docker Elastix 5.2 + IMPACT (MRSegmentator) | 8 | 2534 | 0.93 | **0.591 / 0.575** |
|
||||||
| **niftyreg** | NiftyReg 2.1.1 (reg_aladin, Docker + CUDA) | 9 | 20.8 | 0.93 | **0.292 / 0.338** |
|
| **niftyreg** | NiftyReg 2.1.1 (reg_aladin, Docker + CUDA) | 9 | 20.8 | 0.93 | **0.292 / 0.338** |
|
||||||
| **simpleitk** | SimpleITK 2.5.6 | 9 | 10.6 | 0.93 | **0.651 / 0.641** |
|
| **simpleitk** | SimpleITK 2.5.6 | 9 | 10.6 | 0.93 | **0.651 / 0.641** |
|
||||||
| **synthmorph** | Docker SynthMorph (TensorFlow) | 9 | 24.5 | 0.93 | **0.277 / 0.328** |
|
| **synthmorph** | Docker SynthMorph (TensorFlow) | 9 | 24.5 | 0.93 | **0.277 / 0.328** |
|
||||||
|
|
@ -149,15 +152,15 @@ Per-case MI (nats, unclipped score; **bold** = best for the case):
|
||||||
| case | antspyx | elastix | impact | niftyreg | simpleitk | synthmorph |
|
| case | antspyx | elastix | impact | niftyreg | simpleitk | synthmorph |
|
||||||
|---|---|---|---|---|---|---|
|
|---|---|---|---|---|---|---|
|
||||||
| m6_22GD5VL2_20230724 | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT |
|
| m6_22GD5VL2_20230724 | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT |
|
||||||
| m6_22IRRKU6_20240115 | **0.649** | 0.472 | abort | 0.292 | 0.642 | 0.277 |
|
| m6_22IRRKU6_20240115 | **0.649** | 0.472 | 0.635 | 0.292 | 0.642 | 0.277 |
|
||||||
| m6_23XIMO4M_20230222 | **0.655** | 0.416 | abort | 0.632 | 0.652 | 0.616 |
|
| m6_23XIMO4M_20230222 | **0.655** | 0.416 | 0.478 | 0.632 | 0.652 | 0.616 |
|
||||||
| m6_24V62QQM_20230223 | **0.704** | 0.701 | abort | 0.260 | 0.663 | 0.256 |
|
| m6_24V62QQM_20230223 | **0.704** | 0.701 | 0.697 | 0.260 | 0.663 | 0.256 |
|
||||||
| m6_24V62QQM_20231017 | 0.601 | 0.590 | abort | 0.493 | **0.602** | 0.484 |
|
| m6_24V62QQM_20231017 | 0.601 | 0.590 | 0.556 | 0.493 | **0.602** | 0.484 |
|
||||||
| m6_24V62QQM_20240820 | **0.653** | 0.644 | abort | 0.118 | 0.652 | 0.111 |
|
| m6_24V62QQM_20240820 | **0.653** | 0.644 | 0.531 | 0.118 | 0.652 | 0.111 |
|
||||||
| m6_26TYQI3C_20221027 | 0.659 | 0.653 | abort | 0.609 | **0.695** | 0.590 |
|
| m6_26TYQI3C_20221027 | 0.659 | 0.653 | gate-abort | 0.609 | **0.695** | 0.590 |
|
||||||
| m6_2BBN5XA4_20250411 | **0.577** | 0.287 | abort | 0.069 | **0.577** | 0.061 |
|
| m6_2BBN5XA4_20250411 | **0.577** | 0.287 | 0.450 | 0.069 | **0.577** | 0.061 |
|
||||||
| m6_2C4TKI6H_20240506 | **0.645** | 0.401 | abort | 0.233 | 0.634 | 0.231 |
|
| m6_2C4TKI6H_20240506 | **0.645** | 0.401 | 0.629 | 0.233 | 0.634 | 0.231 |
|
||||||
| m6_2CDNN7BH_20240102 | **0.655** | 0.516 | abort | 0.340 | 0.651 | 0.323 |
|
| m6_2CDNN7BH_20240102 | **0.655** | 0.516 | 0.625 | 0.340 | 0.651 | 0.323 |
|
||||||
|
|
||||||
### Interpretation
|
### Interpretation
|
||||||
|
|
||||||
|
|
@ -179,13 +182,19 @@ Per-case MI (nats, unclipped score; **bold** = best for the case):
|
||||||
single CenterOfGravity start still locks onto local minima on the
|
single CenterOfGravity start still locks onto local minima on the
|
||||||
repositioned ones (`m6_2BBN5XA4` 0.287, `m6_2C4TKI6H` 0.401, `m6_23XIMO4M`
|
repositioned ones (`m6_2BBN5XA4` 0.287, `m6_2C4TKI6H` 0.401, `m6_23XIMO4M`
|
||||||
0.416).
|
0.416).
|
||||||
- **IMPACT (semantic metric) does not run on these pairs under the no-mask
|
- **IMPACT (semantic metric) now runs on these pairs** — the no-mask abort was
|
||||||
rule** — earlier (masked) runs showed it strongest on the hardest cases
|
elastix's sample-validity gate (build default `RequiredRatioOfValidSamples`
|
||||||
(`m6_22IRRKU6` 0.624, `m6_2C4TKI6H` 0.626, `m6_2CDNN7BH` 0.618, median
|
0.25) rejecting the ~14% in-buffer fraction of the CT head-neck FOV vs the
|
||||||
0.511) with a stochastic draw per run; it also was by far the slowest
|
tight T1c crop; the parameter map sets it to 0.1 (see the IMPACT section).
|
||||||
(~14 min/case at the 500/2000 budget, hours/case at 2000/5000). With the CT
|
With MRSegmentator at the 2000/5000 budget it is top-three in MI on 7 of the
|
||||||
head-neck FOV as the fixed volume it aborts ("too many samples map outside
|
8 registered cases (median 0.591): 0.635 on the hardest `m6_22IRRKU6`
|
||||||
moving image buffer", ~10–20% in-buffer) — see the IMPACT section.
|
(masked-era 0.624), 0.697 on `m6_24V62QQM_20230223` (near-best), 0.629 on
|
||||||
|
`m6_2C4TKI6H` — and it shares the production-acceptance of the grossly
|
||||||
|
repositioned `m6_2BBN5XA4` (std 10.7 mm) with antspyx/simpleitk, the only
|
||||||
|
non-intensity tool to do so. One run diverges mid-registration
|
||||||
|
(`m6_26TYQI3C`: the in-buffer fraction falls below 10% at iteration 256 and
|
||||||
|
the gate aborts — a stochastic-draw outcome). It remains by far the slowest
|
||||||
|
(~42 min/case at 2000/5000).
|
||||||
- **NiftyReg (reg_aladin) is mid-pack**: single-start symmetric block matching
|
- **NiftyReg (reg_aladin) is mid-pack**: single-start symmetric block matching
|
||||||
is strong on moderate cases (`m6_23XIMO4M` 0.632, best in class;
|
is strong on moderate cases (`m6_23XIMO4M` 0.632, best in class;
|
||||||
`m6_26TYQI3C` 0.609) but falls into the same tilt local minima
|
`m6_26TYQI3C` 0.609) but falls into the same tilt local minima
|
||||||
|
|
@ -205,9 +214,11 @@ Per-case MI (nats, unclipped score; **bold** = best for the case):
|
||||||
CT head everywhere, so coverage is not the discriminator — MI is.
|
CT head everywhere, so coverage is not the discriminator — MI is.
|
||||||
- **Speed** (median wall, 4 cases in parallel, 24 threads): elastix 6.0 s,
|
- **Speed** (median wall, 4 cases in parallel, 24 threads): elastix 6.0 s,
|
||||||
SimpleITK 10.6 s, **NiftyReg 20.8 s** (v2.1.1, GPU block matching),
|
SimpleITK 10.6 s, **NiftyReg 20.8 s** (v2.1.1, GPU block matching),
|
||||||
SynthMorph 24.5 s (GPU forward pass), ANTs 36.6 s. (IMPACT does not run on
|
SynthMorph 24.5 s (GPU forward pass), ANTs 36.6 s. (IMPACT is by far the
|
||||||
M6 under the no-mask rule; when it did — masked, 500/2000 budget — it was
|
slowest — 2534 s/case median at the 2000/5000 benchmark budget, ~276 s/case
|
||||||
~833 s/case, GPU feature extraction with backprop per iteration.)
|
at the authors' 500/2000; GPU feature extraction with backprop per
|
||||||
|
iteration. No-mask runs are faster per iteration than the masked-era
|
||||||
|
~833 s/case @500/2000 because only the ~14% in-buffer samples are scored.)
|
||||||
- **Stochasticity**: ANTs' Mattes-MI subsamples the image at random, so its
|
- **Stochasticity**: ANTs' Mattes-MI subsamples the image at random, so its
|
||||||
per-case numbers carry run-to-run variance (this table is one draw);
|
per-case numbers carry run-to-run variance (this table is one draw);
|
||||||
SimpleITK's MI (and registration) is deterministic at fixed thread count;
|
SimpleITK's MI (and registration) is deterministic at fixed thread count;
|
||||||
|
|
@ -235,7 +246,7 @@ Median achieved MI (nats) and wall-clock (9 complete-CT cases):
|
||||||
| niftyreg | 0.296 / 23.8 s | 0.320 / 20.2 s | +0.024 | 5/9 |
|
| niftyreg | 0.296 / 23.8 s | 0.320 / 20.2 s | +0.024 | 5/9 |
|
||||||
| simpleitk | 0.651 / 10.6 s | 0.436 / 11.1 s | **−0.215** | 0/9 |
|
| simpleitk | 0.651 / 10.6 s | 0.436 / 11.1 s | **−0.215** | 0/9 |
|
||||||
| synthmorph | 0.241 / 26.0 s | 0.318 / 25.6 s | +0.077 | 5/9 |
|
| synthmorph | 0.241 / 26.0 s | 0.318 / 25.6 s | +0.077 | 5/9 |
|
||||||
| impact | abort | abort | - | - (still aborts; clipping does not fix the FOV geometry, 194–388/2000 samples in-buffer) |
|
| impact | abort | abort | - | - (historical: at the time both aborted on the old 0.25 sample-validity gate; the gate is now 0.1, A/B to be re-done) |
|
||||||
|
|
||||||
**Verdict: no — clipping the CT to [0, 80] HU does not improve the result; do
|
**Verdict: no — clipping the CT to [0, 80] HU does not improve the result; do
|
||||||
not clip.**
|
not clip.**
|
||||||
|
|
@ -286,8 +297,10 @@ python scripts/bench_registration.py --pipelines simpleitk elastix antspyx nifty
|
||||||
# sections below for the one-time setup)
|
# sections below for the one-time setup)
|
||||||
python scripts/bench_registration.py --pipelines synthmorph --n-cases 10 \
|
python scripts/bench_registration.py --pipelines synthmorph --n-cases 10 \
|
||||||
--workers 4 --threads 24
|
--workers 4 --threads 24
|
||||||
# NOTE: impact aborts on the M6 pairs under the shared no-mask rule (the CT
|
# NOTE: impact now runs on the M6 pairs (the parameter map relaxes the
|
||||||
# head-neck FOV maps mostly outside the T1c head crop) — see the IMPACT section
|
# sample-validity gate to RequiredRatioOfValidSamples 0.1) but is by far the
|
||||||
|
# slowest: ~5 min/case at 500/2000, ~42 min/case (2534 s median) at 2000/5000
|
||||||
|
# (--impact-iters / --impact-samples) — see the IMPACT section
|
||||||
python scripts/bench_registration.py --pipelines impact --n-cases 10 \
|
python scripts/bench_registration.py --pipelines impact --n-cases 10 \
|
||||||
--workers 4 --threads 24
|
--workers 4 --threads 24
|
||||||
|
|
||||||
|
|
@ -346,8 +359,7 @@ python scripts/bench_registration.py \
|
||||||
--pipelines synthmorph --n-cases 10 --qa-images
|
--pipelines synthmorph --n-cases 10 --qa-images
|
||||||
```
|
```
|
||||||
|
|
||||||
(9 images each; the incomplete-CT case is skipped. `impact` produces none on
|
(9 images each; the incomplete-CT case is skipped.)
|
||||||
M6 — it aborts under the no-mask rule, see the IMPACT section.)
|
|
||||||
|
|
||||||
## SynthMorph pipeline (Docker) — one-time setup
|
## SynthMorph pipeline (Docker) — one-time setup
|
||||||
|
|
||||||
|
|
@ -442,14 +454,17 @@ conda install -c conda-forge niftyreg
|
||||||
The `impact` pipeline runs the [`vboussot/elastix_impact`](https://hub.docker.com/r/vboussot/elastix_impact)
|
The `impact` pipeline runs the [`vboussot/elastix_impact`](https://hub.docker.com/r/vboussot/elastix_impact)
|
||||||
image: **Elastix 5.2.0** with the [IMPACT](https://arxiv.org/abs/2503.24121)
|
image: **Elastix 5.2.0** with the [IMPACT](https://arxiv.org/abs/2503.24121)
|
||||||
*semantic* similarity metric compiled in as a native Elastix metric, plus a
|
*semantic* similarity metric compiled in as a native Elastix metric, plus a
|
||||||
TorchScript **TotalSegmentator** feature extractor. It is GPU-based and
|
TorchScript segmentation feature extractor (**MRSegmentator** `MRSeg/MRSeg.pt`
|
||||||
|
by default; TotalSegmentator `TS/M730.pt` and other models from
|
||||||
|
`VBoussot/impact-torchscript-models` are selectable). It is GPU-based and
|
||||||
Docker-based (alongside `synthmorph`). It follows the shared rules (full
|
Docker-based (alongside `synthmorph`). It follows the shared rules (full
|
||||||
volumes, no mask, fixed = CT, moving = T1c) — which means it **aborts on the
|
volumes, no mask, fixed = CT, moving = T1c). The metric only evaluates samples
|
||||||
M6 pairs**: the IMPACT metric only evaluates samples that map inside the
|
that map inside the moving buffer, and the CT head-neck FOV (~300×300×251 mm)
|
||||||
moving buffer, and the CT head-neck FOV (~300×300×251 mm) is much larger than
|
is much larger than the T1c head crop (~220×220×150 mm), so ~14% of the samples
|
||||||
the T1c head crop (~220×220×150 mm), so ~14% of the samples are in-buffer at
|
are in-buffer at the auto-init. The image build's sample-validity gate
|
||||||
the auto-init and elastix raises "too many samples map outside moving image
|
defaults to `RequiredRatioOfValidSamples 0.25` and used to abort the M6 pairs
|
||||||
buffer" (see the IMPACT specifics below).
|
on that ("too many samples map outside moving image buffer"); the parameter
|
||||||
|
map now sets it to `0.1` (see the IMPACT specifics below).
|
||||||
|
|
||||||
Prerequisites (check with `RP.availability()`; a missing piece raises
|
Prerequisites (check with `RP.availability()`; a missing piece raises
|
||||||
`PipelineUnavailable` with the exact command):
|
`PipelineUnavailable` with the exact command):
|
||||||
|
|
@ -462,16 +477,23 @@ docker --version
|
||||||
docker pull vboussot/elastix_impact
|
docker pull vboussot/elastix_impact
|
||||||
|
|
||||||
# 3. Fetch the TorchScript model (~123 MB, one-time) into the repo's model
|
# 3. Fetch the TorchScript model (~123 MB, one-time) into the repo's model
|
||||||
# cache (data/impact_models/). M730.pt is the recommended TotalSegmentator.
|
# cache (data/impact_models/). MRSeg.pt (MRSegmentator) is the default:
|
||||||
mkdir -p /mnt/t24/Public/xfr/longitudinal/impact_models/TS
|
# it is trained on both MRI and CT, the right prior for this
|
||||||
curl -L -o /mnt/t24/Public/xfr/longitudinal/impact_models/TS/M730.pt \
|
# cross-modality CT->T1c task (all models share the same 2^l+3 FOV
|
||||||
https://huggingface.co/VBoussot/impact-torchscript-models/resolve/main/TS/M730.pt
|
# architecture, so the parameter map is unchanged between them).
|
||||||
|
mkdir -p /mnt/t24/Public/xfr/longitudinal/impact_models/MRSeg
|
||||||
|
curl -L -o /mnt/t24/Public/xfr/longitudinal/impact_models/MRSeg/MRSeg.pt \
|
||||||
|
https://huggingface.co/VBoussot/impact-torchscript-models/resolve/main/MRSeg/MRSeg.pt
|
||||||
```
|
```
|
||||||
|
|
||||||
Other models (M258/M291/M850, MIND, SAM2.1, …) are selectable with
|
Other models (TotalSegmentator `TS/M730.pt`, `TS/M258/M291/M850`, MIND, SAM2.1,
|
||||||
`--impact-model <HF-relative-path-or-abs>`; see the
|
Anatomix, …) are selectable with `--impact-model <HF-relative-path-or-abs>`;
|
||||||
|
see the
|
||||||
[impact-torchscript-models](https://huggingface.co/VBoussot/impact-torchscript-models)
|
[impact-torchscript-models](https://huggingface.co/VBoussot/impact-torchscript-models)
|
||||||
repo for the full list and FOV/layer tables.
|
repo for the full list and FOV/layer tables. The model choice does not affect
|
||||||
|
the sample-validity gate (below): the "too many samples outside moving buffer"
|
||||||
|
check is geometric (CT head-neck FOV vs the tight T1c crop) and fires at the
|
||||||
|
first sample evaluation, before feature extraction.
|
||||||
|
|
||||||
### How the adapter drives the image
|
### How the adapter drives the image
|
||||||
|
|
||||||
|
|
@ -479,14 +501,16 @@ The image's `CMD.sh` is a generic launcher, so the adapter calls the in-image
|
||||||
binary directly (`/lib/elastix-install/bin/elastix`):
|
binary directly (`/lib/elastix-install/bin/elastix`):
|
||||||
|
|
||||||
1. **Full volumes, fixed=CT, moving=T1c** — both volumes are used as-is (no
|
1. **Full volumes, fixed=CT, moving=T1c** — both volumes are used as-is (no
|
||||||
head box, no mask, no crop / pad, per the shared rules above). **On the M6
|
head box, no mask, no crop / pad, per the shared rules above). The metric
|
||||||
pairs this aborts**: the IMPACT metric only evaluates spatial samples that
|
only evaluates spatial samples that map inside the moving buffer, and the
|
||||||
map inside the moving buffer, and the CT FOV (~300×300×251 mm, head+neck)
|
CT FOV (~300×300×251 mm, head+neck) is much larger than the tight T1c head
|
||||||
is much larger than the tight T1c head crop (~220×220×150 mm), so at the
|
crop (~220×220×150 mm), so at the CenterOfGravity auto-init only ~14% of
|
||||||
CenterOfGravity auto-init only ~14% of the 2000 samples are in-buffer and
|
the samples are in-buffer. The image build's default validity gate
|
||||||
elastix raises "Too many samples map outside moving image buffer: 277 /
|
(`RequiredRatioOfValidSamples 0.25`) aborted on that ("Too many samples map
|
||||||
2000". IMPACT therefore needs a fixed volume whose FOV is comparable to
|
outside moving image buffer: 704 / 5000"); the parameter map therefore sets
|
||||||
the moving crop (e.g. a head-only CT) to work under the no-mask rule.
|
`(RequiredRatioOfValidSamples 0.1)` so the M6 pairs run on their in-buffer
|
||||||
|
samples while a genuinely diverging transform (<10% in-buffer) still
|
||||||
|
aborts.
|
||||||
2. **Auto init (no `-t0`)** — the parameter map sets
|
2. **Auto init (no `-t0`)** — the parameter map sets
|
||||||
`AutomaticTransformInitialization "true"` with method `CenterOfGravity`, so
|
`AutomaticTransformInitialization "true"` with method `CenterOfGravity`, so
|
||||||
elastix aligns the intensity centroids; `AutomaticScalesEstimation` sets the
|
elastix aligns the intensity centroids; `AutomaticScalesEstimation` sets the
|
||||||
|
|
@ -495,9 +519,24 @@ binary directly (`/lib/elastix-install/bin/elastix`):
|
||||||
**Jacobian** mode (differentiable, per the recommended config), 4 semantic
|
**Jacobian** mode (differentiable, per the recommended config), 4 semantic
|
||||||
scales `ImpactVoxelSize` 6/3/1.5/1 mm (scaled to the T1c base spacing) with
|
scales `ImpactVoxelSize` 6/3/1.5/1 mm (scaled to the T1c base spacing) with
|
||||||
**no image resampling** (all-1 pyramid schedules, as the IMPACT docs require),
|
**no image resampling** (all-1 pyramid schedules, as the IMPACT docs require),
|
||||||
ASGD, `Random` sampler. GPU auto-spread across the host GPUs by pid
|
ASGD, `Random` sampler, `(RequiredRatioOfValidSamples 0.1)`. A copy of the
|
||||||
(`--impact-gpu -1` forces CPU).
|
exact parameter map is dumped to
|
||||||
4. **Read-back** — the result `TransformParameters.0.itk.txt` is an
|
`data/m6reg/impact/paramdump/ParameterMap_<model>_<pid>.txt` before every
|
||||||
|
docker run (also recorded as `param_dump` in the result JSON). GPU
|
||||||
|
auto-spread across the host GPUs by pid (`--impact-gpu -1` forces CPU).
|
||||||
|
4. **Intensity standardization happens inside the metric** (no pipeline-side
|
||||||
|
normalization — the model card's "standardize intensities to zero mean and
|
||||||
|
unit variance" is already applied): at init the metric computes
|
||||||
|
whole-volume `[min, max, mean, sigma]` for each image
|
||||||
|
(`itk::StatisticsImageFilter`) and passes it to the model, whose
|
||||||
|
`normalize` branch rescales every extracted patch with the image's own
|
||||||
|
`(mean, sigma)`. It is computed over the whole buffer (CT air included) and
|
||||||
|
identically for fixed and moving, so both modalities' features sit on the
|
||||||
|
same normalized scale. Pre-standardizing the MHA files in the pipeline
|
||||||
|
would be an exact no-op (z-scoring is affine-invariant: the metric
|
||||||
|
recomputes the stats of the image it receives) and would only corrupt the
|
||||||
|
HU scale used by MI scoring and the QA brain-window images.
|
||||||
|
5. **Read-back** — the result `TransformParameters.0.itk.txt` is an
|
||||||
`Euler3DTransform` in the world physical frame (the
|
`Euler3DTransform` in the world physical frame (the
|
||||||
`AutomaticTransformInitialization` is folded in by elastix); the adapter
|
`AutomaticTransformInitialization` is folded in by elastix); the adapter
|
||||||
runs it through the usual `fix_orientation` gate.
|
runs it through the usual `fix_orientation` gate.
|
||||||
|
|
@ -590,29 +629,41 @@ binary directly (`/lib/elastix-install/bin/elastix`):
|
||||||
defaults (`NumberOfSpatialSamples` 5000, `NumberOfHistogramBins` 32).
|
defaults (`NumberOfSpatialSamples` 5000, `NumberOfHistogramBins` 32).
|
||||||
- **IMPACT specifics** (Docker `vboussot/elastix_impact`, Elastix 5.2.0,
|
- **IMPACT specifics** (Docker `vboussot/elastix_impact`, Elastix 5.2.0,
|
||||||
GPU, Jacobian mode):
|
GPU, Jacobian mode):
|
||||||
- **Slow, by design.** Every ASGD iteration runs the TotalSegmentator
|
- **Slow, by design.** Every ASGD iteration runs the segmentation
|
||||||
extractor on the sampled patches with backpropagation. The benchmark
|
extractor (MRSegmentator by default) on the sampled patches with
|
||||||
|
backpropagation. The benchmark
|
||||||
budget (MaximumNumberOfIterations 2000, NumberOfSpatialSamples 5000 =
|
budget (MaximumNumberOfIterations 2000, NumberOfSpatialSamples 5000 =
|
||||||
the elastix default) is ~10× the authors' 500-iter / 2000-sample
|
the elastix default) is ~10× the authors' 500-iter / 2000-sample
|
||||||
Jacobian budget, i.e. on the order of *hours* per case on full volumes —
|
Jacobian budget: measured 2534 s/case (median) on the no-mask M6
|
||||||
the results shown for `impact` are from the previous 500/2000 budget and
|
pairs, ~276 s/case at 500/2000. `--impact-iters` / `--impact-samples`
|
||||||
were not re-run at the new budget. `--impact-iters` / `--impact-samples`
|
|
||||||
trade accuracy for speed.
|
trade accuracy for speed.
|
||||||
- **Aborts on M6 under the no-mask rule.** Like every other pipeline it
|
- **Runs on M6 with a relaxed sample-validity gate.** Like every other
|
||||||
runs fixed=CT/moving=T1c on the full volumes (no mask, no crop / pad).
|
pipeline it runs fixed=CT/moving=T1c on the full volumes (no mask, no
|
||||||
The CT FOV (~300×300×251 mm) is much larger than the T1c crop
|
crop / pad). The CT FOV (~300×300×251 mm) is much larger than the T1c
|
||||||
(~220×220×150 mm), so most of the CT buffer maps outside the T1c moving
|
crop (~220×220×150 mm), so at the CenterOfGravity auto-init only ~14% of
|
||||||
buffer and the metric aborts ("Too many samples map outside moving image
|
the samples map inside the T1c moving buffer. The metric calls
|
||||||
buffer: 277 / 2000" on M6 at the CenterOfGravity auto-init). The
|
`CheckNumberOfSamples()` after every evaluation and the image build
|
||||||
transform contract (T1c→CT) still holds when a run completes; for M6
|
defaults to `RequiredRatioOfValidSamples 0.25`, so it used to abort
|
||||||
pairs a head-only CT as the fixed volume would be needed.
|
("Too many samples map outside moving image buffer: 704 / 5000"). The
|
||||||
|
parameter map now sets `(RequiredRatioOfValidSamples 0.1)`: the gate
|
||||||
|
still aborts a genuinely diverging transform (<10% in-buffer) while the
|
||||||
|
FOV-mismatched M6 pairs run on their ~14% in-buffer samples — exactly
|
||||||
|
the native "only in-buffer samples are evaluated" behaviour. In the
|
||||||
|
2000/5000 re-run the gate still fired once, mid-registration: on
|
||||||
|
`m6_26TYQI3C` the transform drifted until only 468/5000 samples (9.4%)
|
||||||
|
stayed in-buffer at iteration 256 and the run aborted. The transform
|
||||||
|
contract (T1c→CT) holds; `fix_orientation` decides the direction as
|
||||||
|
usual (all 8 completed runs report `auto_inverted: True`).
|
||||||
- **Auto init.** `AutomaticTransformInitialization` (CenterOfGravity) aligns
|
- **Auto init.** `AutomaticTransformInitialization` (CenterOfGravity) aligns
|
||||||
the intensity centroids; no manual `-t0` translation is used.
|
the intensity centroids; no manual `-t0` translation is used.
|
||||||
- **History: single-init convergence.** The semantic landscape has more
|
- **History: single-init convergence.** The semantic landscape has more
|
||||||
local minima than Mattes-MI; earlier (masked, fixed=CT) runs no longer
|
local minima than Mattes-MI; earlier (masked, fixed=CT) runs no longer
|
||||||
nailed the hardest repositioned case (`m6_22IRRKU6` mid 22.7 / std 27.0;
|
nailed the hardest repositioned case (`m6_22IRRKU6` mid 22.7 / std 27.0;
|
||||||
the pre-mask head-box flow got it to mid 8.3 / std 13.2). Under the
|
the pre-mask head-box flow got it to mid 8.3 / std 13.2). The no-mask
|
||||||
current no-mask rule it does not run on M6 at all (see above).
|
MRSegmentator runs show the same pattern: strong MI (0.635 on
|
||||||
|
`m6_22IRRKU6` at 2000/5000) but the pose still misses the concentricity
|
||||||
|
acceptance rule there (mid 26.1 / std 28.6); it *is* accepted on
|
||||||
|
`m6_2BBN5XA4` (mid 15.1 / std 10.7).
|
||||||
- **Stochastic.** Jacobian mode resamples patches each iteration, so results
|
- **Stochastic.** Jacobian mode resamples patches each iteration, so results
|
||||||
vary run to run (see Stochasticity).
|
vary run to run (see Stochasticity).
|
||||||
- **Elastix 5.x differences vs the in-process `elastix` pipeline:** the image
|
- **Elastix 5.x differences vs the in-process `elastix` pipeline:** the image
|
||||||
|
|
|
||||||
|
|
@ -654,7 +654,7 @@ def register_synthmorph(ct, t1c, model="rigid", gpu=None, threads=None,
|
||||||
IMPACT_IMAGE = "vboussot/elastix_impact"
|
IMPACT_IMAGE = "vboussot/elastix_impact"
|
||||||
IMPACT_ELASTIX_BIN = "/lib/elastix-install/bin/elastix"
|
IMPACT_ELASTIX_BIN = "/lib/elastix-install/bin/elastix"
|
||||||
IMPACT_MODEL_CACHE = "data/impact_models"
|
IMPACT_MODEL_CACHE = "data/impact_models"
|
||||||
IMPACT_DEFAULT_MODEL = "TS/M730.pt"
|
IMPACT_DEFAULT_MODEL = "MRSeg/MRSeg.pt"
|
||||||
IMPACT_HF_MODEL_URL = (
|
IMPACT_HF_MODEL_URL = (
|
||||||
"https://huggingface.co/VBoussot/impact-torchscript-models/resolve/main/{rel}"
|
"https://huggingface.co/VBoussot/impact-torchscript-models/resolve/main/{rel}"
|
||||||
)
|
)
|
||||||
|
|
@ -757,6 +757,14 @@ def _impact_param_map(model_cont_path, gpu, iters, samples, voxel_sizes,
|
||||||
f'(ImpactUseMixedPrecision "{mixed}")',
|
f'(ImpactUseMixedPrecision "{mixed}")',
|
||||||
"(ImpactFeaturesMapUpdateInterval -1)",
|
"(ImpactFeaturesMapUpdateInterval -1)",
|
||||||
'(ImpactWriteFeatureMaps "false")',
|
'(ImpactWriteFeatureMaps "false")',
|
||||||
|
# safety gate: abort if < ratio of the spatial samples map inside the
|
||||||
|
# moving buffer. The build default is 0.25, but the no-mask M6 pairs
|
||||||
|
# (CT head-neck FOV vs the tight T1c crop) only have ~14% in-buffer at
|
||||||
|
# the CenterOfGravity auto-init, so elastix aborts at the first metric
|
||||||
|
# evaluation. 0.1 keeps the gate (a diverging transform still aborts)
|
||||||
|
# while letting the FOV-mismatched runs proceed -- the metric natively
|
||||||
|
# evaluates only the in-buffer samples.
|
||||||
|
"(RequiredRatioOfValidSamples 0.1)",
|
||||||
"",
|
"",
|
||||||
"// rigid 6-DOF to be optimized (auto center init, no -t0)",
|
"// rigid 6-DOF to be optimized (auto center init, no -t0)",
|
||||||
'(Transform "EulerTransform")',
|
'(Transform "EulerTransform")',
|
||||||
|
|
@ -820,12 +828,21 @@ def register_impact(ct, t1c, model=None, gpu=None, iters=2000, samples=5000,
|
||||||
``vboussot/elastix_impact`` (Elastix 5.2.0 + IMPACT semantic metric).
|
``vboussot/elastix_impact`` (Elastix 5.2.0 + IMPACT semantic metric).
|
||||||
|
|
||||||
IMPACT (arXiv:2503.24121) replaces the intensity metric with a *semantic*
|
IMPACT (arXiv:2503.24121) replaces the intensity metric with a *semantic*
|
||||||
one: a TorchScript TotalSegmentator model (``TS/M730.pt`` by default) is run
|
one: a TorchScript segmentation model (``MRSeg/MRSeg.pt`` by default,
|
||||||
on randomly sampled patches of the fixed and moving images and the
|
MRSegmentator -- robust multi-modality segmentation on MRI *and* CT, the
|
||||||
|
right prior for this cross-modality CT->T1c task; TotalSegmentator
|
||||||
|
``TS/M730.pt`` and the other models in the HF repo
|
||||||
|
``VBoussot/impact-torchscript-models`` work too via ``model=``) is run on
|
||||||
|
randomly sampled patches of the fixed and moving images and the
|
||||||
registration minimizes the distance between the extracted feature maps
|
registration minimizes the distance between the extracted feature maps
|
||||||
(L2, per the recommended config). This is a genuinely different metric from
|
(L2, per the recommended config). Intensity standardization (zero mean /
|
||||||
the Mattes-MI pipelines above -- it is the cross-modality tool the image was
|
unit variance per the model card) is applied *inside the metric*: at init
|
||||||
built for.
|
it computes whole-volume [min, max, mean, sigma] per image and the model's
|
||||||
|
normalize branch rescales every patch with that image's own (mean, sigma),
|
||||||
|
so no pipeline-side pre-normalization is needed (and would be an exact
|
||||||
|
no-op -- z-scoring is affine-invariant). This is a genuinely different
|
||||||
|
metric from the Mattes-MI pipelines above -- it is the cross-modality tool
|
||||||
|
the image was built for.
|
||||||
|
|
||||||
The pipeline registers on the full CT / T1c volumes (no head box) with no
|
The pipeline registers on the full CT / T1c volumes (no head box) with no
|
||||||
manual centroid init: elastix's ``AutomaticTransformInitialization``
|
manual centroid init: elastix's ``AutomaticTransformInitialization``
|
||||||
|
|
@ -910,10 +927,22 @@ def register_impact(ct, t1c, model=None, gpu=None, iters=2000, samples=5000,
|
||||||
os.path.join(datadir, "Fixed_image.mha"), True)
|
os.path.join(datadir, "Fixed_image.mha"), True)
|
||||||
sitk.WriteImage(sitk.Cast(moving, sitk.sitkFloat32),
|
sitk.WriteImage(sitk.Cast(moving, sitk.sitkFloat32),
|
||||||
os.path.join(datadir, "Moving_image.mha"), True)
|
os.path.join(datadir, "Moving_image.mha"), True)
|
||||||
with open(os.path.join(datadir, "ParameterMap.txt"), "w") as f:
|
param_text = _impact_param_map(os.path.join("/Data/Models", model_rel),
|
||||||
f.write(_impact_param_map(os.path.join("/Data/Models", model_rel),
|
|
||||||
gpu, iters, samples, voxel_sizes,
|
gpu, iters, samples, voxel_sizes,
|
||||||
rescale=schedule, smoothing=smoothe))
|
rescale=schedule, smoothing=smoothe)
|
||||||
|
with open(os.path.join(datadir, "ParameterMap.txt"), "w") as f:
|
||||||
|
f.write(param_text)
|
||||||
|
|
||||||
|
# debug aid: keep a persistent copy of the exact parameter map elastix
|
||||||
|
# is about to run, written BEFORE the docker call so an abort leaves it
|
||||||
|
# behind for inspection (per-pid name: parallel workers do not clobber
|
||||||
|
# each other's dumps)
|
||||||
|
dump_dir = d("data/m6reg/impact/paramdump")
|
||||||
|
os.makedirs(dump_dir, exist_ok=True)
|
||||||
|
family = os.path.basename(os.path.dirname(model_rel)) or "model"
|
||||||
|
param_dump = os.path.join(dump_dir, f"ParameterMap_{family}_{os.getpid()}.txt")
|
||||||
|
with open(param_dump, "w") as f:
|
||||||
|
f.write(param_text)
|
||||||
|
|
||||||
cmd = ["docker", "run", "--rm"]
|
cmd = ["docker", "run", "--rm"]
|
||||||
if gpu >= 0:
|
if gpu >= 0:
|
||||||
|
|
@ -949,17 +978,19 @@ def register_impact(ct, t1c, model=None, gpu=None, iters=2000, samples=5000,
|
||||||
if not keep_scratch:
|
if not keep_scratch:
|
||||||
shutil.rmtree(tmp, ignore_errors=True)
|
shutil.rmtree(tmp, ignore_errors=True)
|
||||||
|
|
||||||
|
family = os.path.basename(os.path.dirname(model_rel)) \
|
||||||
|
or os.path.splitext(os.path.basename(model_rel))[0]
|
||||||
meta = {
|
meta = {
|
||||||
"pipeline": "impact",
|
"pipeline": "impact",
|
||||||
"toolkit": f"Docker {IMPACT_IMAGE} (Elastix 5.2.0 + IMPACT)",
|
"toolkit": f"Docker {IMPACT_IMAGE} (Elastix 5.2.0 + IMPACT)",
|
||||||
"metric": f"IMPACT semantic (TotalSegmentator {os.path.basename(model_rel)}, L2, Jacobian)",
|
"metric": f"IMPACT semantic ({family} {os.path.basename(model_rel)}, L2, Jacobian)",
|
||||||
"transform": (f"Rigid (EulerTransform, ASGD, {n} scales, auto center init; "
|
"transform": (f"Rigid (EulerTransform, ASGD, {n} scales, auto center init; "
|
||||||
f"device={'GPU' + str(gpu) if gpu >= 0 else 'CPU'})"),
|
f"device={'GPU' + str(gpu) if gpu >= 0 else 'CPU'})"),
|
||||||
"convention": "T1c->CT (verified)", "auto_inverted": inverted,
|
"convention": "T1c->CT (verified)", "auto_inverted": inverted,
|
||||||
"fixed": _modality_name(fixed), "moving": _modality_name(moving),
|
"fixed": _modality_name(fixed), "moving": _modality_name(moving),
|
||||||
"seconds_reg": round(dt, 1),
|
"seconds_reg": round(dt, 1),
|
||||||
"docker_image": IMPACT_IMAGE, "model": model_rel, "gpu": gpu,
|
"docker_image": IMPACT_IMAGE, "model": model_rel, "gpu": gpu,
|
||||||
"iters": iters, "samples": samples,
|
"iters": iters, "samples": samples, "param_dump": param_dump,
|
||||||
}
|
}
|
||||||
if keep_scratch:
|
if keep_scratch:
|
||||||
meta["scratch"] = tmp
|
meta["scratch"] = tmp
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue