Data Flow

Label semantics, the ordered transform chain that turns a raw CT volume into a training sample, the batch contract exposed to the model, and the prediction artifacts that leave the pipeline.

From raw CT volume to scored prediction. The custom pipeline (left) and the nnU-Net pipeline (right) take different preprocessing routes but converge on the same probability-volume evaluation surface.

1. Label semantics and the ignore region

Every design decision downstream is a consequence of the label scheme. Three classes are used, and they do not carry equal weight in training:

Value	Name in code	Handling
`0`	Background	Contributes to the loss as a negative, but only where it is a valid label.
`1`	Foreground (surface)	The positive target. Becomes the binary mask and the basis of the distance field.
`2`	Unlabelled / ignore	Removed from both the loss and the metric through a validity mask. Also used as the fill value when padding, so padded regions are never mistaken for real background.

The ignore class is the reason a validity mask exists at all. Annotations cover only part of each volume, so treating unlabelled voxels as background would teach the model that valid surface is background — a much more damaging error than the opposite. Every sample therefore carries an explicit mask saying which voxels are trustworthy.

2. Raw data to training arrays

A carbonised, still-sealed ancient scroll shown beside the same X-ray CT volume virtually unrolled into a flat surface — The sealed scroll and its virtual unrolling from CT data. Source: AIhub.

Scans arrive as 3D TIFF volumes. Because TIFF decoding is slow relative to training throughput, they are converted once to memory-mappable NumPy arrays and the training pipeline reads the converted form:

raw scan volume (.tif)  →  one-time TIFF-to-NumPy conversion  →  <data_dir>/train_images/*.npy
                                                                        <data_dir>/train_labels/*.npy

Conversion is a one-time preparation step, not part of the training loop. The conversion utility is a local working script and is not published with the repository; everything downstream depends only on the resulting <data_dir>/{train_images,train_labels} layout. The case identifier is the filename stem, and that same identifier threads through the dataset, the inference output filenames and the scoring report, which is what makes per-case debugging possible.

3. The transform chain

A training sample is not “a volume”: it is the output of an ordered chain of MONAI transforms. Order is load-bearing, and two properties of the chain matter more than any individual transform.

First, geometry is decided before content. Padding, cropping and augmentation all happen while the label is still a plain three-class map; only once a patch is final does the code derive the binary mask, the validity mask and the distance fields. This keeps the expensive derived arrays proportional to patch size rather than volume size.

Second, every geometry transform is paired with an appropriate fill value. Image padding fills with 0 (empty space), while label padding fills with 2 (ignore) so that synthetic padding can never be scored as a confident negative.

#	Transform	Purpose
1	Load volume and label	Reads the converted arrays for one case.
2	Ensure channel first	Volumes are stored without a channel dimension; adds one.
3	Scale intensity range	Maps raw intensities to `0–1`.
4	Spatial padding (image, fill `0`)	Guarantees the patch is extractable from volumes smaller than the patch size.
5	Spatial padding (label, fill `2`)
6	Build crop label	Constructs the map that drives foreground-weighted sampling.
7	Random patch crop, positive-weighted	Draws a patch centred on foreground. This is essential: uniform sampling would produce almost exclusively empty patches at this foreground density.
8	Augmentation set	See the note below — applied to geometry and appearance, with the scroll axis respected.
9	Extract foreground	Reduces the three-class map to a binary foreground mask.
10	Copy foreground	Retains an untouched copy of the binary mask for the explicit positive-class target.
11	Extract valid mask	Builds the mask that excludes ignore voxels from the loss.
12	Signed distance transform	Produces the distance field the auxiliary head regresses: negative inside the sheet, positive outside, normalised to a bounded range.
13	Unsigned distance transform	Produces a one-sided distance field capped at a maximum distance, giving the main head a graded signal near the surface where the binary label alone is nearly all zeros.

Validation uses a shorter chain The validation pipeline stops at intensity scaling and derives the same label tensors, but skips padding, cropping and augmentation. Validation must see the case as it will be seen at inference time, so any transform that changes geometry is confined to training.

4. The batch contract

This is the interface between data and model, and the place most readers need to slow down. Each sample exposes several keys, and each key has exactly one consumer:

Key	Contents	Consumed by
`pixel_values`	The normalised and augmented image patch.	The network input.
`label_fg_bin`	Binary surface mask.	The binary classification head’s target.
`label_fg`	Signed distance field derived from the binary mask.	The auxiliary regression head’s target.
`label_valid`	Validity mask excluding ignore voxels.	Loss masking, so ignore voxels never contribute.
`distance_unsigned`	Capped one-sided distance field.	Distance-based loss components.
`idx`, `id`	Position in the split and case identifier.	Traceability: which case and which patch a metric belongs to.

Two design choices are worth calling out. The dataset returns two distance representations rather than one: the signed field gives the auxiliary head a well-posed regression target that changes smoothly through the sheet boundary, while the capped unsigned field gives the distance-based loss terms a bounded local signal. And the dataset returns both a binary mask and its distance transform, so the two heads can be supervised from one patch without recomputation.

5. Training / validation split

Cases are partitioned once with a fixed random seed and a small held-out fraction, so a run is reproducible and every experiment sees the same split. The held-out cases are also the ones the sliding-window validation pass scores, which keeps validation cost proportional to a fraction of the dataset rather than to all of it.

6. Prediction artifacts

Inference inverts the chain. A trained checkpoint consumes image patches and writes, per case, two arrays in half precision to keep volumes manageable:

<output_dir>/
├── probabilities/<case_id>.npy     foreground probability volume
└── distance_preds/<case_id>.npy    predicted distance field

Everything after this point — thresholding, morphology, topological repair and scoring — operates on those files rather than on the network. That separation is deliberate: it makes post-processing a pure function of stored predictions, so strategies can be swept and compared without re-running inference. The Evaluation & Metrics page describes that stage, and Pipelines covers inference in context.