01 — Start with the failure mode
The sea can make a bad model look accurate.
In the oil-spill dataset I used, oil occupies roughly 1% of pixels. Most of an image is not oil. A segmentation model can therefore classify enormous regions of background correctly and still miss a meaningful share of the spill while retaining an impressive-looking overall pixel accuracy.
The selected SegFormer model reaches 0.967 pixel accuracy, but that is deliberately not the headline result. The more useful numbers are oil IoU at 0.566 and oil recall at 0.764, reported alongside mean IoU and macro F1. Those metrics force the evaluation to look at the class the system is meant to detect.
The general lesson is not that accuracy is a bad metric. It is that a metric becomes bad when its weighting is misaligned with the cost and rarity of the classes that matter.
02 — Make ambiguity a class
Oil is not just dark water.
Synthetic Aperture Radar is valuable because it works through cloud and at night, but dark patches in SAR are not uniquely oil. Low-wind zones, biogenic films and rain-related effects can suppress backscatter too. If those look-alikes are collapsed into generic background, the model can avoid learning the distinction the operator actually needs.
The project frames the task as five-class segmentation: sea surface, oil spill, look-alike, ship and land. That makes oil-versus-look-alike confusion measurable rather than hiding it inside a binary target.
This framing also changes how qualitative inspection should work. A visually plausible dark-region mask is not enough; it matters whether the model is repeatedly converting look-alikes into false oil alerts.
03 — Use a metric stack
IoU, recall and macro metrics answer different questions.
Intersection over Union asks how much the predicted oil region overlaps the labelled oil region, penalizing both missed oil and over-prediction. Recall asks how much of the labelled oil the model recovers. Macro F1 and mean IoU stop the majority class from completely dominating the summary because each class contributes more evenly.
On the committed 110-image test split, SegFormer mit-b2 produced oil IoU 0.566, oil recall 0.764, mean IoU 0.696 and macro F1 0.802. U-Net and DeepLabV3+ produced lower oil IoU and recall in the same project evaluation. That is why the model-selection argument is expressed in oil-specific terms rather than '96.7% accurate'.
No single number captures every operating preference. A response system that values missing as little oil as possible may choose a different threshold than a review workflow that cannot tolerate many false alarms. The metric stack keeps that trade-off visible.
04 — Keep the split honest
A test score describes one dataset, not the ocean.
The official split is small: 1002 training images and 110 test images. The project records that limitation directly. Model comparisons are meaningful inside that evaluation contract, but they should not be inflated into a claim of universal marine performance.
The dataset also uses VV intensity imagery. That means dual-polarization information is not part of the learned signal. A later system with different sensors, preprocessing, incidence-angle distributions or geographic conditions should be treated as a domain change until evidence shows otherwise.
This is where evaluation language matters. 'Best on this committed test split' is a checkable statement. 'Detects oil spills accurately everywhere' is not.
05 — Test the pipeline boundary
The model is only one part of a geospatial product.
Training runs on labelled image chips, while useful deployment starts with a raw Sentinel-1 scene. The project therefore carries the selected model through ONNX export, scene preprocessing, tiled inference, overlap stitching, GeoTIFF output, polygon vectorization, API serving and a MapLibre interface.
Every transition is another place where an offline metric can stop describing the real system. Preprocessing mismatch can move the input distribution. Tiling can create seams. Georeferencing errors can place a good mask in the wrong location. Vectorization can change the geometry an operator sees.
For production ML, I therefore think of validation as a chain: model metrics establish one link; parity checks and end-to-end scene tests establish the rest.
06 — Use real events carefully
A case study can expose a domain gap without pretending to close it.
The Wakashio case study runs the pipeline on a previously unseen Sentinel-1B scene over the August 2020 spill near Mauritius. The output places detected oil in the expected coastal and lagoon area, which is useful evidence that the geospatial pipeline can operate end to end.
It is not a substitute for a large independently labelled raw-scene benchmark. The project explicitly treats detected area as an approximate lower bound because the raw-scene radiometry and training-chip preprocessing do not line up perfectly. That caveat is the right conclusion from the evidence, not a defect to hide.
07 — Design metrics around the decision
The headline metric is part of product design.
Choosing what to put at the top of a report shapes what the team optimizes. If background-dominated accuracy is the headline, improvements that make the sea easier to classify can look more important than improvements that find oil. If oil IoU and recall are first, the optimization target is much closer to the reason the system exists.
That is the broader pattern I carry into imbalanced ML work: identify the consequential class, expose the confusable alternatives, report the denominator, and choose metrics that make expensive errors visible rather than numerically cheap.
Practical checklist
What I carry into the next system.
- For rare-class segmentation, overall accuracy can be numerically strong while the target class remains weak.
- Report target-class overlap and recall alongside balanced multiclass metrics.
- Represent operational look-alikes explicitly when they are a real source of false positives.
- Keep dataset size, sensor channel and domain limits beside the score.
- Validate the deployment chain after the model: preprocessing, export parity, tiling, georeferencing and vectorization all matter.
Evidence & references