Machine vision

Detecting and counting apples in orchard imagery

Yield estimation from orchard imagery, treated as three connected problems: find the apples, count them, and separate the ones that are touching.

Context: Machine Vision, MSc Robotics, Bristol
Role: Model training and evaluation, five-person team
Approach: Faster R-CNN and YOLO, compared
Code: apple-machine-vision
Report: Group report (PDF)

Top left: orchard image with green detection boxes around individual apples. Top middle: the same image with coloured instance segmentation masks. Top right: cropped examples of individual detected apples, including blurred, occluded and yellow fruit. Bottom: four source orchard images showing red and yellow varieties in varying light. — Detections, instance masks and individual crops from the same orchard scene. The bottom row shows the variety in the source imagery: two cultivars, different light, and a lot of fruit on the ground that should not be counted as fruit on the tree.

Why it is not just detection

A single apple is an easy detection target, but an orchard scene is much harder. The difficulties are structural: fruit hangs in clusters where the boundary between two apples is a soft edge rather than a hard one, foliage occludes most of the crop most of the time, and the same variety photographs completely differently depending on where the sun is. Windfall on the ground looks identical to fruit on the tree and must not be counted.

Counting compounds this. Individually small detection errors accumulate into the yield figure, so a model that looks respectable on a per-image metric can still be well out on the total.

Four panels: the original orchard image, its blurred LAB A channel where red fruit shows as bright blobs, its blurred B channel, and a colour-enhanced version of the original. — Working in LAB rather than RGB. The A channel separates red fruit from foliage far more cleanly than any RGB combination does, which made it a useful preprocessing step and a decent classical baseline to measure the learned models against.

What we built

This was a five-person team; my side was the model training and evaluation.

Training and prediction pipelines for both Faster R-CNN and YOLO against the same dataset, so the comparison is about the architecture rather than the data handling.
A colour-space preprocessing stage in LAB, used both as an input enhancement and as a non-learned baseline worth beating.
Separate evaluation for detection, counting and segmentation, because they fail in different ways and a single mAP number hides that.
A conversion step from polygon annotations to instance masks, which was most of the unglamorous work.

What it showed

The two architectures trade off in the way the literature suggests, with the single-stage detector faster and the two-stage one better on the small and partially occluded fruit that make up the difficult tail of the dataset. For yield estimation that tail is where most of the counting error comes from, so accuracy is worth more here than throughput.

← All projects