A 4D representation of the crop, not a 2D snapshot.
The field reads the crop from flat images, and pays for it twice. In space, fruit hidden behind leaves goes uncounted and the same fruit is re-counted across frames - occlusion is the documented failure mode of 2D detection. In time, a single snapshot can estimate what is ripe today but carries little signal for what ripens next. Our answer to both is one representation: the canopy reconstructed in 3D and tracked week over week - a 4D record - then fused with climate. In space, geometry lets us see around occlusions, never double-count, and recover the metric operators trade on - kilograms, not counts. In time, the same fruit followed across captures, together with the climate that drives it, is what turns a census of the present into a forecast of the future.
-
M.01
3D reconstruction of the canopy
A handheld or rail-mounted camera (an Insta360 works) sweeps each row weekly. We pair an off-the-shelf detector stack (YOLO + SAM) with structure-from-motion and 3D Gaussian Splatting to lift every row into a geometrically faithful, spatially registered 3D scene - each fruit a tracked instance with a position, a size, and a persistent identity across captures - building on recent work in 3D plant capture and photorealistic greenhouse reconstruction.
- → Multi-view: see fruit hidden behind leaves and other fruit
- → 3D identity: no double-counting across frames
- → True volume from geometry → weight via a per-cultivar size→mass model
-
M.02
Maturity staging from the canopy itself
Each tracked fruit is staged green → breaker → ripe from its 3D appearance. Knowing where each fruit sits in the ripening pipeline - not just how many there are - is what lets us forecast a harvest curve rather than a single number. Robustness across glazing, supplemental LEDs, and Mediterranean tunnel light is the open frontier we're investing in.
- → Per-fruit ripeness stage, not a per-row average
- → Census of every plant, not a sample of ten
- → Stable across glasshouses (Northern) and plastic tunnels (Southern)
-
M.03
Thermal-time forecasting
Tomato and pepper ripening is dominantly temperature-driven. We pair the per-fruit pipeline from M.01–M.02 with greenhouse climate data (temperature, daily light integral) to project when each cohort ripens and what it will weigh - kilograms harvestable per zone, per week, with uncertainty bands. Calibration is per-house and improves with every harvest cycle.
- → 1–3 week horizon - the length of the ripening pipeline
- → Disaggregated by zone for labour and logistics planning
- → Continuously recalibrated against the operator's actual harvest logs
This is the whole thesis in one line: estimation and prediction are not separate products but two reads of the same 4D record. Most of the field keeps them apart - images for the present, climate models for the future - and fuses neither; we fuse both, and each sharpens the other. A weekly 4D record is a state representation in the modern ML sense - the kind of input recent vision-and-forecasting research consumes directly: sized cohorts and ripening stages drive the forecast, geometry-and-colour anomalies feed stress and disease detection, and plant-level response to a fertigation change becomes attributable instead of averaged out. Paired with the grower's existing climate and soil sensors, it closes the loop - measure → predict → act → re-measure - and moves the operation from forecasting its produce to understanding and controlling it.
TABLE 01 · Design targets vs. manual sampling
| Manual sampling | Earth Models (target) |
| Coverage | ~ 10 plants / block | Census of every plant we walk |
| Volume / weight | Sparse sample, extrapolated | Per-fruit, from 3D geometry (target ± 5 %) |
| Cadence | Weekly, sparse | Weekly, exhaustive |
| Forecast horizon | Gut-feel | 1 – 3 weeks, calibrated |