A spatial intelligence layer for agriculture
We reconstruct every plant in a greenhouse from a simple weekly camera pass - the crop in 3D, tracked week over week - and fuse that living 4D record with its climate. That lets us both measure what is on the plants today and forecast what a grower will harvest, accurately enough to commit retail volume on. The forecast ships first; the deeper aim is what the data makes possible - a learned model of how living systems grow, ripen, and fail.
Greenhouse growers commit to retailers and labour crews weeks before fruit ripens; today they bridge the gap with manual sampling and gut feel. The research splits this into two problems - estimation (what is on the plants now) and prediction (what will be harvested, and when) - and the literature tends to attack each with thin, single-source input: images alone, or climate alone8. Our thesis is that both improve together when the input is richer - a full 4D representation of the crop: every plant reconstructed in 3D1, tracked through time, and fused with the climate sensors growers already run. A truer census of the present makes a truer forecast of the future, and the same record drives both - toward full understanding, and ultimately control, of the produce. Harvest forecasting is where we start, because it is the problem growers pay to solve; the weekly 4D record it produces is the asset that compounds beneath it.
The 3D reconstruction and the harvest forecast are real, and they ship first - but they are the foothold, not the destination. Underneath them we are accumulating something rarer: a continuously growing, plant-resolution record of how living things develop in space and over time, joined to what they actually go on to do. Text taught machines to write; video is teaching them physics. A dense, spatio-temporal multimodal record of growing systems, is the training ground for something that hasn't existed before: a learned model of the living world3. That is what we mean by Earth Models.
Yield forecasting on greenhouse tomato is the wedge - a problem growers pay to solve, on a crop that lets us validate weekly. It funds the work, and it switches on the data engine. The forecast is the first thing we ship; it was never meant to be the last thing we build.
Every weekly pass extends a 4D archive7 - each plant, each fruit, in geometry, across time - joined to the harvest weights, the stress events, the response to every change in water and light. Not block averages reconstructed after the fact, but the living system measured as it unfolds. This corpus is the asset; the product is how we earn the right to grow it.
Train on that substrate and the geometry stops being a record and becomes a model - one that learns how living systems grow, ripen, respond, and fail. One crop, then many; the greenhouse, then the open field. World models, for the living world. The tomato is simply where it starts.
The field reads the crop from flat images, and pays for it twice. In space, fruit hidden behind leaves goes uncounted and the same fruit is re-counted across frames - occlusion is the documented failure mode of 2D detection8. In time, a single snapshot can estimate what is ripe today but carries little signal for what ripens next. Our answer to both is one representation: the canopy reconstructed in 3D and tracked week over week - a 4D record - then fused with climate. In space, geometry lets us see around occlusions, never double-count, and recover the metric operators trade on - kilograms, not counts. In time, the same fruit followed across captures, together with the climate that drives it, is what turns a census of the present into a forecast of the future.
A handheld or rail-mounted camera (an Insta360 works) sweeps each row weekly. We pair an off-the-shelf detector stack (YOLO6 + SAM) with structure-from-motion and 3D Gaussian Splatting1 to lift every row into a geometrically faithful, spatially registered 3D scene - each fruit a tracked instance with a position, a size, and a persistent identity across captures - building on recent work in 3D plant capture4 and photorealistic greenhouse reconstruction5.
Each tracked fruit is staged green → breaker → ripe from its 3D appearance. Knowing where each fruit sits in the ripening pipeline - not just how many there are - is what lets us forecast a harvest curve rather than a single number. Robustness across glazing, supplemental LEDs, and Mediterranean tunnel light is the open frontier we're investing in.
Tomato and pepper ripening is dominantly temperature-driven2. We pair the per-fruit pipeline from M.01–M.02 with greenhouse climate data (temperature, daily light integral) to project when each cohort ripens and what it will weigh - kilograms harvestable per zone, per week, with uncertainty bands. Calibration is per-house and improves with every harvest cycle.
This is the whole thesis in one line: estimation and prediction are not separate products but two reads of the same 4D record. Most of the field keeps them apart - images for the present, climate models for the future - and fuses neither8; we fuse both, and each sharpens the other. A weekly 4D record is a state representation in the modern ML sense - the kind of input recent vision-and-forecasting research consumes directly: sized cohorts and ripening stages drive the forecast, geometry-and-colour anomalies feed stress and disease detection, and plant-level response to a fertigation change becomes attributable instead of averaged out. Paired with the grower's existing climate and soil sensors, it closes the loop - measure → predict → act → re-measure - and moves the operation from forecasting its produce to understanding and controlling it.
| Manual sampling | Earth Models (target) | |
|---|---|---|
| Coverage | ~ 10 plants / block | Census of every plant we walk |
| Volume / weight | Sparse sample, extrapolated | Per-fruit, from 3D geometry (target ± 5 %) |
| Cadence | Weekly, sparse | Weekly, exhaustive |
| Forecast horizon | Gut-feel | 1 – 3 weeks, calibrated |
FIRST PRODUCT
A grower commits twice before any fruit is in hand: to a marketer for retail volume, and to a labour contractor for the picking crew. Both are paid for in days, not weeks. Under-deliver and you eat penalties or lose the listing; over-produce and you dump ripe fruit at fire-sale prices. We replace gut-feel with measured geometry - the same number drives both commitments.
Retail supply contracts run on volume and consistency. A credible weekly forecast earns better terms - and avoids the fill-rate penalties and lost listings that follow a miss.
Pickers are booked days ahead. Knowing the ripening load by zone means no idle crews on slow days and no over-ripe fruit on heavy ones.
A ripe tomato has days, not weeks. Forecast accuracy is the difference between selling at a contracted price and dumping the surplus into the wholesale market.
The same forecast sizes packing-line shifts, graders, cold-chain, and transport. Get the forecast right and the whole post-harvest chain stops absorbing slack.
A startup learns at the rate of its predict → harvest → compare cycle. Crops with one harvest a year - apples, grapes, wheat - give a single validation event per season; we'd fail once and die. We pick continuously-harvested crops in controlled environments, where calibration is weekly.
Tabletop day-neutral strawberry
Flower-to-ripe in 3–4 weeks, picked every 2–3 days, day-neutral varieties fruit continuously. Tabletop systems hang fruit below the canopy, so occlusion is minimal - the cleanest substrate to converge the perception model fast. We can validate a one-week forecast every week.
Indeterminate greenhouse tomato
An 8–11 month cycle harvested weekly yields 30–40 prediction-vs-actual events from a single crop, with hundreds of fruit per plant. It is the dominant Mediterranean greenhouse crop, the substrate the existing players validate on, and the natural commercial home for the product.
Each direction below has the same shape: feed the weekly canopy state into a learned model, get back a decision a grower used to make on intuition. These are bets, not shipping features - but they are the reason we build the measurement layer in the first place. The shorter the capture-to-action loop, the more the operation shifts from managed to self-correcting.
Each tracked fruit becomes a time series - size, shape, ripening rate per cultivar and per zone. Phenotype data dense enough to inform variety selection and crop steering.
Anomalies in geometry, colour, and growth rate often precede the visible disease signal by days. With a weekly baseline of "normal" per zone, the deviations become detectable.
Irrigation, fertigation, and climate setpoints are decided today against per-block averages. Plant-resolution measurement makes it possible to tie input changes to actual fruit-load response.
Premium-grade volume is what moves price. Per-fruit geometry plus illumination-corrected appearance is the route to forecasting it - once the underlying staging model is robust enough.
The 4D record is the input. Learned models on that state are the engine. Less waste, better decisions, faster response is the output. Yield forecasting is simply the first place we ship it.
If you run a tomato or strawberry operation under glass or tunnel and want a forecast credible enough to commit retail volume on, we're looking for a few partner sites for the 2026 season. Concierge capture, your harvest logs as ground truth, transparent reporting.
contact@earthmodels.ag