topío — system design
A query-based agent that searches landscape-specimen imagery, resolves real-world measurements from a single photo down to the inch, and builds its own dataset — no human in the measuring loop. Approved specimens go straight on air as ch21 library records.
pipeline
Operator asks in plain words: 'blue spruce, 4–6 ft, field-grown'.
Candidate images gathered with license + source recorded.
Blur, collage, occlusion, and multi-specimen frames dropped early.
Monocular depth field estimated for the surviving frames.
Path A: EXIF focal length + sensor size. Path B: reference objects (pavers, brick courses, doors, hands, standard nursery pots).
The two scales must agree within tolerance. Disagreement lowers confidence instead of picking a winner.
Height, spread, caliper resolved in inches with an uncertainty band.
Below threshold → quarantine, not library. The dataset never absorbs a guess.
Record embedded in small chunks, deduped, marked pending review.
Admin approves; courier files it in the station library and it goes on air.
sub-agents & tool contracts
scout
Finds candidate specimen imagery for a query.
grader
Rejects unusable frames before any expensive work runs.
depth
Produces a relative depth field from one photo.
scaler
Converts relative depth into real-world scale two independent ways.
dimensioner
Measures the specimen: height, spread, caliper, leaf/needle length.
librarian
Writes the record, dedupes, embeds, and files it in the library.
courier
After admin approval, files the specimen in the station library and marks it on air.
retrieval
Retrieval engineering
The library is the product. It is chunked for meaning, not for convenience.
- One chunk per meaning unit: species identity, measurement set, habitat note, source/license, visual description. Small (roughly 2–4 sentences), never a whole record in one blob.
- Text chunks embedded for semantic search; image embeddings stored alongside so 'looks like this' queries work next to 'named like this'.
- Vector recall unioned with keyword/filter recall on species, size range, and region — then deduped by specimen id.
- Re-rank the union by query relevance, measurement confidence, and recency of source. Confidence is a ranking signal, not a footnote.
- Every answer cites specimen ids and source URLs. No citation, no claim.
reliability
Reliability engineering
Every tool call is assumed to fail sometimes. The workflow is built to survive it.
- Per-tool budget; depth and scale get the longest, scouting the shortest. A step that blows its budget is cancelled, not waited on.
- Bounded retries with exponential backoff and jitter. Only timeouts, rate limits, and 5xx are retried — bad input never is.
- If a scale path fails, the run continues on the surviving path with reduced confidence. If both fail, the frame is quarantined.
- Filing and publishing carry idempotency keys so a retry can't create a duplicate specimen or a double upload.
- A source or tool failing repeatedly is dropped from the run and reported, instead of poisoning the batch.
safety
Security & safety
Scraped captions and page text are untrusted input, not instructions.
- Fetched captions, alt text, and page copy are wrapped as data. Instruction-shaped content is stripped and logged, never executed.
- Schema validation on every tool boundary: URL allowlists, content-type and byte-size checks, real image sniffing, EXIF sanity ranges.
- Measurements must be physically plausible for the species before they leave the agent. Impossible numbers are rejected at the boundary.
- Least privilege per sub-agent. Only courier can write to the station library, and only with an admin approval token. Nothing else has publish rights.
- Images without usable license provenance never enter the library.
observability
Evaluation & observability
If it isn't traced and scored, it isn't shipped.
- One trace id per query, spans per sub-agent and per tool call, with inputs hashed, outputs summarized, and retries visible.
- Per-tool success rate, p50/p95 latency, retry rate, and failure taxonomy.
- Tokens, image ops, and storage rolled up to cost per accepted specimen — the number that actually matters.
- A held-out set of hand-measured specimens scores mean absolute error in inches and calibrates the confidence gate.
- A model or prompt change ships only if MAE and accepted-specimen yield hold on the eval set.
product
Product & UX
A query bar, a wall of specimens, and one approval button.
- The whole surface is one input. Ask for a specimen and size range; the run starts and streams progress by step.
- Each result shows the image, dimensions in inches, the uncertainty band, which scale path was used, and its evidence.
- Low-confidence items sit in quarantine, visibly separate from the library. Nothing silently becomes truth.
- Admin reviews pending specimens, approves in batches, and filing to the station library happens automatically after that.
- Humans approve. They never enter a dimension by hand.
station library handoff
Once a specimen is approved, the courier files it into the ch21 station library: one record per request, idempotency key so a retry can't double-post, and the remote id written back onto the library record. If no external network endpoint is configured, the courier runs in dry mode and logs the exact payload it would hand off.