Research preview · v0.8.5
Explore the engine
OXID SENSORY · THE SENSES

It reads the world so your database can understand it.

Sensory is perception for Oxid. It turns unstructured reality, source code, PDFs and email, into structured, ontology-aware, vector-searchable knowledge your database can reason over.

See how it readsView the roadmap
codebase · pdf · emailLayer B (LLM) · experimental
›RAW WORLD
crates/parser/lib.rs
spec_v3.pdf
Re: SLA targets.eml
docs.oxid.dev
Sensory reads · extracts · resolves
STRUCTURED KNOWLEDGEvector-searchable
parse_streamDEPENDS-ONTokenBuffer
§2.3 LatencyPART-OFspec_v3.pdf
alice@oxid.devSENTRe: SLA targets
THE INGESTION PIPELINE

From raw reality tocoherent knowledge.

Nine steps, five that matter, turn messy input into facts your database can trust: parsed by real tools, materialised for an EL⊥ reasoner, traced to their source, and gate-checked for coherence before anything commits.

01

Parse, deterministically

A real parser reads each source: tree-sitter for code, operator-level PDF, RFC email. Zero LLM, hallucination-free.

02

Extract entities & relations

Functions, types, sections, people and threads become typed individuals with real edges: declaredIn, calls, partOf, sentBy.

03

Materialise the closure

Oxid-DB reasons in EL⊥, so Sensory forward-chains the transitive rollups (partOf*, dependsOn*) and asserts them as flat facts.

the RL materialiser
04

Embed & bridge

Every chunk is embedded into a per-project vector collection and hung off its real entity: similarity and structure in one store.

05

Classify & gate

The reasoner checks disjointness consistency; then a five-gate harness runs the competency questions. A gate failure fails the run.

runs on Oxid-DB
Layer A · deterministic and gate-verified: the same pipeline ships today for codebase, pdf & email
THE MODEL · WHY IT'S DIFFERENT

Two truthful products,one store.

Most pipelines build a tag tree and call it an ontology. Sensory keeps two things honest and separate (a retrieval index and a reasoning ontology), then composes them at query time.

Retrieval index

Vectors plus a navigable structural tree. Answers find content like X, under Y: embeddings hung off real entities via hasChunk and a materialised partOf closure.

vectors + partOf tree

Reasoning ontology

A clean TBox, class hierarchy and disjointness, that supports inference about what the content is. Every term forced through an upper ontology so it lands in the right category.

BFO + IAO anchored
THE PAYOFF · COMPOSED AT QUERY TIMEstructure × role × containment × similarity
›FIND?fnWHERE?fnIS-APublicApi⋀?fndependsOn*auth≈"token refresh"

Vector-search every PublicApi function that transitively dependsOn the auth module: one source of truth, four kinds of query at once.

partOf, never subClassOfBFO + IAO upper anchorrigid types vs anti-rigid rolesdisjointness assertedfive-gate quality harnessHTTP-only, decoupled from Oxid-DB
SHIPPING TODAY · THREE SENSES

Three senses already read into the graph.

codebase is the reference sense: a full Rust symbol graph with real calls and roles. pdf and email ship the same way: a deterministic structure graph, gate-verified, queryable in OxQL the moment ingest finishes.

  • Rust symbols, modules & dependency edges as typed individuals
  • partOf & dependsOn rollups materialised for an EL⊥ reasoner
  • Every fact traceable to its exact source line, by design
  • Composable: structure × role × containment × vector similarity
Built & gatedLayer B · opt-in
oxid-sensory codebase ./repo
$oxid-sensory codebase ./repo
→preflight · mxbai-embed-large · dim 1024 ✓
→walk · 1,284 files · rust entities, 20-lang chunks
✓842 symbols · declaredIn · calls · implements
✓partOf* / dependsOn* materialised
✓classify consistent · gates 1–4 pass
›FIND?fnWHERE?fnIS-APublicApi
Built & gated
oxid-sensory codebase

The reference sense. Rust entities, calls & roles from the AST; AST-aware chunking across 20 languages.

Built & gated
oxid-sensory pdf

Text PDFs into a Document → Section → Block tree, operator-level parsing. Hosts the opt-in LLM Layer B.

Built & gated
oxid-sensory email

A real correspondence graph: people, threads & time from .eml / mbox, resolved deterministically from headers.

ON THE ROADMAP · BUILT VS. NEXT

Honest about what's real and what's next.

The deterministic core ships today. The LLM layer is real but opt-in and unproven at scale; a couple of senses are still just a name in the docs. Here's the line, clearly marked, so you always know what's solid.

Layer B · semantic extractiontext → typed, grounded entities · the opt-in LLM pass
SHIPPING · OPT-IN · EXPERIMENTAL

The opt-in LLM pass on top of the deterministic core: it drafts entities into a closed 10-type taxonomy, verifies every one against its exact source span, gates on confidence, and only asserts what the reasoner keeps coherent. The contract is deliberate: grounded and auditable, not “true.” Real and built; factual quality is proven only on a labelled eval corpus so far.

text → typed entitiesspan-verified groundingclosed 10-type taxonomyconfidence-gatedprecision/recall eval gate
oxid-sensory webNOT STARTED

Crawl and distill the live web into the same grounded, ontology-aware graph. Named in the docs, not started yet.

entities beyond RustPLANNED

Today only Rust yields a full symbol graph; the other 19 grammars chunk-and-embed. Extending entity extraction to more languages.

OXID SENSORY · THE SENSES

Give your database a way to perceive.

Point oxid-sensory at a repo, a PDF or a mailbox today and it reads straight into the graph. The world is unstructured, and Oxid is learning to read it.

Run oxid-sensory Read the design docs
codebase, pdf & email ship today · Layer B (LLM) is opt-in & experimental · web is next