ASK A QUESTION
REQUEST PATH
QUESTION
↓
L1 KV EXACT CACHE
↓
BGE-SMALL EMBEDDING
↓
VECTORIZE + SAFETY GUARDS
↓
HIGH-CONFIDENCE MATCH?
YES→CACHED ANSWER
NO→LLM FALLBACK
EXACT CACHE
~0.8s
Current benchmark average
LLM FALLBACK
~13.4s
Current benchmark average
EXACT COST
132×
Modeled reduction vs LLM miss
SEMANTIC HIT RATE
60%
Current 10-pair benchmark
HOW IT WORKS
Three-tier inference path
01
L1 · Exact KV
Instant reuse for identical normalized questions.
02
L2 · Semantic cache
BGE-small + Vectorize + subject/number/operation guards.
03
L3 · LLM fallback
Generate and store a new answer when confidence is insufficient.