Triage
A pipeline that takes messy customer input and turns it into structured data: an urgency score, a category, a two-line summary. When something is urgent, the webhook and the alert e-mail go out on their own.
You throw free text or JSON at Triage and get back an answer forced into a fixed schema: urgency from 1 to 10, a category, a summary. Because the schema is enforced, the result is always machine-usable. No "sometimes it returns broken JSON" problem.
I didn't want to pay the LLM twice for the same input, so responses are cached in Redis. The same Redis also keeps the per-tenant rate limits. Records land in PostgreSQL for the audit trail.
If a request crosses the urgency threshold, the pipeline moves the work off the main flow and fires a webhook plus an e-mail. There's a small Next.js dashboard on top: live feed, success rate, cache hit rate.
- Made one Redis carry both the LLM cache and the per-tenant rate limits.
- A second component to rate-limit with, and a second thing to keep alive.
POST /api/v1/ingest
→ 200, 340ms, cache miss
{
"urgency": 8,
"category": "billing",
"summary": "Card was charged twice, customer wants a refund.",
"actions": ["webhook", "email"]
}
- Schema-enforced output, no broken JSON
- Redis: LLM cache + per-tenant rate limits
- Urgent items fire webhook + e-mail on their own
- Spring Boot, clean layered architecture