Apr 2026
Message queue publisher reconnect loop
Publisher entered an infinite reconnect cycle without backoff after broker restart — fixed with exponential retry and connection health checks.
Apr 2026
Service port not injected at container startup
Requests to a downstream service failed silently because the orchestrator wasn't injecting a required environment variable at runtime.
May 2026
Internal auth check caused infinite requeue loop
An internal service was returning 401 for valid internal calls. Jobs were nacked and requeued indefinitely with no visible error surfacing to the user.
May 2026
Workers terminated mid-job by orchestrator scale-down
Event-driven autoscaling sent SIGTERM to workers processing active jobs. No graceful drain was implemented, leaving documents stuck in a processing state.
May 2026
Document structure retrieval overflowed model context
A tool returning document headings was including full section content alongside — saturating the context window on large documents and truncating responses.
May 2026
Heading search index silently empty after ingestion
Two chained bugs: an embedding queue timeout fired silently before headers were enqueued, and a downstream consumer overwrote the index type field — preventing correct routing entirely.
May 2026
Schema draft — three independent failures masking each other
Frontend, backend, and inference layer each failed for unrelated reasons. Fixing one revealed the next: a missing stream subscription, an async type mismatch, then an insufficient token budget.
May 2026
Session token expired silently mid-session
Short-lived auth tokens expired during idle periods with no background renewal. Returning to an open page after inactivity produced a network error instead of a transparent re-auth.