AI Lead Engine

Problem

Qualifying a lead with an LLM takes seconds, so scoring has to run out of band. But the product’s value is a live experience: submit a lead and watch it get qualified without a refresh. The default way to get that — a single-page app polling an API — adds a frontend stack and a second codebase to maintain. The goal was live updates without that overhead.

Requirements

• Score and qualify leads with an LLM and return structured qualification output.
• Run scoring asynchronously so the UI never blocks on model latency.
• Push results to the user the moment they are ready.
• Deliver live updates without building or maintaining an SPA.

Constraints

• LLM scoring latency is variable and outside the app’s control, so the UI cannot wait on it synchronously.
• The team is Rails-first; a separate JavaScript application would add operational and maintenance cost disproportionate to the interaction complexity.
• Users keep a page open while leads process, so results have to arrive over a connection that already exists rather than through repeated polling.

Architecture

Built on Rails 8. A lead submission enqueues a Solid Queue job that calls the LLM through OpenRouter; on completion the job broadcasts the result over Turbo Streams via ActionCable directly into the open page.

submit lead ──▶ Solid Queue job ──▶ OpenRouter (score + qualify)
                                          │
                                          ▼
                       broadcast Turbo Stream ──▶ ActionCable ──▶ live DOM update

The server renders HTML and pushes fragments over WebSockets; there is no client-side framework rendering state.

Engineering Decisions

• Solid Queue rather than an external broker. Rails 8 runs database-backed jobs, so the queue is one fewer service to deploy and monitor and shares the app’s Postgres durability guarantees.
• Scoring is a background job; results stream back. The LLM call never sits in a request, and the result reaches the user through a broadcast when it finishes, so variable model latency never blocks interaction.
• Hotwire instead of an SPA. Turbo Streams and ActionCable deliver live updates from server-rendered HTML, which keeps the whole feature in one Rails codebase.

Tradeoffs

• WebSocket connection state. Live delivery means managing ActionCable connections and broadcasts, accepted in exchange for removing an entire SPA and its build/tooling.
• Server-rendered reactivity covers this interaction cleanly; a highly interactive client with local state would still favor a JS framework — not the case here.

Failure Modes

• OpenRouter error or timeout — the job retries; the UI holds a pending state until a result streams in rather than showing a failure.
• Client disconnects mid-job — the result is persisted on completion, so it is present on reconnect instead of being lost with the socket.

Scaling

Scoring throughput scales by adding Solid Queue workers. Because results are pushed over existing WebSocket connections rather than polled, more concurrent users add broadcast fan-out rather than multiplying inbound request load.

Monitoring

Solid Queue job state lives in Postgres, so backlog, retries, and failures are queryable alongside application data. ActionCable connection counts and broadcast delivery indicate whether live updates are reaching clients; a rise in pending leads with flat completions points at the model integration rather than the transport.

Security

The OpenRouter API key is server-side only and never reaches the browser. Lead data is sent to the provider over TLS. ActionCable streams are scoped per record so a client only receives broadcasts for leads it is authorized to see, preventing cross-tenant result leakage over the socket.

Outcome

A workflow where leads are scored asynchronously and results stream into the browser as they complete — a live, server-rendered experience delivered from a single Rails codebase without a separate frontend application.

Lessons Learned

Rails 8 and Hotwire make live AI features achievable without a second stack: Solid Queue absorbs the asynchronous work and Turbo Streams delivers the update, so the system stays one codebase while the interaction feels immediate.