If an educator clicks generate and the browser just freezes behind a spinner, the system feels broken even when the backend is technically working.
Paper Pilot solves that by separating request handling from long-running LLM and PDF generation work.
01The Original Problem
Holding the HTTP request open for a multi-second LLM call made the UI feel frozen and pushed the API toward timeouts.
// before
const paper = await deepseek.generate(spec);
res.json(paper);
// after
await genQueue.add('generate', spec);
res.status(202).json({ assignmentId });02Why a Queue-Based API Works Better
The API now validates input, persists the assignment, enqueues a BullMQ job, and responds quickly. All expensive model and PDF work happens in a dedicated worker process.
- The browser gets a response in under 300 ms.
- Long-running generation does not block request threads.
- Retries and failure handling live in the queue where they belong.
03Progress Instead of a Black Box
A queue alone is not enough. Users still need feedback, so the worker emits stage-level progress through Redis Pub/Sub and Socket.io rooms.
- Analyzing input
- Building the prompt
- Generating questions
- Parsing and validating output
- Saving and rendering the PDF
04Validation Has to Happen Twice
LLM output is untrusted. Paper Pilot validates the generated structure before persistence and again before PDF rendering so malformed output cannot leak further downstream.
05The Main Lesson
- If the work is slow, do not pretend it is a normal request-response flow.
- Background workers plus real-time progress create a better user experience than blocking spinners.
- Schema validation is mandatory when model output feeds production workflows.