Most AI projects stall not because the technology fails, but because the scope is too wide. Here is the four-week plan we use with clients to put a useful assistant in front of real users, safely.
Pick one painful, measurable problem
The best first project is narrow: one team, one workflow, one number you want to move. Customer support questions, internal policy look-ups and document summaries are strong candidates, because the answers already exist somewhere and success is easy to measure.
Write the goal down in a single sentence, for example: "Answer 60% of billing questions without an agent, with 95% accuracy." That sentence will settle every debate that follows.
Week 1: gather the knowledge and real questions
Collect the documents the assistant should rely on, such as help articles, policies and product notes, and fix anything that is outdated. Then export 100 to 200 real questions from your inbox or chat logs. These become your test set.
Week 2: build a grounded prototype
Set up retrieval so the model answers only from your content, and ask it to cite the source for every answer. Run the full test set every day and score the results. Most of the improvement comes from better documents and better search, not from clever prompts.
Week 3: add guardrails and a human hand-off
Teach the assistant to say "I am not sure" and pass the conversation to a person with a short summary. Block topics it must never handle, log every conversation, and give your team a simple way to flag bad answers.
Week 4: launch small, measure, then expand
Release to 10% of users or to a single channel. Track resolution rate, accuracy on sampled conversations and customer satisfaction. When the numbers hold for two weeks, widen the rollout and pick the next workflow.
Mei Lin
AI Engineer
Mei designs retrieval systems, agents and evaluation pipelines. Before IntelliCraft she worked on search quality, and she insists that every AI feature is measured before it ships.
