Put AI to work on real problems, not demos

1. 1

   A real problem, or another demo?

   A demo runs on cases someone picked. Ask which of your cases it was tried on, and what it got wrong.

2. Why 2 waits for 1Nothing you spend on the next question matters until the problem is worth solving this way.

3. 2

   Build cost, and the monthly bill after launch?

   The build price is the number every firm will discuss. Models bill per use, so the running cost never stops and almost nobody has it ready.

4. Why 3 waits for 2You cannot judge either number without knowing what the thing must achieve for you.

5. 3

   How will we know it is working?

   Not a dashboard. Your own cases, a score on them, and a number below which the feature is switched off.

6. Why 4 waits for 3Measurement is meaningless if the data cannot legally go to where the model runs.

7. 4

   Does our data train someone else’s model?

   Where it goes, and who may train on it, sits in the providers’ contracts. A badge on a website is not the answer.

8. Why 5 waits for 4All of it is temporary if you cannot run the thing without us afterwards.

9. 5

   What happens when you leave?

   Ask now for the list of artifacts you would keep, in your own accounts, on the day the engagement ends.

All five are answered below, in roughly this order. The order is ours rather than yours, so if yours is different, that is worth saying on the first call.

1. Before any code

   The evaluation set is your cases, not ours

   An eval is a fixed set of real inputs with the answers you would accept, scored automatically whenever anything changes. Yours is built from tickets, documents or transcripts you already have. A case the team cannot agree on is itself a finding.

   We size the first set at 150 to 300 cases,
   each with an accepted answer and why.

2. Before any code

   The pass bar and stop number, in writing

   One score to clear before launch, and one below which the feature is switched off. Both are written down before we build, along with the test set and the metric each is measured on.

   NIST AI Risk Management Framework,
   the US standards agency’s guidance.
   MEASURE 2.1 and 2.3: test sets and
   metrics documented, criteria
   demonstrated under conditions
   like deployment.

3. During the build

   We measure what a reviewer cannot see

   A person checking output catches confident nonsense. They do not catch the answer the model failed to give, the record it skipped, the flag it did not raise. Those misses are the more damaging kind and the ones a score has to count.

   Separate scores for wrong answers given
   and correct answers missed.

4. After launch

   The same score runs on live traffic, scheduled

   Models drift: the same input produces different output as the provider updates the model. A score taken once before launch says nothing about the week after, so the same set runs again on schedule.

   NIST AI RMF MEASURE 2.4 and 2.6:
   behaviour monitored in deployment,
   residual risk inside tolerance.

5. After launch

   Every reported bad answer becomes an eval case

   A complaint is only useful if it changes the score. Every bad answer a user reports enters the evaluation set that week. The same failure is then measured on every run rather than remembered.

   NIST AI RMF MEASURE 3.3:
   users can report problems,
   fed into the metrics.

• It gives a customer a wrong answer, confidently.

  OWASP, the security standards body, lists this as LLM09 Misinformation. It is false or misleading output that appears credible, and OWASP names reputational damage and legal liability as consequences.

  What we commit to

  The wrong-answer path is written down before launch. It names what the user sees, who is told, and how the case enters the evaluation set that week.

• Somebody hides an instruction inside a document and the model obeys it.

  OWASP LLM01 Prompt Injection and LLM05 Improper Output Handling. A guardrail is a check that runs around the model, not a promise about it.

  What we commit to

  Untrusted text is never handled as instructions. Tool access is allowlisted, output is escaped, and you see the tests rather than a claim about them.

• It reveals something from another customer.

  OWASP LLM02 Sensitive Information Disclosure. What leaks is personal data, financial details, health records or security credentials.

  What we commit to

  Retrieval, where the model is handed documents to read, is filtered by the permissions your application already enforces. Nobody can see through the model what they could not already see in the product.

• It is given the ability to act, and acts too freely.

  OWASP LLM06 Excessive Agency. Agency is the ability to call functions and other systems, and the damage scales with that list.

  What we commit to

  Actions are enumerated, scoped to the minimum, and anything irreversible stops for a person. An agent, a model that calls tools in a loop, gets a written list of what it may call.

• The bill runs away in a bad week.

  OWASP LLM10 Unbounded Consumption. Every call to the model is billed, so nothing but a ceiling stops a runaway loop.

  What we commit to

  A per-request cost ceiling and a monthly cap, with an alert that fires before either one is reached. One bounds a single bad request, the other bounds a bad month.

• We stop working together and you cannot run it.

  If the AI feature is the slow part rather than the wrong part, that is performance work rather than this engagement.

  What we commit to

  Code and infrastructure, prompts, the evaluation suite, the runbook and the cost dashboard live in your own accounts throughout. Nothing has to be handed over at the end.