The FDE Toolkit
Chapters 10 and 14 argue that GenAI value is captured workflow by workflow, with evidence, not enthusiasm, as the gate. This appendix supplies the four working artifacts that discipline requires. They are deliberately short: each fits on a page, and each has been stripped to the questions that change decisions. Copy them, adapt the vocabulary to your organization, and treat them as living documents that improve with every pilot.
A.1 The Pilot Charter (One Page, Written Before Any Building)
A pilot without a charter is a demo with a budget. The charter forces the six decisions that determine whether a pilot can ever produce a deployment decision, and it is written before the first prompt.
| Field | What to Write | Example (invoice-coding copilot) |
|---|---|---|
| Workflow & owner | The specific recurring task and the single named person with authority to change how it is done. | Coding incoming supplier invoices to GL accounts; owner: AP team lead. |
| Baseline | Current cost, cycle time, and error rate, measured, not guessed, over a defined period. | 4,100 invoices/month; 6 min avg handling; 2.9% coding-error rate (April audit). |
| Success metric & threshold | The one number that triggers a scale decision, and the level it must reach. | ≥95% of AI-suggested codes accepted unchanged; handling time under 2 min. |
| Eval set | Where the golden examples come from and how many (see A.2). | 60 real invoices from last quarter, incl. 15 known-tricky cases; correct codes from senior clerk. |
| Permission line | What the system may do alone vs. what requires a human, stated as actions. | Suggests code + confidence; clerk approves. No auto-posting in pilot. |
| Kill criteria & review date | The evidence that ends the pilot, and the date the decision meeting is already on calendars. | <85% acceptance after two prompt iterations, or any hallucinated vendor data; review June 30. |
The two fields teams resist are the baseline and the kill criteria, and they are the two that matter: without a baseline no result can be interpreted, and without pre-agreed kill criteria no pilot ever dies, it just fades into the 90-percent graveyard Chapter 2 described.
A.2 The Eval Design (Worked Example)
Section 3.2 introduced the minimal eval; here is the complete recipe as a worked example for a customer-support drafting assistant. The same skeleton adapts to any text-producing use case.
Step 1: Collect. Pull 50 real, recent tickets, stratified deliberately: roughly 30 routine, 10 hard (multi-issue, angry customer, policy edge case), 10 that should not be answered by AI at all (legal threats, safety issues, VIP escalations). The third group tests the most important behavior: knowing when to hand off.
Step 2: Define pass/fail per dimension. A draft passes only if it clears all five: Factual, every claim about policy or the customer's account is correct against source documents; Complete, addresses every issue raised in the ticket, not just the first; Safe, refuses or escalates the ten handoff cases; On-brand, tone a supervisor would send unedited; Grounded, cites or quotes the policy passage it relied on. Binary per dimension beats 1-10 scores: graders agree on pass/fail far more than on the difference between a 6 and a 7.
Step 3: Grade. First pass: an LLM judge applies the rubric to all 50 (cheap, repeatable). Second pass: a human, the support lead, not the project team, audits 15 of the judge's grades weekly, including every failure. If human and judge disagree more than ~10% of the time, fix the rubric before trusting the judge.
Step 4: Use. Every change, prompt wording, retrieval settings, model version, runs the full set before shipping. Track the pass rate on a visible chart. The eval is also your regression alarm for vendor model upgrades (Section 5.5) and your evidence pack for the governance review (A.4).
Cost reality check: building this eval takes the support lead about a day, and running it costs under a dollar in tokens. It is the highest-return day in the entire project.
A.3 The ROI Model (With the Cost Side Filled In)
Most GenAI ROI claims quote the benefit side and wave at the costs. Here is the honest arithmetic, using the invoice-coding pilot from A.1. Substitute your own numbers; the structure is the point.
| Line | Item | Amount |
|---|---|---|
| B1 | Time saved: 4,100 inv/mo × 4 min saved × €0.60/min loaded cost | + €118,000/yr |
| B2 | Error reduction: coding errors 2.9%→1.2%, × €35 avg rework cost | + €29,000/yr |
| C1 | Inference: ~2,500 tokens/invoice at mid-tier rates | − €1,800/yr |
| C2 | Build: 6 person-weeks (builder + AP lead time) incl. eval construction | − €28,000 one-off |
| C3 | Integration & security review (ERP connector, key management) | − €15,000 one-off |
| C4 | Change management: training, floor support, two process revisions | − €12,000 one-off |
| C5 | Run: monitoring, eval upkeep, quarterly model-upgrade regression | − €14,000/yr |
| Year-1 net (B1+B2 − C1 − C5 − one-offs) | ≈ + €76,000 | |
| Steady-state net (recurring only) | ≈ + €131,000/yr |
Three lessons generalize. Inference (C1) is almost never the cost that matters; people costs (C2-C5) dominate, which is the 10-20-70 rule showing up in the budget. The benefit lines are real only if the saved minutes are redeployed, Chapter 2's evaporation problem, so pair the model with an explicit answer to "what will the AP team do with the time?" And the steady-state line is contingent on C5 actually being spent: an unmonitored system's benefits decay silently as inputs drift and models change.
A.4 The Governance Intake (Use-Case Register, One Row Per Deployment)
Chapter 13's obligations become operational through one unglamorous artifact: a register with a row for every AI use case, filled in at intake and updated at each review. Ten fields cover what boards, auditors, and (for EU-touching deployments) regulators ask.
| # | Field | The question it answers |
|---|---|---|
| 1 | Use case & owner | What does it do, and who is accountable by name? |
| 2 | Autonomy class | Assistant, Automation, or Agent (Chapter 7 behavioral test)? What actions can it take alone? |
| 3 | Data touched | Personal data? Confidential data? Which tier of vendor agreement covers it? |
| 4 | EU AI Act tier | Prohibited / high-risk / limited / minimal, and in which jurisdictions it operates (incl. US state exposure). |
| 5 | Injection surface | Does it process untrusted content? Does the lethal trifecta apply (Chapter 3)? |
| 6 | Eval & pass rate | Link to the A.2 eval; current pass rate; date last run. |
| 7 | Human oversight | Where is the human in the loop, and what do they see when overriding? |
| 8 | Monitoring | What is logged, who reviews it, on what cadence, and what triggers escalation? |
| 9 | Labeling | Is output disclosed as AI-generated where required (EU transparency, China labeling rules)? |
| 10 | Review & sunset | Next scheduled review; conditions for decommissioning. |
Run intake as a fifteen-minute conversation, not a compliance ambush: the goal is that filling the row is easier than avoiding it. A register like this is also the fastest possible answer to the two questions executives increasingly face from boards: "where are we using AI?" and "how do we know it is behaving?" If Chapter 14 gave you the offense, this page is the defense, and organizations that run both are the ones for which the technology compounds instead of surprises.