How to Know When an AI Pilot Is Worth Scaling
Read Between the Roles | By Erin Harris
Pilot activity is not a business result
AI pilots are easy to celebrate because they create visible activity. A new assistant goes live. Employees try it. A demo impresses leadership. A few early tasks happen faster. None of those facts answer the question that matters for the next budget decision: should this become part of the operating model?
AstraZeneca's Head of AI for IT posting, dated October 8, 2026 (R-261565), emphasizes use-case prioritization, investment decisions, adoption, governance, reusable capabilities, and measurable value. This is an AI-for-IT leadership role; the original Operations IT strategy posting from the earlier research was not recovered. The verified responsibilities point to a useful discipline for any organization: a pilot should be designed to inform a decision, not simply prove that technology can work.
Start with a baseline, not a promise
Imagine a 20-person services firm testing AI-assisted proposal drafting. The team can create a first draft in minutes. That sounds like a success until managers discover they spend more time checking scope, pricing, client commitments, and factual accuracy.
Before the test begins, record the current process. How long does a proposal take from request to approval? How many revisions occur? Where does work wait? What does quality look like? What labor is involved? Without a baseline, the organization has no reliable way to tell whether the pilot improved the whole process.
Define success and failure before enthusiasm takes over
Choose a small number of measures tied to the business outcome. For proposal drafting, those might include total cycle time, revision count, manager review time, error rate, employee experience, and win rate where the sample is large enough to interpret responsibly.
Also define failure signals. For example: confidential information appears in an unapproved system, correction time exceeds the time saved, a required control cannot be enforced, or employees consistently bypass the workflow. A pilot should be able to fail clearly enough that leadership can learn from it.
Use three decisions: scale, revise, stop
Scale when the improvement is meaningful, the controls are sustainable, and the economics still work after review and maintenance are included. Revise when there is a specific bottleneck that can reasonably be corrected. Stop when value remains unproven relative to cost, risk, or disruption.
A stop decision is not wasted effort. The purpose of a pilot is to reduce uncertainty. Learning that a use case is not ready can prevent a much more expensive rollout.
Do not measure only the first step
AI can make a draft appear quickly while shifting labor downstream. It can automate classification while increasing exception handling. It can create more leads while overwhelming sales with poor-fit inquiries. Measure the entire value stream from trigger to business outcome.
Include implementation cost, software cost, review time, maintenance, training, incidents, rework, and the cost of errors. That is closer to the real economics than a simple estimate of hours saved.
The leadership lesson
My operations experience taught me that managers need checkpoints, not just initiatives. A new process should have an owner, a target, a review date, and a decision. AI pilots deserve the same discipline.
The question is not, “Are we doing AI?” It is, “What uncertainty did this pilot reduce, and what should we do next because of what we learned?”
Bring the question into your business
EH Synergy helps businesses define AI opportunities, establish baselines, design bounded pilots, and evaluate results against business outcomes before scale. If your organization has several AI experiments but no clear scale, revise, or stop criteria, the portfolio may need a decision framework more than another pilot. Learn more at EH Synergy’s consulting contact page.
Source and editorial disclosure
Original employer source. Posting date: October 8, 2026. This dated analysis does not guarantee that applications remain open. The business scenarios are hypothetical. Recommendations and organizational implications are independent Erin Harris / EH Synergy analysis, not evidence of internal employer failures, an industry-wide trend, or an employer affiliation.

