Module 7 · AI in the Real World — Use Cases & Value
Build vs. Buy & Measuring ROI
65 min
Learning objectives
- Decide when to use an off-the-shelf product or vendor API versus build a custom solution
- Estimate ROI for an AI initiative including ongoing run costs
- Sequence an initiative from proof of concept to pilot to scaled rollout
Build, buy, or call an API?
Most organizations should buy or call an API before they build. Off-the-shelf tools and vendor APIs let you ship in days, not months, with no ML team required. Building custom only pays off when the capability is core to your competitive advantage, when no product fits your need, or when data sensitivity or scale economics demand it.
Build vs. buy — The choice between developing an AI capability in-house (build) and purchasing a product or calling a vendor API (buy).
| Factor | Lean toward Buy / API | Lean toward Build |
|---|---|---|
| Time to value | Need results in days/weeks | Can invest months |
| Differentiation | Commodity capability (e.g., transcription) | Core to competitive edge |
| In-house ML talent | Little or none | Strong, sustained team |
| Data sensitivity / control | Vendor handling acceptable | Must keep data in-house |
| Volume economics | Low/moderate volume | Very high volume where per-call API cost dominates |
Default to buy or API; build only when the capability is a genuine differentiator or no acceptable product exists. Building 'because it's interesting' is the expensive path.
Analogy
Build vs. buy is like housing. Most companies should rent (API) or buy a finished home (off-the-shelf) — fast, predictable, someone else maintains it. You only build a custom house when your needs are truly unusual and you can fund the architects, construction, and decades of upkeep.
Estimating ROI honestly
ROI compares the value an initiative creates against its full cost. The discipline is to count all the costs — not just the build — and to be conservative about benefits. Many AI business cases look great until the ongoing run costs and human-oversight costs are added.
Return on investment (ROI) — (Benefit − Cost) / Cost over a defined period. For AI, 'Cost' must include ongoing API/usage fees, monitoring, and human review — not only the initial build.
- Quantify the benefit: time saved × loaded labor cost, error/loss reduction, or revenue uplift — using conservative estimates.
- Total the cost (TCO): build or license, integration, data preparation, API/usage fees, monitoring, and human oversight.
- Account for accuracy: if the model is right 90% of the time, factor in the cost of handling the 10% it gets wrong.
- Compare over a realistic horizon (e.g., 12 months) and add a sensitivity check ('what if usage doubles?').
Example — A simple support-automation ROI
An assistant deflects 30% of 10,000 monthly tickets. Each deflected ticket saves ~10 minutes of agent time at a loaded cost of $0.50/minute, so 3,000 tickets × $5 = $15,000/month saved. Ongoing run costs: API usage $2,000 + monitoring/review $1,500 = $3,500/month, giving a net of ≈ $11,500/month on a run-cost basis. A complete ROI must then also amortize the one-time build and integration cost over its expected life — a project that looks great monthly can still be negative in year one once build cost is included.
Watch out
The most common ROI mistake is counting only the one-time build cost and ignoring recurring run costs — API/usage fees, monitoring, retraining, and the human time to review outputs. A model that looks cheap to build can be expensive to operate.
From proof of concept to scale
Don't go straight to a company-wide rollout. Sequence the investment so you spend small to learn fast, and only scale what has demonstrably worked. Each stage has a different question and a different exit decision.
Proof of concept (PoC) — A small, throwaway test of whether the AI approach is technically feasible for the specific problem — before production investment.
Pilot — A limited but real deployment with actual users and data, used to measure value and surface risks before scaling.
| Stage | Question it answers | Exit decision |
|---|---|---|
| Proof of concept | Is this technically feasible at all? | Does it work well enough to try with real users? |
| Pilot | Does it deliver value safely with real users and data? | Are the metrics and risks acceptable to expand? |
| Scaled rollout | Can it run reliably and cost-effectively at full volume? | Sustain, optimize, or sunset based on measured ROI |
Spend small to learn, then scale what works: PoC proves feasibility, pilot proves value and surfaces risk, rollout proves it holds up at volume. Define the metric and the kill criteria before each stage starts.
Knowledge check
Quick practice — not part of your exam score.
When does building a custom AI solution make more sense than buying a product or calling a vendor API?
Which cost is most commonly omitted when estimating AI ROI, leading to overstated returns?
What is the correct sequence for responsibly introducing a new AI initiative?
Sign in to track your progress and mark lessons complete.
Sign in