AI EngineeringZero to ProductionHome·About·What’s new·Contact
FDE Interview Prep · FI2

Deployment Scenarios

Building the system is half of a forward-deployed engineer's job. The other half is getting it adopted, trusted, and kept running inside a client's organisation. The deployment round tests that half: launches nobody uses, rollouts that must not hurt anyone, incidents in regulated industries, and systems that were never meant to be integrated with. This page works eight of those scenarios end to end.

⏱️ ~2 hours🧩 8 worked questions🎯 Essential→Tech-lead

Learning objectives

  • Diagnose a stalled deployment from data on how people actually use it, not from opinions.
  • Design a staged rollout with explicit gates, a rollback path, and a kill switch.
  • Handle a production incident in a regulated client: contain first, then investigate, then prevent.
  • Size a deployment's cost and capacity out loud, with numbers an interviewer can check.
📋 Before you startThis page builds on: The FDE method · MLOps & LLMOps foundations. Jumping in? Skim those first — or open the recap.
30-second recap
  • The FDE method: Work backward from the client's outcome, ship the thinnest end-to-end slice, and iterate in a tight feedback loop with real users.
  • MLOps & LLMOps foundations: Treat models, prompts and data as versioned artifacts with tests, deploys and monitoring — the same discipline as software, plus evaluation.

1 · What the deployment round is testing essential

Plenty of strong engineers fail this round because they answer every question with more engineering. Adoption is low? Improve the model. The client is nervous? Add a feature. But the hardest problems in a deployment are rarely in the code. They're in how people work, what the client's risk and compliance teams will accept, how the old systems behave, and whether anyone can tell the new thing is actually helping. An interviewer who asks "adoption is stuck at 12%, what do you do?" is not looking for a prompt-engineering fix. They want to see you find out why before you build anything.

Think of a deployment as a promise you make to the client: this will make a specific job better, safely, and you'll be able to prove it. The deployment round checks whether you can keep that promise when reality pushes back. The FDE method gives the core instinct — start from the outcome, ship a thin slice, iterate in a tight loop with real users. These questions add the operational side: how to roll out without harming anyone, how to respond when something goes wrong, and how to show value in terms the client cares about.

One shape covers most answers on this page. Measure the current state with real data (usage, outcomes, baseline). Diagnose the cause by talking to the people involved and reading the data together. Stage the change so it can be reversed: shadow, a small group, then wider. Gate each stage on agreed numbers. Prove the result against the baseline, in the client's language. Strong candidates say the first two steps out loud before proposing anything.

measure and diagnose before you change anything — then stage, gate, prove Measure usage · baseline Diagnose talk · read data Stage shadow → small Gate agreed numbers Prove vs the baseline gate fails → roll back a stage Measure → Diagnose → Stage → Gate → Prove. The first two steps happen before any change. Every stage can be undone, and moving forward needs numbers both sides agreed in advance.
🗺️ How to read this diagram
  • Read left to right. The two light blue boxes — Measure and Diagnose — come before you change anything.
  • Stage means the change goes out in reversible steps: shadow mode, then a small group, then wider.
  • The amber Gate is the decision point: move forward only if the agreed numbers hold. The dashed arrow is the rollback path when they don't.
  • The green Prove box is how the deployment ends: a result compared with the baseline, in terms the client's leadership understands.

In short: find out why before you build, make every step reversible, and finish with proof against where you started.

How to use this pageSame layout as FI1: the prompt as an interviewer would say it, what they're really testing, a weak answer, and a strong answer worked in full. Cover the strong answer and talk through your own for three minutes first, out loud, with a timer. Then compare.

2 · Launched, and only 12% use it essential

The question"We launched an AI assistant for the client's 600 claims handlers six weeks ago. Only 12% use it weekly. The client's sponsor is asking whether to cancel the contract. What do you do?"

This is the most common deployment question, and the number is meant to scare you into building something. The trap is assuming the product is the problem. Low adoption has many causes, and most aren't technical: people weren't trained, the assistant lives in a separate tab they forget to open, it helps with a task they rarely do, their manager still measures them on the old workflow, or one bad answer in week one cost it their trust. The interviewer wants to see you find the real cause with data and conversations before proposing a fix — and notice that the sponsor's question is really about value, not usage.

❌ Weak answer"I'd improve the answer quality, add some features users asked for, and send a reminder email encouraging everyone to use it." — Three guesses and no diagnosis. If the real blocker is that the tool sits outside the claims system, better answers won't move the number at all.
✅ Strong answer
  1. Split the 12% from the 88%. Usage logs first: who uses it, for which tasks, how often, and where people drop off. Do the regular users share a team, a manager, or a type of claim? Is anyone who tried it once never coming back?
  2. Watch people work. Sit with five heavy users and five people who stopped. Ask what they used it for and what made them stop. Watching beats a survey — people describe their work differently from how they do it.
  3. Match the cause to a fix. If it's outside their workflow, embed it in the claims screen. If it helps with a rare task, point it at the most frequent one. If trust broke early, show sources with every answer and fix the failure that hurt them. If managers don't back it, agree with leadership on what "using it" should change in their metrics.
  4. Answer the sponsor in outcomes. Compare the 12% with similar non-users on the metric the contract was signed for — say, claims closed per day or time to first decision. If regular users are 20% faster, there's a value story and an adoption problem. If they aren't, there's a product problem, and you say so.
  5. Agree a short recovery plan. A four-week plan with one or two targeted changes, a pilot team, and a number to hit, reviewed weekly with the sponsor.

Why this lands: you diagnosed before you built, you used both data and observation, you matched specific causes to specific fixes, and you turned the sponsor's "should we cancel?" into a question about measured value. Being willing to say "if users aren't faster, the product is the problem" earns more trust than defending it.

Go deeper: why good agents fail to land · change management · adoption metrics that aren't vanity · iterate in the feedback loop

3 · Rolling out a logistics agent intermediate

The question"A logistics client wants an agent that reassigns delivery routes when a driver calls in sick. A bad reassignment means missed deliveries and contract penalties. How do you roll it out?"

The interviewer wants a rollout plan where nobody gets hurt while you learn. An agent that takes real actions — moving routes, notifying drivers — can't be tested only in a lab, because the lab doesn't have Monday's real absences and traffic. But you also can't switch it on for every depot and hope. The answer is to increase the agent's autonomy in stages, each with a clear gate and the ability to step back down.

❌ Weak answer"We'd test it thoroughly in staging, then launch it to all depots with monitoring." — One big step, with no way to learn on real data before the agent can cause harm, and no definition of "working" that would trigger a rollback.
✅ Strong answer
  1. Shadow mode first. The agent sees every real sick call and produces a reassignment, but only logs it. Dispatchers keep working as normal. Compare the agent's plans with what dispatchers actually did, and with what happened on the road.
  2. Assist mode at one depot. The agent proposes; a dispatcher approves, edits, or rejects with one click. Track approval rate, how much they edit, and the outcome for approved plans.
  3. Limited autonomy. The agent acts alone only inside a safe envelope — for example, moves of up to 5 stops between drivers on the same shift — and asks a human for anything bigger. Widen the envelope as the data supports it.
  4. Gate each stage on agreed numbers. Example: move from shadow to assist when the agent's plans match or beat the dispatcher's on-time rate for two weeks; move to autonomy when approval without edits is above 90% and zero approved plans caused a penalty.
  5. Build the way back. A feature flag per depot to drop back a stage instantly, a kill switch that returns everything to manual dispatch, and an alert if on-time delivery falls below the baseline.

Why this lands: you increased autonomy gradually, learned from real data before the agent could cause harm, defined gates in numbers instead of feelings, and built the rollback before you needed it. "The agent earns autonomy" is the phrase to remember.

Go deeper: pilot → scale with graduation gates · human-in-the-loop matched to risk · flags, shadow mode & the kill switch · bounding every agent run

4 · The bank's compliance team found a violation intermediate

The question"You've deployed an assistant that drafts customer emails for a bank's relationship managers. Their compliance team just found a sent email in which the assistant promised a guaranteed investment return — a regulatory violation. What do you do in the next hour, the next day, and the next month?"

The three time frames are the point. The interviewer is checking that you know the order: contain the harm first, then find the cause, then prevent it from happening again — and that you keep the client's compliance team on your side throughout. A candidate who jumps straight to "I'd fix the prompt" has skipped containment while the problem may still be happening.

❌ Weak answer"I'd update the system prompt to tell the model never to promise returns, and add that example to our tests." — It might help eventually, but nothing stops the next bad email being sent today, nothing finds the other emails like it, and nothing explains how a draft with a guarantee got past a human.
✅ Strong answer
  1. Next hour — contain. Switch the feature to a safe mode: drafts held for compliance review, or the feature paused for that product area. Tell the client's compliance lead what you've done. Preserve all logs; don't change anything you'll need to investigate.
  2. Next day — scope and explain. Search every draft the assistant produced for similar language (guarantees, promised returns, "risk-free"). Count how many were sent. Trace the bad one: what context and instructions it got, why the model wrote it, and why the human who sent it didn't catch it. Give compliance a written timeline and the full list of affected emails so they can decide on any customer remediation.
  3. Next month — prevent. Add an output check that blocks prohibited claims before a draft is shown, using rules plus a classifier, with compliance owning the list of banned phrases. Add the failing cases to the evaluation suite. Make the review step harder to skip, for example by highlighting regulated phrases for the reviewer. Agree with compliance how incidents are reported from now on.
  4. Reopen gradually. Turn drafting back on in stages, with compliance sampling the output, until the agreed error rate holds.

Why this lands: containment came first, you scoped the problem before fixing it, you treated the human review as part of the system that also failed, and you made compliance an owner of the fix rather than an opponent. In regulated industries, how you handle the first incident decides whether the deployment survives.

Go deeper: grounding & the output contract · the approval gate & audit trail · AI incident response · AI governance & compliance

5 · A 15-year-old ERP with no API advanced

The question"The client's purchase orders live in a 15-year-old ERP with no API. Their IT team won't touch it — it's stable, and the person who knew it best has left. Our assistant needs to read order status and eventually create orders. How do you integrate?"

Legacy systems are where many deployments stall, and this question checks whether you can work around one without breaking it. The constraints are deliberate: no API, an IT team that's protective (and right to be), and a system nobody fully understands. The interviewer is listening for how you'd read data safely, how you'd treat writes very differently from reads, and how you'd earn IT's trust rather than going around them.

❌ Weak answer"We'd build a bot that logs into the ERP screens and clicks through them, for both reading and creating orders." — Screen automation can be a last resort, but using it for writes on day one, on a system nobody understands, is how you create corrupt orders that IT then has to clean up.
✅ Strong answer
  1. Find what already exists. Old systems usually have more ways out than people remember: a database you can read, scheduled export files, a reporting tool, a message queue other systems already use. Ask IT what other systems already connect to it.
  2. Reads from a copy, never from production. Use a read replica or the nightly export, so the assistant can't slow down or lock the live system. Be clear with users about how fresh the data is ("order status as of 6am").
  3. Put an adapter in front. A small service that turns the ERP's data into a clean, documented interface for the assistant. If the ERP is ever replaced, only the adapter changes.
  4. Writes come later, through the ERP's own rules. Start with the assistant drafting an order that a person enters or approves. When you automate, use whatever import path the ERP already supports for bulk loads, so its own validation runs. Use screen automation only if nothing else exists, and only with a person checking.
  5. Make IT a partner. Show them the read-only design first, agree on load limits and a maintenance window, and give them a switch to turn the integration off. Their trust is what unlocks write access later.

Why this lands: you looked for existing exits before inventing one, you separated low-risk reads from high-risk writes, you isolated the legacy system behind an adapter, and you treated the protective IT team as allies with a veto. That's how integrations with legacy systems get approved in real enterprises.

Go deeper: integration patterns for legacy systems · the strangler-fig architecture · reading legacy data safely

6 · The AI is slower than the manual process advanced

The question"The client measured it: invoice processing with our AI takes 9 minutes on average. Their manual baseline was 7 minutes. They're unhappy. What happened, and what do you do?"

A worse number than the baseline is uncomfortable, and the interviewer wants to see you treat it as data rather than something to explain away. Averages hide a lot. Maybe most invoices are faster with the AI but a few take far longer. Maybe people spend the saved time double-checking the AI because they don't trust it yet. Maybe the measurement itself compares different things. You need to break the number down before you can fix it — and you need to be honest if the AI really is worse for some kinds of work.

❌ Weak answer"The users are still learning, so it'll get faster. And the AI is more accurate, which matters more than speed." — Possibly true, but you've given the client nothing to check, and claiming better accuracy without having measured it sounds like an excuse.
✅ Strong answer
  1. Check the comparison is fair. Same invoice types, same period, same people? If the AI group got the harder invoices, or the baseline was measured differently, fix that first.
  2. Look at the distribution, not the average. If 80% of invoices take 3 minutes with the AI and 20% take 25, the problem is a specific group — say, multi-page invoices or a certain supplier's layout — not the whole system.
  3. Break down where the time goes. Waiting for the model? Fixing extracted fields? Re-checking everything by hand because people don't trust it? Each has a different fix: speed up the slow step, improve extraction for the hard group, or show confidence so people only check uncertain fields.
  4. Route around the weakness. Send the invoice types where the AI is slower straight to the manual process, and keep the AI where it's clearly faster. A partial win you can measure beats an overall loss.
  5. Agree how success will be measured. With the client: time per invoice by type, error rate after processing, and the share of invoices handled with no edits — tracked weekly against the baseline.

Why this lands: you checked the measurement, broke the average into groups, found where the time actually goes, and were willing to route work away from the AI where it loses. Clients trust an FDE who says "it's worse for these invoices, so we won't use it there" far more than one who argues with the numbers.

Go deeper: metrics execs care about · routing work by type · evaluation & testing

7 · Size it before you sign it professional

The question"The client has 2,000 support agents. Each handles about 40 tickets a day, and they want an AI draft on every ticket. Roughly what will it cost per month, and what capacity do we need at peak? Talk me through it."

This is an estimation question, and the interviewer cares more about your method than the exact answer. They want to hear you state your assumptions, do simple arithmetic out loud, keep track of units, and then say which numbers matter most and how you'd reduce them. Getting within a factor of two with clear reasoning is a strong answer; a precise-looking number with no reasoning is a weak one.

❌ Weak answer"It depends on the model and usage — we'd need to run a pilot to know." — True but useless. Contracts and budgets are set before pilots finish, and the client asked for an estimate.
✅ Strong answer
  1. Volume. 2,000 agents × 40 tickets = 80,000 drafts a day.
  2. Tokens per draft. Assume a 1,500-token system prompt plus 6,000 tokens of ticket history and retrieved articles = 7,500 input tokens, and a 400-token draft.
  3. Cost per draft. At an illustrative $3 per million input tokens and $15 per million output: 7,500 × $3/1M ≈ $0.0225, plus 400 × $15/1M = $0.006, so about $0.0285 — call it 3 cents — per draft.
  4. Monthly cost. 80,000 × $0.0285 ≈ $2,300 a day; × 22 working days ≈ $50,000 a month.
  5. Peak capacity. 80,000 drafts over 8 hours = 10,000 an hour on average. With a peak twice the average, that's ~20,000 an hour ≈ 330 requests a minute, or about 2.5 million input tokens a minute — which you check against the provider's rate limits before launch, not after.
  6. Biggest levers. Input dominates (~80% of cost). The fixed system prompt can be cached; trimming retrieved context from 6,000 to 3,000 tokens cuts cost by about a third; drafts nobody needs right away can go through a cheaper batch path. Then compare with the value: if a draft saves even one minute of a $30-an-hour agent's time, that's $0.50 saved against $0.03 spent.

The same arithmetic as a small script, so you can change assumptions and see the effect immediately:

Lab FI2.1 
cost_estimate.py# Back-of-envelope sizing for an LLM feature. Prices are ILLUSTRATIVE — use your vendor's sheet.
USERS          = 2_000      # support agents at the client
TASKS_PER_USER = 40         # tickets each, per working day
INPUT_TOKENS   = 7_500      # system prompt 1,500 + retrieved context 6,000
OUTPUT_TOKENS  = 400
PRICE_IN, PRICE_OUT = 3.00, 15.00   # $ per million tokens (illustrative)
PEAK_FACTOR    = 2.0        # busiest hour vs the average working hour
WORK_HOURS, WORK_DAYS = 8, 22

def per_million(tokens, price):
    return tokens / 1_000_000 * price

tasks_day  = USERS * TASKS_PER_USER
cost_task  = per_million(INPUT_TOKENS, PRICE_IN) + per_million(OUTPUT_TOKENS, PRICE_OUT)
cost_day   = tasks_day * cost_task
peak_rpm   = tasks_day / WORK_HOURS * PEAK_FACTOR / 60

print(f"tasks/day        {tasks_day:,}")
print(f"cost per task    ${cost_task:.4f}")
print(f"cost per day     ${cost_day:,.0f}")
print(f"cost per month   ${cost_day * WORK_DAYS:,.0f}")
print(f"peak requests/min {peak_rpm:,.0f}")
print(f"peak input tok/min {peak_rpm * INPUT_TOKENS:,.0f}")
▶ How this works
  1. Every assumption is a named constant at the top, so the conversation with the client becomes "which of these numbers is wrong?" rather than an argument about the total.
  2. per_million keeps the units honest: token prices are quoted per million tokens, and this is where estimates usually go wrong by 1,000×.
  3. It prints 80,000 tasks a day, about $0.0285 per task, about $50,000 a month, and about 333 requests and 2.5 million input tokens a minute at peak — the same numbers as the spoken answer.

Try this: halve INPUT_TOKENS and see how much the monthly cost drops, then double PEAK_FACTOR and see what happens to the rate you'd need from the provider.

Why this lands: clear assumptions, arithmetic anyone can follow, units checked, the peak rate compared with provider limits, the biggest cost driver identified, and the cost compared with the value. That last comparison is usually what gets the budget approved.

Go deeper: capacity estimation · unit economics · prompt caching as a cost lever · bulk jobs: too slow and too expensive

8 · "Every Monday at 9am, p99 latency is 10×" professional

The question"The client says the assistant is fine all week, except Monday mornings around 9am, when p99 latency jumps to ten times normal and some requests time out. What's going on, and how do you fix it?"

A problem that repeats on a schedule is a big clue. Something happens every Monday at 9am that doesn't happen at other times. It could be users: everyone logs in at once after the weekend, often with a backlog of weekend tickets. It could be other systems: a weekly batch job, a cache that expired over the weekend, or a scheduled re-index. The interviewer wants you to use the pattern to narrow it down, look at the data from that window, and fix the cause instead of just adding capacity.

❌ Weak answer"Add more servers and raise the timeout." — More servers don't help if the bottleneck is the model provider's rate limit or a cold cache, and a longer timeout just makes users wait longer before failing.
✅ Strong answer
  1. Look at that hour. Compare a Monday 9am window with a Tuesday 9am window: request volume, number of 429 (rate-limited) responses from the provider, cache hit rate, queue depth, and how long each step of a request takes.
  2. Check the common causes. A login surge plus a weekend backlog that pushes requests over the provider's rate limit, so retries pile up. Caches (prompt cache or retrieval cache) that expired over the weekend, so every early request pays the full cost. A weekly job — re-indexing, reporting, or another team's batch — that shares the same rate limit or database.
  3. Fix the cause you find. Retries with backoff and jitter so they don't arrive all at once; a queue that smooths the burst instead of failing; warm the caches before 9am; move batch jobs out of business hours or onto a separate rate-limit budget.
  4. Protect the user experience. Give interactive requests priority over background ones, show progress instead of hanging, and fall back to a smaller, faster model when the main one is overloaded.
  5. Prove it next Monday. Set an alert on Monday 9am p99 and compare it with the previous four weeks.

Why this lands: you used the schedule as evidence, compared the bad window with a normal one, named the specific shared resources that fail under bursts, and fixed the cause rather than adding hardware. The "warm the cache before 9am" detail shows you've seen this happen before.

Go deeper: rate-limit thundering-herd outage · latency SLO missed under load · graceful degradation · queues to absorb bursts

9 · Design the kill switch before launch tech-lead

The question"You're the tech lead for a client deployment that goes live in two weeks. The client's CIO asks: 'If this goes wrong at 2am, what happens?' Give them an answer."

The CIO is asking whether you've planned for failure, and a tech lead should have an answer ready before anyone asks. An LLM feature can fail in ways normal software doesn't: a model update changes behaviour, the provider has an outage, a bad document gets indexed, or a prompt change goes wrong. The interviewer wants to hear a plan someone can follow at 2am without you: how failure is detected, who is woken up, how the feature is switched off or rolled back, and what users see while it's off.

❌ Weak answer"We have monitoring, and our team will fix any issues quickly." — The CIO hears "we haven't planned it." Who gets alerted? What can they do at 2am? What do users see in the meantime?
✅ Strong answer
  1. Detect. Alerts on what matters: error rate, latency, failed safety checks, and a sudden change in output quality from the sampled evaluations. Each alert links to a runbook.
  2. Respond. A named on-call rota covering both teams, with a clear escalation path to the client.
  3. Switch off safely. A kill switch that anyone on call can use in one step, without a deploy, that sends users back to the manual process or a static fallback. Test it before launch and again every quarter.
  4. Roll back. Prompts, model version, and retrieval index are all versioned, so you can return to the last good combination in minutes. Changes go out gradually behind flags, so most problems are caught at a small percentage of users.
  5. Tell people. Users see a clear message and the fallback, not an error. The client gets an incident note within an agreed time and a written review afterwards.
  6. Practise. Before go-live, run a short drill: simulate a provider outage and a bad prompt release, and time how long it takes to detect and recover.

Why this lands: you gave the CIO a plan someone can follow at 2am — detect, respond, switch off, roll back, communicate — and you proved it with a drill. "We tested the kill switch last Tuesday" is the sentence that makes a CIO sign off on go-live.

Go deeper: deployment & rollback · production go-live checklist · progressive delivery strategies · AI incident response

🪜 Practice ladder beginner → industry

  1. Beginner: answer §2 (12% adoption) out loud in three minutes, naming Measure → Diagnose → Stage → Gate → Prove as you go.
  2. Easy: for §3 (logistics rollout), write the exact numbers you'd use as the gate between each stage, and what triggers a rollback.
  3. Core: change the assumptions in Lab FI2.1 for a deployment you know and present the monthly cost and peak rate in two sentences.
  4. Stretch: write the one-hour / one-day / one-month plan from §4 for a different incident — an assistant that leaked one customer's data into another's answer.
  5. Hard: run a mock with a peer playing the client's protective IT lead in §5. Get them to agree a read-only integration without going around them.
  6. Industry: write the 2am runbook from §9 for a real system you've worked on, then run a timed drill with a teammate.

✓ Checkpoint — you can move on when you can…

  • Diagnose low adoption from usage data and observation before proposing a fix.
  • Design a staged rollout — shadow, assist, limited autonomy — with numeric gates and a rollback path.
  • Respond to a compliance incident in the right order: contain, scope, prevent.
  • Estimate a deployment's monthly cost and peak capacity out loud, and name the biggest lever.
  • Explain to a CIO what happens if the system fails at 2am.

Knowledge check check yourself

✓ Knowledge check

In an FDE interview, you're told an AI assistant has only 12% weekly adoption six weeks after launch. Why is "improve the model" a weak first answer, and what should come first?

Show answer
Most low-adoption causes aren't model quality: the tool may sit outside people's normal workflow, help with a rare task, lack manager support, or have lost trust after an early bad answer. Start by splitting users from non-users in the usage data and watching both groups work, then match the specific cause to a specific fix. Answer the sponsor in outcomes — compare regular users with similar non-users on the metric the contract was signed for — so the decision is about measured value, not usage alone.
✓ Knowledge check

What are the stages of a safe rollout for an agent that takes real actions, and what decides when it moves to the next stage?

Show answer
Shadow mode (the agent decides but only logs, compared with what humans did), then assist mode (the agent proposes and a human approves or edits), then limited autonomy inside a safe envelope that widens over time. Moving forward is decided by numbers agreed in advance — for example, matching the human on-time rate for two weeks, or approval without edits above 90% with no harmful outcomes. Every stage has a feature flag to step back down and a kill switch to return to the manual process.

Go deeper across the course

7The Forward Deployed Engineer method EL4Rolling agents into an org CS8Case · legacy AI retrofit CS7Case · regulated industry
© 2026 studybydoing.in · AI Engineering: Zero to Production · All rights reserved. · About · Privacy Policy · Terms · Contact
Educational content, provided as-is and without warranty. Code samples are examples — review, test, and adapt them before using in production. See the Terms of Use & Disclaimer. Use at your own risk.
© studybydoing.in