When Outsourced DevOps Builds It and You Get the Pager

The demo went fine. Terraform in the vendor’s GitLab, an EKS cluster with three node pools, a pipeline that deploys on merge, green ticks against every line of the statement of work. The handover pack arrived a week later: a PDF architecture diagram, a Confluence page called “Runbook (WIP)”, and an invite for a two-hour walkthrough. Their engagement ends on the 30th. Yours does not. Your first 3am page will be about a component you have never watched deploy.

You are not being precious about this. What you are describing is a specific and predictable failure: outsourced DevOps sold as a build project, when the thing being built is not a project. Infrastructure is a running system, so a contract that ends at handover transfers the expensive half of the work without transferring the knowledge needed to do it. Operating a platform costs more over three years than building it. The vendor was paid for the cheap half.

The fix is not a better diagram. It is a short list of conditions, some contractual and some technical, and most can be pushed for from below by an engineer with no budget authority. One is a pass/fail test that takes a day.

Why handover works for software and not for infrastructure

Hand someone a compiled application and they have the application. Hand someone a platform and they have a text description of a live thing that keeps changing state without them. What makes it operable is procedural knowledge, and it accumulated in the people who built it: that the node pool has to be drained in a particular order or the ingress controller goes with it, that the RDS parameter group was changed by hand during the migration and never went back into code, that a change to the shared VPC module touches four services and should not be applied at 5pm on a Friday. Four months of being wrong in public produced that. A walkthrough does not move it.

The industry’s most-copied operations model refuses the simple handoff outright, which is worth citing when someone tells you this is just how vendor work goes. Google’s SRE book describes the Production Readiness Review: before an SRE team takes on a service it must meet “accepted standards of production setup and operational readiness”. The review runs through engagement, analysis against a reliability checklist, an improvements phase where developers fix what the checklist found, training with hands-on exercises, then onboarding in which responsibility transfers progressively while the people who built it stay available as backup. That is five stages of transfer with the builders still in the room, all of it inside one company. Your vendor proposed a slide deck and a final invoice.

A diagram also describes topology at a moment, while the real definition lives in the cloud API and in the state that maps your code to it. OpenTofu’s docs are blunt: state stores “bindings between objects in a remote system and resource instances declared in your configuration”, and the tool “cannot function without state”. If that state sits in the vendor’s account, you inherited a running platform rather than a reproducible one, which is worse to own.

Three versions of this that genuinely work

Bringing in an outside firm is not automatically a mistake, and plenty of internal teams could not have built what you now run. Three shapes work, and all three share one feature: somebody with knowledge is still around after go-live.

Greenfield build with your engineers embedded. The vendor brings people who have done it before; two of yours sit inside the build with commit access, on the same tickets, in your repository. Acceptance is your engineer performing the deploy, not a vendor demo. Strongest version, and it costs more, because two of your people are half-allocated for a quarter.

A defined migration with a runbook and a paid support tail. Scope has a real end: move 40 services out of the colo, replace the hand-built CI. The runbook is a deliverable, and for 60 to 90 days after cutover your team holds the pager as primary with the vendor secondary, priced monthly against a response target. The tail is where knowledge moves, because it moves under real failures.

Co-managed operations with a real split. The vendor keeps named functions, usually overnight monitoring and first-line response; you keep change, architecture and anything needing product context. Written function by function: who performs it, who holds admin rights, who gets paged, which budget it comes from. Vague co-management is the worst of both. Specific co-management is often right for a team of six that cannot staff a rotation.

What none of these is: build-and-leave with a documentation deliverable. If nobody named is reachable in month four, you are not in one of the three.

The handover artifacts that actually transfer capability

Most handover checklists list documents, and documents are cheap to produce. Ask for these six instead, each of which either works or visibly does not.

  • Infrastructure as code in your repository, with a working plan and apply from a clean checkout. Not a copy delivered at the end: your GitHub or GitLab org from the first commit, vendor engineers as members of it. Clean checkout means a machine that has never run it, provider versions pinned, and a plan that returns no unexplained drift.
  • Runbooks for the top ten failure modes, picked by asking what has broken and what would hurt most: failed deploy and rollback, certificate expiry, disk exhaustion, database failover and restore, a DNS change gone wrong, secret rotation, cluster upgrade, a queue backing up, a cost spike, a leaked credential. Commands and decisions, not prose.
  • A documented escalation path. Who is primary and secondary, who declares an incident, who may restart production unapproved at 4am, and which vendor number still works during the tail.
  • Dashboards and alerts that map to those runbooks. Every alert links to the runbook section that resolves it, and any firing alert with no runbook is an open item. Fastest way to audit a handover pack, and it takes an hour.
  • An architecture decision record for anything non-obvious. AWS’s guidance defines an ADR as a document describing “the architectural decision, its context, and its consequences”, and notes the structure “focuses on the reason for the decision rather than how the team implemented it”. That is what stops a later engineer overruling a choice they do not understand. One per question a new joiner asks: why Kubernetes and not ECS, why self-hosted Postgres when RDS exists.
  • A live rebuild exercise, covered next, because it is the only one of the six that cannot be produced by writing quickly the night before.

Documentation is easy to treat as optional, and DORA’s research argues against that: its 2022 analysis put the estimated performance lift from continuous integration at roughly 34% for teams with below-average documentation against about 750% for those above average. Model estimates from a self-reported survey, so read the ratio as direction rather than physics. The direction is not ambiguous.

The rebuild test, which settles it without anyone losing face

The best single acceptance criterion for an infrastructure handover is this: one of your own engineers destroys a non-production environment and recreates it from the repository, with the vendor not in the room. If that fails, the handover is incomplete regardless of what the contract says or how thorough the documentation looks.

Run it like an exam, not a workshop. Pick staging, and pick one of your engineers, ideally not the strongest, because the test is whether the system is transferable rather than whether that person is clever. Fresh clone on a machine that has never run the code. The vendor stays in another room, and questions go to them in writing so you have a log. Then destroy, rebuild, and time it.

The log is the real output: every question your engineer had to ask is a documentation defect with a name attached. The failures repeat across engagements.

  • Resources created by hand in the console and never imported, so the code does not describe the environment it manages.
  • Remote state in the vendor’s bucket, or on the laptop of whoever ran the last apply.
  • Secrets typed in once, so the rebuild produces a stack that comes up and cannot talk to anything.
  • A DNS zone or TLS certificate in an account nobody at your company can log into.
  • Unpinned provider versions, so a plan on a clean machine no longer matches the one from six months ago.
  • A bootstrap step, usually creating the state backend and the CI identity, that exists only in one person’s shell history.

Put it in the contract as an acceptance gate with money behind it: ten to twenty percent held back, released when your engineer completes a destroy and rebuild plus a restore from backup unaided. Vendors who have done this properly agree quickly, because they will pass. Resistance to the rebuild test is itself the finding.

If the vendor holds root, DNS or the secret store, you do not own the platform

Escalate this one hard. Ownership of a cloud platform is the ability to lock everyone else out of it, and teams routinely discover after final payment that they cannot. AWS is explicit about root credentials. Its guidance says not to create access keys for the root user “because the root user has full access to all AWS services and resources in the account”, requires MFA on the root user of an organisation’s management account, and describes centralising root access so member accounts hold no root credentials at all. None of that is available to you if the management account was opened on the vendor’s email domain under their partner agreement, and Google Cloud and Azure have the same problem in their own vocabulary. Before the final invoice is paid, get this in writing:

  • Cloud organisation or tenant on your company’s identity, billed to you directly, root or global admin credentials in your own vault, MFA devices you physically hold.
  • Domain registrar and authoritative DNS zone in accounts you control. The most common point of captivity and the easiest to overlook.
  • CI and artifact registry in your organisation, with the deploy identity federated from your own identity provider rather than a long-lived key you cannot see.
  • Secret store in your subscription, vendor holding access rather than ownership: Secrets Manager, Key Vault or your own Vault. Not a shared password-manager item, not a variable in their CI.
  • Remote state in a bucket in your account with locking enabled, and audit logs written where the vendor cannot delete them.
  • Every vendor engineer signing in through your identity provider, so offboarding is one revocation, plus one break-glass account excluded from that federation and held by you.
  • A rotation clause: on a stated date at the end of the engagement, every credential the vendor touched is rotated. Copies persist on laptops without anyone intending harm.

Why the two-day training session at the end does nothing

The usual remedy for a thin handover is a training session, and it is close to worthless for one reason: it is a lecture about procedural knowledge. Hearing someone describe a failover does not let you perform one at 2am with a manager asking for updates. Three formats do work, and they cost the vendor nothing extra when agreed at the start rather than bolted on at the end.

Pairing during the build. Your engineer works the vendor’s tickets while decisions are being made, which is when the reasoning is visible and cheap to explain.

Your engineer writes the runbook; the vendor reviews it. Invert the usual direction. If your person writes it they have to understand it, and the gaps surface while the vendor is still being paid. When the vendor writes and your person reviews, the review becomes a spell check and everyone signs off on something nobody can execute.

Reverse handover. In the final weeks your team presents the architecture back to the vendor, section by section, and the vendor corrects them. Then run a game day: the vendor breaks something in staging without warning, your engineer holds the keyboard, the vendor answers only when asked. You find out which parts your team cannot operate yet, which is what a checklist hides.

What you can do about it from where you sit

You are an engineer, the contract is signed, and the person who signed it does not want to hear they bought the wrong thing. Four moves, none of which need you to be publicly right about someone else’s decision.

Write the gaps down factually, and send them up rather than across. One page to your manager, not the vendor. Three columns: the missing artifact, the risk, who absorbs it. “No runbook for database failover. If the primary fails we are improvising a restore none of us has performed. Lands on me and two backend seniors.” No adjectives. It also protects you when the first bad outage arrives and someone asks why nobody flagged it.

Quantify the operational load in hours per week. Log it for two weeks: every alert, every access request, every “can you look at the pipeline”, every hour spent reading someone else’s Terraform. Then total it. “Operating this has averaged eleven hours a week across three engineers since cutover, roughly a quarter of an engineer nobody budgeted for, none of it in a sprint” moves a manager. “The handover was bad” does not. If the number comes out small, you have learned something too.

Propose a support tail or co-managed period with a number attached. Never bring a complaint without a purchasable option: 90 days, second-line only, your team primary, a stated number of hours a month, funded from the retention if there is one. Managers can act on a scoped proposal. They cannot act on dissatisfaction.

Use the rebuild test as the neutral gate. It is not an accusation, it is a test with a result. What to say: “Before we sign off I would like to run one acceptance check. I rebuild staging from the repo, with their team answering questions in writing. If it comes up clean we have confirmed we can operate this; if not, we get a list of fixes while they are still engaged.” Nobody has to be wrong for that to be reasonable, and a vendor who objects has to explain the objection.

Two things not to do. Do not refuse to operate it, because the outage lands on you whoever was right. And do not quietly become the hero who keeps it running, because invisible effort reads as a system that works fine.

Measure the aftermath with delivery metrics, not with resentment

Six weeks after the vendor leaves, the argument becomes whether the handover was adequate, and whoever has numbers wins it. DORA publishes five software delivery metrics, grouped as throughput and instability: change lead time, the time for a change to go from committed to version control to deployed in production; deployment frequency; failed deployment recovery time, the time to recover from a deployment that fails and requires immediate intervention; change fail rate, the ratio of deployments requiring that intervention; and deployment rework rate, the ratio of unplanned deployments caused by a production incident. DORA also warns against using these to compare teams or applications, or setting them as fixed targets. Use them on one service, in your own context, before and after the handover.

Add two local measures: alert-to-runbook coverage, as a percentage of firing alerts with a documented response, and operational hours per week from the log above, split into planned work and interruption. Track it for a quarter, then be honest about the result. If lead time and change fail rate held steady and the hours are modest, your complaint was about workload distribution, which is real but is a staffing conversation. If recovery time doubled and rework climbed because nobody understands the deploy path, you hold evidence of a handover defect, and the next contract gets written differently.

If your conclusion is that this work was never a project, and what you needed was experienced platform engineers inside your own team rather than a vendor who builds an environment and departs, that is the model AB7 Solutions works in: staff augmentation and contract staffing for platform, SRE and infrastructure engineers who sit in your repository, your standups and your rotation, plus managed SOC and cybersecurity cover for the overnight monitoring a small team cannot honestly staff. Named seats, with rotation notice and handover overlap agreed up front, because operating knowledge has to live with someone still there next quarter. Send us your handover pack and alert list and we will say where the operational gaps are, including when the honest answer is one internal hire rather than us. Call +1 321 341 7733, email ab@ab7solutions.com or director@ab7solutions.com, or start at www.ab7solutions.com.

Sources: DORA, DORA’s software delivery metrics (definitions of the five metrics, and the warning against comparing teams) and Documentation Quality (2022 research on documentation amplifying technical capabilities; self-reported survey data); Google, Site Reliability Engineering, “The Evolving SRE Engagement Model” (Production Readiness Review phases and progressive onboarding); AWS, Root user best practices for your AWS account and ADR process; OpenTofu, State.

Leave a Comment

Your email address will not be published. Required fields are marked *