Lesson 22: Capture Incident Learnings Into the SOP (Close the Loop)
Every team comes out of a serious operational failure a little smarter – at least for a while. They pick up details about what broke, why it broke, and exactly what worked when things finally got fixed. But this sharp, actionable knowledge? It’s fleeting. Give it a week, and most of it’s already fading.
There’s a short, incredibly valuable window right after any big incident. In those hours and days, the people who lived through the mess have a clear sense of what actually happened. The exact sequence, all the messy bits, the little things that turned a disaster into a quick fix instead of a drawn-out pain – it’s all right there, fresh and vivid.
But time does what it always does. The team reviews the incident, writes up a post-mortem, logs some action items, and then everyone moves on. A few weeks later, the rich, hard-earned knowledge has dulled. At best, it turns into a vague memory. Under stress, people end up guessing, swapping stories, or hunting for half-updated documents.
You get the usual: “I think last time we did this…” or, “Someone from the old crew handled it, but I’m not sure how…” or, “Pretty sure there’s a doc somewhere – maybe?”
This is how lessons die. Not with a bang – just one forgotten nuance at a time. Eventually, what should have made the next incident less painful is impossible to find, because nobody put it somewhere it would last.
So, what’s the fix? It isn’t fancier post-mortems or more complex documentation. It’s a rule: within 72 hours after an incident (or even a drill), the top couple of key lessons go into the standard operating procedure (SOP). Not next week – before the adrenaline wears off, while the details are crisp and the team hasn’t scattered to the next emergency.
What really matters: If you don’t get those specifics into the SOP fast, they turn into folklore. Teams that get this right don’t just bounce back when things break – they get better.
By now, if you’ve been following this series, you’ve probably got most of your operational system mapped out, documented, and tested. You’ve got reversal plans, SOPs, and drills that expose all the cracks. What you need at this stage is a habit: every incident should tighten your system. Not just patch things up, but actually improve for next time.
If an incident doesn’t update the SOP, you just burned time and stress for nothing. Next time, someone will hit the same gap, fumble through the same confusion, maybe even improvise the same fix – but if it’s not written down, nobody actually learns.
How you close this loop is dead simple: update the SOP within 72 hours. Add the new learning, tag the update with the incident ID, and broadcast it in whatever channel your team was using during the incident. It’s this habit that marks a team that improves from one that just survives.
A Real Example: The Hidden Nuance
This isn’t just theory. Take this: a team “solved” a recurring issue with a hotfix. It worked, but only because one person knew a subtle trick in the process. That was never written down.
Next time the problem hit, that person was gone. The team followed the steps they had, but missed the nuance – and the fix failed. It took hours longer. The knowledge existed, but only in one person’s head.
So, what did the team do next? They added the step to the SOP. Wrote it in simple, bulletproof language, so anyone could follow it cold. Marked it with the incident date and ID. Then they not only checked that the fix still worked, but assigned a backup to run it every month. Not just a review – an actual run-through, making sure the knowledge stuck.
Result? Next incident, anyone on the team could handle it, first try, no escalation. Time to resolve? Normal. Not thanks to heroics, just because the fix was written down and visible.
That’s the 72-hour rule in action: make hard-earned fixes easy to repeat, every single time.
How to Actually Close the Loop
Don’t overcomplicate it. Teams screw this up by making documentation a giant task, something you put off while catching up on everything else. Stick to this:
Step one: Within 72 hours, pick the top one or two fixes worth putting in the SOP. Not every lesson – just the most important couple. If you already ran a quick retro, you should know what these are.
Step two: Make sure the SOP says when to use it, right at the top. Was the trigger unclear before? Fix that first.
Step three: Write the step(s) into the SOP. Use clear, short language. If it’s a “gotcha” – a special sequence or decision – spell it out so the next person doesn’t need inside knowledge.
Step four: Update the audit trail. Add the date, the incident ID, so you know what changed, when, and why.
Step five: Announce it in the channel where the incident was handled – not just in the doc repo. That channel is where people are paying attention, and when they’re most likely to absorb the change.
The hardest moment: Editing When You’re Tired
Right after an incident, nobody really wants to slow down and update the SOP – you just want to be done. You want to move on. But this is when the details are sharp, and it’s when the update matters most.
That’s why the rule has to be hard and specific: 72 hours, assigned to a real person, and treated as part of closing the incident – not optional homework.
The useful mindset: “The incident isn’t closed until the SOP is fixed.” The resolution, the post-mortem, whatever else – none of it counts unless the learning makes it into the playbook.
The Honest Lesson
It’s easy to say “we’ll capture the lessons later” – to write action items or talk through next steps, then drift right back to business-as-usual. But unless someone actually updates the SOP, the real, useful knowledge stays hidden in someone’s head or in a doc nobody reads when it matters.
Editing the SOP right away isn’t flashy. It takes a quiet kind of diligence – the kind that understands the next team shouldn’t have to learn what you just did the hard way.
Updating the SOP after an incident is a small act of generosity. The team that lived it is leaving a map for the next team handing the same situation. That’s real operational maturity.
So, after your next incident? Block half an hour. Pull in whoever owned the fix, and get the lesson written down where it counts – before it fades into team folklore.
Next time, I’ll dig into “Automate One Manual, Repetitive Step” – see you there.