Complete an incident plan for a small software team

An incident plan must help your team handle interrupted work in an orderly way. Choose an important workflow in your software and describe how its failure is recognised, who makes decisions and what evidence is needed to resume operation.

Name the affected workflow

In this fictional example, incident I-72 affects a customer portal. The homepage responds, but new requests cannot be saved completely with an attachment. Request A-72 and file F-72 are test examples only. This is not an outage observed here.

For this case, homepage availability alone is not the criterion for closing the incident. The plan names the complete request workflow, including file storage, role checks and subsequent work. It also records which changes are allowed during the response.

Fill in a response card with roles

Google SRE describes clear roles, a current working record and confirmed handovers. Small teams may combine several roles in one person. It must still be clear who coordinates, who changes the system and who shares information.

The table describes planned steps, rather than a recovery already performed. Define a deputy and an available escalation contact. The working document must remain accessible if the portal is down.

Plan communication as a separate task

Name who informs affected people and the services involved. A status update states the known impact, current response progress and next agreed update point. A suspected cause is identified as a hypothesis; an unsupported recovery estimate is not passed on as a promise.

When people change over during the incident, the current state is handed over and acceptance is confirmed. The handover also includes unsuccessful actions and outstanding checks. Sending a message alone does not establish that the new coordinator has taken over.

Distinguish resumption from follow-up work

Before resuming, the team checks the affected workflow in a suitable environment using the agreed test cases. Whether returning to an earlier state is possible also depends on data changed in the meantime; a restart alone is not evidence of recovery.

Record who approves resumption and which outstanding tasks remain under observation. The subsequent root cause analysis can improve the plan, but need not be equated with the initial stabilisation. Practise the card using an approved scenario and correct missing contacts, access or checks.

Sources

• Google SRE: Managing Incidents