Written on 02:29 reading time

“If you want to make God laugh, tell him about your plans.” Woody Allen

Happy Friday the 13th.

The single incontrovertible fact in anything is that things are going to go wrong. Oh, you can try and plan for it. You can develop resiliency, elasticity, scalability and all the other ty’s you want, but things are still going to go wrong. Hypervisors are going to fail, deployments are going to botch things even though CI said it wouldn’t, message queues are going to go haywire, and things are going to crash.

Brent Chapman gave a great talk at O'Reilly’s 2008 Velocity conference titled “Incident Command for IT: What We Can Learn from the Fire Department.” After looking at this presentation, our team started using ICS anytime things went south. At first, ICS can seem a little heavy weight especially in the heat of the moment. You might honestly think “Do we need to use ICS for this? Let’s just figure out why the message queue isn’t draining!” And sometimes, that’s the right thinking ...

Read more