Keys To Successful Postmortems – Part II: Conducting the Meeting

Posted on April 13, 2011 Updated on September 3, 2014

In Part I, we talked about setting up the postmortem meeting.  The postmortem is part of a project or release close out and is also beneficial (and should be used) at the close of any security or production support incident, etc., as part of the RCA (root cause analysis).  Bottom line is that any time a project, incident or initiative is closed, there’s an opportunity to capture any lessons learned and leverage the knowledge going forward.

I’m a big fan of the ‘Plus/Delta’ framework for capturing postmortem findings. In this framework, a Plus is any practice, process or finding that worked well and should be kept or repeated in the future.  Conversely, a Delta is any practice, process gap or risk that resulted in a problem and should be changed or fixed.  Identify Pluses and Deltas for each phase of the SDLC/Project Life-cycle:

• Initiating
• Planning
• Executing
• Controlling
• Closing

You can use this framework to conduct the meeting, as well.  Keep it simple; for each phase, discuss and document all Pluses and Deltas.  For incident postmortems/RCAs, you can be flexible on the specific phases (as the SDLC or Project Life-cycle may not be appropriate).  For example, incident postmortems can make use the following phases:

• Identification
• Communication
• Short-Term Resolution (how was the problem fixed?)
• Long-Term Resolution (how do we make sure the problem doesn’t happen again?)
• Closure

The point is to go through the incident or project in a logical, chronological fashion, capturing Pluses and Deltas.

FACILITATION

Here are some guidelines to consider in facilitating the meeting:

• Who Should Facilitate the Meeting: The facilitator should NOT be a member of the project or incident team. The reason that the facilitator shouldn’t be someone involved in the project or incident is that you want THOSE resources to contribute content, while the facilitator is responsible for guiding the meeting, keeping it productive and acting as…..well….as the facilitator. It’s also best to have a neutral party facilitating the meeting in the case of a not-so-successful project or issue resolution.  This avoids anyone thinking that, for example, the developers got off easy because the facilitator was the lead developer on the project.  The facilitator should be someone familiar with the team who can keep the meeting controlled and productive. You can facilitate the meeting if necessary, but it’s always better if it’s someone at the same level as those on the postmortem team.
• Pre-Meeting by Functional Teams:  The postmortem team should not just “show up” for the meeting and figure it out as they go along.  Each functional team (e.g., the Development team, the QA team, the network team, etc.) should meet on their own beforehand to discuss their specific observations.  During this pre-meeting, each functional team should use the framework below to document their findings, then publish these findings to the rest of the postmortem team in advance of the meeting.  This will help stimulate thought and discussion, resulting in a more productive postmortem.  Strongly encourage all participants to review the combined findings beforehand.  The facilitator can also combine all the findings into a draft document and send it to participants in advance as well as using the document on the overhead to guide the meeting.
• Documenting: Use an overhead projector and a laptop to avoid any double-work in documenting the findings.  If that’s not an option, use a white board or easel and get everything written down and into the document as soon as the meeting is over (and throw all that paper away).  Only document findings that will result in an ‘Action Item’ or a very clear ‘Lesson Learned’.  For example, documenting that the QA team did “a really great job” feels great but it has no pragmatic benefit going forward.  The point is to create a deliverable that is useful beyond the project or incident.

GROUND RULES

Establish and communicate the ground rules in advance of the meeting and review them as the meeting begins.  Make sure you follow the same rules for each postmortem.  The facilitator MUST be very clear on the ground rules and tactfully enforce the rules while not quashing discussion (particularly with the ‘No Problem-Solving’ rule; that is the hardest to enforce, I guarantee!).  Here are some ground rules to consider:

• No Problem-Solving: The purpose of the postmortem is to capture Action Items and Lessons Learned and NOT to solve all the problems that came up during the project or incident.  Problem Solving sessions are Action Items and should be conducted outside of the postmortem and will include all stakeholders in resolving the particular issue or process gap.  While you will DEFINITELY have people in the meeting who’ll want to tear into a problem and start kicking around ideas, the facilitator has to stop this and instead, note an Action Item to set up a meeting to tear into the problem and NOT do it during the postmortem (if you ever want the meeting to end, anyway!).  Trust me – this is the one rule you’ll be enforcing right up to the end of the meeting, so be very tactful and positive about this – again, you do NOT want to quash discussion and have people shut down/not participate.
• Processes, Not People: When focusing on some of the Deltas, avoid focusing on the actions of specific people.  Focus on the process gap and not the person.  If someone’s actions caused a problem, this is evidence of a process gap that allowed something bad to happen.  If someone falls through a hole in the floor, firing the person who fell through the hole WILL get rid of the “faller” but the hole remains; fix the damn hole.  Besides, focusing on the people will lead to bad blood both during and after the meeting.  Don’t allow this to happen.
• No Distractions: This one is obvious.  The “no sidebars” rule is important and, to my mind, goes without saying.  I also recommend a ‘no Blackberries/iPhones’ rule.  With the sole exception of someone who has a specific reason, production support issue, etc., that necessitates that they keep their phone near them during the meeting, instead set up a “coral” (i.e., a shoe box or something) to house everyone’s phones and allow a break every 45 minutes / 1 hour for people to retrieve and check their phones.  This keeps the meeting focused and is respectful of other people and their time.
• Action Items: Action Items have ONE owner, who is accountable for ensuring the action is completed.  It’s true that Action Items usually have many stakeholders/participants, but only ONE person accountable for delivering – period.  Also, ‘TBD’ is NOT a due date.  Due dates have specific dates, months and years.  You can always change a due date but do NOT let anyone walk out of the meeting with an Action Items assigned to more than one person and with no firm due date.
• Follow the Framework: Make sure you follow the framework in order to maintain focus.  For example, for a project you’ll start with the ‘Intiating’ Phase and discuss what went well and should be repeated/kept (Plus) and what went badly and needs to be fixed/changed (Delta).  The idea is to keep the framework simple and clean and focus on identifying the action items and lessons learned.

Basically, your facilitator ensures that the brainpower in the room produces a solid list of Action Items and Lessons Learned and makes excellent use of everyone’s time.  The facilitator should ask for anonymous feedback after the close of the meeting – what could have been done better/different?  Another good opportunity to capture some learnings.  Also, allowing others to facilitate and avoiding the task yourself is a good way to build leadership skills in your team.  If your team is new to this process, you facilitating the first few sessions to establish precedence is worth considering.

In Part III, we’ll discuss the postmortem deliverable and I’ll include a sample.

© 2011, Mark E. Calabrese

Keys To Successful Postmortems – Part I: Setting Up the Meeting

Posted on April 8, 2011 Updated on September 3, 2014

Part of closing out any project, outage, service restoration team effort or any initiative whether succesful or otherwise (and it’s even more critical on the “or otherwise”) is the postmortem.  Too often, this milestone in the ‘Closing’ phase of a project is either ignored or not fully exploited.  I’d like to share some guidelines on conducting a succesful postmortem that net useable results.

While the guidelines below are not meant to be all-inclusive (I like to keep my posts short; no more than 2-3 scrolls as my friend Greg Hubbard at Cenage wisely advised).  Use your judgement on what to add, modify or ignore.  In Part I, we’ll focus on the meeting itself (timing, participants, etc.):

• Setting Expectations: Make it known either at the start of a project, early in the SRT (service restoration team) effort during an outage or just as a general principle of your management style that you will hold a postmortem after project or issue closure.  This will help ensure that people will take note of the good, the bad and other learning opportunities.
• When To Meet: Schedule the postmortem shortly after successful project go-live, or as soon thereafter as is practicable.  The key is to schedule the meeting so that doesn’t interfere with any critical post go-live activities but soon enough to capture any learnings while they’re still fresh in everyone’s minds.
• Where to Meet: Pick a conference room that has lots of room and ideally, lots of natural lighting.  You want to keep people awake and talking in order to get good input that can be used well after the project is completely closed out.  Make sure there are plenty of white boards or easels with tear-away Post-It sheets as well as markers that work.  Finally, have some coffee, water, pop (or ‘soda’ as you east coast people insist on calling pop) on hand.  I’m not a big fan of food at a meeting, but that’s your call.
• Conference Calls: Ideally, you want people to be there in person but these days that’s not always practical or possible.  Do the best you can.  In Part II, we’ll discuss how to ensure good engagement and input from those dialing in.
• Who To Invite: Include everyone on the project team, especially business partners who were critical to both the decision-making process as well as in any UAT, JAD sessions, requirement gathering/approvals, etc.  Ultimately, you want to include everyone who did the “real work” on the project, as well as any key stakeholders or project influencers.  If you find that the list of attendees is becoming unwieldy, try to limit it to those who have the most knowledge of what went right and wrong on the project and also make sure they are people who will SPEAK UP AND GIVE THEIR OPINIONS!
• Timing and Duration: Set aside at LEAST two hours and try to hold the meeting on a Tuesday or Wednesday morning.  Mondays tend to be busy and Fridays, people have other things on their minds (or may be working from home).  I prefer mornings, as afternoons can be “sleepy time” for some people, so earlier in the week and early in the day is optimal.  I like 9AM until 11 or 12.

In Part II, we’ll talk about conducting the postmortem meeting and in Part III, we’ll talk about the deliverables and putting them to work for you, your team and your organizations beyond the project so that you truly CAN leverage lessons learned.  Stay tuned.

© 2011, Mark E. Calabrese

Some Thoughts (and Some Frameworks) on Agile

Posted on April 4, 2011 Updated on September 3, 2014

I had an interesting coffee meeting today with Gregg Wheeler with Solstice Consulting.  Solstice is increasingly becoming known as a solid Agile partner, both in applying the principles to their work and also in helping technology teams apply these principles to their own methodologies.  They’re hosting a workshop in April – find more information here.

Gregg and I discussed how Agile is now viewed not as the label to “keep things the same” as was the fad with many technologists in the 1990’s.  Now, both technology AND business teams are seeing the value in applying these principles that allow the business-technology partnership to quickly adapt to and exploit business opportunities before the competition.  Below are a few frameworks that are worth posting, reading and if applicable at YOUR shop, are worth swiping.  All of them are broken down into that most magical of numbers – three:

THREE ROLES ON THE AGILE TEAM

1. Product Owner: Plain and simple, this is the business partner who has the business need and decision-making authority.
2. Scrum Master: The “project manager” or team lead, whose primary function is to remove obstacles to progress and to coordinate discussions, decisions and where necessary, activities.
3. Project Team: The people doing the work.

THREE ARTIFACTS FOR THE AGILE TEAM

1. Product Backlog: The prioritized “wish list” of enhancements, features, maintenance items, etc., that are required to meet the businesses needs.
2. Sprint Backlog: The next level of granularity, consisting of a breakdown by task with ETC (estimate to complete)
3. Burn-down Chart: The breakdown of tasks against the available hours.  For example, if a given Agile cycle is two weeks or X hours, the burn-down chart shows how the available time in the cycle will be consumed by the tasks within the Sprint Backlog.

THREE TYPES OF AGILE TEAM MEETINGS

1. Sprint Planning: Breaking down the Sprint Backlog into the requirements that will ultimately drive the tasks.
2. Daily Scrum: Led by the Scrum Master; these are typically 5-10 minute “stand up meetings” where the Scrum Master asks three key questions: 1.) What did you do YESTERDAY?; 2.) What is planned for TODAY?; and 3.) What OBSTACLES to progress need to be removed (the role of the Scrum Master, as noted above).
3. Demo and Retrospective: The team meets with the Product Owner to demo the work to date and to perform a brief postmortem to identify lessons learned and action items with SINGLE owners (a single point of accountability) and with DUE DATES (e.g., not “TBD”)

I think it’s worth repeating the three questions that are at the heart of the Daily Scrum.  Again – they are:

1. What did the team to yesterday?
2. What will the team do today?
3. What obstacles must be removed to ensure team success?

I really like the idea of the 5-10 minute “stand up” meeting.  I first read about these in Patrick Lencioni’s ‘Death by Meeting’ – a book I highly recommend as it’s full of good info and written in Lencioni’s parable-style that makes for a better read in my opinion.

Agile is a solid set of principles that, when employed by the right type of team, can not only provide exceptional business value to the “business” part of the business-technology partnership, but also creates a fun and rewarding working environment!

© 2011, Mark E. Calabrese