Mel Bost is a Principal Consultant in BOT International’s PMO Practice who specializes in PMO best practices, project lessons learned, and program management. He is experienced in all aspects of project and program management, including strategic planning, design thinking, knowledge management, risk management, capital project audit and control, contactor audits and business process analysis.
Download PDF Transcript of Podcast
Note: This is a transcription of a podcast. It has not gone through a professional editing process and may contain grammatical errors or incorrect formatting.
Related Podcasts: Patterns of Behavior affect Projects & Lesson Learned in Project Management
Transcription of Podcast
Welcome, everyone. This is Joe Dager, the host of the Business 901 podcast. With me today is Mel Bost, he’s a principal consultant in BOT International PMO practice, who specializes in PMO’s Best Practices, Project Lessons Learned and Program Management.
Prior to venturing into the world of consulting, Mel worked for a number of large national and international companies, including Exterran Corporation, ConocoPhillips, ARCO, and Ford Motor Company. He is the author of the blog, “Mel Bost, PMO Expert,” which addresses the structure, activities and behavior of a program management office environment. After all that Mel, could you just start out and break down something simple. What does BOT International stand for?
Mel: Joe, thank you. Thank you so much for the opportunity to talk with your audience today and to give them some background on my Project Closeout and Lessons Learned practice area. BOT is an organization that’s been in existence for about 10 or 15 years. Our founder, Mark Price Perry, was an IBM employee for a number of years, working in the Asian markets, Japan, and in that area, helping them set up program management offices, PMOs, which is an organizational structure that most mature project groups use to ensure repeatability and consistency of project delivery.
Now, BOT is interesting because when Mark was working with these groups in Japan they said “We want you to put together a PMO for us, but we want you to build it first of all. We want you to operate it for us for a while until it gets to a good, steady state, and then we want you to transfer that knowledge and the ability to operate it to our organization.” That’s where BOT comes from. It’s an acronym for build, operate and transfer. Not a lot of people are familiar with that. A lot of IBMer’s may be familiar with that, but it’s something that this organization has carried forward as part of its marching orders.
Joe: You’ve been recently globe-trotting around the world to places like Dubai and Panama. Working, teaching and training on a certain, specific area of Project Management. You alluded to Lessons Learned and Project Closeout. Give me the elevator speech on that portion of it and what that is.
Mel: You’re right, Joe. I have been to Dubai to do some training. I spent 10 days with the Panama Canal Authority in 2011, training their project engineers and Contracted Administrators in their Construction Division on Project Closeout Lessons Learned. I originally got involved in Project Closeout when I was with ConocoPhillips. Back in 2002 and 2003, when Conoco and Phillips merged, they asked me to join a group in that organization to set up IT Shared Services Program Management Office, PMO. As we put together the methodology and how we wanted to accomplish this, over time, we began to realize that we were developing a lot of knowledge, a lot of information, a lot of data about projects, but we weren’t capturing it very well. We certainly were not sharing it with other project managers who needed to hear this. I was assigned to go out and put together a MS SharePoint type system to capture, document and share lessons learned. That was my first foray into the real lessons learned category.
I’ve found, in my work with other PMOs that it’s the mature organizations who have tackled the basics of project management who, now, are beginning to feel as they need to document, capture and share those lessons learned. That’s basically what my practice has been centered on in the last 8 or 10 years, a little bit more of the basics. What I’ve done is developed a framework that most organizations and smallest groups could use to capture lessons learned. I have a small, one page, Word document template, and basically, it follows what most people might have heard of as the after-action review format. Those of your audience who may have had some military service probably have come across that term, the AAR, which is After Action Review. It means that after every engagement or initiative, the group pulls back, and they say, “What went right, what didn’t go right?” Basically, there are four questions you ask yourself. What was the expected result? What was the actual result? What was the gap between the two, and what can we learn that we can carry forward to the next initiative? That’s a brief background of how I got involved in this.
Joe: Pretty tough to put a project to bed. One of the first things when I think about a closeout is when do you know when a project is done?
Mel: That’s a very good question. Some groups are bad at trying to close out projects. They want to leave them open. They want to make sure every bill is paid. They want to make sure that every resource is accounted for. That’s one reason that we have projects. Projects, as defined, are an initiative that has a start and finish date. What we try to do is we try to get the stakeholders to agree, “Yes, the major objectives have been concluded for this particular project, and yes, we’re ready to close it out.” It’s very important to have a closeout session for a project. I find that a lot of organizations talk about doing closeout and the post-mortem examinations of what happened. Very few are good at that. Project Closeout is an important aspect.
It’s interesting that when I was in Panama, a lot of their project work there with the expansion of the Panama Canal had to do with individual contracts. In other words, they had contracts with third parties who would come in and do a major portion of the work. I found that the framework that I had put together applied equally well to closing out contracts as it did to project. You’re exactly right, knowing when and how to close out a project is very important. Really, I believe it’s the responsibility of the project manager to know how to carry that out and actually carry through because the organization, sometimes, will leave these open-ended and not take the appropriate action.
Joe: When we talk about a PMO and a project manager in an office, when you open that, is that just an ongoing system of people that handle projects that their formal job is project management, and there’s groups of three or four of them that may work together on a project, or they may all work individually on projects and just be the project manager?
Mel: PMO’s have a lot of different shapes and forms out there, depending upon the size of the organization that’s trying to employ it. The PMO’s that I have worked with have ranged from large PMO’s that not only develop the standards, best practices and methodology for the whole organization; they also will provide project managers who will carry those projects forward for the business function. There are other groups who take a little bit different approach. They’ll have a centralized group they call a PMO, that’s involved in making sure that there’s repeatability that every project that’s done follows the same path, and there’s consistency of delivery. In that particular PMO, they might leave the project managers in the business or functional areas where the stakeholders live that are going to benefit from those projects. So, you’ve got sort of a centralized and decentralized aspect to PMO’s, depending upon the way the organization wants to set it up.
In my experience working for ConocoPhillips from 2003 and 2008, we were, first of all, part of a merger process. That’s a special situation; where you’ve got two large organizations that bring their own set of systems, their own set of applications. The first couple of years of that process are involved in systems integration and applications rationalization. After you get through with that piece of it, then you begin to see their business functions that have formed from this merger that need our assistance in carrying out this project. The PMO might morph a little bit from being just a systems integration applications rationalization type group, to one that’s serving those business functions. During that whole process, it’s important to have somebody who’s looking at standards and best practices and bringing some of those practices into the organization.
In my experience, there are a couple of major functions or competencies, capabilities that most PMO’s don’t start out with, but because of the feedback they receive on how well they’re delivering projects and the outcomes, the two that come to mind that I’ve most often worked with after a PMO’s been in effect for a very long time, are vendor management and risk management. Oftentimes, these days, a project is really the coordination of a third party to come in and do some work for your organization. That’s become more and more of a practice out there because most organizations don’t do a lot of work internally now. They employ third parties or contract people. I found that to be particularly the case with something like the Panama Canal expansion; because they wouldn’t possibly have all the excavation, concrete work and all the technology development.
The other area is risk management. A lot of PMO’s begin to grow into risk management because they recognize that they’re original assumptions for the project didn’t incorporate or didn’t change much throughout the project. There were a lot of risks that they might have encountered that were not planned. The answer to your question, I think, is it depends on the organization. It depends on whether they want to employ a centralized or decentralized approach to where the project managers reside and, also, who is going to do the basic work of best practices and bench-marking. Bench-marking your organization by going out and talking to other PMO’s or other similar groups is part of a major function of PMO’s.
Joe: When I think about this, I think that there’s a project manager out there that manages a project without necessarily having the technical skills to deliver the project. Is that true?
Mel: Not really, I think one of the ways that we try to choose project managers, especially if they are in a business functional area, is that they have a lot of expertise in that particular business context. They may not have all the project management skills, but that’s where the PMO comes in to help lead that the project manager. Most project managers, and say an IT, a specialized IT type of environment for a PMO; those people will probably be pretty well trained on methodologies for carrying out projects. That’s just the nature of how IT has evolved over time. It was back in the 1985-86 time period, where IBM went out and developed what’s called the Software Capability Maturity Model. Many of your audience may be familiar with that, and that became sort of the standard by which a lot of these IT PMO’s were put together as having a Capability Maturity Model; that gave him some feeling for how they wanted to mature. I wouldn’t say that a lot of project managers lack the capability. I think the Project Management Institute, the international standards organization that a lot of the project managers become certified under; they certainly try to instill in project managers out there the need for those technical skills.
Joe: To go back to the Lesson Learned and the Closeout; in your description there, it was driven by the need for developing a better way to do projects, is what drove this Project Closeout. Would that be correct, to be able to take that task and knowledge and make it explicit to other parts of the PMO, other project managers?
Mel: What I was talking about there is, we had a couple of significant projects that, at the end of the project, we found out we had made the same mistake in all three projects. Someone asked the question, “Was this particular mistake analyzed, and was anything done about it after the first project?” It wasn’t, and here came the second project, and we did the same thing again. A good example of that I lived through was a couple of SAP projects, where the data conversion stage had not been handled very well. The particular project manager who handled that project had not defined enough detail in the activities, and therefore, data conversion took longer, in other words the schedule was extended, and there was more cost expended than the budget. That happened three different times with projects until somebody said, “Wait a minute, you know, we’ve seen these three projects, each had a problem with data conversion. Is there anything we can learn that we can capture and pass onto future SAP project managers?” I think that’s where a lot of these lessons learned comes from as organizations have had some redo, remake, rework in some of their projects over time, and they finally recognize that they need to be taking some action which improves that process. In fact that leads me into the part I wanted to bring up, is the continuous improvement aspect of this.
The framework I’ve put together has a closed loop, where if you have a process like Project Management that produces a result and you capture lessons learned, what I like to do is make it an actionable lessons learned. By actionable, I mean have it documented in such a way that someone else in the organization could pick up that lesson learned, and if they were not knowledgeable about the process, they could implement it. It wouldn’t have to be the same project manager who developed that lessons learned. There’s a feedback process here that I think your audience can envision. Where you have a process like Project Management, you have a result, you have lessons learned, you have your feedback back to the original process, and you make an improvement to your Project Management process. That’s the thing I was teaching to the Panama Canal and also the groups in Dubai when I was over there, is looking at it from a continuous process improvement standpoint and understanding what actionable means. It goes back to having this little template that I talked about before. There’s a real simple template that has four sections to it. It’s what was the expected result, what was the actual result, what’s the gap between the two, and what is the lesson to be learned. If you fill that out in a judicious way so that you define how we didn’t achieve the expected result and what the details are, then you come out with an actionable lesson learned at the end that someone can take back and improve the process. I think that’s really what lessons learned are all about, is their activities or events that you can use to improve a process. I know you and I both are process thinkers, we’ve lived with process for years and years. I think that’s where a lot of your audience is headed, probably, is to be more in terms of process thinking and how can I improve this process.
Joe: I think we go right along today back to Dr. Deming. When I looked at your background material it reminded me just not of PDCA, it really reminded the similarities of Dr. Deming’s Profound Knowledge. Is that how you developed it? Are you a Deming guy?
Mel: No. I didn’t develop it from Deming; I developed it from having experience at working with groups. Lessons learned when you first talk about lessons learned in organizations, most organizations and most project managers looked upon lessons learned as a way to attach blame for what happened in a project. That was the first reaction that we got in the ConocoPhillips situation. Whenever we tried to say let’s capture lessons learned from this project, a lot of people said, “Wait a minute. I’m not sure I really want to cooperate because all it’s going to do is point fingers, and attach blame for something that wasn’t handled correctly.” It took a long time to get over that particular hump. The people who stepped forward and really wanted to implement this were the ones who became leaders in that PMO because they were the individual risk takers who saw that it was to their advantage to make these changes, to go forward and to put it out there for other people to capture and to share. The feedback loop that I put together was not necessarily a Deming loop. Although, it could be construed as having come from his practices, I think.
The important thing that I’ve taken from working with PMO’s, with individual project managers, is that every one of them wants to get better. They want to do things in a better way. They want to improve what they’re doing. They’re looking for all types of input from all types of people like you and me who have something that they can grasp onto and improve the outcomes of what they do every day. That’s where this continuous process improvement comes from. It can be almost from saying, “Hey, I have a personal improvement objective that I’m going to make sure that what I produce for this organization is first rate, it’s up-to-date, it has the latest thinking, and if there’s a way to improve it, I’m going to be receptive to that.”
Joe: The part that jumped out to me when we were looking at that was Dr. Deming’s Profound Knowledge and the part on psychology. You discussed something in terms of patterns of behavior.
Mel: Right.
Joe: How do patterns of behavior relate to Project Closeout?
Mel: Let me tell you where patterns of behavior fit in. Really, there are two parts to lessons learned. There’s, first of all, what I call the single project case where you’re trying to focus on just a single project, but there’s a whole other class of projects. Suppose you have five projects, and they’ve all been subject to what I call the same project environment. Project environment means you have the same structure; you have the same policies, procedures in place. People follow those procedures to carryout projects. One of the interesting things about looking at the project environment is you can find patterns of behavior among projects that are subject to that same project environment.
It goes back to the age-old question that we’ve all asked for many, many, many years. Does structure influence behavior. What that means is, do the policies and procedures I put in place for an organization that people abide by when they carry out these project management processes, do they have an impact on the outcomes in the way those project pings behave. The answer is, undecidedly, yes. There have been a lot of examples and a lot of work that’s been done in that particular area. That’s where, in fact, the most leverage comes from lessons learned. If you can discern the patterns of behavior among, say, five projects who are subject to that same project environment, you can back up one level and say, “if I make a change in the structure, or the policies, or procedures that people are following then I can leverage and make changes to the performance of those five projects.”
Let me give you a good example from my experience in the PMO. When we first put our PMO together, we had a project justification process that said every project had to go through a project plan, a schedule, an economic analysis, an examination by a committee before it was approved. Some of the people reacted to that and said, “Wait a minute. I’ve got such a simple project. It doesn’t involve that much money. Is there a distinction we can make between having to go through the entire business case, the economic case, and so forth, for those smaller projects?” As a PMO organization, we put into place a two-tiered structure for evaluation and justification of projects. The one tier said, every project, over say $5,000,000.00 had to go through the full justification process; a project plan, economic analysis committee approval. Any project that was less than 5,000,000 had to have just a project plan; it had to have signoff by the stakeholder, the sponsor who was going to be responsible for the delivery of that project. That $5,000,000.00 was sort of an arbitrary figure. The structure we put in place. Do you know what it did? It drove the behavior of some of the project managers to say, “Gosh, I’ve got this $8,000,000.00 project, but if I divided it into two $4,000,000.00 projects, I can get under the requirement to do this full analysis.” See what I’m saying?
Joe: I figured that one out as soon as you said it.
Mel: That’s a good example of where structure influences behavior. You have to be very careful of that in organizations. I’ve seen situations where you might have five projects, they’re all subject to that same project environment and every one of them has a similar behavior, but they also could be improved if you understood what it was about the policies that drove those project pings.
Another good example I can give you from my experience. Suppose you have a project that’s got a strong security content. You go to the group and infrastructure that’s responsible for security. You say, “OK. I want to keep my cost down on this project. So, I just want to borrow your security analyst whenever I need him on this project. So instead of having a dedicated resource that would cost me money for the project, I’m just going to borrow that security analyst whenever I need him for the project.” Now what happens? If you get in the middle of a project, all of a sudden, all these viruses attack the corporation. All the security people are off trying to solve those virus problems. They’re not available to help your project. Therefore, you don’t have the security people available at the right time to make a real impact. It costs you schedule time. It costs you increased cost time because you don’t have them available. It was a conscious decision on the part of the organization not to use dedicated resources to use these borrowed resources. That’s another example of where structure influences behavior. You might have five different projects that are subject to that same project environment. When you go back and look at each one of them, you found out that they exceeded budget, they exceeded the time to get the project done, and they were all for the same reason. That’s where patterns of behavior come into play here.
A lot of project managers, especially those that I see who go on to lead PMO’s, really need to be cognizant of the policies, structure and procedures they put in place for an organization. Oftentimes, they might drive unintended consequences. That’s where those people need to by real systems thinkers. They need to think in terms of, if I put this initiative out there, or if I put this new policy out there, what’s the behavior going to be like, what am I going to see coming back to me? That’s a step ahead from just lessons learned from a single project. That’s where patterns of behavior are important.
Joe: You really complicate things. I don’t think I can just go out and buy a piece of software and run my projects anymore, can I?
Mel: It depends on if you’re the person making that decision and making that purchase. If you’re going to make it unilaterally for your usage and your particular organization, you can do anything you want to. If you’re making that decision for a $25,000,000.00 – $30,000,000.00 organization that’s relying on that piece of software to carry them forward for the future work, you need to have some diligence, some standards to work by. That’s another area that I am finding a lot more people interested in, is a technology development. So many projects, now, have an aspect of technology development in them that you need to be able to manage that and direct that as you go to a project. That’s one reason I’m doing so much work on project risk management right now. I’m finding that if you conduct technology development in the right way, you can actually have a controllable risk and an uncontrollable risk. In other words, controllable risk is those aspects of development for which we know something about, and we have a reasonable appreciation that we’ll complete on time and within budget. We’ll be able to do those things. The uncontrollable aspect of that risk is things that we don’t know about, but which we should continue to learn about throughout the project.
Rather than having just an aspect of risk that’s over a project, if it’s technology development, is try to divide it into two pieces. What’s controllable; what do I know about that I can control and have some knowledge about. What’s uncontrollable, where I may need to bring special expertise in? These are all aspects of PMO’s that maybe some of your smaller organizations or smaller audience groups may not be so interested in. I’m sure, on a personal basis if they do a project, there’s always a certain amount of technology development, even if they decide what pencils are we going to use, or what mobile equipment are we going to use to communicate, are we going to use twitter for updates? It’s that kind of technology, today, that’s impacting projects that people don’t think of as technology development, but they’re just as impactful as having a Panama Canal project where you’ve got huge dam installations to produce power to move those new locks and those new facilities that are being put in for that third set of locks. That was an interesting experience for me. I learned so much about the fact that so much of the new Panama Canal expansion, that’s supposed to be finished in 2014, was really technology development for electrical, control systems, new lock gates, and so forth that they didn’t have in place before.
Joe: What knowledge are you supposed to capture? Do you capture everything? Is everything a lesson learned? How do you decide what you need to capture for the closeout?
Mel: That’s a good question. I’m writing this book which is going to be out shortly. In my blog, I talk about this process in capturing and documenting lessons learned. Really, what I’m defining there is what I call significant events. A significant event is something where you may have a major deviation between expected result and actual result. That would probably be easy for a project manager to sit down with his project team and list 50 things that ought to be improved. I say take the top 5 or 10 where you’ve had a significant deviation between expected and actual result. Focus on those 5 or 10, if you’re able to do that, you’ve got the old 80/20 rule. Eighty percent of the value to be gained is probably in the first five or six lessons learned significant events that you identify. The rest of them trail off very quickly, but there can be a lot of things.
If an organization has a robust risk management plan in place, they are 90 percent of the way toward having a lessons learned process. They’ve already defined some potential risks. If those risks get triggered in a project and they have a mitigation plan, they’ve already identified some significant events. The definition of risk and carrying that out for a project, having that risk identified, and if it’s triggered, if it’s significant and you’ve got a mitigation plan, that ought to go right at the top of the list of lessons learned. When you say how do you know, what do you need to capture all the knowledge? I’d say pick the top 5 or 10 lessons learned, something where you had a major deviation between expected and actual result.
Joe: My next question is how do you share that knowledge? How is that shared so that knowledge is made explicit to other people?
Mel: Different organizations have handled that in different ways. Certainly some of the larger organizations, we have, especially the large oil companies have put knowledge management systems into place. Even companies like Schlumberger, which is an oilfield service organization, they were one of the first companies to put together a knowledge management system where they actually go in and put it into a database. Depending on how sophisticated the organization is, I’ve even seen a couple instances in NASA where organizations have developed what they call a knowledge-based risk. In other words, they’re using their knowledge database to inform their risk managers of previous project problems before the initiate a new project. I think it’s up to, once again, the individual organization.
In the case of ConocoPhillips, when I worked there, one of the first things we tried to do was to have a breakfast forum among our project managers and invite some of the project managers who were just completing projects to talk about lessons learned. We videotaped those, and we put them into a SharePoint system, so anybody goes out and look at those. We promoted that to all our project managers, especially those who were going to initiate a new project that was similar to one that had just been completed. Today, we have a project server system out there from Microsoft project that incorporates a lot of that and allows capture of lessons learned.
Joe: You’ve mentioned your blog and some material in your blog before. Could you tell someone how to find your blog and what’s in it?
Mel: The blog I affectionately call, “The Mel Bost PMO Expert,” but if you follow those words, it’s just M-E-L-B-O-S-T-P-M-O Expert, www and then Mel Bost PMO Expert. That’s the easiest way to get to it. Since January of 2010, I’ve tried to write about as I said the structured development behavior activity performance of the PMO. A lot of times, I will see things in my own life that trigger me to want to write something that I think is applicable to PMO’s. Good examples, situations from patterns of behavior.
There’s a real good systems archetype out there that came from Wal-Mart and Proctor & Gamble many years ago. You may have heard this story; it’s called “Accidental Adversaries.” It’s where two organizations, which ought to be working closely together to achieve a common goal, focused too much on their individual things they want to get done; therefore, short-circuited that goal. I’ve seen that happen in PMO’s. We had a situation a few years ago when I was part of an IT project office. We were going through a merger, and we were told to use this outside consulting firm to put together our IT project office. We gave them the mandate, help us put together a methodology, but we don’t want it to be exactly what your outside methodology is we want something new. No matter how much we worked with them, everything they came out with for project methodology looked so much like what they had been employing with their other clients out there. That was a good example of accidental adversaries. It’s where two organizations ought to be working closely together to achieve a common goal, but each is focusing on their own particular aspect. They never get together as a common team. It’s things like that I’ve written about. I think your audience will find a lot of very interesting information and advice and some of my experience with the Panama Canal that they can use in their everyday project work.
Joe: Is that the best was to get a hold of you?
Mel: They can get in touch with me by email, mbost, m-b-o-s-t@botinternational.com. That’s another way they can contact me, m-b-o-s-t@b-o-t international.com. BOT International typically tries to work with projectmanager.com now, to offer some of those one-day courses. If your audience wants to go to our BOT International website, they can see a number of those courses in PMO Setup, Project Closeout and Lessons Learned and PPM, which is called Project and Portfolio Management. That’s where you have a portfolio of projects and how do I prioritize them, how do I classify them, how do I make sure the organization is doing all the projects they need to do to move forward. So that’s where I’m headed.
Lean Sales and Marketing: Learn about using CAP-Do
Special Marketing with Lean Book and Program offers on Facebook