Communication of Software Risks

One of the interesting challenges in managing a project is estimating technical risk.

Here is an example of a classic technology risk scenario:

The development team is building an online transaction system that sends orders from an e-commerce system to a warehouse. The e-commerce system is internal, the warehouse is external. The warehouse works on an AS400 system and no one on the team has ever worked with AS400 platforms before. There is a defined web services layer between the two systems which should hopefully hide the workings of the AS400 system behind the layer.

This is a pretty typical high level description of requirements from a manager, a request for proposal, or an email.

Most developers look at this and start analyzing all the things that could go wrong...there are dozens of risks just in this one paragraph.

A project manager might realize that there are some risks involved and if the PM has a technical background might be starting to ask some risk based questions that show some understanding of the potential problems. The worst case scenario is a non-technical PM who thinks that this project is the equivalent of ordering pizza from Domino's - you simply call and expect the order delivered by the guaranteed delivery window.

Let's assume that the entire team is on the ball, everyone's working together and the team comes together to review the project plan. A risk matrix gets created and the team comes up with the following risks:

Risk: Web Service might not work
Probability: Medium
Impact: High

Risk: AS400 might have performance problems
Probability: Medium
Impact: Medium

Risk: Transactions might not work across the two distributed systems.
Probably: High
Impact: Low/Medium

and so on....

This is the critical point where there is a massive communication gap between what a typical PM thinks a risk impact is and what a developer think of a risk impact.

A PM may think, "Well we have some risks so we should allocated some contingency in the budget. This is a high risk project, so we'll add in 30% contingency to our estimate and a month onto the time line."

A Developer may think: "If this server cannot move data fast enough, we're dead in the water. It won't matter how much time or money you throw at it because short of hauling out the entire AS400 out the warehouse we're toast."

Big difference in risk!

I've seen projects where there were mature team members and did all the traditional risk analysis that is recommended on software projects. But the project still failed because there were fundamental differences in assumptions in what "High Risk" actually meant to cost, time line and project success.

Here are some common reasons why there are differences in assumptions around risks.

1. Team members don't all have the same assumptions on impact of a risk. For example, a task is estimated at 3 days but the impact of the risk is "High". So what does that mean to the estimate? Add 10%? Add 50%? What about 10X? In my experience, project managers in particular do not understand the sheer magnitude of software risks - they'll see a risk as "High" and put in contingency of an additional 30% and think they're good to go.

A 10X risk impact is not unheard of - I've seen it myself on many projects. We had one project where one web service call was supposed to take 3 days and took 30 days because of technical complications.

2. There are some risks that result in project failure because in some cases requirements are not variable. For example, financial data cannot be more or less correct - its either right or its wrong. So getting to 80% right isn't really valuable - if you cannot hit the 100% mark then your software project fails. So if you have a risk that the data isn't going to be right then it isn't so much that your project went over budget or took too long to deliver - its a total failure because it cannot meet the basic requirements. Same thing with external software dependencies - for example we had a case where we were integrating with a web services framework and there was a bug in one of the supplied web services from the vendor. It wasn't a matter of "let's just find a work around" - the project was effectively dead in the water until that bug was solved.

Project managers tend to be solution oriented and optimistic, so while they like the idea of contingency, they tend to shy away from the concept that a risk might actually kill their project with no way to get around it.

3. The "I don't know" estimate will get quantified too early on in the process or not be listened to at all. If your team is saying, "I don't know" as a project manager, what is your approach? Many PMs are forced into providing estimates too early by management, so they simply fudge the "I don't know" into a contingency or pad the estimate. They think that by taking the developer's estimate and doubling it they've been given themselves a massive breathing space to worry about the problem later.

The cone of uncertainty on typical projects is a lot bigger than most PMs realize - Steve McConnell's work on estimation pegs the variance at 1600%!



If the team doesn't have a common understanding of how to quantify the impact of risks, then everyone can be communicating using a traditional risk process but the shared understanding of the impact of any risk will tend to be different.

Here are some recommendations for Project Managers for creating an improved risk communication culture:

1. Vary the risk based on expected uncertainty level and the phase of the project. For example, if you're in the planning phase the expected uncertainty level is high. If someone on the team says, "Well we don't really know how that's going to work but we have a plan" that's probably OK. If you hear the same thing in the middle of QA, then you should be very concerned and adjust accordingly.

2. Have the team translated risks from labels ("High", "Medium", "Low") to numbers where possible. In addition, don't assume that one set of labels means the same across the project. For example, a "High Risk" HTML issue might be something very different than a "High Risk" web services integration issue or performance issue.

3. Identify the requirements that are absolute and unmovable and pay particular attention to risks that are around them. For example, in some cases performance is an absolute requirement (e.g. it must serve 30 pages per second) and sometimes it isn't (it must be "fast"). If your requirements are absolute then any risk that hits those requirements will not just cause a delay, an increase in cost, etc. - it will kill your project.

4. Pay close attention to areas of the project where there is limited flexibility or options. Compare the flexibility of a creative front end vs. a bought software system. If someone doesn't like the the front-end, you can rebuild it, change it, etc. It might cost more, create a time impact, etc. but you've got lots of flexibility for change. If you buy Microsoft Content Management Server and it has a bug in it your options are pretty limited.

5. "I don't know" must be a viable option in any discussion. In addition, the response to "I don't know" should be to go find out - be careful about arbitrary padding of estimates simply to factor it into your plan.

6. Pay close attention to body language, talent and historical experience. Relationships matter in software teams and a team that has worked together before has the advantage that the PM starts to recognize the subtleties of individual communication styles and to understand how each person reacts to different questions or risks. Is John an optimist or a pessimist? When someone says, "I don't know" are they being cagey or honest? Teams who have worked together start to intuitive find out these non-verbal cues.

7. Review risks on a regular basis so that you can refine your strategies to mitigate them.

Hopefully that helps you in creating a more pro-active risk based approach and to be aware that just because people the team are calling somet "high risk" they may not actually mean the same thing.

Developer Weasel Words

As a development manager or project manager, you here a lot of weasel words and excuses from your staff or from external consultants who are trying to hose you into believing things are better than they are. In many cases, developers use these phrases to even convince themselves that things are better than they are, resulting in chronic late delivery and poor quality.

So beware the following phrases from your teams:

• It should work: this usually means that it doesn't. It also means that it was probably not tested properly as the result is current undetermined. The word "should" should be taken out every developer's vocabulary - it either does or it doesn't.
• I just need: beware of that word "just". Its a belittling word meant to make things smaller than they are. Just take the phrase "I just need to write this component" to be "I need to write this component" and already the magnitude of the work involved grows. Developers tend to be chronic under-estimators and the use of the word "just" is a sign of that mentality.
• Almost done: this is also a weasel word. When a developer tells you things are "almost done" ask for the specific tasks that are left over immediately. In addition, keep in mind that projects do not progress linearly - the last 10% is always about 40-50% of the work of the total project. I've seen projects that are chronically late be "almost done" for 3 months.
• It was tested: this also usually means it that wasn't tested properly. Ask for a test plan and the specific tests that were done. If the developer cannot produce these with sufficient evidence that PROVES that it was tested, then it wasn't tested.
• It must be an environment/configuration/deployment problem: this may be actually true, but it usually points to a larger stability problem. If you cannot build and deploy reliably then why would you have confidence that the code works?
• If things go smoothly: this I hear a lot, e.g. if we don't hit any snags then we can be done by Friday. Guess what - you're likelihood of hitting a "snag" by next Friday is probably high and given the lack of risk based management, the team has probably got no mitigation or contingency strategy. Then next week, you'll hear the next phrase in our list, "Yes, it could have been done if it weren't for that Snag we had".
• Yes, but, as in "Yes it can be done, but": this means it cannot be done. Tell your staff to just come clean and say, "No it cannot be done". Another variant on this is, "Yes it could have been done, if it weren't for marketing, requirements, technical risk, etc." This simply shows that your developers work in an idealistic world where things never go wrong.
• We'll make up the time at the end: if you're already late by the end of requirements, you're likely going to be even later by the end if you simply keep going on the same track. In my experience, teams don't dramatically faster as they hit their stride. Even if there is some efficiency, its nowhere enough to make up for lost time.
• I've got it done, but I need to build a few more components for you to be able to see it: then its not done! Encourage show and tells, code reviews, unit tests, etc. so that code is visible as soon as possible. Use slicing models so that you can see pieces of the application in weeks, not months. In addition, code that isn't checked in should never be counted - that means that someone cannot build it sufficiently to share it. It only counts if you can verify it. Ideally, its not "done" unless there are sufficient unit tests, a build script, and a document that someone walking into the code repository could check out the project run a build and have all the unit tests pass. Then the code is done - anything less is mythical.
• I've got it done - I just need to integrate it: The word "Integrate" is a big weasel word. Think of a web service that adds two numbers together. The algorithm is one line of code. But the integration work is huge. In addition, integration usually means the first time that disparate teams are bringing code together which is always cause for issues. Don't under-estimate the integration, especially in today's world of distributing computing, web services, etc.
• It Worked on my Machine!: programmers use this excuse to downplay a bug. The reality is actually the opposite - it means that you have an intermittent bug which is by far the worst kind of bug to have in your application. You want bugs to fail quickly and consistently - any variant such as "That's Weird", "That didn't happen yesterday", "That must be a data problem", etc. is admitting you have a bug that cannot be easily duplicated.

My recommendations to reduce the amount of excuse making from your team:

• Encourage a culture of honesty and team work: You get these excuses when developers are hiding things, and sometimes this is because you've created a culture that encourages hiding because you don't want an honest answer.
• Be ruthless with your quality and talent standards: don't excuse poor talent, bad management or chronic late delivery. If you create a culture where talent isn't rewarded the bar isn't kept high then you'll be excusing the team to continually strive for excellence.
• Expect more than just code: measure performance based on estimation, quality, delivery and team work as well as pure code quality. If you have a developer who produces great code but cannot deliver on time then that's not a great developer.
• Increase visibility and shrink delivery cycles: if you have to show your work on a constant basis and deliver on 2 week iteration cycles, your excuses tend to go away. You either deliver or you get found out pretty quickly. Use show and tells, code reviews and continuous integration to see what people are doing on a constant basis.
• Don't give half credit for 50% done - its either done or not done: If your tasks cannot be managed this way, then you should split up your tasks until you can work this way.
• Establish what "Done" really means: for example, at a minimum "Done" should mean checked into source code control and able to build into the current branch. If you're doing Test Driven Development, it should also mean all tests run successfully. If you have specific performance criteria, then its not "Done" until the performance tests pass.
• Use counting techniques to measure wherever possible: this is a great suggestion from McConnell's book on estimation. The more you can count in units, the more accurate you estimation. So if you can count the number of pages, web services, objects, databases, tables, stored procedures, tasks, etc. that are left to accomplish then you can measure them more easily than if its a big blob of work. If your requirements aren't well defined enough to count objects, e.g. you don't know how many web pages you're building in your web site, then you're really not in a position to estimate your ship date.
• Don't sucker, manipulate or bully your team: If as a project manager, you resort to traditional management tactics such as playing games, being political, establishing a blame culture, or bullying your team you'll lose your credibility and simply encourage lying. A tortured prisoner will tell you anything you want to hear - the same goes with development teams.

If you have a project that operates in the open, has a culture of honesty and establishes a high performance bar, you'll find that peer pressure as well as some overall guidance will get risks, problems and bugs out in the open. If when you discover these problems people work as a team to fix them instead of blaming each other then every problem solved becomes a victory and not a blame opportunity. You'll get better answers and improved morale on the team as you set clearer performance expectations.

How to be an Effective Stakeholder - Part 2

The Role of the Stakeholder
Part 1 - Introduction

Stakeholders who are effective on projects understand their valuable role. This post articulates what effective Stakeholders bring to the table.

There are a few different types of Stakeholders:

• Direct Customers: if you're lucky, you get end-users as stakeholders. These are great to have because you know that these users represent actual end users who will use the application on a daily basis.
• Domain Owners: some stakeholders are involved in software projects because of their domain expertise. Software developers are good at building software, but they're usually not domain experts as well (although some will claim to be). So if you're building a financial application, its important to have someone on the project who knows the minute rules on accounting. Not only is it important to have general expertise, but it's equally important to have domain experts who know the specifics of the business. There are dozens if not hundreds of hidden business rules in any business and in many cases they are only learned through oral tradition. Domain owners provide this expertise.
• Customer Group Representatives: if you cannot get direct customers or if your user base is large, then you need to have Stakeholders who can represent the needs of the customer. Be careful about this in that it's important that the Stakeholder is an accurate voice for the customer and doesn't have their own agenda.
• Business Representatives: general Stakeholders that represent the actual business. If the business is selling soft drinks, then it's a good idea to have an understanding of how the actual business works and how your application will fit into and support the business of selling soft drinks. In any business, there are also specific experts who own global business areas such as marketing, security, privacy, branding, and customer service. These horizontal service areas are not specific to any vertical project but cross all projects being delivered.

Regardless of the type of Stakeholder, here is what I'm looking for them:

• Decision Making: a Stakeholder's primary responsibility is to make a decision. Should we charge the credit card before we ship the product or after? Should the name field be 100 characters long? Should the colour of the button be purple or blue? There are hundreds of decisions that a software team must make and your role as a Stakeholder is to make the call when asked.
• Approvals: similar to Decision Making, your job as a Stakeholder is to approve documents, decisions, milestones, etc. When I was a Stakeholder on a project with Accenture, they would send me test cases to review. I would simply reply with a single word, "Approved." Especially with external vendors, teams need a formal mechanism for sign-off before they can move forward.
• Opinion and Feedback: in building web sites in particular, there are lots of subjective decisions on what a web site should look like, how it feels, whether it represents the brand, etc. As a stakeholder, your primary job is to have an opinion that can be translated into actions, revisions and improvements. An example of a poor opinion: I saw a graphic design present a screen design to the Creative Director. His opinion was, "I want exactly what you have, but just bigger and bolder!" How is someone supposed to action that opinion? A good opinion is, "I hate that text - its boring and its too long." It's just as subjective but at least you can action it.
• Solicitation of Feedback Amongst the Broader Community: Most Stakeholders represent their department, a user community or a business unit. Key deliverables should be brought back to this broader community for feedback. As a Stakeholder, your job is to help explain, sell the concepts being presented and capture feedback that can be actioned by the team.
• Be Supportive Externally and Demanding Internally: Stakeholders should be evangelists for the team as it helps boost confidence for the team, helps provide buy-in from the great organization, etc. Internally, Stakeholders should be demanding like a good customer. Stakeholders should be keeping the bar high and making sure that the software team feels that they have to meet a high bar for quality and performance.

What frustrates software teams to no end is when Stakeholders are not doing their job. Here are some examples of where Stakeholders drop the ball:

• Showing up to Meetings Unprepared: if you cannot bother to take 15 minutes to read through a technical document that someone spent 2 weeks writing, what does this say about your level of respect for the team?
• Being Late with Approvals or Feedback: when I manage projects, I put review and feedback periods in the project plan. Stakeholders have deadlines for turn around of feedback. Yet time and time again, I've had Stakeholders miss their deadlines with really lame excuses. If you miss your deadlines for reviewing a document, what kind of commitment can you expect for higher risk activities such as building new algorithms or leveraging the latest technologies?
• Abusing the Team: this symptom seems particularly bad with external teams being managed by internal Stakeholders. There must be something in management training that tells middle managers to treat external vendors abusively in order to squeeze them for another 5-10% in performance. Unfortunately for many smaller and less mature vendors, the practice works which only reinforces the practice. However, in my experience a mature software team won't take abuse and understand the difference between having high expectations and simple bullying.
• Poor requirements: if you're supposed to be a Domain Expert, then you need to be able to know your domain. Ideally, your domain is well articulated through existing documents, manuals or practices. If you're supposed to be providing requirements to the software team, then these requirements should be well defined and articulated. Here is an example of poor requirements: we were supposed to build an online store for a client, but no one had given any thought to basic requirements such as tax rules, catalogue structure, content requirements, fulfillment, inventory rules, etc. When we asked basic requirements such as, "What are you current inventory rules?" the answer we got back was, "Well, we don't really have any rules - what do you suggest?". Argh!
• Us and Them: I would highly recommend for Stakeholders to embed themselves into the team as much as possible. The more that Stakeholders are part of the team, the better the communication. If you find yourself viewing yourself as the "customer" and not part of the team, figure out how to get yourself better integrated. Take the team for drinks, start coming to meetings, and support the team by doing your job. When Stakeholders build credibility with the delivery team, it means that when they actually need to put some pressure on them to deliver they'll have credibility to spend instead of simply crying wolf.

I've been on projects as a Stakeholder, a project manager, a technical architect and a sponsor. When the Stakeholder brings their value and integrates highly into the team, it makes a massive difference in the success of a project. If the Stakeholder is flying in and out, using politics and manipulation to get what they want, or being abusive in order to squeeze the team, it tends to put the entire project at risk. As a Stakeholder, I feel responsible for the entire success of the project and I'm not just there to provide input and walk away from the scene. In addition, I don't view software projects as ordering McDonald's whereby you simply throw in your requirements and drive up to the pick-up window to get your order!

Problem with Shoutwire Tag

I've posted bugs previously relating to the little Digg and Shoutwire tags that allow you to vote. The generated CSS tends to be quite buggy and these tags don't play nice together, e.g. having a Digg and a Shoutwire button next to each other is impossible.

I discovered yesterday that having the Shoutwire Vote for Me tag breaks my blog template completely now. I'm not sure why it does, but having the Shoutwire TAG means the rest of the page gets mangled with pictures not showing properly, ad words not showing up, etc. Taking it out instantly restores the blog template to its proper format, so it must the generated HTML. It fails in both IE 7 and Firefox.

I've gone through my blog postings and stripped out all the Shoutwire buttons as a result. You can still go to Shoutwire.com and if you see a posting of mine there you can vote for it on their main site.

How to be an Effective "Stake Holder"

Part 1- Introduction

Every software project has "Stakeholders". There are other terms used for this role such as "Internal Customer", "Steering Committee Member", "Department Head", "Boss", "Sponsor", "Champion", etc. Sometimes stakeholders are formally identified (usually by assigning them to a project directly) but there are also lots of informal stakeholders as well.

While the context for these articles is software development projects, in my experience being an effective Stakeholder is consistent across all projects and the same guidelines apply consistently for other projects as well, e.g. marketing projects, facilities projects, etc. As a Director of Information Technology, I play the role of Stakeholder on some projects and been on the delivery side on other projects. I've worked on projects that are internally delivered as well as worked as an external consultant. So I have seen the Stakeholder role from a number of different angles.

This introductory post gives you an introduction to the role of the Stakeholder and what is expected from you if you have been assigned as one. In future posts, I'll provide some specific techniques, guidelines and suggestions for managing and influencing project teams for Stakeholders.

Your Boss Comes to You with a New Project...

Here are some really bad but unfortunately typical ways that Stakeholders get assigned:

• Your boss has been assigned the role and doesn't want to do it. So he or she delegates it to you. You can continue to delegate down if you choose to do so. I've actually had this happen to the point where the original stakeholder was the VP of Marketing and by the time the project had started a junior copy editor showed up saying, "I was told that I'm to represent marketing on this project"!
• You're in a role of authority and someone on the project thinks you should be consulted. For example, you're the Director of Marketing and someone in HR decides that they want to change the web site to have job postings. You get roped in because you've issued the policy that no one changes the web site without your input. You're now a Stakeholder!
• The project is half way through and someone realizes they need your input. This happens to Operations teams a lot - 80% of the way through development, someone on the team realizes that they need 10 more servers! As the person owning this non-budgeted purchase you're now a Stakeholder!
• You're asked to join or are already part of a committee that regularly meets to discuss general issues. For example, many organizations have branding committees, privacy committees, project review committees, etc. Sometimes projects will make the entire committee a Stakeholder. I had this experience once working on a project building a government web site - there was a permanent "web committee" with 7 members on it who had to meet on every issue and come to consensus on every change. The entire committee was a Stakeholder.

If you find yourself getting labelled as a Stakeholder in any of these scenarios, beware! They are high risk and put increased risk on the project.

Recommendations

Here are some recommendations on when to sign-up to be a Stakeholder.

• Be empowered and qualified to be a stakeholder. The primary role of a Stakeholder is to make decisions and provide input. If the project is expecting a VP level Stakeholder and they get the VP's secretary, they're not getting a stakeholder who is qualified or empowered to make decisions.
• Avoid Stakeholder proxies. Stakeholder proxies are people who show up to all the meetings, collect a lot of notes but then cannot make decisions on their own. If the answer to questions asked of you is typically, "Well I need to ask my boss whether we're allowed to do that" then tell your boss that the project really needs the boss to be the Stakeholder, not you.
• Be prepared to commit a minimum of 2-3 hours a week to the project. If you cannot commit to the time, then having you in the loop just creates additional communications overhead for the project team.
• If you are primary stakeholder or if you are providing extensive documentation, business expertise, or testing support then expect this time commitment to be considerably higher. For example, we had a project to deliver a new B2B order entry system. One of our stakeholders was the person who ran the B2B business. During requirements gathering, she was involved in the project about 20-30 hours a week. During development, she was involved about 10 hours a week to clarify requirements, approve changes, etc. During testing, she was involved 40-50 hours a week to help test key functionality, provide clarification on missing requirements, facilitate testing with other people in her department, etc.
• Stakeholders should be established up front and given ample warning of expectations. You should know you're on a project from the beginning even if you won't be needed until further down the line. In addition, you should have a context for the entire project and not just be isolated to your area.
• Beware of people randomly assigning you as a Stakeholder through conversations in the hall, informal emails, etc. It may come back at you if the project turns south.
• The inverse of the above is don't assign yourself as a Stakeholder simply because you're in a position of power. Simply inserting yourself into a project is not an effective way to be a Stakeholder. In addition, don't take credit for projects where you just went to a single meeting - that's not being a Stakeholder!
• If you work in a traditional departmental structure, switching to a project based model will require radical changes in how department heads interact with project teams. Many Directors assume that if a single resource of their department is on any project, they automatically get to have a say in every project decision. Even worse, some Directors simply reserve the right to sit on the sidelines and snipe at the project teams at any time at random. You'll see this in emails coming from Directors that start with, "I noticed this project is trying to use MY resource (equipment, people, brains, etc.) and before I ALLOW YOU to do so, I must REVIEW AND APPROVE this project". Don't assume that just because you're a Director you're automatically a Stakeholder on every project!
• Avoid projects where there are many Stakeholders, especially if there is no Stakeholder hierarchy or specific areas of ownership. Here is a good sign of too many stakeholders - if there are more stakeholders than project team members!
• Try to be on projects where you have a personal interest in their success. This creates investment, accountability and passion. Part of your role as a Stakeholder is to be a champion, so having a personal stake in the success of the project makes a huge difference for you and the project team. If your career is on the line because of a project, do you think you're going to be a great Stakeholder and do what it takes to be responsive? Lesson to executives - make your Directors bonuses tied to project targets, goals and successes and not just the running of the department.

Depending on where you work, you will more or less control over the projects in which you are responsible. However, all this means is that the techniques you use are more or less political There are lots of ways to get on and off a project - some of them explicit and some of them implicit (More on this in future posts). In addition, if you have no choice but to be the Stakeholder for a project then there are things you can do to improve your contribution to the project and ensure a greater chance of success.

Stay tuned for future posts for more recommendations on how to be a better Stakeholder...

Why Agile Software Projects are Good for Customers

We have two software projects that are going on at the Foundation. Project #1 is just finishing. It was not managed by me and it was done using a traditional waterfall death march type methodology. Project #2 is being managed by me and we're using a traditional Agile project management approach.

Comparing the two projects really shows the different in the benefits and obligations of the customer in an agile project:

• Project #1 is 100% over original estimate for no apparent reason. The customer gave up trying to even get a date from because they haven't seen any deliverables, don't even really understand the scope of work and just want it done.
• In Project #1, there was about 1 month just spent trying to define requirements. There were dozens of meetings trying to nail down field mappings with no code being written in the meantime (which starts to explain why the project is late). The right people were never in the room and the developers got different answers from different people.
• In Project #1, despite the investment in time, the output in terms of requirements documentation and certainty of scope was quite poor. Despite the investment, the requirements still changed up until the last week before testing including major business rule corrections.
• In Project #2, we've taken a traditional Agile PM approach. We include the customer in every meeting (usually once a week) and we are using a combination of use cases and wire frames to document requirements. This is the first time in the organization that the customers have seen either format so its a bit of a learning curve in terms of how to read them. But it took about an hour of reading through use cases before the customer fundamentally got the value of such a document.
• In Project #2, I got my lead technical architect to start prototyping on day one. By week 2 of the project, we already have a prototype writing in .NET using SQL Server 2005. The data model is about 50% done. But before even the documentation is done we have a working model.
• In Project #2 for project management we've taken an Iteration Cycle of 2 weeks as the basis for development. We have an architecture/planning/requirements spike for the first 2-3 weeks but have already started coding. Every Iteration has a plan to deliver a list of use case implementations and related wire frames as fully functional code. We have formal show and tell sessions at the end of each cycle.
• In Project #2, we have a proper project plan with real estimation. Estimation is done at the wire frame and use case level and tasks are broken down to about 1-3 days in duration. In my experience anything more granular is a waste of time and anything more becomes unpredictable and subject to error.
• In Project #2, we will be using Test Driven Development and Customer Testing all the way through the project as well as Continuous Integration.

The experience so far from a customer perspective has been really interesting so far:

• Customer participation on Project #2 has been extremely high and involved. The customer role has been really well defined and our stakeholders have been good at providing the right level of input.
• There is a good trust relationship and a sense of team - it isn't a US vs. Them approach. We're not delivering software for a customer - we're all as a team delivering on a project.
• Our ability to shift priorities and change the schedule is highly responsive. For example, our marketing department is working with the team to figure out the content for the application and its taking a lot more time than expected. So we've started on the back end logistics sections and left the front-end until the end in order to give the marketing folks the most leeway. By having check ins every two weeks we can easily adjust.
• In addition, because the application is built in vertical slices instead of horizontal ones, we don't have to have 100% of one tier done before moving to the next one. We can simply move from front end to back and hook together some disconnected pieces as they come available.
• The project is simply more fun and more of a team based approach.

Some recommendations to software project teams in treatment of their customers (whether internal or external) in an agile context:

• Don't patronize your customer because they've never done software projects before. Don't treat them like idiots because they've never read use cases or UML diagrams or database models before. If you take an hour to introduce the concepts and travel light, you'll find that customers "get it" rather quickly and then see the value of these requirements tools.
• Always remember that the customer is the end user and not a stakeholder representing an end user. For example, many Directors of Marketing act as the "customer" in web project teams. They're not. They're representing a customer. Everyone on the team needs to always focus on the end customer and stakeholders always have to be able to filter out their own biases and represent the end user successfully.
• Customers will buy into an agile project management approach if it produces results - period. If you cannot deliver software on time or its poor quality, an Agile approach won't save you and will quickly degenerate. Shorter iteration cycles means there is no where to hide, so talent/experience and ability to deliver are critical early success factors.
• Show and Tells are critical to the Agile Project. We use to have them once a week on Fridays. Make Show and Tells Fun! We use to dim the lights and use the projector and have popcorn and snacks. Even if its some boring application, hype it up and celebrate it.
• Customers should be the leading evangelists as well - some customers have been taught to be the evil task masters and try to crush development teams because they think it gets the team to work harder. I can tell you from the other side of the team that it doesn't work - developers work less, produce poorer quality and flip the bozo bit on you in short order.
• One of they values of an Agile project team is transparency and team work. Transparency is a double edged sword - customers have to be prepared to hear the bad news with the good and to help the team prioritize effectively. Good customers develop a relationship with the team that allows them to be trusted - its critical in order to encourage honest dialogue.

Overall, so far Project #2 is doing much better than Project #1. It has lots of risks and we may not be able to deliver all the functions asked for in the time line. But the requirements are well understood, communication is open and use cases are prioritized. We've already had conversations about what parts of the scope could drop if needed and contingency plans if we go late. Customers are in the loop at all times and a core member of the team.

Agile Just Feels Natural

Have done agile development for most of my career, I find it interesting when I first introduce the concepts to new developers. We're just starting to introduce it to people in my department, and we have got started with reading through the usual standard books by Ambler, Fowler, Beck, et al and I've start acting as the Agile PM on some key software projects.

What I find interesting is when I introduce it to developers, the universal reaction seems to be the following:

1. What a breath of fresh air - this is how we should have doing software all along.
2. I've already been using these techniques but didn't have a methodology to explain it and wasn't rewarded for it. I find that developers who have been raised in the classical SDLC have secretly rebelled against it and agile methodologies simply validate the rebellion.
3. The biggest flaw as most software developers experience with the traditional waterfall model (or worse the traditional death march just get it done by a fixed deadline approach) is that they now have some confidence in their work and ability to deliver against timelines.
4. When I find developers revert back to their old bad habits, they almost always run back into trouble and then slap themselves in the head of going back to hiding their risks, not being iterative and letting projects slip into death marches.

See my next post on why Agile is Great for Customers...

How to Use Microsoft Project Efficiently

I've worked with a lot of project managers who don't really know how to use MS Project efficiently. I was trained well on MS Project because in the consulting business, you have to be able to build million dollar project plans in days for sales pitches. Our PMO figured out a number of ways of using Microsoft Project more efficiently.

Here are my recommendations:

1. Get rid of the GANTT chart by dragging it to the right. Use the extra space for some additional columns.

2. Go to Window and click on Split. This will bring up the Task Form dialogue box at the bottom. You will find that entering data in the task form is much more efficient and visible than any other method.

3. Go to Tools->Options and click on Schedule. Change the Default Task Type to Fixed Duration and click off the box marked "New Tasks are Effort Driven". The reason is that when you're estimating, you'll find in most cases its easier to think in duration and allocating a % of a persons time across that duration than thinking about person hours. For example, instead of thinking of a task taking 25 hours, its easier to think about a task taking 3 days at 100%. Its also easier to think about a task as taking 2 people x 3 days, especially if you're days are 7.5 hours (do you think of a task as taking 50% of a week or 18.75 hours?)

4. Don't put in any absolute dates in your plans. Make all your dates based on dependencies except for the basic project start date.

5. For tasks that you want to go in parallel, use the SS predecessor type to make sure that multiple tasks start at the same time.

6. Go to Format->Text Styles and click on Item to Change. Select Critical Path and change the colour to something obvious (I use BOLD RED). This will help you in making sure that every task has the appropriate predecessor set - if it doesn't, you'll notice that the critical path stops half way through your project plan. It should highlight tasks from the beginning to the very end of your project.

7. Lets assume you have a project plan that is way past your target timeline and you want to now parallel your effort by adding more people. Many project managers will start creating streams of tasks in parallel with complex dependencies. But here is a simpler way if all you want to know is how many people you need to complete the work on time - simply take your tasks, change them to fixed work and increase the percentage to 400%, 500%, etc. until you get your timeline in the ball park. If its 500%, that means you need 5 full time developers to get the work done.

NOTE: the one draw back to this approach is that it doesn't tell you when each task will be done. But in my experience, this is not that useful as development projects tend to have 50 tasks that can largely be done at any time over a longer period and you simply spread the load across a pool of developers until the work is done. If there are specific tasks that are dependent, then set them up as a seperate stream. But if its simply a pool of tasks that need to get done this technique is much faster and simpler to manage.

8. Don't allocate overhead resources to tasks (e.g. your PM, your TA, your QA lead, your stakeholders, etc.) unless you have very specific requirements. In most cases, you can estimate an overhead resource at a % allocation across an entire project phase. For example, I'll allocate a PM at 30% for the duration of the project, a TA for 20%, etc. This again is a bit of a cheat - if you want to plan that a PM is going to be busy at 100% on Wednesday and 20% on Thursday and 40% on Friday, then this technique fails. In my experience, this is not the way overhead resources work and in the end having your project plan reflect this micro-detail is not worth the work to maintain it.

9. Here are a few ways to calculate contingency:

• Click on Insert Column and pick Cost1. Right click on it and select customize field. Then click on Formula and type in Cost*1.10. This will calculate contingency at 10%. If you want 25% contingency then use Cost1*1.25. Make sure that Calculation for Task and Summary Rows has Rollup selected and Sum.
  
  NOTE: There is a bug in Project 2003 where it sometimes does recalculate fields properly. If you notice this, then go to Tools->Options->Calculation and click Calculate Now. This will recalculate the field.
  
  


• Create a task called Contingency and set a price for it. This is good for setting an arbitrary contingency amount.


• Create a task called Contingency and assign a resource called Contingency at sufficient hours to drive up the price. The advantage of this approach over the one above is that it means your total hours assigned also has contingency built it and your hourly rate stays consistent with what it would be without the contingency assigned.


• Create a task called Schedule Contingency and set the duration to an amount of time with no resources attached. Bury the task where you think the schedule risk is going to occur - I usually put it into the QA phase of the project as this is the mostly likely area where you'll need the contingency.

10. Save often. I've had too many cases where Project crashes and there is no automatic recovery. In addition, there is only one level of Undo in Project 2003 (this is fixed in 2007) so if you make some significant changes its tough to revert them.

11. Use the keyboard as much as possible. Learn keyboard shortcuts. Don't drag things around when you can copy and paste. Don't use your mouse to indent when you can click on Alt-Shift-Right Arrow. Typing in predecessors manually instead of clicking on the link task button in the toolbar generally is faster.

Hopefully, that will help speed up your MS Project work dramatically. If you work this way, you'll find that planning with MS Project is feasible. If you're doing it the traditional way by dragging grantt bars around you'll find a major speed improvement working this way.

Low Tech Approaches to Requirements Gathering - Part 2

For those who missed Part 1, I wrote on how to use low tech tools such as paper, Post-IT Notes, whiteboards, etc. to gather requirements.

In this post, I'll describe how to better use tools such as Office, Project, Visio, etc. to gather requirements.

Here are my recommendations about approaching tools such as these in an agile context:

1. Use "Just Enough" tools. Don't use Rational Rose for UML diagramming if Visio will do quite nicely.

2. Buy the absolutely best tools for the job. In theory, you can write .NET applications using Notepad and command lines tools and save a few bucks on licenses. But the productivity will be poor and learning curve will be worse. Don't allow your organization to scrimp on tool costs - in my experience if its the right tool for job, paying the $500-1000 for a particular tool is worth the cost.

3. Invest in tools that are standardized. If you're working in Java, use Eclipse. If you're working in .NET, using Visual Studio. If you're an Oracle DBA, get Toad in your office. If you want a source control tool, get VSS, Subversion or CVS and not Seapine. That way when you go hiring for new developers, they've probably used your tools before.

4. Get really good at using the tools. It sounds like a basic suggestion, but I don't know how many project managers I've worked with don't really understand Microsoft Project. If you set up project in particular ways such as exposing the dependency columns, splitting the window to have the resource allocation on the bottom instead on the side, getting rid of the GANTT chart altogether, etc. productivity goes up considerably. Same thing with Word - the first thing I do with any document sent to me is look at how many Styles are being used and typically I find hundreds of them. Whomever wrote the document didn't understand how styles work, why they improve productivity, etc.

5. Buy tools that are easy to learn. Again, another dumb suggestion but I look for tools that a junior developer or programmer can understand. In many cases, this means buying more tools as the simplest ones can be single purpose. I've rarely come across a monster tool that works as a really good Swiss Army Knife for all occasions. The profiler I use though is stupid simple - its from RedGate and you load it up, point it at your project and start profiling in about 30 seconds. Its dumb simple to learn and cheap as well.

6. Have everyone have access to all the tools and have everyone knowledgeable about them. For example, I've worked in organization where only the Senior Project Managers were "allowed" to have Microsoft Project. If you weren't a Senior PM, then you had to either use Excel or find a Senior Project Manager to help you build a project plan. Technical resources should be able to open up a project plan as well - if you want your team to be accountable for deliverables, telling them that they're not important enough to see the plan sends the wrong message!

7. Automate work as much as possible. Use Continuous Integration, Test Driven Development, build scripts, O/R mapping layers, code generators, documentation generators, etc. where appropriate to automate drudgery work. Invest in time sheet systems, collaboration spaces, source code control systems, etc. where appropriate instead of tracking things manually.

8. Remember that every document, artifact and tool costs money to buy and to maintain. Every software license has maintenance costs. Every document takes up space on a file server. Artifacts have to be managed, secured, backed-up, etc. So travel light as product just enough documentation. Use just enough tools to do the job which also usually means use the ones that are the most productive. Remember that emails are documents as well - if you're finding that you're getting hundreds of emails a day, figure out how to get your team to be more concise, more action oriented and spend the majority of their time producing deliverables instead of documents!

9. Get the fastest computers you can afford. Get two monitors minimum for every developer. Get a good video card. If you have to wait 20 minutes to do a build, trying to do agile development is going to be tough. If paying an extra $500-1000 will cut that build time to 5 minutes then I would pay the money as it will result in massive productivity improvements on a daily basis.

Low Tech Approaches to Requirements Gathering: Part 1

One of the tenents of agile development is just enough documentation. I would recommend adding another suggestion: just enough tooling.

The boss comes to you and says, we need a project plan for this large software project. Budget is about $1 million and I need it done in 6 months. Figure out what is required, how many resources you need and what kind of scope can be delivered.

So how many of you have already reached for Microsoft Project?

Forget it for now - come back to it after you've tried some more low tech approaches. Here are some suggestions in order of really low to medium tech approach:

1. Talking: before you start visualizing, diagraming, or white boarding, start talking! Some people talk and diagram at the same time (I do) but I've been to too many meetings where an architect gets up to the front of the room and the team sits there for 30 minutes waiting for the person to finish his fancy whiteboard picture.

2. Piece of paper: I love white boards, but they're hard to carry to a bar, a coffee shop, an airplane, etc. So keep a sketchpad, a napkin, etc. and use it as your white board.



3. White board: the number one tool for an agile PM. If you want to record your white board, take a digital picture of it!

Note: Make your white boards portable. Get the white boards that are on wheels or make them small enough that you can move them from desk to desk. We used to have our release checklist on white boards and it worked great - we could leave it up permanently (without the PLO in a meeting room) and then we would erase things off the board as they got done. This also stops the need for dedicated planning rooms - simply bring the tools with you.



4. Big sheets of paper and Post-IT Notes: Ever done a web site map with a big sheet of paper, some markers and post-it notes? Try it and you'll never go back to using Visio to do site maps again.

Note: If you want to be really advanced, use coloured Post-IT Notes. Get Red, Yellow, Green and you can use them to connote priority, level of difficulty, etc.

5. Paper templates: if you find yourself redrawing, create some templates in word and then print them off. For example, don't draw yourself a calendar each time - simply print off one in outlook and then start writing on it.

All of these techniques mean no use of a computer (except for create a couple templates). You can use these techniques in any environment that has access to an Office Depot.

Low tech approaches to planning, requirements gathering, design, etc. have some specific benefits:

1. No license costs: while this isn't such an issue with Project, if your development tool standard is Rational Rose, Together Soft or some other expensive IDE where every user must fork out thousands to build a UML diagram, a project plan or a document then using lower tech approaches will have dramatic cost savings.

2. Encourage collaboration: if you get people to talk, to collaborate and to physically draw, use construction paper, or write the collaboration happens naturally.

3. Take away bias: If you're a technical architect, you think in objects. If you're a database designer, you think in tables, stored procedures, queries, etc. If you're a project manager, you think in gantt charts, dependencies and resources. So which toolset do you want to work with together? Post-it notes makes everything give up their biases and focus on real requirements.

4. Speed: How fast are you really with Project, Visio, Word, etc. I'm a power-user on all these tools but I still find whiteboards are faster.



5. Freedom to reboot: People get attached to their work. In addition, if you've spent the past 2 hours making sure the lines all connect properly to the boxes in Visio or making sure that you've leveled your resources in Project you really don't want to relive that experience in the same afternoon. But whiteboards are really easy to reboot and people don't get as attached to Post-IT notes.

6. Multiple models: If you use computer based tools, its more difficult to use multiple models. You need Visio for your wireframes, you need Word for some documents, you need Project for your gantt, etc. The amount of tools required to support a multiple-model approach. Alternatively, you get sucked into the idea of using one BIG GRANDIOSE TOOL - enter in the Rational Rose, MDL, Case Tools, Heavy IDEs, etc. that try to be all things to all people and cost a fortune in the process.

One of the challenges with low-tech approaches is how to translate them into digital, cleaned-up format for posterity. Here are some basic suggestions:

1. Don't bother: every document requires upkeep, so don't bother capturing every design decision in a fancy document unless its really necessary. Low-tech documentation is more about the process of thinking through drawing then creating fancy documents. Keep the top 10% of what comes out and dump the rest.

2. Take a picture: take a picture of your white board - you've now got documentation! Put it on your WIKI, store in Word documents with notes, etc. and you're good to go.

3. Data-entry: sometimes you want to capture the notes, diagrams, etc. in a digital file so it can be further manipulated. So at the end of the meeting, take the 20 minutes it requires in most cases to data-enter it using the right tool. Better yet, get your junior or admin assistant to do it for you - why have a $100K technical architect creating circles in Visio when a $40K secretary could do it for you?

Note: this is probably an out-sourcing opportunity as well. Do some white boarding, take a picture and then send it overnight to diagrammed in India. Walk in the next morning with your fancy new diagram complete with colour coding and code stubs created!

Once you've done some design work in a low-tech manner, then use a computer based tool if you need it. See tomorrow for part 2 where I discuss how to use the computer in a low tech way and also how to use the existing tools more efficiently.

I'm Somehow Ranked 6 for the Google Phrase "Buy Books"

I have an Amazon affiliate store (see the link to the right). It has a few books in it, but not much more. I have never received a sale through this link.

Yet my friend Jordan pointed out today that if you search on "Buy Books" the link to my store shows up as ranked number 6 in Google's list!

How is this possible? What does this really say about Google's ability to filter results? Is it simply because I'm a random Amazon store?

Check out Funny or Die

Check out the new site from Will Farrell, Funny or Die.



While YouTube has volume, the sheer amount of it means that it suffers from mediocrity. If you search for something specific you'll find some content that is good, but just browing for entertainment is pretty limited given the sheer amount of garbage content.

Funny or Die has some distinct advantages that give it benefits of YouTube and traditional broadcast television:

1. Star Power - lots of videos from Will Farrell, Jimmy Fallon, etc.
2. Original Content - YouTube has lots of content that is just copies of other media e.g. videos, music, etc.
3. Cutting Edge Humour - this sight makes Saturday Night Live look pedestrian by comparison.
4. Some selective choosing - the site has a rating setting of "Immortal" which basically means that Will Farrell put it up and so no one can rate above him. This isn't a bad thing - this means that there is some real selective editing on this site and its driven through a particular perspective. This makes it different than a generic search engine.

Check it out - some great content worth viewing...