Why Agile Software Projects are Good for Customers

We have two software projects that are going on at the Foundation. Project #1 is just finishing. It was not managed by me and it was done using a traditional waterfall death march type methodology. Project #2 is being managed by me and we're using a traditional Agile project management approach.

Comparing the two projects really shows the different in the benefits and obligations of the customer in an agile project:

• Project #1 is 100% over original estimate for no apparent reason. The customer gave up trying to even get a date from because they haven't seen any deliverables, don't even really understand the scope of work and just want it done.
• In Project #1, there was about 1 month just spent trying to define requirements. There were dozens of meetings trying to nail down field mappings with no code being written in the meantime (which starts to explain why the project is late). The right people were never in the room and the developers got different answers from different people.
• In Project #1, despite the investment in time, the output in terms of requirements documentation and certainty of scope was quite poor. Despite the investment, the requirements still changed up until the last week before testing including major business rule corrections.
• In Project #2, we've taken a traditional Agile PM approach. We include the customer in every meeting (usually once a week) and we are using a combination of use cases and wire frames to document requirements. This is the first time in the organization that the customers have seen either format so its a bit of a learning curve in terms of how to read them. But it took about an hour of reading through use cases before the customer fundamentally got the value of such a document.
• In Project #2, I got my lead technical architect to start prototyping on day one. By week 2 of the project, we already have a prototype writing in .NET using SQL Server 2005. The data model is about 50% done. But before even the documentation is done we have a working model.
• In Project #2 for project management we've taken an Iteration Cycle of 2 weeks as the basis for development. We have an architecture/planning/requirements spike for the first 2-3 weeks but have already started coding. Every Iteration has a plan to deliver a list of use case implementations and related wire frames as fully functional code. We have formal show and tell sessions at the end of each cycle.
• In Project #2, we have a proper project plan with real estimation. Estimation is done at the wire frame and use case level and tasks are broken down to about 1-3 days in duration. In my experience anything more granular is a waste of time and anything more becomes unpredictable and subject to error.
• In Project #2, we will be using Test Driven Development and Customer Testing all the way through the project as well as Continuous Integration.

The experience so far from a customer perspective has been really interesting so far:

• Customer participation on Project #2 has been extremely high and involved. The customer role has been really well defined and our stakeholders have been good at providing the right level of input.
• There is a good trust relationship and a sense of team - it isn't a US vs. Them approach. We're not delivering software for a customer - we're all as a team delivering on a project.
• Our ability to shift priorities and change the schedule is highly responsive. For example, our marketing department is working with the team to figure out the content for the application and its taking a lot more time than expected. So we've started on the back end logistics sections and left the front-end until the end in order to give the marketing folks the most leeway. By having check ins every two weeks we can easily adjust.
• In addition, because the application is built in vertical slices instead of horizontal ones, we don't have to have 100% of one tier done before moving to the next one. We can simply move from front end to back and hook together some disconnected pieces as they come available.
• The project is simply more fun and more of a team based approach.

Some recommendations to software project teams in treatment of their customers (whether internal or external) in an agile context:

• Don't patronize your customer because they've never done software projects before. Don't treat them like idiots because they've never read use cases or UML diagrams or database models before. If you take an hour to introduce the concepts and travel light, you'll find that customers "get it" rather quickly and then see the value of these requirements tools.
• Always remember that the customer is the end user and not a stakeholder representing an end user. For example, many Directors of Marketing act as the "customer" in web project teams. They're not. They're representing a customer. Everyone on the team needs to always focus on the end customer and stakeholders always have to be able to filter out their own biases and represent the end user successfully.
• Customers will buy into an agile project management approach if it produces results - period. If you cannot deliver software on time or its poor quality, an Agile approach won't save you and will quickly degenerate. Shorter iteration cycles means there is no where to hide, so talent/experience and ability to deliver are critical early success factors.
• Show and Tells are critical to the Agile Project. We use to have them once a week on Fridays. Make Show and Tells Fun! We use to dim the lights and use the projector and have popcorn and snacks. Even if its some boring application, hype it up and celebrate it.
• Customers should be the leading evangelists as well - some customers have been taught to be the evil task masters and try to crush development teams because they think it gets the team to work harder. I can tell you from the other side of the team that it doesn't work - developers work less, produce poorer quality and flip the bozo bit on you in short order.
• One of they values of an Agile project team is transparency and team work. Transparency is a double edged sword - customers have to be prepared to hear the bad news with the good and to help the team prioritize effectively. Good customers develop a relationship with the team that allows them to be trusted - its critical in order to encourage honest dialogue.

Overall, so far Project #2 is doing much better than Project #1. It has lots of risks and we may not be able to deliver all the functions asked for in the time line. But the requirements are well understood, communication is open and use cases are prioritized. We've already had conversations about what parts of the scope could drop if needed and contingency plans if we go late. Customers are in the loop at all times and a core member of the team.

Agile Just Feels Natural

Have done agile development for most of my career, I find it interesting when I first introduce the concepts to new developers. We're just starting to introduce it to people in my department, and we have got started with reading through the usual standard books by Ambler, Fowler, Beck, et al and I've start acting as the Agile PM on some key software projects.

What I find interesting is when I introduce it to developers, the universal reaction seems to be the following:

1. What a breath of fresh air - this is how we should have doing software all along.
2. I've already been using these techniques but didn't have a methodology to explain it and wasn't rewarded for it. I find that developers who have been raised in the classical SDLC have secretly rebelled against it and agile methodologies simply validate the rebellion.
3. The biggest flaw as most software developers experience with the traditional waterfall model (or worse the traditional death march just get it done by a fixed deadline approach) is that they now have some confidence in their work and ability to deliver against timelines.
4. When I find developers revert back to their old bad habits, they almost always run back into trouble and then slap themselves in the head of going back to hiding their risks, not being iterative and letting projects slip into death marches.

See my next post on why Agile is Great for Customers...

How to Use Microsoft Project Efficiently

I've worked with a lot of project managers who don't really know how to use MS Project efficiently. I was trained well on MS Project because in the consulting business, you have to be able to build million dollar project plans in days for sales pitches. Our PMO figured out a number of ways of using Microsoft Project more efficiently.

Here are my recommendations:

1. Get rid of the GANTT chart by dragging it to the right. Use the extra space for some additional columns.

2. Go to Window and click on Split. This will bring up the Task Form dialogue box at the bottom. You will find that entering data in the task form is much more efficient and visible than any other method.

3. Go to Tools->Options and click on Schedule. Change the Default Task Type to Fixed Duration and click off the box marked "New Tasks are Effort Driven". The reason is that when you're estimating, you'll find in most cases its easier to think in duration and allocating a % of a persons time across that duration than thinking about person hours. For example, instead of thinking of a task taking 25 hours, its easier to think about a task taking 3 days at 100%. Its also easier to think about a task as taking 2 people x 3 days, especially if you're days are 7.5 hours (do you think of a task as taking 50% of a week or 18.75 hours?)

4. Don't put in any absolute dates in your plans. Make all your dates based on dependencies except for the basic project start date.

5. For tasks that you want to go in parallel, use the SS predecessor type to make sure that multiple tasks start at the same time.

6. Go to Format->Text Styles and click on Item to Change. Select Critical Path and change the colour to something obvious (I use BOLD RED). This will help you in making sure that every task has the appropriate predecessor set - if it doesn't, you'll notice that the critical path stops half way through your project plan. It should highlight tasks from the beginning to the very end of your project.

7. Lets assume you have a project plan that is way past your target timeline and you want to now parallel your effort by adding more people. Many project managers will start creating streams of tasks in parallel with complex dependencies. But here is a simpler way if all you want to know is how many people you need to complete the work on time - simply take your tasks, change them to fixed work and increase the percentage to 400%, 500%, etc. until you get your timeline in the ball park. If its 500%, that means you need 5 full time developers to get the work done.

NOTE: the one draw back to this approach is that it doesn't tell you when each task will be done. But in my experience, this is not that useful as development projects tend to have 50 tasks that can largely be done at any time over a longer period and you simply spread the load across a pool of developers until the work is done. If there are specific tasks that are dependent, then set them up as a seperate stream. But if its simply a pool of tasks that need to get done this technique is much faster and simpler to manage.

8. Don't allocate overhead resources to tasks (e.g. your PM, your TA, your QA lead, your stakeholders, etc.) unless you have very specific requirements. In most cases, you can estimate an overhead resource at a % allocation across an entire project phase. For example, I'll allocate a PM at 30% for the duration of the project, a TA for 20%, etc. This again is a bit of a cheat - if you want to plan that a PM is going to be busy at 100% on Wednesday and 20% on Thursday and 40% on Friday, then this technique fails. In my experience, this is not the way overhead resources work and in the end having your project plan reflect this micro-detail is not worth the work to maintain it.

9. Here are a few ways to calculate contingency:

• Click on Insert Column and pick Cost1. Right click on it and select customize field. Then click on Formula and type in Cost*1.10. This will calculate contingency at 10%. If you want 25% contingency then use Cost1*1.25. Make sure that Calculation for Task and Summary Rows has Rollup selected and Sum.
  
  NOTE: There is a bug in Project 2003 where it sometimes does recalculate fields properly. If you notice this, then go to Tools->Options->Calculation and click Calculate Now. This will recalculate the field.
  
  


• Create a task called Contingency and set a price for it. This is good for setting an arbitrary contingency amount.


• Create a task called Contingency and assign a resource called Contingency at sufficient hours to drive up the price. The advantage of this approach over the one above is that it means your total hours assigned also has contingency built it and your hourly rate stays consistent with what it would be without the contingency assigned.


• Create a task called Schedule Contingency and set the duration to an amount of time with no resources attached. Bury the task where you think the schedule risk is going to occur - I usually put it into the QA phase of the project as this is the mostly likely area where you'll need the contingency.

10. Save often. I've had too many cases where Project crashes and there is no automatic recovery. In addition, there is only one level of Undo in Project 2003 (this is fixed in 2007) so if you make some significant changes its tough to revert them.

11. Use the keyboard as much as possible. Learn keyboard shortcuts. Don't drag things around when you can copy and paste. Don't use your mouse to indent when you can click on Alt-Shift-Right Arrow. Typing in predecessors manually instead of clicking on the link task button in the toolbar generally is faster.

Hopefully, that will help speed up your MS Project work dramatically. If you work this way, you'll find that planning with MS Project is feasible. If you're doing it the traditional way by dragging grantt bars around you'll find a major speed improvement working this way.

Low Tech Approaches to Requirements Gathering - Part 2

For those who missed Part 1, I wrote on how to use low tech tools such as paper, Post-IT Notes, whiteboards, etc. to gather requirements.

In this post, I'll describe how to better use tools such as Office, Project, Visio, etc. to gather requirements.

Here are my recommendations about approaching tools such as these in an agile context:

1. Use "Just Enough" tools. Don't use Rational Rose for UML diagramming if Visio will do quite nicely.

2. Buy the absolutely best tools for the job. In theory, you can write .NET applications using Notepad and command lines tools and save a few bucks on licenses. But the productivity will be poor and learning curve will be worse. Don't allow your organization to scrimp on tool costs - in my experience if its the right tool for job, paying the $500-1000 for a particular tool is worth the cost.

3. Invest in tools that are standardized. If you're working in Java, use Eclipse. If you're working in .NET, using Visual Studio. If you're an Oracle DBA, get Toad in your office. If you want a source control tool, get VSS, Subversion or CVS and not Seapine. That way when you go hiring for new developers, they've probably used your tools before.

4. Get really good at using the tools. It sounds like a basic suggestion, but I don't know how many project managers I've worked with don't really understand Microsoft Project. If you set up project in particular ways such as exposing the dependency columns, splitting the window to have the resource allocation on the bottom instead on the side, getting rid of the GANTT chart altogether, etc. productivity goes up considerably. Same thing with Word - the first thing I do with any document sent to me is look at how many Styles are being used and typically I find hundreds of them. Whomever wrote the document didn't understand how styles work, why they improve productivity, etc.

5. Buy tools that are easy to learn. Again, another dumb suggestion but I look for tools that a junior developer or programmer can understand. In many cases, this means buying more tools as the simplest ones can be single purpose. I've rarely come across a monster tool that works as a really good Swiss Army Knife for all occasions. The profiler I use though is stupid simple - its from RedGate and you load it up, point it at your project and start profiling in about 30 seconds. Its dumb simple to learn and cheap as well.

6. Have everyone have access to all the tools and have everyone knowledgeable about them. For example, I've worked in organization where only the Senior Project Managers were "allowed" to have Microsoft Project. If you weren't a Senior PM, then you had to either use Excel or find a Senior Project Manager to help you build a project plan. Technical resources should be able to open up a project plan as well - if you want your team to be accountable for deliverables, telling them that they're not important enough to see the plan sends the wrong message!

7. Automate work as much as possible. Use Continuous Integration, Test Driven Development, build scripts, O/R mapping layers, code generators, documentation generators, etc. where appropriate to automate drudgery work. Invest in time sheet systems, collaboration spaces, source code control systems, etc. where appropriate instead of tracking things manually.

8. Remember that every document, artifact and tool costs money to buy and to maintain. Every software license has maintenance costs. Every document takes up space on a file server. Artifacts have to be managed, secured, backed-up, etc. So travel light as product just enough documentation. Use just enough tools to do the job which also usually means use the ones that are the most productive. Remember that emails are documents as well - if you're finding that you're getting hundreds of emails a day, figure out how to get your team to be more concise, more action oriented and spend the majority of their time producing deliverables instead of documents!

9. Get the fastest computers you can afford. Get two monitors minimum for every developer. Get a good video card. If you have to wait 20 minutes to do a build, trying to do agile development is going to be tough. If paying an extra $500-1000 will cut that build time to 5 minutes then I would pay the money as it will result in massive productivity improvements on a daily basis.

Low Tech Approaches to Requirements Gathering: Part 1

One of the tenents of agile development is just enough documentation. I would recommend adding another suggestion: just enough tooling.

The boss comes to you and says, we need a project plan for this large software project. Budget is about $1 million and I need it done in 6 months. Figure out what is required, how many resources you need and what kind of scope can be delivered.

So how many of you have already reached for Microsoft Project?

Forget it for now - come back to it after you've tried some more low tech approaches. Here are some suggestions in order of really low to medium tech approach:

1. Talking: before you start visualizing, diagraming, or white boarding, start talking! Some people talk and diagram at the same time (I do) but I've been to too many meetings where an architect gets up to the front of the room and the team sits there for 30 minutes waiting for the person to finish his fancy whiteboard picture.

2. Piece of paper: I love white boards, but they're hard to carry to a bar, a coffee shop, an airplane, etc. So keep a sketchpad, a napkin, etc. and use it as your white board.



3. White board: the number one tool for an agile PM. If you want to record your white board, take a digital picture of it!

Note: Make your white boards portable. Get the white boards that are on wheels or make them small enough that you can move them from desk to desk. We used to have our release checklist on white boards and it worked great - we could leave it up permanently (without the PLO in a meeting room) and then we would erase things off the board as they got done. This also stops the need for dedicated planning rooms - simply bring the tools with you.



4. Big sheets of paper and Post-IT Notes: Ever done a web site map with a big sheet of paper, some markers and post-it notes? Try it and you'll never go back to using Visio to do site maps again.

Note: If you want to be really advanced, use coloured Post-IT Notes. Get Red, Yellow, Green and you can use them to connote priority, level of difficulty, etc.

5. Paper templates: if you find yourself redrawing, create some templates in word and then print them off. For example, don't draw yourself a calendar each time - simply print off one in outlook and then start writing on it.

All of these techniques mean no use of a computer (except for create a couple templates). You can use these techniques in any environment that has access to an Office Depot.

Low tech approaches to planning, requirements gathering, design, etc. have some specific benefits:

1. No license costs: while this isn't such an issue with Project, if your development tool standard is Rational Rose, Together Soft or some other expensive IDE where every user must fork out thousands to build a UML diagram, a project plan or a document then using lower tech approaches will have dramatic cost savings.

2. Encourage collaboration: if you get people to talk, to collaborate and to physically draw, use construction paper, or write the collaboration happens naturally.

3. Take away bias: If you're a technical architect, you think in objects. If you're a database designer, you think in tables, stored procedures, queries, etc. If you're a project manager, you think in gantt charts, dependencies and resources. So which toolset do you want to work with together? Post-it notes makes everything give up their biases and focus on real requirements.

4. Speed: How fast are you really with Project, Visio, Word, etc. I'm a power-user on all these tools but I still find whiteboards are faster.



5. Freedom to reboot: People get attached to their work. In addition, if you've spent the past 2 hours making sure the lines all connect properly to the boxes in Visio or making sure that you've leveled your resources in Project you really don't want to relive that experience in the same afternoon. But whiteboards are really easy to reboot and people don't get as attached to Post-IT notes.

6. Multiple models: If you use computer based tools, its more difficult to use multiple models. You need Visio for your wireframes, you need Word for some documents, you need Project for your gantt, etc. The amount of tools required to support a multiple-model approach. Alternatively, you get sucked into the idea of using one BIG GRANDIOSE TOOL - enter in the Rational Rose, MDL, Case Tools, Heavy IDEs, etc. that try to be all things to all people and cost a fortune in the process.

One of the challenges with low-tech approaches is how to translate them into digital, cleaned-up format for posterity. Here are some basic suggestions:

1. Don't bother: every document requires upkeep, so don't bother capturing every design decision in a fancy document unless its really necessary. Low-tech documentation is more about the process of thinking through drawing then creating fancy documents. Keep the top 10% of what comes out and dump the rest.

2. Take a picture: take a picture of your white board - you've now got documentation! Put it on your WIKI, store in Word documents with notes, etc. and you're good to go.

3. Data-entry: sometimes you want to capture the notes, diagrams, etc. in a digital file so it can be further manipulated. So at the end of the meeting, take the 20 minutes it requires in most cases to data-enter it using the right tool. Better yet, get your junior or admin assistant to do it for you - why have a $100K technical architect creating circles in Visio when a $40K secretary could do it for you?

Note: this is probably an out-sourcing opportunity as well. Do some white boarding, take a picture and then send it overnight to diagrammed in India. Walk in the next morning with your fancy new diagram complete with colour coding and code stubs created!

Once you've done some design work in a low-tech manner, then use a computer based tool if you need it. See tomorrow for part 2 where I discuss how to use the computer in a low tech way and also how to use the existing tools more efficiently.

I'm Somehow Ranked 6 for the Google Phrase "Buy Books"

I have an Amazon affiliate store (see the link to the right). It has a few books in it, but not much more. I have never received a sale through this link.

Yet my friend Jordan pointed out today that if you search on "Buy Books" the link to my store shows up as ranked number 6 in Google's list!

How is this possible? What does this really say about Google's ability to filter results? Is it simply because I'm a random Amazon store?

Check out Funny or Die

Check out the new site from Will Farrell, Funny or Die.



While YouTube has volume, the sheer amount of it means that it suffers from mediocrity. If you search for something specific you'll find some content that is good, but just browing for entertainment is pretty limited given the sheer amount of garbage content.

Funny or Die has some distinct advantages that give it benefits of YouTube and traditional broadcast television:

1. Star Power - lots of videos from Will Farrell, Jimmy Fallon, etc.
2. Original Content - YouTube has lots of content that is just copies of other media e.g. videos, music, etc.
3. Cutting Edge Humour - this sight makes Saturday Night Live look pedestrian by comparison.
4. Some selective choosing - the site has a rating setting of "Immortal" which basically means that Will Farrell put it up and so no one can rate above him. This isn't a bad thing - this means that there is some real selective editing on this site and its driven through a particular perspective. This makes it different than a generic search engine.

Check it out - some great content worth viewing...