Chris Woodill

Bug in Microsoft Project 2010 – Resource Usage

2010-11-23T15:10:00.001-05:00

In Microsoft Project 2010, you can create a custom calendar to set your day as something different than the default 8 hours per day. In this example, I changed the working time to 7.5 hours per day.

So far, no problem. However, look what happens when you view the Resource Usage Tab for the same task:

That’s not what I want - I said 7.5 hours per day is the working time! It should be 7.5 hours per day, not 8 hours for the first 4 days and 5.5 hours for the fifth day.

IT Manager Survey Results So Far…

2010-11-01T20:56:00.001-05:00

About 2 months ago, I started a survey on IT Management. The following are the results so far after about 40 responses. If you would like to participate in the survey, please feel free to do so by following this link: https://www.surveymonkey.com/s/ITManagerSurveyWeb.

Here are the highlights so far…

The US Customs Rules have not Kept Up with the Realities of Global Knowledge Work

2010-09-20T11:42:00.001-05:00

As a global consultant, I travel from time to time to the United States. When you go through customs, you are questioned to ensure that you are not “working” full-time in the United States without a green card or visa.

The basic rules for "working" in the United States when you interact with airport customs seem to be as follows:

You are not allowed to be a "worker" without a visa or a green card
To get either a visa or green card you have to be employable in the US for an extended period of time, e.g. permanently in the case of the green card and at least a year in the case of a visa
Getting a NAFTA visa is based on a list of in demand occupations such as teacher, computer programmer or occupational therapist
If you do the work in another country they will let you visit your customer in the united states on occasion. There is no specific rule that is spelled out but in my experience if you go down for a couple days in a month you are ok but if you are going down too often or stay too long the customs officer will start suspecting you are doing "work" in the United States.
You are allowed to go into the United States on business if you state that you are in “Sales”, e.g. to close a deal.

So those are the basic rules, formulated decades ago before the Internet era. The rules clearly show a very mid 20th century view of "work":

"Work" equals hours put in against a job requiring a very specific boxed in role. The worst thing to tell a customs officer is that you are a "business analyst", "trainer", "strategist", or "consultant". "Work" equals a job that an average customs officer had heard of, e.g. farmer or secretary. Try explaining what you do as an "Information architect" and you will be met with suspicion.
"Goods" equals tangible imports like lumber, cars, etc. What if your "good" is a company's online strategy that they just paid you 150k to develop? What if your “good” is something that you can store on a USB key in your pocket?
"Labor market" is based on the notion that you have to be in the country in order to work. It misses the point that I can do millions of dollars of knowledge work without any concept of borders.
“Work” means working on premises. I can work just fine for a US company as long as I work for a Canadian company and ship goods into the United States. However, if I want to spend 2 weeks deploying the “product” on site, then I’m going to get questioned.

If the goal is to prevent labor from competing with local “workers”, these types of restrictions don’t really provide much support in the digital global knowledge working realm. Really, the rules basically are a restriction on face to face meetings but place no restriction on me being able to compete first hand with local resources from remote access points. Instead, they just put a barrier to innovation and physical collaboration as a complement to existing means of digital collaboration. It means that American companies with offices in other countries (Canada being a great example) need to obtain visas for employees who are jumping across the border for short term stays.

Surely, in the 21st century, we should update this approach and recognize that INCREASING global knowledge workers ability to collaborate across borders is essential to higher productivity, better solutions, and a better economy.

Given that reality, I would propose a very easy to obtain, very low restriction visa that would enable short term deployments (say 2-4 weeks) across the border. The governments on both sides could charge a fee and use a Nexus type approach to clearing consultants for the usual security risks. Once they have the pass, they could go down at any time to deploy solutions, meet with customers, and collaborate with stakeholders with the assumption that 80% of the work would still be done outside the country but that at crucial times there is a requirement for global knowledge workers to collaborate together. If the United States wants to be the centre of global innovation, it should enable these global workers to actively choose the US as their physical meeting hub for planning sessions, deployments to data centers, training, and collaboration sessions. This would create a high degree of network effect that would put it at a significant advantage over more restrictive countries such as China, Russia, Middle Eastern countries, etc.

Zachman Framework Certification No Longer Valid?!

2010-09-20T10:57:00.001-05:00

Gartner is reporting that after almost 30 years of being the de facto Enterprise Architecture framework for government, aerospace, and other large scale enterprises the Zachman Framework has collapsed as John Zachman is retiring.

At first glance, it appears that John A Zachman is closing up shop and about to retire, having severed his business partnerships, and appearances are that his son, John P Zachman, will take over the IP trademarks and, as alleged by a former business partner Stan Locke, seek to place the Zachman IP within a COTS package.

A second, and more penetrating glance, into the actual lawsuit itself reveals alleged deceptive practices by the defendant, Stan Locke, and claims of serious and reprehensible actions that are alleged to damage the brand of Zachman.

The capitulation by the defendant in a surprising lawsuit, Stan Locke, will not resolve the concerns or Zachman brand ownership concerns as the damage has already occurred. This, in turn, raises questions about how the Zachman brand will regain control after relinquishing that control for so long.

What does this mean for Enterprise Architects that have spent their time, energy and significant financial resources getting certified and becoming practitioners in the EA field? Based on the current state, it is not clear how this will be sorted out.

As of right now, certification programs have been suspended, John Zachman is exiting the business and the entire framework is now surrounded by legal troubles over who controls the brand, the IP, etc.

Copying Data from Project 2010 to Word is Actually Worse than 2007

2010-09-08T08:12:00.001-05:00

In Project 2007, copying data from Project to Word was quite painful, resulting in some funky formatting that was hard coded to whatever colors, fonts, etc. you had in your project file. In addition, when you copied dates, the default format was 9/8/2010 12:00pm which unless your plan is VERY time sensitive is not typically want you want to show in a word document.

So in Project 2010, the Microsoft team seems to have fixed some things but broken others in an effort to improve these problems in 2007.

So take this dummy project plan I created:

I want to move this into Word so that I can include it in a Statement of Work, Project Charter, etc. What I want is this data in a simple table – I don’t need GANTT charts or anything else…just a basic list of tasks and durations.

So now if I copy this into Word, this is what I get:

Not a bad representation of the plan – certainly better than the output in Project 2007. However, the biggest challenge here is the indenting – Task 1.1, 1.2, etc. are indented using 4 Spaces instead of a TAB, a proper Word paragraph indent, etc. If you want to change the size of the indent in word, you’re now stuck with doing a search and replace for 4 spaces and either replacing them or converting them to TABs or Word paragraph indents.

In Excel, there is a further problem – date formatting. In Excel, the same plan looks like this:

Again, not bad and certainly an improvement over MS Project 2007. However, let’s suppose that you want to change the date format to a simple date without the day in it, e.g. 8-Sept-10. In order to convert the cell into a proper date, you have delete the "Wed " from the dates for each day of the week. Then you can format your dates anyway you like in Excel. In addition, you have the same problem with indents as Word – indents are handled with 4 Spaces instead of real indent marks, making them tough to remove.

So from my perspective, while the formatting is prettier when copying data from MS Project 2010, its actually less useful and harder to convert to something I can manually format and change in Word or Excel myself. This is ultimately what is needed – if you are copying and pasting into another document, you will want to use the parent documents fonts, indenting, date formatting, etc. and not Project’s default settings.

Another Apple IPOD Touch OS - A New Set of Weird Bugs

2010-08-01T18:08:00.003-05:00

I have now upgraded my IPOD Touch to IOS 4.0 and already I have experienced some odd bugs:

1. The default Wall Paper is a picture of rain hitting a glass window - not really a bug, but my first instinct was to assume that my IPOD had been dumped in water or had been exposed to humidity and it was warning me!

2. The genius mix (which I have blogged about previously as a major source of bugs) seems to at least work, but the genius database makes some odd choices...it put Glenn Miller into my "Classic Rock" mix and Madonna into my "Indie Rock" mix for example.

3. When you upgrade your OS, it backs up your existing media, wipes your OS and then restores your media through a sync. I had 26 images on my IPOD from my vacation and the restored pictures are there but they are now super pixelated low resolution!

Zachman vs. TOGAF – Enterprise Architecting for Change

2010-02-13T12:05:00.001-05:00

In the past several years, I have been working with the Zachman Framework for classifying and organizing architecture artefacts. Zachman is quite popular as a framework in large enterprises such as governments, big banks, utilities, etc. because it is seen as a comprehensive way to manage the architecture process.

In the past year or so, I have been switching over to TOGAF which is a competing EA framework. In comparing the two frameworks, I have found it to be fundamentally more useful in my line of work for a simple reason – TOGAF is a process for managing architecture change, while Zachman is a taxonomy for capturing the current state of an enterprise (exhaustively seems to be the goal).

Roger Session’s wrote a good article comparing various EA frameworks, and I agree with his conclusion – Zachman is really a taxonomy and TOGAF describes processes.

The TOGAF process is described by the Architecture Development Method (ADM) :

The ADM has some interesting features that make it more relevant to evolving enterprises:

The ADM is fundamentally iterative, both within a single cycle and through repeated iterations of the full cycle.
TOGAF provides a prescription for storing, classifying and managing architecture artefacts and solution building blocks at various levels of detail and maturity, promoting an incremental approach to enterprise architecture.
Along with the ADM, TOGAF prescribes many tools for capturing requirements, defining/designing architectures and implementing governance practices.
The process is explicitly business and stakeholder focused – there is a recognition that architecture has to have a business purpose and not just be for its own sake.
TOGAF explicitly allows borrowing from other frameworks, adapting and tailoring the framework to suit the needs of the business, etc. So if you still want to use your Zachman Grid, you can continue to do so.

Zachman, in contrast, is about capturing state in a structured way. It provides no guidance on HOW to do architecture design, HOW to introduce change or HOW to improve the architectural maturity of an organization. It simply provides the model for storing architecture artefacts.

Zachman also has no guidance for incorporating processes into an architecture state – there is no spot in the grid for defining change management processes, operations processes (e.g. ITIL), project management processes, etc. TOGAF in contrast compliments these processes quite nicely:

In my work, I am finding TOGAF to be an effective framework for working with stakeholders through strategic planning and architecture work, especially for organizations who are actively transforming themselves. It allows me to define a process for managing the architecture change in a structured manner through an iterative change cycle. Stakeholders see a PROCESS that involves them through change and growth in their organization’s maturity.

My old URL is now obsolete

2009-11-15T21:44:00.001-05:00

Before Yahoo Mail became a big deal, it didn’t have its own URL (its currently mail.yahoo.com). The URL was www.yahoo.com/r/m1. This URL hasn’t been promoted or used for about 10 years now but it still worked. My brain’s automatic typing reflexes simply remember that URL as a habit.

As of this week, it finally died! Now when I without thinking type www.yahoo.com/r/m1 I get this error message:

I guess it will take me a while to remember mail.yahoo.com as a replacement…

New Details on SharePoint 2010 from Microsoft

2009-07-14T07:14:00.001-05:00

Microsoft has released a new site on SharePoint 2010.

In the site, you will find information on SharePoint 2010 for developers and it professionals.

There are some interesting announcements in video format here that are worth exploring. In the developer video, they show SharePoint integration into Visual Studio 2010. There are a few new bits in here:

Manifest editing tools in Visual Studio
Solution Package Designer in Visual Studio
New web part editor for Visual Studio
LINQ will now support SharePoint lists.
Developer Dashboard will help by providing diagnostic information when developing web parts in SharePoint.

Will Google Chrome OS Be Any Different than LINUX, BEOS, Network Computer, etc.?

2009-07-08T07:13:00.001-05:00

It was widely announced that Google will be releasing a new operating system called the Chrome OS that will be initially deployed to NetBooks but will hopefully compete with Windows on the desktop.

Yawn…shades of 1995 when Java OS, Oracle OS, Network Computer etc. tried to do the same thing.

If you want a OS replacement for Windows on your NetBook, you can get it today – its called LINUX! Acer, Dell, etc. all have Net Books that will come with LINUX today. And if you want to run Google’s browser, you can download it for free.

Can you even call Chrome OS a new operating system when you’re just repackaging LINUX? KDE and GNOME aren’t operating systems, so why is Chrome?

"The operating systems that browsers run on were designed in an era where there was no Web," the blog post said. "So today, we're announcing a new project that's a natural extension of Google Chrome--the Google Chrome Operating System. It's our attempt to rethink what operating systems should be."

Huh? Windows Vista, Windows XP, Mac OS, and LINUX were all designed with the web in mind.

COREL had the same vision in mind when they created COREL LINUX. They had WordPerfect, CorelDraw, etc. (even at the time much more sophisticated applications than Google Spreadsheet is today) and they couldn’t sell any more office licenses because Microsoft had a lock on the desktop and was killing them in the office market. So they made a big splash (and almost killed the company) investing heavily in desktop LINUX, the supposedly Windows killer.

BE Operating System was an entirely new OS developed in 1991. Unlike the Chrome OS or Corel, it wasn’t a rehash of LINUX with a new set of apps thrown on top of it. It was an entirely brand new operating system from the kernel upwards. It had the same ambitious target – replace Windows with a better operating system. It had a niche following originally as a replacement to the Apple Operating System (since the BE OS initially only ran on Power PC Chips and not Intel) and then as a Windows replacement. The company could not gain enough market share and eventually was bought out by Palm.

The same reasons why these operating systems didn’t survive will cause major challenges for Google. I found this table from Microsoft to challenge the proposition of the Network Computer:

	PC	Network Computer
Operating System	Windows	Proprietary / Incompatible
Mobile Support	Excellent	None
Applications	> 100,000	Limited / Incompatible
Development Tools	Choice	Limited to Java
Device Support	Thousands	Proprietary / Incompatible
Device Model	Windows	None / Incompatible
Manufacturers	2,500	Few
Networking	Choice	Requires high speed net
Availability	Now	Limited
Printer Support	Thousands	None
Scalability	Excellent	Poor

With using LINUX, Google can solve some of these issues (scalability for example). But even OS X for example has less third party support than Windows and far less partners and applications and it is fairly mature. Google will have none of these advantages.

Challenges with Sharing Excel Workbooks in SharePoint

2009-07-05T14:22:00.001-05:00

In Excel 2007 (and 2003), you can share a workbook. This allows users to simultaneously update the same file:

This feature is used in many enterprises for collaboration, especially in departments such as Finance or Planning where there are lots of shared figures, budgets, plans, etc.

There are some interesting features that allow for some rich collaboration scenarios simply within a single Excel file:

Each time that you save the shared workbook, you are prompted with the changes that other users have saved since the last time that you saved the shared workbook. If you want to keep the shared workbook open to monitor progress, Excel can update you with the changes automatically, at timed intervals that you specify, with or without saving the workbook yourself.
When you save changes to a shared workbook, another person who is editing the workbook might have saved changes to the same cells. In this case, the changes conflict, and you are prompted with a conflict resolution dialog box so that you can choose which changes to keep

When moving to SharePoint as a document management platform, this feature is no longer supported:

More than one user cannot simultaneously make changes to a shared workbook that is stored on a Windows SharePoint Services 3.0 site. If you want to store your workbook on a Windows SharePoint Services 3.0 site, you should do so only after the collaboration effort through sharing is complete.

Document libraries by design only allow one person to edit a document at the same time. In addition, they support check-in and out to provide locks on documents during the editing process. When a user grabs a file for editing from SharePoint, they edit a file on their local machine and then when done it is saved back to the central document repository.

Excel Services allows for running of Excel files within the SharePoint server, but only in read only fashion. Excel Services is great for read only dashboards, charts and graphs but isn’t a collaboration service.

For more information on the different options for Excel collaboration, see Microsoft’s article on the subject.

Bad Search Synonyms

2009-06-28T08:15:00.001-05:00

I was on TheServerSide.NET which is one of the leading Microsoft .NET community sites and typed in “SharePoint”. The result I got was this:

I thought that was kind of amusing, so I tried a few other Microsoft product names:

What We Know About SharePoint 2010

2009-06-17T21:45:00.001-05:00

Based on the various blog posts and articles, here is what we know from the publically available information on SharePoint 2010:

This is just a preliminary list – Microsoft will be publishing more information when the public beta is released this summer. Stay tuned as more details about the next release comes out.

How to Use Microsoft Project: Fixed Cost

2009-06-12T07:16:00.001-05:00

One of my colleagues ran into a major Project usability problem when he started filling in the cost column manually. He actually did it by right clicking on a cell in the cost column and selecting “fill down” which he thought might like in excel just copy the formula. In Project, this copies values and overrides the calculations normally done, e.g. cost = rate * number of hours.

Here is what Project assumes when you type in a cost into the Cost Column:

1. Normally, the cost column is calculated based on the rate (in this example $100 / hr) * the number of hours.

2. If I type in a number into the cost column, I can replace the calculated cost with a new cost – in this case, I have specified it to be $1200.

3. Project takes the cost I typed in and calculates the DIFFERENCE between takes the original calculated cost and what I typed in as a FIXED cost, e.g. in this case $400. If you insert the Fixed Cost Column, you will see this value.

4. If you do not understand what Project has done, it becomes really confusing as to how to get rid of the fixed cost. Simply changing the duration or hours doesn’t remove the cost. For example, if I change the duration to 2 days it doesn’t recalculate the cost to be $1600 – it keeps the $400 in fixed cost and makes the total cost $2000.

5. If you have added a fixed cost my mistake, the simple way to fix the problem is to set the fix cost to $0.00 manually. This resets to the default behaviour and now your cost is calculated normally. See below – now that I have changed the fixed cost to $0 I can easily change the duration and the cost is calculated correctly.

I have been using Microsoft Project for ages and didn’t know this behaviour existed. I found this problem because one of my colleagues was not an experience Project user and started filling in values like in Excel. However, this is nothing in Project that warns you or points you to the solution – it took me digging through help and google to figure it out.

If you are new to Project, keep this simple lesson learned in mind – do not manually change the cost column. Let Project always calculate it for you. If you have a specific fixed cost for a task, then insert the Fixed Cost column and type it in there. This will avoid a lot of confusion.

Yet Another Facebook Bug

2009-06-11T07:25:00.001-05:00

I have posted previously on bugs in Facebook, and I ran into another one just this morning.

Facebook has a friend suggestion feature that lists people who you might know based on how close they are to your existing friends. It’s a great way to find people but I found a bug in it. It presents you a list of potential friends and for each one there is a little X that you can click to take the person off the list (e.g. it was a bad suggestion and you don’t know them). Here is the bug: if Facebook runs out of further suggestion, only the text disappears and the picture stays behind.

If you go back to the home and come back in, the whole lists resets and the problem goes away.

SharePoint vs. Confluence Comparison

2009-06-07T14:37:00.001-05:00

I was at a client site this week and they currently have Confluence and are moving to SharePoint 2007 and wanted to understand the differences between the two products. I have personally implemented both products in different environments and like both of them. Each as its strengths and weaknesses.

Here is a side by side comparison:

	Confluence	SharePoint 2007
Company	Made by Atlassian in Australia. In my experience, Atlassian has wicked customer service, a great support network, etc.	Made by Microsoft. SharePoint is one of Microsoft’s most successful products and Microsoft has made a major commitment to the product. Support in my experience is standard product support but doesn’t have the tender love and care you get from Atlassian staff.
Consulting Support	Limited, but in some markets quite good. Look for open-source experts as Atlassian has strong ties to the open source community. The software itself is Java based.	Huge consulting market from small shops to major consulting companies such as Accenture. One of the major pluses of the platform is the talent pool to help implement and configure it.
Price	Free if you are a non-profit company and you ask them nicely. Server license is available ranging from $800-12000 depending on number of users.	Pricing starts at about $5000 but for intranet cost is per named user at about $75-94 a user. 200 users would cost about $20,000 vs. about $4,000 for confluence.
Search	Good with confluence content and Microsoft documents. No federated search model included. Did you mean feature allows for synonym searches.	Enterprise level search engine with relevance ranking, synonyms, large scale searches (up to 500,000 documents easily and up to 50 million in theory), and federation model. Out of the box search can index other web sites, file servers, etc. and plugins through third party products can index other systems such as OpenText, Oracle, SAP, etc.
Look and feel customization	If you’re changing the logo or basic colours you’re fine. If you want to create a whole new interface, you will find it a challenge. Confluence uses SiteMesh templates which are a set of XML/HTML files that need to be customized directly. In my direct experience (and I have done lots of HTML editing), wading through the library of templates is not a trivial undertaking.	SharePoint supports customization at multiple levels through separation of layout, look and feel, and master pages. SharePoint Designer is provided as a GUI based tool for creating new themes as well. Template system is significantly simpler than Confluence in my experience to customize.
WIKI	Stronger than SharePoint in supporting WIKI mark-up. Rich text editor is not bad but not as strong as SharePoint.	WIKI template provided as a site template. WIKI mark-up is limited to embedding links in text, but rich text editor make WIKI mark-up less needed in most cases.
Security	LDAP based. Authorization model is 100% stored in Confluence, so groups need to managed within the product. Single Sign On is not provided.	SharePoint is Active Directory based by default. Single Sign On is provided with Windows using Internet Explorer – users already logged into Windows are automatically logged into SharePoint. Groups in SharePoint can map to Active Directory groups which makes management somewhat easier than Confluence.
Web Parts	Confluence supports a plugin model and there are some available. Product availability is small in comparison to the third party web part market for SharePoint.	Long list of web parts are supplied out of the box for SharePoint. Massive third party market for Web Parts including integration with major vendors such Oracle, SAP, OpenText, etc. Web Parts can be easily built in .NET if custom parts are required.
Document Management	Version control, linking and embeddable. However, there is no check-in/out, content types, workflow, etc.	Much stronger in SharePoint including features such as check-in/out, content types, meta-data, integration with Office 2007, workflow, authorization models, etc. that are typical of an enterprise document management system.
Integration with Office	No support.	Very strong integration as long as you’re on Office 2007.
Other Features	Blogs, discussion forums, etc. are provided.	Too many features to name, but SharePoint is an enterprise portal with dozens of additional features including: reporting integration, dashboards, dozens of web parts, workflow, forms server, etc. See this link for a complete list of features.

Comparing the two products is a bit of an apples to orange comparison because the markets for each are quite different. Confluence is primarily a collaboration tool for small to medium size enterprises. SharePoint is an enterprise portal that includes collaboration but also dozens of other portal features.

In my direct experience, SharePoint is more user friendly for end-users – creating pages, changing content, managing documents seems to be more intuitive than in Confluence. If you’re a power WIKI user then its less of an issue but I handed it to an HR department and found they struggled with it. SharePoint requires a lot of pre-configuration to get it right but if its set up right I find the adoption tends to be easier.

If you really want a strong WIKI collaboration tool and don’t have much money then Confluence is a very good product. The support you will get from Atlassian is awesome and they are constantly improving it. However, if you’re looking at enterprise collaboration, document management, search, etc. then in my experience you may be looking at an upgrade to SharePoint in the long run as your user base becomes more sophisticated.

White House Data Feeds are Baby Steps

2009-05-22T08:29:00.001-05:00

The White House announced to great fan fare that it would be releasing data feeds of its public information collected by hundreds of agencies within the US government. They have launched the first iteration of this service.

My interest in this feed is to use this data to build mash-ups, visualization applications, and web applications that use this data. There are many such applications that use existing data feeds such as Amazon data, Google Maps data, Microsoft Live data, etc. in interesting ways. Amazon in particular has had an XML based REST API available for years (in fact Amazon sells a book on how to use it) For example, Frucall provides a mobile comparison application that uses a variety of publically available APIs to compare products across retailers and present them to your mobile device.

So with hundreds of data feeds, imagine the visualization, comparison, calculation and searching applications that could be built on US government data. As well, imagine then directing these applications to mobile devices, XBOX 360, IPhones, Blackberries, etc. in formats that work ideally for those platforms.

Unfortunately, based on what has been launched, this vision is not really feasible. Here is what I can see from what is currently available:

There are only about 50 data feeds. Given the number of US agencies, this is a very small sample of the data available.
The site provides data in a variety of formats such as XML, CSV, KML (google earth) and ESRI (GIS data). The feed that would be useful for apps is the XML feed, as explained by their own site (“Better suited for consumption by automated applications capable of handling raw XML files”). There are only 8 feeds in XML format available.
The data is not a feed, they are files in most cases. If you click on the links, you get either a zip file containing some XML data or a site with some links to XML files.
In some cases, the XML files are RSS feeds. While this might be great if you want content, the “data” is really just unstructured content.
There are no APIs, no REST and each feed has a completely different format. There is zero consistency even for things like unique identifiers, delivery format, or XML structure. There are also generally no available XML schemas – you just get raw XML.
Its not clear how data is published in a timely fashion. In most cases, you get a directory of files but there is no information on how often they are published.

So while its great to see the information being public, the US government has a long way to go before they have a useful API that could be used for application development purposes. Given that Googgle. Expedia, Amazon, etc. have had these APIs for many years, its a lot of fanfare and not a lot delivered for this service debut – dumping a bunch of files onto a directory was a victory in 1999, not in 2009.

Billboard Gone Bad

2009-05-13T14:59:00.001-05:00

I was downtown the other day and snapped this with my cell phone camera. Apparently, this bill board needs Windows Activation!

SharePoint is not a Database!

2009-05-12T17:38:00.001-05:00

As users of SharePoint start using lists they start taking advantage of a very intuitive and highly configurable information management tool. Users who have been using Excel or Access in the past start seeing SharePoint as a kind of rapid application development platform for building structured data applications. However, this is where SharePoint’s list architecture breaks down because simply:

SharePoint is not a database!

Part of the problem is that business users designing lists are not familiar with basic database concepts - they just want to store their information. Part of the problem is false advertising by consultants and/or Microsoft that SharePoint is a miracle platform that can do anything - it does some things well but replacing your SQL server or even Access database isn't one of them. The other problem is there is no really good way for surfacing bi-directionally data from a database into SharePoint – the business data catalogue works only in read only mode and does not allow for updates back to the underlying data store.

Here are a list of some features that a database has that SharePoint does not support - the moment you need these features you will need a real database.

Primary And foreign keys: one of the most basic concepts in a database is the enforcement of a unique identifier (either system generated or user supplied) that can identify a record and link across table in parent child relationships.
Transactions: if two changes are required to go together and the second change fails you want the first change to be undone (rolled back).
SQL language: complex queries can me written in code using a standard language. Queries can be saved and repurposed.
Stored procedures: database programmers can write complex routines that can be called as reusable code blocks.
Indexing and query optimization: used to improve performance, indexing allows the database designer to pre-index specific fields that are frequently used in queries to improve performance. In addition, most databases have optimization engines that based on what you are trying to fetch will optimize how the data is retrieved.
Large data and binary fields: most modern databases allow you to store large binary files such as video files, large volumes of text, images, etc. in the database. SharePoint can be made to store large files as documents but you only get one per record and a generic binary object.
Access outside of SharePoint: lists are accessible outside of SharePoint but only through XML or programming interfaces. Lists are not great data stores to be used for line of business applications written independently of SharePoint because the integration is relatively poor in comparison to running a SQL query on a database.
Cascading deletes: if a parent record is deleted, its children should also be deleted. In most databases, this can be configured to happen automatically or else reject the delete of the parent until the children are deleted first. This eliminates the risk of orphan child records.

These features are considered basic in databases – they have been around for decades and even the most primitive databases such as Access or FileMaker Pro have most of these features.

Lists are great for flat, changing content especially when attached to unstructured documents. At least in the current version, lists are not substitutes for databases and shouldn’t be considered so. If you need a database, then you are better off building a proper database and using SharePoint as a front-end for surfacing that data through reports, dashboards, etc. or as a host for data entry components such as InfoPath forms or custom build web parts.

Hotspot Shield and Hulu No Longer Work Together

2009-05-05T21:11:00.001-05:00

Sites like Pandora and Hulu block IP addresses not originating from the United States. The solution for the past 6 months or so has been to use Hotspot Shield which proxies your IP address behind an US address.

As of today, this no longer works. When you load up a video, you get the following:

Pandora still works thankfully…

NOTE: There seems to be a solution to this problem – thanks to the comment below! If you clean out your browser cache and then reconnect to hotspot shield it seems to work. I suspect this is a temporary solution until Hulu starts blocking this new batch of servers.

When is the old world media going to get that the Internet is global and find a solution for all this rights territory nonsense?

Why SharePoint Projects Fail

2009-05-01T07:05:00.001-05:00

Microsoft Office SharePoint Server 2007 (affectionately known in our circles as “MOSS”) is now at what should be at the peak of adoption. The product has been on the market since 2007 and there is a new version coming in a year.

Yet with many of our customers, we find that MOSS has been implemented but barely gets off the ground. There a few reasons why this seems to be happening:

MOSS is championed by IT and then too tightly controlled. For example, we have customers who have installed MOSS but will only use it for out of the box collaboration – no custom code, no document management, no BI, etc. If you want an out of the box team site, then that’s part of the “service” but otherwise you will have to wait.
MOSS starts in a single department and then never makes it out. A department picks MOSS for their needs but then there is not enough support or sponsorship in the organization to roll it out further.
MOSS is implemented skunk works style on a single box. The implementation is more successful than originally envisioned and what was a “proof of concept” starts getting treated like production. This sets up the assumption that MOSS is “Rapid Development” and eventually the box falls over under strain.
MOSS is implemented by IT but there has been no planning for adoption and training. MOSS sits there with a bunch of services – users simply don’t know how to use them and go on using other more reliable forms of collaboration such as email, file shares, etc.
The implementation gets mired in legal, HR, or branding issues as corporate citizens start to have the ability to create their own content. Similar to the IT clamp down, the organization clamps down on the services that MOSS can provide such as blogs, wikis, My Sites, etc. because they do not have a good model for managing the risk of employees contributing their own content.

Why does this happen? What is missing from most MOSS implementations? In my experience, there are a few common causes:

Too much involvement by IT, not enough involvement from business stakeholders.
Too much focus on MOSS as a generic toolbox (document management, collaboration, search, etc.), not enough focus on MOSS as a way to automate or enable true business services.
Poor infrastructure planning to enable scaling to the enterprise.
Lack of training, adoption and change management.
Lack of a killer app for the particular needs of the business
Not enough support from executive champions
No governance model to set expectations on how MOSS will be operated, changed and maintained.

If you are an IT Director or department champion for SharePoint, I strongly encourage you to think about these issues BEFORE you start an implementation. We see many cases where organizations start thinking about these issues after the first implementation has already failed and the organization has soured on the platform because there was not enough investment in managing the project and an eager IT department ready to roll out a service.

Separating the Humans from the Robots

2009-04-20T07:05:00.001-05:00

With a new Terminator movie coming out, I started thinking about the ways that on the Internet now we need to try and distinguish the humans from the robots. There is already a “war” going on out there between humans and sophisticated robots for control of Facebook, Hotmail, Gmail and other web 2.0 services.

One of the more annoying “fronts” in this war is CAPTCHA (“Completely Automated Public Turing Test to Tell Computers and Humans Apart”). This reverse Turing test is what is used on many sites to test to see if you are human and not some automated robot.

The problem is that these tests are like antibiotics – the more they get used the more sophisticated the robots. Early CAPTCHA tests were easily broken by bots so they have evolved into more and more sophisticated tests. Now the number of hoops and the readability of these tests are so bad that human beings can barely pass them.

The challenge is how to make a test easily answerable by a human being but yet heuristically difficult enough to foil a software program.

Google is experimenting with new techniques to try and simplify the experience of verification – see below.

Even using this technique, Google has to specifically pick the right images:

However, while the images will be randomly rotated, they will be carefully selected. First, Google will exclude images for which its own computers can readily identify the top, such as photographs showing landscapes with blue sky (easily detected), text (easily recognized) or portraits of people (there are many facial recognition applications on the market). Then, it will screen out images that humans find too difficult to orient (for example, abstract art, or overhead views, that don't have a readily identifiable top) by conducting a sort of opinion poll.

Google is relying not just on individual results but collective results. The heuristic picture composed from millions of clicks is what will power the analytics so that Google can reliably detect bots instead of humans.

Live Earth Bird’s Eye View

2009-04-17T07:10:00.001-05:00

One of the cooler features that sets Live Earth/Maps apart from Google Maps is the birds eye view.

Take a look at this building in Google Maps:

Now take a look at the Bird’s Eye view of the same building in Live Earth/Maps.

The four views are available if you hit the rotate button in the interface – these are actual photographs taken from different angles.

The difference in understanding heights of buildings, features around buildings, etc. is significant – imagine if you’re doing some house hunting and you see a neighbourhood you like. Wouldn’t you like to see the increased detail and vantage points that you get from this orthogonal view instead of just the top down view you get in Google maps?

Adsense and Evil Java

2009-03-28T10:43:00.001-05:00

Adsense in general does a good job of targeting ads based on the content of your site. I have adsense running on my blog and I find that it does a reasonable job of putting ads that are somehow connected to my content. If I’m writing about Microsoft Project, the reader gets ads on Microsoft Project.

Adsense seems to have one basic problem – it cannot distinguish between positive and negative content. Here is some content that I found on a random site:

Java is EVIL, it's the work of the devil, made to collect your
souls and to turn you into brainless slaves of hell.
If you like Java, you will lose your soul to be damned
for eternal torment in hell.Demons and devils will feast on your soul
for all eternity.
So stop using it now to save your soul.
Or the holy inquisition will come for you and burn your flesh to save
your soul, because the devil must not collect more souls or nobody can
prevent Armageddon.
Java is evil, using it is BLASPHEMY , only heretics and
witches like Java.And heretics and witches shall BURN !
Java = programming language written by SATAN in Hell's Kitchen in 666 minutes ! ! !

I of course didn’t write this and for the record, I have used Java for many many years and I don’t think I’m going to Hell. I posted this to prove a point – Adsense will now show ads that promote java products. This is a sample of what Adsense delivered on this site:

So the question is for advertisers – do you really want your company being linked to content such as this simply because it has a few key words in it? Is there a way to heuristically analyze content to filter this out (or even better, target the negative content for example to anti-Java advertisers?)

How To Use Microsoft Project: Task Type

2009-03-28T10:28:00.001-05:00

One of the more confusing aspects of Microsoft Project is Task Type. Task Type can be set to "Fixed Units", "Fixed Work", or "Fixed Duration" for each task.

How to Set Task Type

You can set task type in a couple ways:

1. If you select a task or tasks and go to task information (under project in the menu or right click and its there as well) you will find task type in the advanced tab.

2. If you go to window and click Split when in the GANTT chart view, you get a Task Form at the bottom where you can set the Task Type.

If you select all your tasks, you can set the Task Information for every task to the same Task Type – this will lead to less confusion when you are adjusting your plan because all the task types will be consistently set.

Effort Driven

There is also a check box (which is checked by default) that says that tasks are “Effort Driven”. When this is checked, it is assumed that work stays constant even if you adjust the other two measures. So for example, if you change a 40 hour task from 1 week duration to 2 weeks duration, project will keep the 40 hours of work but decrease the number of resources required to compensate (only 50% of a resource in this case).

This may not be what you want and I find particularly with using Fixed Duration having this turned off is highly desirable because want to adjust the work, not the units. For example, what I want is to tell project, “I want to schedule this task with 1 person for 2 weeks” and have it calculate work to be 80 hours. If I adjust the schedule to be 3 weeks, I want the work to increase by 50%. Similarly, if I am working in Fixed Duration and I double the resources, I want the duration to stay constant and for the work to double. This will only happen if “Effort Driven” is turned off.

How Task Type Works

Any task can be adjusted using three measures: Units, Duration or Work. Because each measure is dependent on the other, at least one of the other two must be impacted. By changing the task type, you are telling project which measure you are going to adjust and which measure should be automatically re-calculated. Below is a chart that shows how Project reacts based on the task type:

Field that you change	Field calculated if the task is Fixed Units	Field calculated if the task is Fixed Work	Field calculated if the task is Fixed Duration
Work	Duration is recalculated	Duration is recalculated	Units are recalculated
Duration	Work is recalculated	Units are recalculated	Work is recalculated
Units	Duration is recalculated	Duration is recalculated	Work is recalculated

Basic Tips for Using Task Type Efficiently

Here are some basic tips to using Task Type efficiently and to avoid confusion:

Pick the task type that makes sense for you and use it consistently. I tend to think in Duration with Effort Driven turned off first and what I want is to calculate the number of hours (and therefore cost) required to do the work. Other PMs like thinking in work instead and want to adjust the schedule instead. Pick an approach before you start and be consistent with it.
You can switch the task type without impacting your plan. Therefore, feel free to change the task type for the specific scenarios below.

Scenarios to use Specific Task Types

Fixed Duration

I used Fixed Duration when I have a good understanding of the schedule and the team and I want to calculate cost/effort. A good example of this is doing a plan for requirements gathering interviews. I know that I have 2 weeks of interviews, 1 week of documentation and 2 days of client review and I know that I need a Business Analyst and a Technical Architect in each one at 100%. Fixed Duration works really well because I can adjust the schedule really easily and the work effort is calculated for me.

Another good use for Fixed Duration is % allocations for overhead resources. Project Managers for example tend to be used on an overhead basis for the duration of the project on a fairly constant basis. If the project takes longer you will need more project management. It’s also easier to think of assigning a PM at 30% to a project for the entire duration than to say, “I need 85 hours of project management for this project). With Fixed Duration, you can simply assign the PM for the entire duration and adjust the % allocation and the work effort is automatically calculated.

Fixed Work

A good use of Fixed Work is when you want to try and fit a fixed amount of hours into a fixed schedule. By fixing the work, you can adjust the number of resources you need to impact the duration. For example, let’s assume you have 1280 hours of work and you want to try and jam that work into a fixed 8 week period. With Fixed Work turned on, you can set the allocation of resources to 400% in order fit the fixed amount of work into a duration you wish.

Fixed Units

I generally find that using Fixed Units is my least desirable option simply because in most real world scenarios adjusting the number of resources required is the most flexible option.

One scenario where Fixed Units might be appropriate is for department planning. In a department, the number of fixed units is constant (the number of full time employees) and as a Director you want to know how much work your team can produce. What tends to happen in these scenarios is the queue of work is constant and the duration flexes – you end up telling the poor person at the end of the priority list that their project won’t get done until next year.

In this case, setting Fixed Units would allow you to lock in the resource pool based on the size of your department and then adjust the work to calculate the resulting schedule. Based on the schedule, you would then be able to tell your stakeholders how many projects can be done in the coming year.

I hope that provides some guidance on how to use this key function in Microsoft Project – I have found even very senior Project Managers do not fundamentally understand how Duration, Effort and Units work together and how to model them in Microsoft Project. If you understand the conceptual framework and the rules that Microsoft Project uses, you will find that there are less errors in your plan, less confusion and less tearing your hair out when you find Project adjusts your numbers in ways you didn’t intend.

Check Out the New Live Writer

2009-03-22T21:16:00.001-05:00

As part of the latest Microsoft Instant Messenger update, there is a Live Writer tool now included as part of the Windows live toolset.

It’s free and having now written a couple posts with it, I can attest that it’s quite good:

It has a single interface for editing blog posts but can publish to multiple sources, e.g. Blogger, Wordpress, Typepad, etc.
Editing using a real client is much easier than the Blogger HTML content editor. I always found it frustrating to use it because it created some odd HTML choices and formatting mistakes.
Spell check uses the word spell check and it highlights your spelling errors immediately.
You can save a local draft so if you editing offline you can save your post and publish later.
Images and videos are inserted locally and only published at the end. This makes it faster to create your post, fiddle with the images, add or remove them, etc. because you don’t have to do the upload until your post is finalized.
The tool has the ability to insert a windows live map.
There are a number of plugins available online for insert other types of objects such as flickr pictures, silverlight streaming applications, files, facebook photos, etc. The plugins insert the appropriate HTML into your blog posts for each type.

This will definitely make my life easier as a blog writer…thanks Microsoft!

How to Use The Fill Handle in Excel

2009-03-22T21:05:00.001-05:00

One of the key features in Microsoft Excel is its ability to fill data based on existing values. Excel is smart enough to extrapolate and auto-fill values based on your existing ones.

Click on the black box in the corner of any cell and drag. Excel will fill the cells automatically. The fill handler works both horizontally and vertically.

Here are some tips on how to use the fill handle for values:

Type in a month in either full form (e.g. January) or abbreviated form (e.g. Jan) and drag and Excel will fill in the subsequent months.
Type in a day of the week in either full name (e.g. Monday) or abbreviated form (e.g. Mon) and fill. The filled cells will contain the correct days of the week in sequence.
Type in any value into a cell (e.g. 1, 2, etc.) and when you fill in with the same value.
Type in a sequence of values in cells, e.g. 1 in B2, 2 in B3, etc. and select multiple cells and then drag the fill handler. The cells will be filled in with the sequence. This works just as well with intervals such as 2, 4, 6 etc. as well as 1,2,3.
If you hold the control key when dragging the fill handle, you will see a plus (+) sign and the value selected will increment by 1 in each cell.
If you right click on the fill handle, it pops up a dialogue where you can explicitly instruct excel how to fill the data.

The Fill Handle can also be used to fill formulas. Here is an example:

The product in c2 is a formula =A2*b2 and the sum is a formula A2+b2. Now let’s assume that you have a series of values and you want to have the same formula repeated:

If you select the cell with the existing formula you can use the fill handle to automatically fill down the right formula for the other rows. It automatically figures out that as you move down rows it should adjust the formula so that A2 * b2 becomes A3 * B3, C4 * B4 and so on.

There is also another way to fill in the cells. If you select the cell you want to copy and then select a region and paste, it will fill the values in the same way. It again automatically adjusts the formula so that it makes sense based on the new row that it now occupies. This works both horizontally and vertically.

One last tip on filling – there are cases where you will want the value in your formula to not change as it moves from row to row. Take this example:

In this case, if you simply do a standard fill in Excel this is the result:

Why did this not work properly? The reason is that Excel assumed that B1 * B3 should be B2 * B4 in the second row and B3 * B5 in the third row. However, what we wanted is for the first number to stay constant and only the second row to change.

The solution to fixing constants is to name them. Named values in Excel do not change when filling. To name a cell, click on the cell and then in the left box type in a name for the cell.

Then when you create the formula, use the name instead of the cell address, e.g. instead of B1 * B3 we will now set the formula to InterestRate * B3. By using a named cell, Excel assumes these values are constants in formulas and doesn’t adjust them automatically.

Proposed ISP Tax Shows Canadian Content Producers Do Not Understand the Internet

2009-02-18T18:33:00.004-05:00

Decades ago in Canada, funds were created to encourage television production of Canadian Content. These programs were funded by the cable companies and regulated by the CRTC. These funds were created to share "Canadian Stories" in a sea of American television content that because of their inherent scale could be produced easier and cheaper.

The CRTC is now entertaining a proposal to create a similar fund through taxation of ISPs (and therefore all Internet subscribers) to fund Canadian television producers so they can continue to produce "Canadian Stories" created by Professional Content Creators. This proposal is supported by industry lobby groups such as ACTRA, Society of Composers, SOCAN, etc.

These folks want to tax every Internet subscriber (e.g. practically everyone in Canada) 3% of revenue (which of course means the price will go up by 3% to compensate) from the ISPs that provide us Internet Access.
This proposal shows a complete misunderstanding of the difference between a regulated, limited, and centralized Television Industrial Complex model and the Internet - a fundamentally distributed, decentralized and long tail model of content distribution.

Here are some fundamental differences that these lobbyists fail to acknowledge in their proposal:

In the television age, the definition of "Content" was professional created, broadcast content. The assumption in this proposal is that Colin Mochrie, the CBC, and other CanCon stars own the culture and the MILLIONS of content creators on Facebook, YouTube, etc. in Canada are not qualified to tell their own stories.
In the television age, the definition of "Content" is VIDEO content. On the Internet, there is no limitation and content can be everything from maps, images, videos, flash, applications, etc. Content can also be a mix from a wide variety of sources - how do you regulate content creation when it is mashed up collections from all over the world?
In the television age, there were major barriers to content creators because of the limited shelf space. Canadians need a way to get on the air simply because they couldn't compete with American content producers. But on the Internet, this model makes no sense because anyone can put up a web site, the costs are cheap and the number of channels are unlimited. What is the barrier to making money on the Internet other that POPULARITY? There is a simple way for Canadians to support their own culture - visit the web site and support it through ad revenue like everyone else does on the Internet.
In the television age, exporting content to other countries was extremely limited because the distribution channels were controlled by cable channels in the country. The Internet has no such barriers - if Canadians want to share their stories they can gain an audience world wide without any restriction.- In the television age, there is no integration of "Application" and "Content". Monetization of content was through a single model - distribution broadcast supported through advertising. In the Internet, the two are mixed together and there are many monetization models available. The monetization mechanisms are also distributed, e.g. I can be creating "Canadian" content that is monetized through Google AdSense in the United States. The selling, re-packaging and syndication of content through dozens of channels is both an opportunity and a challenge for Canadian content producers to claim for themselves. But the solution isn't to bypass these channels and simply tax the consumer - this is just a return to the simplified television monetization model.
In the television age, the justification for this fund was that in order to compete with American content producers there needed to be funds to create a level playing field and encourage the telling of "Canadian Stories" because the distribution channel for content was limited both in capacity (number of channels) and revenue potential (only the top 10-20 television shows make the money). However, the Internet market is completely the opposite - the market is a long-tail, unlimited distribution channel and there are lots of examples where relatively small content distributors are making money by targeting micro-niche audiences by keeping their costs relatively low and providing a highly valuable target audience for advertisers or by providing content through subscription that funds the cost.

The CRTC is in the business of regulating traditional centralized content broadcasting and is therefore bought into the fundamental model assumptions of the television age. Canadian Television Producers who are in favour of this model seem to be wanting to keep their existing model and apply it to the Internet, because if they simply wanted to make money from content there are easier more direct Internet models available that do not require a tax on consumers. The Canadian Producers advocating for this proposal and the CRTC if it accepts it simply are trying to implement the old television age model and apply to the new Internet content model so that these folks do not have to migrate to a new way of creating, distributing, monetizing content. In addition, the entire proposal continues to question the fundamental value of funding "Professional Content Producers" as the bastion of Canadian culture when there are potentially millions of Canadians who are creating "Canadian Stories" themselves on a daily basis through blogs, Facebook, etc. Do we really need Gordon Lightfood, Colin Mochrie, the CBC, etc. to have a monopoly on culture creation when anyone in Canada can easily tell their own stories?

Kincardine Cable TV Community Channel

2009-02-14T10:24:00.008-05:00

I was up in Kincardine and had a major nostalgia moment. They still had the low resolution cable access announcements feed going - I haven't seen one of these in about 20 years!

I particularly love the last picture - low resolution announcement board has links to web sites in it....

For those who are young enough never to have seen one of these, here are some details:

The reason for the garish colours is back in the mid 80's, these systems were limited to about 4-16 colours depending on the system used. There are only so many look and feel options when your choice of colours is BRIGHT BLUE, BRIGHT RED, OBNOXIOUS GREEN, BLACK AND WHITE.
You'll notice that all the letters are capitalized. This was pretty common in early computer systems - the resolution was so low that trying to display lower case letters made them hard to read.
There was only one choice of font and it was always that big blocky font that you see here.

Maybe we should along with the rural broadband fund establish a government program to upgrade small towns cable access feeds so they can have access to those fancy TV guide scrolling ad versions we have in the big cities?

Using Technology to Avoid Stealing Songs By Mistake

2009-01-15T21:05:00.003-05:00

In 1960, The Chiffons published He's So Fine:

In 1970, George Harrison published the song My Sweet Lord:

In 1976, a lawsuit ended with Harrison having to pay over $500K in damages to the original song's publisher for subconsciously plagiarizing (ultimately, Harrison got the last laugh by ultimately buying the publisher).

Just recently, Joe Satriani has taken Coldplay to court for the same infringement for allegedly copying his original song If I could Fly.

So my question is, could there not be a technology solution to this problem? With online radio sites having matching algorithms based on wave analysis, chord progressions, etc. could there not be similar analysis done to create a checksum hash for every published song that if matched against a new song would be deemed too similar?

It would be a lot easier than relying on someone's subconscious, just purely random coincidence or some judge's interpretation of what music is to award royalties.

Or even better yet, we would go back to the days of Mozart, Bach and Vivaldi when musicians copied each other liberally and were able to advance their art without fear of reprisals from copyright lawyers.

Applying the Software Development Life Cycle in a Web 2.0 Context

2009-01-01T18:53:00.004-05:00

For the past 20 years or so, the "Software Development Life Cycle" has been the process for defining, designing, constructing, and deploying applications. Various methodologies for building software have evolved over time, e.g. Waterfall methods, spiral method, rational unified process, extreme programming, etc. that prescribe different processes for development of software. However, all of them regardless of methodology have some common basic assumptions that I believe have now changed in a Web 2.0 world. These basic assumptions are as follows:

Software development involves engineering code artifacts that implement some sort of requirements specification. In waterfall methods, the requirements are defined up front. In more iterative or agile methods the requirements are iterated through cycles. But in all these methodologies, the primary purpose of developing software is custom developing software based on these requirements.
The "Application" is a deployable, atomic set of artifacts that can be deployed through a series of steps to a particular environment. The "Application" is used by a client that engages with the application. Traditionally, the user of the application does not get to change or redefine the application except through very rigidly defined configuration mechanisms.
Software development historically has required a large amount of testing in order to ensure that a) there are no bugs in the software and b) the software complies with requirements. Software is tested before it is released to users.
Software development is done by a team of experts that understand how the software works, how it can be modified and how to manage the overall development cycle. Software developers and end-users of the software do not really collaborate on building the software - end-users consume the software as it is released.

In the media, the old traditional form of Push media is being replaced by user generated content, collaboration, and Pull from users. The distinction between producers and consumers has been blurred so that online news hubs become more like information radiators than centralized distribution points.

Software development, in my opinion, is going through a similar revolution to the mainstream media. It is happening in a slower fashion because the software "industrial complex" is still highly engaged in generating software through the traditional means and because it generally works especially for large enterprise applications. However, in certain spaces and where consumers of software are engaged, the traditional model of a software development team pushing out software that is consumed by a user is starting to break down.

In order to make my point, let me give you a few examples:

Software applications can be created via "Mash-ups" that allow end-users to combine existing application components into new applications.
Portal applications such as Microsoft SharePoint are blurring the distinction between application components and user modifiable objects. For example, a Document Library in SharePoint part of an application or is it a user object?
Cloud based services are now being released and combined in a variety of different methods.
Clients such as Microsoft Office 2007 are now sophisticated enough to serve as application development platforms in themselves. For example, a traditional "reporting application" can now be built quite easily using Excel 2007 pointing at a data warehouse.

In addition, users expectations have changed dramatically has Web 2.0 has encouraged them to collaborate, blog, share information and build their own customized spaces. When a 12 year old can over night create their own blog, organize a protest online, share pictures, integrate Google Maps, and add dozens of widgets to their personalized home page, the idea of waiting 3 months for the next software "release" by an ivory tower software team quaint at best and non-responsive at worst.

In the corporate world, IT departments are struggling with this challenge in application spaces that encourage collaboration and customization with end-users. Business users do not want to wait 3 months for a static report - they want a drill down BI tool that allows them to build their own "application" (whatever that now means). In the portal world, IT departments are struggling with how much control they provide their end-users to redefine the underlying structure of meta-data, document types, user profiles, etc. In the SharePoint world for example, its difficult to tell where the "application" stops and user generated content begins because the platform is designed to encourage modification and collaboration by office workers.

On the other hand, the risks and challenges that the software development life cycle has helped solve over the past 20 years have not disappeared because we have moved from a code centric, monolithic, and IT controlled software development cycle to a collaborative, distributed and user centric model:

How are changes "deployed" when an office user can simply make a change in production to a content type, a work flow, a user profile definition?
How are the impacts of changes "tested"? For example, I can change the layout of my blog, add widgets to it, etc. but can I really test those changes before they become visible to the world (in my blog software, I have the ability to "preview" which is sort of a test but not quite the same as a proper QA environment)?
How are application componentized when changes are done by potentially hundreds of users working collaboratively? What constitutes a "release"? For example, on Facebook, I create a Facebook Page that has a set of content, some groups, some events, etc. but there is no way to package these changes up into a deployable unit.
How are changes versioned and backed-up? I have this problem with my blog for example. If I make changes to my blog "application" through changing the layout, modifying the HTML, etc. there is no easy way for me to preserve these changes as a version. I cannot roll-back to a previous version either.

Software development methodologies have good answers to these challenges and over the past 20 years have matured to address these issues through test driven development, source code control, build management, project management, quality assurance, etc.

As an industry, we have a similar challenge to the mainstream media - how do we adapt these worth while processes so that we can continue to deliver high quality software to our customers while adapting to an environment where our users are going to be creating software with us in a highly collaborative manner? How do we recreate the software development life cycle for a Web 2.0 world?

Key IT Challenges for 2009

2008-12-30T19:55:00.004-05:00

Here are some of the key IT challenges I see for 2009 based on feedback from our customers, the media, and my own understanding of key technologies coming on the scene. In addition, these challenges reflect the broader economy which is going to go through a global recession in 2009, impacting IT budgets in a number of different ways (see my previous post on this issue).

Challenge #1: Maximizing Existing IT Investments

In the past 2 years, there have been a significant amount of investments made in new technology platforms that were supposed to empower office workers to higher productivity and less dependance on IT. Products like content management systems, portals, data warehouses, analytics, CRM, ERP, etc. all have become increasingly mainstream over the past couple years as the economy boomed and CIOs were looking to make long term investments.

This year, in this time of restraint, I see a key challenge for all IT departments in maximizing existing investments that were made over the past two years. For example, how many companies have invested in SharePoint and are only using it to power their intranet? I have many customers who are looking to see how can they can justify their SharePoint platform investment through more advanced applications. For consultants, this is an opportunity to sell more vertically integrated, business centric solutions and move away from selling the latest technology and this will continue for at least another year.

Challenge #2: The Proliferation of Virtual Machines

The virtual machine as a server platform has really hit the IT mainstream. Virtualization has been around for years but thanks to a very mature platform with VMWare ESX and an upcoming push from Microsoft Hyper-V, the use of virtual machines is expanding rapidly.

Virtual servers have started replacing physical ones, first with non-core systems such as development servers to now in some cases full production environments.

One of the key challenges with Virtualization is that because it becomes so easy to spin up a new server, the number of servers proliferates, leading to VM sprawl. Management of virtual servers doesn't easily map from the physical world especially when doing tasks such as asset management (how do you manage a virtual asset). IT managers will spend a lot of time in 2009 coming to terms with these challenges.

Challenge #3: Web 2.0 Goes Mainstream

As technologies go mainstream, the language, terms and concepts that were reserved for leading edge technologists land in the hands of business managers, stakeholders and consumers. Unfortunately, they get them typically without the deep technical or even conceptual understanding.

This happened a couple years ago with the concept of "Portals". Everyone wanted a portal. Portals went from being these esoteric technologies to jargon words in marketing magazines. The same is happening now with concepts like "Blogs", "Wikis", "Web 2.0", etc.

I'm now receiving RFPs from government agencies with requests to "use Web 2.0 technologies" with no rationale, purpose or objectives. Some manager has now heard of Facebook and so now wants it on their web site. As technology consultants, the challenge will be seperating the myth from the fact, selling the real effort it takes to use these technologies effectively and trying to guide stakeholders to realize relevance from what has now become a Fad.

Challenge #4: Storage

Even a few years ago, it was a big deal to have a Storage Area Network (SAN). SANs were reserved for large organizations with lots of data. With the proliferation of media files, volumes of emails and documents, and no discipline when it comes to archiving in most organizations, managing the ever increasing demand for more storage is becoming a major headache. With hard drives getting increasingly cheaper the perception that storage is nearly free is pervasive (I can go buy a terrabyte backup drive now at BestBuy for about $100). Unfortunately, corporate storage is nowhere near as cheap when the requirement includes high performance, highly redundant and low power consuming. For example, at my last organization we spent almost $50,000 implementing a SAN in order to gain faster I/O access for our core databases. The SAN has 28 drives in it, is highly redundant, etc. but even with its several terrabytes of storage it will fill up rather quickly if not managed well.

Challenge #5: Search and Document Management

I mean search here in the broadest sense - finding stuff. Go try and find a document on your corporate intranet. Try and locate a file on your file server. Try and understand the taxonomy or meta-data structure for your document repository (assuming your organization has even defined one).

Creating documents is now so easy that they have proliferated. In addition, there is typically in most organizations very poor meta-data strategies for indexing data and immature archiving strategies to remove old documents. For example, at my last position we had a file server with about 4 terrabytes of documents. We did a scan of the thousands of documents on that server and found that over 80% had not been touched in the past 2 years. Even if they are not being accessed they act as an information clog when trying to find important documents.

Document management used to require very expensive and proprietary software such as OpenText, Documentum, HummingBird, etc. and was reserved for companies such as law firms who had very regulated and structured document repository. But with lower cost tools such as SharePoint you can quite easily now implement some decent meta-data taxonomies and start to catalogue, version control and archive documents in a methodical way.

The biggest challenge isn't the technology - its getting commitment from the business to organize the thousands of old documents and migrate them into the new structure. This is a massive effort and requires significant involvement from the business. While there are some tools to help with this effort, the clean-up effort of document migration is still a continuing challenge.

Online Trends for 2009

2008-12-28T14:24:00.005-05:00

Here are some online trends to look out for in 2009.

Advertising Takes a Beating

With the sluggishness of the US economy, advertising will decline significantly and online advertising in particular will be hit hard. Online advertising will still continue to grow but at a much slower pace.

This slowdown in growth will put pressure on web 2.0 applications with lots of content but lower conversion rates. Applications such as Facebook, Flickr, YouTube, etc. will need to focus more heavily on monetizing all those collaboration interactions that millions of people now do for free every day.

There will also be a continued emphasis on Cost Per Action instead of Cost Per Click to link advertised costs to conversion. As the cost of online advertising increases (for example the cost of buying keywords) there will be increased pressure on Google and other advertising platforms to show direct linkage between advertising and conversion to revenue. This is where online advertising as a channel has had an advantage over traditional media channels.

Internet TV is Finally a Reality

The "watch video on your computer" revolution started about 10 years ago, but until this year watching the equivalent of television via the Internet really wasn't possible. Instead, users have sacrificed the video quality that even low definition television provides for increased collaboration, crowd created content, etc. In addition, the selection for mainstream content has been limited online until the last year or so.

In 2009, the compromise on quality and selection will fade away. Video sites such as YouTube, Facebook, etc. are all upgrading their video streaming so that the videos uploaded by users are streamed at much higher resolution. In addition, sites such as Hulu now stream hundreds of mainstream television programs in almost broadcast quality. While not able to yet compete with HDTV, video on the Internet will now be viable as a full-screen, sit in the living room type experience.

Part of the challenge has been the source video - consumer digital cameras tended to be lower resolution video than television. However, the latest digital cameras produce very good video quality including my Panasonic camera that can shoot HD video in wide screen format.

Web 2.0 will Become Mainstream

Don Tapscott's new book on the Net Generation points out that when technology becomes boring and we stop talking about it then the next level of innovation is possible. Web 2.0 will simply become The Internet as usual and we'll stop talking about it as this new found technology revolution (until we think of the next one of course).

Applications like YouTube, Facebook, etc. are now so mainstream that your mother uses them. Applications such as Facebook, MySpace, YouTube, etc. have all already demographically shifted to the mainstream from being in the hands of the power users, the young and the affluent.

Mobile Applications Will See Massive Growth

With the launch of the IPhone and its inevitable competitors, the phone now becomes a viable application platform. IPhone is as significant for mobile applications as IPod was for downloadable music.

Another mobile device seeing large sales is the Amazon Kindle. It allows users to purchase books and view them in a mobile device. The idea of e-books has been around for over 10 years but it takes a device like the Kindle with its complete vertical integration to drive the adoption.

Geotagging and Live/Google Earth

As the amount of content increases, one of the simplest ways to filter content is through geography. In addition, with the maturity and increased use of mapping interfaces such as Live Earth, Google Earth, Google Maps, etc. both as destination sites and as mash-ups, geotagging of content, pictures, videos, and people will be a vital way of filtering content in 2009.

One of the simplest examples is the integration of Panaramio with Google Maps. In Google Maps if you look up any address it will show you related photos from Panaramio, all geotagged on upload by location.

Change of Strategy From The RIAA/MPAA

For the past several years, the global culture industry has been taking people to court over copyright infringement. Approximately 30,000 people have been sued in the US for copyright infringement over the past 5 years. The impact on file sharing has been no real impact on file sharing but has made the RIAA one of the most hated organizations worldwide.

It appears that after 5 years of harassment law suits, the RIAA has decided to change its strategy and use a softer approach. In addition, new technology and business partnerships will allow the media conglomerates to start making money off their content no matter who uploads it. For example, Universal now stands to make over $100 million in ad revenue through a deal inked with YouTube that uses search terms to identify user contributed music videos that map to Universal's artists.

Casual Entertainment Over Hard Core Gaming

The success of the Nintendo Wii (sold out again this Christmas!) as well as other casual games such as the Sims and Spore will continue to provide online entertainment for the masses. Collaboration, casual games and customization will continue to push the mainstream entertainment market. Games like RockStar, Wii Fit, etc. will continue to provide mainstream entertainment while leaving more hard core games to cater to the hard core 18-30 male demographic.

See the above note as well on platforms like IPhone and Facebook - these are perfect jumping points for casual gaming as they cater to an audience that might play a game of solitaire but isn't going to be spending hours with Call of Duty.

The casual gaming market is still in massive growth mode and this will continue in 2009.

Difference, Story Telling and Community

This isn't a trend, but it will continue to be true in 2009 as it always has been: story telling, difference and community building continue to drive the content creation process.

The revolutionary part of the Internet is the ease at which these tools can be employed for someone to tell a story, create some innovative content and enable a community. The growth of blogs shows that everyone has something to say and the size of the audience it generates is how well it connects to its community of interest. Determining "success" depends on the objective - it may simply be distributing family pictures or it may be making your opinion heard by millions as Barak Obama did in 2008.

The Key Challenge in 2009: Conversion

I have a friend who is very passionate about informing parents about Toy Recalls. She has an active blog, she uses Twitter and Facebook and she is able to get her message out to several hundred viewers a day. When we discussed her "success criteria", she seemed pleased that she had grown this community from zero but her next question was, "ok so now what?". This is I think the key question for all this collaboration activity going on - how do we convert the energy of all this discussion, file sharing, photo sharing, tagging, blogging, etc. into some sort of concrete action so that its simply not just hanging out (although that is not a bad goal in itself).

It's as if we have got people off the couch and started them talking again. So now the community is sharing, talking, collaborating. So now what happens? How do the tools available provide methods to convert this community to action?

This is the future growth opportunity for application platforms like Facebook and you can see the evolution of these tools going in this direction already - applications like the Causes application for example are a first attempt to translate a community of action into a concrete action (in this case donating money to charity).

Corporate IT Departments Trends for 2009

2008-12-28T10:58:00.005-05:00

Corporate IT departments are, at the best of times, the internal consulting organization for their organizations and provide strategic technology services to help propel the overall success of the business. The role and objectives of the corporate IT department will change in 2009 as the economy continues to lag, technologies mature and organizations that saw "Web 2.0" as bleeding edge now start to adopt some of more mainstream applications of online technologies.

Here are some of the key trends I see in 2009 for the corporate IT department. For consultants such as myself, this will have a major influence on how we can continue to provide high value to our internal IT partners by providing services and solutions that contribute to the value proposition for the corporate IT department in 2009.

Corporate IT Departments Will Lag Behind in Upgrades

With budgets tightening, corporate IT departments will take the hit in upgrades for both hardware and software. IT departments will be asked to do more with less and those departments who may have planned on an Office 2007 upgrade, a SharePoint 2003 to 2007 conversion project, a .NET framework 1.1 to 2.0/3.0/3.5 upgrade project, server refreshes, etc. will be asked to defer them to save costs.

Scrutiny on Capital and Cash Flow

Typically, large scale expenditures can be amortized over a 3-5 year period. For example, if a Director IT needs to purchase $500K in new servers, they can typically budget these over a 3-5 year amortization period so they don't get hit with the entire cost in year one. However, from a cash flow perspective, the money still gets spent and in cases where cash is tight (e.g. credit crunches) these types of cash outlays will be dramatically curtailed.

As a result, this will lead to arrangements with providers where costs can be paid on a monthly basis, e.g. leases, software as a service, outsourcing, financing, fee for service, etc. where cash flows out of the organization in a more even way than a large capital expenditure.

Long Term Consulting Under Attack, Short Term Consulting Will Increase

As budgets tighten, the following tends to happen during the budget planning cycle:

Full time hires are deferred
High cost consultants are targeted
Projects have to show very quick ROI

If you are a $200 / hr consultant that has been working on a project for 2 years and the project will continue for another 2 years, then expect that cost to be reviewed and for department managers to try and squeeze out consultants and replace them with cheaper internal resources.

However, because full-time hires become blocked, the only way to get new work done in the short term tends to be through direct consulting costs. In addition, because the business does not have long term visibility they will be hesitant to take on a full-time resource as a long term commitment. If the business stakeholder can justify the ROI in the short term, they will tend to outsource the work simply because the Corporate IT department cannot hire any full time resources to do the work. In addition, outsourcing the work as a fixed cost to a consulting firm is less risk and the cost is directly attached to the cost of the project, making it more tangible for a business stakeholder than a resource hire.

Incremental Optimization

For both revenue and profit, the goal for 2009 will be to optimize as much as possible. For example, technologies such as SEO, analytics, data mining, reporting, CRM (especially for those who already have an existing platform), channel management, marketing optimization, etc. will sell very well to organizations who have over the past 2-3 years have been growing their customer base but now need to focus on retention, optimization of marketing dollars and efficiency. Technologies that can help do this will continue to get the green light as they produce ROI faster than brand new initiatives and they are less risk for a CIO who is under budget pressure.

Layoffs for Many, Indispensable Positions for a Few

In tough times, there are typically layoffs. Depending on the speed of action required, layoffs are many times done without proper planning and project/application coverage (the worst case is where you see companies having to hire back the people they just laid off on contract!). For those who are laid off, it means that they are now looking for a job. But for those who are left standing, they can find themselves now indespensable where there used to be three people who knew how to fix an application and now there is only one. In some cases, the one left is a consultant!

For those who are in this position, it provides leverage to those left behind. If you are the only developer who now has access to source code and can run a build, you were kept because they couldn't afford to dump you. If you can stretch to grab the work that was being done by other developers and demonstrate the flexibility and tenacity to help clean up the mess inevitably left behind by these changes then this will be looked at favourably by the company.

For consultants, there are opportunities to fill the gaps left behind in the rush to lay off staff through short term and long term consulting arrangements, especially if the consultant has had previous experience with the applications in the organization. There are also opportunities to pitch alternatives to lay-offs without adequate transition through outsourcing arrangements. A service company can provide flexibility in resource ramp-up or ramp down that is difficult for an internal department to manage especially in a time of crisis. In addition, a consulting company has the advantage of being able to work in stealth, allowing for a Corporate IT Director for example to quietly outsource key functions to an outsource provider while planning a lay off so as to avoid disruption and continue service levels.

Opportunity for Entrepreneurialism

There is an opportunity for IT departments to re-invigorate through a focus on entrepreneurialism. For those IT leaders who can show the business a clear plan to short term revenue, increased profit, etc. this will tend to get more attention than more themes such as scalability, efficiency, etc. Entrepreneurs focus on delivering high value services on a shoe-string budget - this is what is required in an economic down turn.

Business stakeholders will also be more apt to listen when they are also hungrier for more revenue. For example, many sales businesses will find their traditional sources of revenue tapped out and will be looking around for new revenue sources. If the Corporate IT Department has some creativity and can partner effectively, they can help provide some great new ideas for using Web 2.0 technologies, creating online campaigns, optimizing CRM data, using data mining techniques, etc. to squeeze more revenue out of existing channels or to open up new ones.

For consultants, this is the ONLY way to sell business in 2009 - either demonstrate how to squeeze more revenue/profit out of existing channels or show how a very small investment in a new channel can lead to large growth potential. Be prepared to take risks and be entrepreneurial with your customers in how they pay for your services, how much risk you take on, etc. and this will also be attractive to customers who are hesitant to spend money.

Focus on Business Solutions that Make Revenue or Profit, Not Technology

This is always the case with business stakeholders, but in times of restraint this will be even more pronounced: Corporate IT Departments who are focused on generating concrete business centric solutions that create revenue or profit opportunities will be prioritized.

If you are a Corporate IT Department or one of its service providers and your focus is heavy technology buzzword projects like: SOA, Cloud Computing, Web 2.0, Architecture, Software Development Life Cycle, Enterprise Data Warehouse, ERP, etc. then you will be in trouble in 2009. For example, at one of the banks here in Canada for example, I know that their list of thirty major IT has been cut to less than ten for 2009.

Projects where existing horizontal technologies in place are now leveraged to create additional value will have significantly more priority. For example, we have many customers who have previously bought Microsoft SharePoint 2007 but aren't leveraging it effectively or maximizing its use. Incremental investments in adding verticalized and targeted applications on top of this already existing investment will show quicker pay off than investments in additional infrastructure technologies.

Business Intelligence is More Than Just a Fancy Graph

2008-12-21T16:32:00.004-05:00

"Business Intelligence" is a hot topic these days. There are companies that produce business intelligence products such as Microsoft, Business Objects, Dundas Charts, and so on that focus on visualization of data with the approach that by providing a simplified view of complex data it will enable key stakeholders to make decisions.

A good example is Dundas Charts - the company produces an amazing set of widgets and controls for building dashboards, charts, and visualization of data.

The price is right too - you can buy these fancy BI controls for a license of about $1000.

However, in my experience with Business Intelligence applications, I have come to the following basic conclusion:

Visualizing data is relatively easy.
Obtaining the data in an enterprise environment is hard.
Defining what the data means is even harder.

Let me give you an example. Wait Times are calculated in Ontario and presented to the public in order to show accountability for the funds being invested into improving access to surgeries.

Here is what the data looks like:

As you can see, the indicators are quite simple. The data could have been visualized in a number of different ways such as a red-yellow-green style KPI, a guage, charts, etc. Implementing this would simply involve taking a control library like Dundas Charts and feeding it the above data to get a graphical representation. However, visualization is actually not that valuable - the average person does not need a graph to understand a basic set of numbers.

Obtaining this particular set of data is hard. It requires the synthesis of raw data coming in through automated feeds from about 150 hospitals into a central database that is then scrubbed to match patient records and wait times records together. The cost of building this application to collect the data was millions of dollars - the cost of producing the PDF file containing the numbers is significantly less.

Defining the original business rules, definitions and targets for how wait times data was to be collected was even harder. What does a "Wait" actually mean? There are in Ontario two different wait periods (called Wait One and Wait Two). Wait One is defined as the time it takes for you to get the appointment with the doctor who provide the diagnosis. Wait Two is defined as the time it takes between the point where the doctor provides the diagnosis and the point where you get the procedure. This is what the current Wait Times application tracks - Wait Two data. Different provinces have different definitions and track different wait intervals. In addition, you will notice that only certain types of procedures are represented. These were chosen through government priority - another period of business analysis that took several years to define, prioritize and fund. Similarly, the "targets" were defined through another complex process using clinical experts who invested significant time to analyze and provide recommendations on what was the clinically appropriate targets.

If you are looking at a report, a set of numbers, etc. you are looking at the tip of the iceberg - the amount of energy, work and thought that goes into defining the data and then obtaining it is enormous in comparison to building a fancy graph. In the case of Wait Times, producing the report now is done by running automated processes. The cost of the underlying system to obtain the data cost millions of dollars and took years to implement.

So the next time you look at a report, appreciate the amount of work that has gone into it. And as an IT professional, the value to the customer is in helping them to define the data and then obtain it from a variety of data sources. Produce the actual report is simple in comparison.

Top 10 List of Bad Movie Computer Moments

2008-12-03T20:32:00.003-05:00

I watched Eagle Eye over the weekend, and it was fun but just terrible in its use of the old boogie man plot device of the computer taking over the world. In appreciation, here are some of my all time favourite bad computer movie moments.

1. Eagle Eye (2008): an all powerfull, all seeing surveillance computer tries to take out the entire US government but can be defeated by poking it in the eye with an metal rod.

2. The Net (1995): Sandra Bullock plays a notorious hacker who throughout the entire movie mispronounces the word MODEM (as if it rhymes with bottom) and uses a Macintosh and a drag and drop interface to hack computers! If only it were that simple.

3. Jurassic Park (1993): the computer that controls the park is apparently UNIX (identified by the super genius girl who apparently learned it in grade school). She then proceeds to operate it using this 3D flying interface!

Jurassic Park - Unix System Scene
by unixguru

4. Independence Day (1996): Jeff Goldblum and Will Smith design a computer virus that can infect an ALIEN COMPUTER via TCP/IP using a MacBook! Apparently alien invaders use the Internet but without the McAfee subscription.

5. WarGames (1983): Matthew Broderick and Ally Sheedy hack into the government using a 300 baud modem (with accoustic coupler for visual impact). The super computer is a room full of tape drives and flashing lights that is defeated by forcing it to play chess until it figures out the concept of a stalemate. "The only way to win is not to play".

6. Mission Impossible (1996): Standard computer animations like file uploads, logins, etc. are really boring to look at. In Mission Impossible, a standard macbook was used but the OS was suped up with some fancy animations. Why have a progress bar when you can have the screen whiz around and scream at you (even though its supposed to be top secret)?

7. You've Got Mail (1998): Email clients are also very boring. Who wants to watch someone reading email. This movie is a great example of how special effects people take great liberty to make computer applications much more interesting with animations, sound, etc.

8. Star Wars (1977): The computers in Star Wars are not bad actually, although the amount of blinking lights in the room that the Droids hang out in for most of the movie are a bit much. The demo on how to destroy the the death star is quite primitive (they look a bit like the original asteroids video game) but at least looks somewhat plausible.

9. Golden Eye (1995): Thanks to some product placement, all the computers are running OS 2/Warp. If the Russians were running Windows, maybe they could have had a chance against 007.

10. Swordfish (2001): Hugh Jackman is a hacker who puts together viruses using some sort of 3D visualization software. The one thing he does get right is the repeated swearing at his computer while he works.

Best Practices for Implementing a Learning Management System

2008-11-20T13:57:00.005-05:00

I have just co-written a white paper that is now available through Brandon Hall.

Here is the description of the paper - click the link if you would like to purchase it or stay tuned to my blog for further ideas and writings.

Description

Imagine you've been given the job of researching and finding a suitable learning management system (LMS) for your company. Where do you start?

Everyone knows that selecting the wrong LMS can have terrible financial repercussions and hurt the quality of your learning initiatives. What is often forgotten, though, is that selecting the RIGHT learning management system but implementing it poorly can be equally painful.

This report, written by Gary Woodill, Ed.D., director of Research and Analysis at Brandon Hall Research; David Fell, vice president of Business Development at Operitel Corporation; and Christopher Woodill, Vice President, Solutions and Strategy at Navantis Corporation, is the first in a four-part series on the process of implementing an enterprise LMS.

A typical LMS can take months, if not a year or more, to implement and can cost hundreds of thousands of dollars. Getting it right is critical; getting it wrong can be a problem for both you and your business. Essentially, the process involves four stages:

Stage 1: Planning and Evaluating Business Needs for an Enterprise LMS
Stage 2: Purchasing an Enterprise LMS
Stage 3: Implementing the Selected LMS
Stage 4: Maintaining, Supporting, Operating, and Governing an LMS

This report focuses on Stage 1 and addresses the following:

Developing a business case for investing in a learning management system
Considering alternatives to an LMS
Articulating a vision and scope for the entire project, resulting in developing a “project charter”
Developing an implementation strategy and project management plan that includes presenting the business case, assessing the LMS's impact on the organization, and changing management strategy
Developing the communications and marketing plan for the project
Identifying all stakeholders for an enterprise LMS
Conducting a requirements gathering exercise, including developing “use cases” for each business unit and group of stakeholders

If you have thoroughly carried out the above suggestions, then you are ready to prepare and send out an RFP, investigate products and vendors, and move toward purchasing a learning management system. It’s also important to note that getting this first stage right provides a firm foundation for success in all the later stages.

Some thoughts on the Zachman Framework

2008-11-19T20:07:00.003-05:00

I have been doing some reading on the Zachman Framework in the context of some work we are doing with the government. The ZF is popular in government circles as an enterprise architecture framework and they routinely ask for experience using it within their RFPs, especially for roles such as technical architect or business analyst. Having had lots of work in the enterprise architecture space but none specifically using the Zachman Framework, I wanted to see for myself what it was about and understand how it could support my own needs in presenting strategic plans to a client.

Having now read a few books and posts on the subject, here is my own reaction to it:

Written in 1987, it definately has that architectural ivory tower approach that assumes that if we could only document every explicit assumption in our enterprise we would have this perfectly functioning enterprises. Having worked in lots of different enterprises, I don't know any that actually lived up to this ideal and they functioned quite well despite not having every prescribed artifact nailed down.

I love the simplicity of the model: 6 roles and 6 interogatives. It's the simplicity that makes it "universal" - it's like saying any problem can be solved by combining asking questions and a planning/building life cycle. While I would say this is true, its also quite vague and doesn't provide a lot of specific techniques or approaches to enterprise architecture other than a method of organization.

I like the approach in that its business centric and quite non-technical. I could quite easily explain the framework to a project manager, a business analyst and an industry stakeholder. You do not need an engineering degree to get the ideas here and the expertise is delegated to filling in the particular models.

I really like the integrated approach that forces architects to really think about more than just code, servers, and technology. The framework puts a heavy emphasis on logistics, business cases, scheduling, business rules, and people. The "HOW" part of the framework is only 1/6 of the picture which I think is about right.

The concept of a normalized architecture, e.g. a unique spot for a piece of information is an interesting one but hard to accomplish in practical terms. In my experience, its definitely a good ideal - I have had too many cases where the requirements document, the data model and the network diagram are changed independently and they no longer match.

One challenge with the organization of models in ZF is that they are very model focused instead of feature focused. In many cases, you want to organize your requirements, code, etc. by feature so that you can take it out - this seems inherently different to do in ZF because your feature is now spread out in 36 different buckets of artifacts. Trying to take a feature out or change it means finding it in your master universal schedule, your master universal network diagram, your master use cases, etc. (Ideally, you could automate the entire repository of requirements so you could slice and dice in multiple dimensions so that you could search your whole picture by "sliver")

There is a certain religious zeal about this framework from some authors - the fundamentalist belief that if you can complete all the spots on the grid that you now have this nirvana requirements repository and we can all go home until the next change request. In real life of course, the nirvana never happens and you have to cut corners like crazy to get anything out the door. While this is a nice framework, in practice many medium to large organizations could not fill in every bucket down to every explicit assumption and maintain the repository with any execution speed. This is probably why this framework is popular amongst governments, large scale industries (aerospace for example) where you have years to gain value out of your architecture investment but not used by smaller or faster moving organizations.

There are some ways you can approach the ZF with an agile approach. Scott Ambler has a good article on this here.

In general, for a framework that evolved in the mid 80's, I was pleasantly suprised at how relevant it is in today's context. I could conceivably apply the same principles in some of the enterprise planning work I do today in very different technology contexts than what were around in 1987, e.g. portals, internet, rich client applications, data warehouses, etc. I could also see using it as a basis to analyze and document a very specific domain model instead of an entire enterprise.

How Can You Vote for a VP Who Uses Yahoo Mail?!

2008-09-19T21:51:00.002-05:00

Today, it was discovered that Sarah Palin's Yahoo account was hacked. Various emails were discovered including government business.

There has been speculation in the media that she was using Yahoo to conduct business so that she could avoid being under government freedom of information requests.

http://news.yahoo.com/s/nf/20080918/tc_nf/61961
http://wikileaks.org/wiki/Sarah_Palin_Yahoo_inbox_2008

So my basic question is this: regardless of whether you want to vote for this woman because of her stance on abortion, moose hunting habits, etc. why would you EVER vote for some so security illiterate that she would use a yahoo mail account with a weak password for government business? This is privacy 101 here - do we really want to trust this woman with the keys to nuclear codes, mission briefings, etc.?

Someone should tell her it's probably not a good idea to store CIA documents on USB keys or leave unshredded documents in her garbage can while they're educating her on basic privacy management.

Hug A Developer

2008-09-15T16:51:00.001-05:00

I just love this video...

Enjoy!

Good Example of How Tricky Security Is - Even for Banks

2008-08-25T20:38:00.008-05:00

I signed up for online visa account access today at CIBC, and found a basic security flaw in their registration process.

Here is their form:

The flaw is the last question - the previous balance. The flaw is a simple one - if you call up VISA on the phone and give them the information above, the phone system provides your previous month's balance!

It just goes to show that even a security conscious bank can make easy mistakes when it comes to security and the moral of the story is a simple lesson: make sure you analyze all your access channels for possible leakage of authenticating information before you demand it online.

Live Earth vs. Google Earth - Which One Is Better?

2008-08-25T07:46:00.008-05:00

I decided to run a comparison between Google Earth and Live Earth. Here are some of my findings.

Google Earth runs only as a separate executable. Live Earth runs as an ActiveX control within the live maps interface. Microsoft's approach is more integrated but it also means that you cannot run the application full-screen.

Google Earth seems to find locations better especially when they are vague. For example, I typed in "yonge and dundas toronto ontario" and Google Earth found the intersection. Live Earth couldnt find it and simply suggested businesses which weren't even at that location.

Google Earth's ability to handle misspelled words is much better as well. For example, I typed in "young street toronto ontario" and it was still able to find Yonge Street as an exact match. Microsoft Live Earth could not find it.

Microsoft Live Earth splits the search box between businesses and locations. The search assumes that you will split the business name and location into the two boxes. For example, I typed in "eaton centre toronto ontario" into the business box and it asked me for a location. In Google Earth, typing in the same search query finds me the exact match.

The search box in Google Earth is on the left hand side. In Microsoft Live Earth the search box is at the top of the screen. This makes a big difference if you are using a wide screen monitor. Microsoft's map ends up being very wide. Google Earth's map ends up being nice and square.

Microsoft's satellite pictures seem to be more up to date and detailed. Looking at downtown Toronto, the Google Earth satellite picture shows certain buildings under construction which I know are now complete while the Microsoft picture shows them finished. See also the picture of Monarch Park below and you'll notice a significant difference in picture quality and resolution.

Microsoft Live Earth

Google Earth

Both Microsoft Live Earth and Google Earth have 3D modeled buildings. However, Microsoft Live Earth's are vastly superior both in number and in quality. Google Earth shows about 5-10 buildings in Toronto where Microsoft shows hundreds.

Google Earth: Not Much of a City Scape

Live Earth: Now That's a Downtown!

Both applications take up a large amount of RAM to run. Looking at Task Manager, Google Earth was taking up to 800 megabytes of RAM and Internet Explorer was taking up to 600 megabytes. I assume this must be for caching of data.

For smaller towns, both applications seem to be equally poor. For example, searching for Port Hope, Ontario yields a poor grainy image in both cases likely because the high resolution satellite imagery is simply not available.

Google Earth

Google Earth is much richer in terms of content overlays and more flexible in being able to turn them on and off. My personal favourite is the Panaromio overlay which geo-maps personal photographs to locations from hundreds of thousands of users around the world.

Microsoft Live Earth has a feature called "Birds Eye" which does a map of photographs to the map taken from different angles. This feature is not available at all in Google Earth.

Microsoft Live Earth has very nice traffic reporting features. I was able to check the 401 highway in Toronto, see the busiest spots to avoid and also it provides construction updates as well. Google Earth does not have this feature at all.

Both applications provide directions. However, Google Earth has a feature that allows you to "play" the route. The camera will follow the route through the directions showing you in 3D how to drive to your destination.

Breaking the Law of Triple Constraints

2008-05-20T19:02:00.004-05:00

In classical project management methodology, there is a concept called the Law of Triple Constraints:

The law states that as two of the constraints are changed, there must be an equal change on the third constraint while still maintaining the same level of quality.

For example, if scope is increased and schedule decreased, this MUST have an impact on the number of resources required. If resources are decreased, this MUST have an impact on either scope decreasing or schedule increasing.

However, in practice, this law is not really a law at all, or rather it is based on idealistic assumptions.

These assumptions are as follows:

1. The process for running the project has been 100% optimized.

2. The process for running the project is 100% known and not subject to any possible innovation to improve the picture.

3. Talent on the project is not changeable or all talent is the same.

4. The amount of management is precisely what is required to manage the project.

The reality is that in most projects these assumptions are simply not true. In addition, this model promotes a mechanistic, deterministic approach to managing projects that lacks an hope for innovation, efficiency or reduction in management.

Here are ways you as a stakeholder or project manager can "break" the Law of Triple Constraints without sacrificing quality:

1. Re-examine your estimates and make sure they are not overly padded. Some contingency is always required in order to handle unexpected events. However, on a $1 million project, the difference between 10% contingency and 30% contingency is $200K - is it really necessary?

2. Challenge the team to be innovative about process, documentation, testing, builds, etc. There are better ways to doing any project - no team is 100% optimized. I've seen simple changes such as using automated unit testing that makes massive differences in effort and actually improves quality. Depending on the type of project, there is innovation to be had.

3. Do you really need to build something or can it be bought?

4. Be ruthless about gold-plating on all sides. Developers, stakeholders, creative directors, etc. all like to add changes, rounds of revisions, additional documents, new features, etc. and that extra 10-20% makes a difference over the course of a large project.

5. Improve communication and lower documentation overhead. For every 300 page use case document you have to write, edit, review and then update make sure they are necessary artifacts and they have value. I was on a project once where the wire frame document cost $100K to produce!

6. Make sure you have the right talent for the job. For hard problems, you may think that having your junior flash developer be your technical architect will save you some money but in my experience the opposite is true - get the best talent you can possibly afford because they pay for themselves in much higher value than the additional cost in salary. For easy jobs, don't take your technical architect and make him your excel macro monkey - it will frustrate him/her and is a waste of their talent.

7. Get Creative! On every project, there are ways to innovate. It may be in the implementation of a particular module. It may be that you re-examine the problem you're solving and come up with something more elegant than when you originally built your plan.

8. Make your team entrepreneurial. Encourage open communication, risk taking, fast decision making, and low document overhead. Have the right level of process and no more than necessary. Give everyone ownership in the project and let them understand the big picture.

9. A simple tip: make sure your team members know deadlines! It sounds really dumb, but I have noticed a big difference in teams where they can see the entire plan and understand the context of deadlines than if a person is just handed a task piecemeal and told to complete it blind.

I've seen project plans that are bloated and I have seen some that are way too lean. Starting with bloat and cutting down I find is easier to manage than starting lean and trying to increase to handle the bare minimum requirements. Start with a robust, conservative and full process plan and then take a good look at it and see if there are some trimmings to be had.

One final note - this is not a panacea and in my experience, you can achieve a savings of somewhere between 0-20%. Don't expect that you can cut your project timeline in half. However, on a $1 million dollar project that's still $100-200K and on a 6 month timeline that's still a month of time saved.

As a project manager, you can choose what you do in terms of managing stakeholders' expectations. Typically, I use a conservative estimate to start and then I try to get innovative once I'm executing. If the risks of being creative and entrepreneurial pay off, then I look like a rock star with a project ahead of schedule, a happy team and satisfied customers.

But if you are a project manager who simply adheres to a conservative approach and obeys the Law of Triple Constraints like dogma, then you will be missing opportunities to innovate!

With $15B in Valuation Can Facebook Now Afford Some Quality Assurance?

2007-10-31T07:48:00.000-05:00

I find it amusing that Facebook despite its high valuation and increased interest from Microsoft is still amateurish at times. One of the areas I find is in usability - there are some basic issues that any modern (or even 10 year old site) would handle better than facebook but haven't been addressed as the site grows and scales.

Here are a few examples!

Group Management

One of the areas that is really week in Facebook is group management. It seems that the whole assumption around being a group admin is that your group has about a dozen people.

For example, here is a simple but telling bug - try to remove a member in your group. About 2 months ago, this feature didn't seem to work at all. It now works, but there is no way to search for members! So in our case we have 1900+ members, and if we want to remove or block a member we have to scroll through 70 pages worth of names!

Sorting of Lists

Another example is the basic way that Facebook sorts lists. By default your friends list is sorted in alphabetical order by last name. If you do a search for Classmates, the sort order is the same - but this is not always what is needed. For example, I want to know recently updated Classmates because I've already scanned the list previously. There is no easy way to do this in Facebook.

Paging of Lists

Another basic usability issue - paging through lists doesn't actually page properly! If you click next, you'll see people who were on the previous page.

Inviting People to Groups

In groups, you can invite your friends to join. In previous "versions" of Facebook, it would only show friends in the invite list who you hadn't previously invited. Now it seems to show all your friends but if you click on the friend who has been previously been invited, it pops up a warning and doesn't allow the action. How is an improvement?

All of these are basic usability issues related to scale - what works for a friends list of 20-30 people doesn't scale as an interface if you have 2000 friends, group members, etc. But as Facebook tries to be a "platform" it needs to be able to address these concerns - these are pretty basic and most applications whether Web 2.0 or traditional client server have basic functions for sorting, searching and filtering lists!

Are AJAX enabled forms really worth the trouble?

2007-10-16T07:45:00.000-05:00

We have just launched our new Tribute Application for creating customized cards for donations in honour or memory of someone you love.

We used Microsoft .NET and the new AJAX library for development of our forms. Our developer teams became enthusiastic supporters of using AJAX styled controls for creating a sense of immediacy in the interface.

However, what was interesting was the new set of bugs that cropped up simply because of now using AJAX controls. Here is an example: the above form has two controls for postal code and zip code and based on which state or province is picked, we flip between them so that you can have client side validation, proper field lengths, etc. Sounds simple enough, but we went through quite a bit of work and bug fixing to make this work.

Here are a few bugs that we found in the process as examples:

1. When you click the back button the right control isn't always displayed.
2. We had styling/layout issues in Mozilla.
3. When you picked a state and then switched to province the value would disappear.
4. The switching of state was generally quite slow on some clients.

In the "old days", the interface would have been really simple - a text box submitted to a server side script. Very few presentation layer bugs because there wasn't really anything going on in the presentation.

So the question becomes - is it really worth it? For Yahoo Mail it might be, but for an average data entry form is it really that crucial?

Now that we're planning phase 2 on our project, I would be much more cautious about going gung-ho on AJAX simply because it dramatically increases the testing/bug fixing time for what amounts to a simple user interface convenience.

Software Development Metaphors

2007-09-21T07:13:00.000-05:00

There are lots of discussions on Software Development "methodologies". They are codified sets of practices on how to do software development "correctly". When you get down to the root of each of these methodologies, the root of all of them is a metaphor. Before you adopt a methodology such as XP, Scrum, RUP, Software Factory, etc. you should really ask yourself what metaphor you believe your particular software development practices fit into and what metaphors fit your particular organization's culture. If you start there instead of at the "here is my fancy new toolbox of processes" then adoption will be higher.

So here is a list of popular metaphors used to explain the practice of creating software...

The Traditional Software Development Metaphors

Software Development as Factory

Heavy emphasis on frameworks (assembly line) and components (widgets)
Use in large scale development projects (hundreds of developers) or highly distributed projects (outsourced)
Highly centralized control of processes, tools, etc.

Software Development as Engineering

Heavy focus on estimation techniques
Fixes made in planning are less expensive than after “construction” has started.
Highest project management priority is being “in control”.
Software development as a science, e.g. objective, measurable.
Belief that Software Development is an immature or imprecise version of engineering.

Software Development as Model/Architecture

Architect as “GOD”
Code is simply implementation of a model
If the software doesn’t work, either the model is wrong or the code doesn’t implement the model properly.
Tooling can automatically generate application from the model.
Software models a business, a process, etc. in design. Goal of software is to accurately reflect real life processes.
Need specialized modeling languages, expertise, and tools

Software Development as Workflow/Process

Projects as repeatable processes.
Workflow is defined up front, and people are compartmentalized.
Emphasis on distribution, encapsulation and hand-off.
Project managers are process champions – they make sure the rules are followed.

These metaphors have formed the bedrock of software development. They represent the traditional way of viewing software development as well as project management in general. They have enabled a whole toolbox of techniques, tools and templates for managing projects and software development.

The Radical Software Development Metaphors

These metaphors have become in vogue thanks to various agile methodologies, reactions against the traditional view of software and frankly a lot of failures for the traditional metaphors to come to grips with the complexities of creating software.

Software Development as Craft/Voodoo/Martial Art

Focus on simplicity, elegance, and craftsmanship
Toolbox approach
Apprentice / Master / Guru model
Software is highly dependent on people
Software “Design” provides guide to picking solutions

Software Development as Game Playing

Games are processes without the prescription
Games can be open or closed, cooperative or competitive
Rules of the game are enforced, but outcome is unknown
Games also include collaborative activities such as sports, playing music, dance, etc.

Software Development as Book Writing/Composition/Poetry

Iteration, drafts, reviews, etc.
Focus on creativity within an established structure.
Objective is known, but product is not well defined.
Challenges in forcing creative process into rigid scope, time lines and budget are recognized.

Software Development as Learning/Experiment/Invention

Modeling in software is about learning
Documentation, requirements gathering is a means to collective learning
Focus on establishment of common language
Prototyping, experiment, measuring as a way to learn

Which one fits your culture, your organization needs, and your skill set? If you're a start-up company, then the software factory metaphor might not be the best fit. However, if you're building printer drivers with third party outsourcers in India, then perhaps it is the right model.

If you start with basic metaphors, the toolbox and methodologies such as RUP, XP, Scrum, Waterfall, etc. start to make more sense - they're expressions of the underlying metaphor. In addition, if you want to evangelize a new software development methodology (for example, those who are trying to evangelize the use of XP in a traditional RUP organization) starting with a discussion around metaphors and basic understanding of software development might help the discussion, especially with non-technical users.

One last suggestion - this discussion is a great interview topic for Project Managers. If you want to hire an "Agile" PM, then start the interview with the list of metaphors and ask them which one they believe fits their style of working. If they pick one metaphor, then get them to explain how their PM toolbox (GANTT charts, risk matrices, status reports, etc.) fit into the metaphor. The traditional toolbox can be used in an agile context if the underlying metaphor is right - there are ways to use MS Project even if you believe that Software Development is more like writing a book then building a bridge.

The DigiBarn and Facebook - A Killer Way to Meet Old School Geeks?

2007-08-27T19:44:00.001-05:00

The Digibarn is a museum of old computers. They have a great virtual collection of every computer that I've ever owned and then some.

Here is just a small sample of weird computer systems I've owned or used extensively over the years:

Facebook currently has lots of curious personality matching tests such as "Which Disney Princess Are You?", Celebrity matching applications and personality comparison tests.

So here is a killer app idea for meeting old school geeks - why not combine the power of the Digibarn with the social network for Facebook to create a profile based on computers you owned over the past 20 years? Imagine having a date with someone who also owned a TI 99/4A, just like you? Imagine the nostalgic conversations about GOSUB, sprites and tape drives you could have over dinner?

At least it would be better than that Zombie Facebook application floating around these days...

Take that First Step Toward Becoming a Legend (in IT Governance)!

2007-08-23T07:01:00.000-05:00

I received this spam from Oracle the other day and I nearly fell off my chair laughing about how marketing people can really get things wrong if they try to apply consumeristic jingoisms to IT related products:

Do people in "IT Governance" (whatever that is supposed to mean these days) really need such a confidence boost that they need to be pandered to by suggestions that the systems, memos and processes they put in place will make them "legends" and their companies "Admired by your industry, as a leader in IT governance"?

And as an executive, imagine if the first impression of your company's brand was a "leader in IT governance" or that the "company's legacy" was really good IT governance?

I hope I never work for that company... :)

Best Logging Strategy using Microsoft Enterprise Library?

2007-08-21T21:59:00.000-05:00

The Microsoft Enterprise Library has a really good Logging Application Block that allows you to make your logging infrastructure configurable. By simply typing in the following code:

Logger.Write("log this!");

you can log to a destination configured in your config file.

The Microsoft Enterprise Library out of the box provides the following destinations for logging messages:

Email
File
Database
Message Queue
Text File
WMI Event
Custom location

I took our web site and our main global error handler and did a dump routine of all exceptions to the log and it worked great. But the discussion in our team arose - where should we log? Here are some of our ideas as we debated the issue:

The least error prone seems to be the flat file. The custom location seems the worst option simply we would hope that our proprietary code is less tested in less real world scenarios than Microsoft's Enterprise Library components.
The idea of logging to the database is attractive from a reporting perspective, but everyone on the team felt like it was risky given that the most typical scenario for an error in the application itself was a database related error. I suppose if we had the logging database seperate from our application database then this would reduce the risk, but we don't have the resources for such a move, so if our application were to kill the database it would take down any logging sources there as well.
Even with the flat file, it is possible that it will fail. A simple scenario for it failing would that the file is read-only or the ASPNET process doesn't have permission to access the file. So we need at least two logging mechanisms to provide redundancy. The Enterprise Library allows for this - you can configure more than one listener for every log message being written.
Our second challenge with using flat files is that they no provide a notification mechanism like email. So if there is an exception, the file has to be constantly monitored or periodically checked. Therefore, combining an email message and a flat file backing store seemed like a decent combination. If the email fails, then we may not know for a day or two but at least the error will be logged. If the file fails, then at least we'll get the email.
The event log is not a bad option either and its generally easier to monitor as most monitoring tools already handle event log monitoring. The one issue with using the event log that I have found is that if you use an event log source that doesn't exist the logging will fail without any notification to you. Your error will simply disappear and won't be logged. You can use any existing EventLog source or create a new one in the registry for your custom logging needs. In addition, you have to make sure that the ASPNET account has permission to write to the event log.

That's where we are at so far - its not quite bullet-proof yet but its getting there. If you have any further suggestions or thoughts, leave them as a comment!

Arcade Game Prices are Finally Catching Up With Inflation

2007-08-21T21:39:00.000-05:00

I've been playing arcade coin-op games since I was a little kid. When I started playing these games, they were 25 cents a throw. If you were good at them of course your quarter could go a long way. For example, there was a game called WonderBoy that I could play for about 45 minutes on one quarter.

Somehow despite 20 years of inflation, the price of an arcade game was stuck at 25 cents. Eventually, the prices went up by multiple credits being required, especially for newer games. The current arcade games seem to cost on average $1-2 per contest.

But if you went to the back of the arcade, you could still typically find an old Galaga, Gyruss or Pacman sitting in the back still running for a quarter a throw.

Well not so fast anymore - I was in one of the last arcades left in the city and these older games have now been replaced by emulators which allows the arcade owner to change the price and also house multiple video games in a single console. This makes sense - given the size in memory of any of these old video games, you could house hundreds of them on a single memory chip.

So I had fun today playing the old favourites, but I'm a bit disappointed that it cost me 50 cents. I guess after 20 years of dwindling attendance and increasing costs, the prices have finally gone up.

Thankfully, I have MAME which allows me to play any old video game for free!

Is Google Analytics Having Trouble Keeping Up?

2007-08-17T07:35:00.000-05:00

Since I started this blog, I have been using Google Analytics to measure traffic to the site. In general, its a fabulous service and works really well, until recently.

I've noticed that in the past month or so, Google Analytics seems to have trouble keeping up with the traffic. I've noticed that the numbers reported are lower than expected and stay that way for 1-2 days and then as presumably the engine crawls through the right logs in queue, it catches up.

One way to tell for sure whether you're having a similar problem is compare your Google Analytics numbers to your Ad Sense numbers. Ad Sense seems to be more accurate and is updated in real time. Google Analytics is queue based and so tends to need some time to catch up. I've found in recent days that Ad Sense gives me a better indication of short term stats than Analytics.

For my blog, this is not a huge problem. But this is yet again a mark against the "software as a service" model - would you want to trust your corporate infrastructure to a queue in the cloud that can get bogged down because of an influx of unexpected growth?

Ad Words and Negative Content

2007-08-13T23:58:00.000-05:00

The great site Worse Than Failure had a few images today of examples where legitimate ads were being shown side by side with negative content:

Having ad sense on my blog, I have noticed the same thing. I wrote a couple posts on the failure of Canada's tax software, and if you go to this post the ads displayed are for tax software!

Isn't this a pretty basic requirement for online advertising, e.g. don't put my ad for Levi's Jeans on a web page where the content is "I hate Levi's Jeans!"? Why after 10+ years of online advertising has the experts at Google and others figured out how to targets ads to the right content. In my example, the right ads would be those marketing to people reading about the failures of the tax system, e.g. tax lawyers, H&R block, etc. and not tax software providers.

Project Planning in an Agile Project

2007-08-13T22:37:00.000-05:00

Tate Stuntz has an interesting post on his blog arguing that Project Plans are obsolete and hard to maintain when doing Agile projects. In my experience on agile projects, project planning and Gantt chart making definitely has to change, but there are still ways I think you can make MS Project and Gantt charts work for you in an agile context. Here are my recommendations based on my experience:

Learn to use MS Project efficiently (see my previous post for recommendations). I've seen PM's take 2 weeks to build a project plan - if this is you then trying to work this way in a 2 week iteration cycle means you're not moving fast enough. You have to be at the point where you could re-boot your project plan in a matter of hours so that you can adjust quickly.
Use MS Project for planning and not for tracking. If you're working with short iteration cycles, you'll see improved tracking by simply management by walking around. In addition, measuring project velocity is more valuable than percentage complete since in an Agile project the finish line is not always defined.
Use a project plan for iteration planning at a macro-level and not for iteration tracking within an iteration cycle. What I like doing is maintaining a project plan with a simple list of user stories in each 2 week iteration. That allows me to track dependencies between user stories pretty easily. At review time at the end of an iteration, we discuss the next iteration with our stakeholders and reset expectations by re-forecasting the plan (which takes no more than an hour of work). That gives stakeholders a good understanding of what we currently see as in and out of scope and when features may be coming up. This is highly valuable and I find is critical for buy in for stakeholders - if the response from the team is simply, "we can only see to the end of the next iteration cycle" this tends to make business users very nervous.
If you need pretty Gantt charts, one of my favourite methods is to use Visio - it provides pretty Gantt charts for presentations without the fuss of Project.
I tend to use the whole team as a big pool for planning iterations. This gets you out of the micro-detail of having to assign one user story to one person, another user story to another person and so on. If I have 4-5 developers on a team, that gives me 70 hours x 4-5 developers for a total iteration budget of 280-350 hours. I can then do a good estimate of each user story that includes planning, development and QA with the team. We simply add up a bunch of these stories into the budget and that becomes the iteration. The way to do this in MS Project is to take your steps and instead of assigning individuals assigned to each task and running them in parallel try simply taking a generic role of "developer" and assigning them at 500% to every task in sequence. You can then tell how much time you're going to spend across the entire team without having to fuss with all the micro-dependencies.
For preliminary planning, use low-tech Gantt chart tools such as whiteboards, post-it notes, etc. I can create a Gantt chart without MS Project using a white board, some markers and post-it notes. This works great without the overhead of the tooling getting in the way. If you want to preserve it, simply take a digital photo of it and store the picture.

I think if you follow some of these techniques as a Project Manager, you can still sleep at night without feeling completely out of control and yet move faster and leaner. In addition, you can use project planning to help sell the concept of Agile to stakeholders who aren't as comfortable with the XP style of development and give them the comfort of a traditional project plan without locking yourself into the traditional bureaucratic approach to Project Management.

Good CIO Level article on Silverlight

2007-08-13T07:50:00.000-05:00

CIO Magazine posted a good summary on Silverlight. We used the technology in a Proof of Concept to build a kiosk for our donor hall and from a user experience perspective it's a great toolset. CIO is right to point that Version 1.1 is the version you want if you want to target the platform as it supports the .NET framework and includes support for LINQ and web services.

Compared to Flash, the biggest advantage I see is that XAML is true 3D. I remember building things like car configurators for BMW and to try and get Flash to do fake 3D was a huge challenge. Silverlight is real 3D which means that doing animations that include zooming, tiling, sliding, and using 3D models could lead to some very interesting interfaces.

I'm interested to see how this platform evolves - I can see this tool becoming a real hit in the business applications space. Think of all the VB.NET intermediate developers out there who struggle with creating a really nice interface to an otherwise functional application. With a little bit of creative design and some nice tooling, Silverlight could turn a boring non-user friendly database application into something much more intuitive.

Low Tech Approaches to Requirements - Part 3

2007-08-10T10:42:00.000-05:00

For those who missed Parts 1 and 2, see the following previous posts:

Part 1: Different Techniques on using non-tech tools for requirements gathering
Part 2: Using existing tech tools in simpler ways to speed up requirements gathering.

One of my staff sent me this image today and I thought it was a great example of how to use low-tech approaches to conversations. He simply took a screen shot of a screen, threw it into MS Paint and then started making changes with the developer beside him.

Between them in 5-10 minutes they had a "language" for what the marks on the screen mean:

Red lines through something mean to get rid of it
A box with text means to insert a new column
A line on a checkbox means that on checking it something will happen
etc.

So instead of using UML, fancy wireframes, or some bureacractic documentation process, try simply taking a screen capture and throwing it into Paint!

Managing Configuration Files and .NET

2007-08-08T20:49:00.000-05:00

I've been using the .NET config files lately, and its been rather frustrating lately because of the confusion that arises when running a bunch of different ways to access the same code.

In my scenario, I have a class library called DBAccess.dll and I call it in the following ways:

1. From a web site using the ASP.NET built in web server launched from VS 2005.
2. From a web site running in IIS.
3. From a forms application
4. By running TestDriven.NET using a seperate test assembly.
5. From a windows service.

In all these cases, I'm expecting to get configuration values through the main configuration file for connection string so that it can be used by Microsoft Enterprise Library.

With so many options, .NET doesn't make it easy for you because of the differences in naming conventions on the default configuration file, e.g. in the case of a web app it expects a web.config, a forms app expects an app.config and if you run the NUnit application it wants the assembly config file, e.g. DBAccess.dll.config. Maintaining all these configuration files is a nightmare!

So in sorting through this, I came up with a few basic strategies to combat this configuration file overload:

If you add an App.Config file to your class library project, it will automatically translate this into Assembly.Dll.Config for you.
For NUnit, it expects a config file that is rooted to your test assembly. It does not read any dependent assemblies config files, app.config, web.config, etc. even if its the same directory of your assembly. Alternatively, you can add configuration information directly into the app.config file of the NUnit executable (either the GUI or the Console) but this isn't a very portable approach.
If you have an App.Config file and an assembly config file, .NET will always read the App.Config file. It does not by design read your assembly specific config files. There is some code floating around that can do this for you if you want to have more distributed configuration mechanisms that are attached to assemblies instead of executables. This is one way to mitigate the problem of re-using an assembly across multiple projects where some are web projects and some are forms projects.
Another great alternative is to use proper build files instead of the default studio builds. Either MSBuild or NANT will allow you the flexibility to store a single configuration file and rename it and then deploy it to the appropriate destination. The tricky part would be that you may need a configuration file named Assembly.Dll.config for your unit tests if you've incorporated them into your build process and a final App.config or Web.config for running the executable.
Another great idea that I have used before is to use the Microsoft Enterprise Library to specifiy an external configuration file. This removes the naming problem with your configuration files in that you can have a seperate external file that represents the master configuration file and simply have Web.Config, App.Config, Assembly.Dll.Config point to your external file. This works quite nicely in my experience as long as you remember to add the external file to your project and to configure studio to always copy it into your build directory.
You can build your own configuration loader. I have done this with custom defined configuration sections by having code in the custom section that contains a path to an external configuration file. You can arbitrarily load up any configuration file in code and grab whatever sections out of it. This is really useful for custom configuration sections where you want to provide code that saves and loads the configuration file with an arbitrary name. One important note with this strategy - there are cases where the Enterprise Library does the configuration loading for you, e.g. the DatabaseFactory will load connection strings for you automatically. In these cases, you cannot provide a configuration section or source as an object so you're stuck with the App.Config, Web.Config, etc. as a starting point.
The ConfigurationManager in .NET allows you to open the Machine.config file. You might want to peer into this file directory to look at global values instead of your local App.config values.
You can use a complete different configuration framework - no is forcing you to use App.config just because it's there (although most people do so). For example, look at Nini as an option (although I haven't tried it yet!)

I hope that provides some ideas on how to manage configuration files in a way that will allow you to have more re-usable configuration schemes than a basic App.config or Web.config file so that it can be used across multiple projects and different application contexts.

Test Driven Development When Writing New Code

2007-07-24T21:45:00.001-05:00

I have been doing TDD for several years now, and I can still not get around the recommendation to test first to the point where it breaks the code, write enough code to pass the test, and then go to the next test. In my experience this tends to work with mature code bases where there is a chunk of existing code that is being changed. However, I'm not sure this approach works for new applications that are being built from scratch. In my experience, a less purist approach seems to work for me:

Write enough code to get something at least working minimally. If you think of a use case, I'm not sure there is much point in writing gobs of test cases before you even have a primary path through the use case. The XP methodology talks about an architectural or design spike, but in my experience there is also a coding spike. For example, it might be the first iteration that instead of writing code + tests we just write as much code as we can to push through the primary path as widely as possible instead of trying to be exhaustive in our testing.
I use at least a basic unit test in NUnit as an entry point in all code debugging. That enforces the practice of having tests but also provides a useful way to gain entry into the code without web pages, windows forms, or a global main method. Using TestDriven.NET, you can simply right-click a test and run it right in studio using the debugger, NUnit, NCover, etc. and I find this higly efficient when writing new code.
There does seem to be a drop off point where writing more and more tests provides less value, as in any testing effort. I'm seen some overly puritanical approaches to TDD where it was insisted that every boundary condition was to be coded up as a unit test no matter how unlikely the scenario or risk. In my experience there is a point where coding these scenarios is not worth the benefit of the test. In addition, I have found that in some cases it is still easier and better to manually test something than to try and code it up as a test.
I definately like the idea of using build scripts and NUnit tests when making modifications to code. However, In my experience the tests actually don't change as much as the underlying code. In some cases there are new requirements but in many cases they are implementation changes. As long as the existing tests are still green after the change, then no new tests are required.
There seems to be a trade-off between quick progress and the feeling of "being on a roll" and the pace of TDD which tends to be plodding but regular. Again, I think a less purist approach is best - at some points you feel your creative juices flowing and you want to plow through 2-3 pages of code in one go without writing exhaustive unit tests for each little method. Other times if you're being more cautious you want to test the living hell out of every line of code. As a mature developer, you have to feel the right pattern depending on the code you're writing.
Lastly, I think its critical that at a minimum unit tests are written before code is checked into source control. That way if you have continuous integration the unit tests will run on every build. It seems also just a courtesy - you might have insight on why you left out some tests but your team mates might not. So before checking in code make sure there are tests to go with it. I find also that the unit tests are really a good description in code of the requirements so for new code its really helpful to have them as a guide.

For code changes in a larger mature application, I like the traditional TDD process of making new tests so that they fail, writing code to fix the tests and then regression. But for coding applications from scratch or starting a new major code project, I find a looser style tends to produce better results.

So with any agile technique, don't get to ideological about the approach - in my experience, TDD is best used as an approach and not a rigid perscription.

Example of Fat Code

2007-07-24T21:44:00.001-05:00

Why exactly does the Google Toolbar need to be 47 megabytes?! It's a tool bar for heaven's sake...

Agile Tools for .NET Part 2

2007-07-23T07:41:00.000-05:00

This is a follow up to my previous post on Agile Tools for .NET. I got to the office and I noticed a few more tools that I missed in my previous post.

Confluence

Atlassian's professional WIKI solution, it works great with JIRA and provides a really nice WIKI solution. For those who want to travel light with documentation, WIKI's provide a nice fit for technical documentation, build documentation, internal designs, etc. as they easily accomodate rapid changes, are completely web based and allow for automatic version tracking.

Data Dictionary Creator

One of the principles of Agile is make your code self-documenting. What about making your database self-documenting? You can do this in SQL 2005 in T-SQL by adding meta-data and descriptions to each field in your database. The Data Dictionary Creator tool does the same thing through a nice GUI and allows for export and import of the documentation. You can then document your database and store the meta-data in source safe for version control and safe keeping. In addition, you can export the descriptions to an excel file for instant documentation!

Wisdom Screen Hunter

Not really a development tool, but how many times have you been creating documents and wanted a quick screen capture? Forget using the PrntScrn and fiddling in paint - use this tool. Its free and allows you to capture any arbitrary rectangular area as well as any active window. Simply hit F6 and off you go.

Agile Tools for .NET

2007-07-22T20:04:00.000-05:00

The following is a list of tools for doing agile based development with Microsoft .NET. There are quite a few tools out there that will accelerate your use of continuous integration, test driven development, source control integration, etc.

I have used all these tools - most I use on a daily basis. They are highly recommend and most are either low cost or free.

JIRA

Best tool for task management. For more details see my review of JIRA from a previous post. It's bugzilla on steroids and is much better and simpler than most windows based tools. It's java based, but there is still no reason not to use it for .NET based development.

NUNIT

NUnit is an implementation of the XUNIT framework for .NET. It allows you to create unit test classes that can be run by either a GUI or console application. NUnit will also integrate into your build process so that on each build, you run through your automated tests to make sure they still pass.

NANT

NANT is a port of the ANT build tool from Java. NANT allows for build scripts based on simple XML files. NANT can do things such as compile code, deploy assemblies, manage directories, start and stop processes, run executables, etc.

NCover

NCover is a code coverage tool for .NET. It will instrument your assemblies when they are built to analyze each line of code in your assembly and whether it was actually executed (exercised). If you combine this with NUnit, you can then tell how much your unit tests actually go through every path of code.

dotTrace or Ants Profiler

Either tool will do, but personally I like Ants Profiler better and you can get it as a bundle with Ants Loader for cheaper. Both tools do the same thing - they profile either memory or performance data for your executing code. You can profile an ASP.NET web site, a Winform application or a Windows Service. What you will get in the case of memory profiling is the ability to snapshot at any point of execution a picture of what is in memory. This can help with objects not getting garbage collected properly or where you're running into memory issues. For performance profiling, the tool will tell you the amount of time each line of code took to execute. This can be useful for performance testing in that you can see the breakdown of time for a method to isolate the particular lines of code that are taking the longest.

Cruise Control .NET

Cruise Control is a port of the java application of the same name. It is a continuous integration server which will fire a build on every check into your source code repository. It can run either a NANT script or an MSBuild script or simply tell Visual Studio 2005 to do a build. It then stores the status of each build in a little intranet site that you can monitor and it can email you notifications as well. This provides an automated way to integrate code and test to make sure it builds and passes all your unit tests.

My Generation

My Generation is a code generator. It ships with a number of templates for generating O/R mapping code for frameworks such as Gentle, NHibernate, DAAB, etc.

TestDriven .NET

This is one of my favourite tools. It allows you to integrate your unit testing right into studio. Want to run one of your NUnit Tests? Just right click on the test code and up comes a Run with option that you can then run with the debugger, NCover, NUnit, etc. The one I use most is run with Debugger as it means you can step through the code hit by each of your unit tests.

ANKH

Ankh is a subversion add-in for Visual Studio 2005. For those who are using subversion as your source code control system (and I would recommend subversion in general over Visual Source Safe) it allows you to update and commit code from the solution explorer window. My only problem with this tool is that its quite slow especially if you have large projects, so you may find it easier to check in with external tools after a while. But for those who are just getting used to subversion, its a great first step.

Getting to the Finish Line in an Agile Project

2007-07-19T21:50:00.000-05:00

One of the great potentials of Agile software development is flexibility in changing requirements. We currently use agile methodologies and have scheduled 2 week iterations. We have show and tells with stakeholders every two weeks where we show them where things are at and gather new user stories, functionality changes and other feedback to plan for the next iteration.

One of the challenges we have found with this approach is that the project quickly turns into perpetual development. One of the advantages of a traditional waterfall model is a solid arc in development and a very specific deliverable - you have a good definition of when you're done. In an environment where the requirements evolve, the time lines are fluid and the stakeholders are not that experienced at software projects we are finding that this means that the project never seems to end! Every two week iteration, stakeholders simply pile on new requirements and the launch date drifts away.

In a consulting environment, this isn't such a bad thing if you're on a time and materials engagement - the client simply gets to spend their money as long as they want. But in an internal development team, this poses a challenge because most stakeholders aren't really aware of how much money they are spending and they simply keep monopolizing the team. In the meantime, there are other projects or priorities queueing up in the background.

Based on what we have found works, here are some practical recommendations on how to combat this problem:

Establish clear business objectives and as many solid functional requirements as possible. There is a happy medium between XP style iterative programming and a rigid waterfall model. Try to put some acceptance criteria that can be measured so that the entire team has a common understanding of the target.
Remember the law of triple constraints - of cost, resources, and time changing any of these impacts the others. Make sure that stakeholders are aware of the real cost (even if it is an internal cost) through an internal budgeting model. In our organization, we have a shared cost model in IT but we can allocated the shared cost to any department we choose. In cases where we see departments monopolizing our time we can allocate more cost to them. This gives internal stakeholders an incentive to prioritize and conserve resources.
Make stakeholders compete with each other. This is pure politics but it works. The easiest way to stop a project is to simply tell another stakeholder that they're waiting in the queue behind someone else. There will be a pretty quick meeting between stakeholders where one will start to put a lot of pressure on the other to get their project done quickly or to get out of line altogether.
Time box and resource constraint the project explicitly. We do this quite a bit with reporting needs which seem to continually evolve ad nauseum. When we start these types of projects, we establish a budget of 2-3 iterations and that's the budget. The requirements can evolve but there continues to be pressure on the stakeholder to get it right the first time or else lose their window.
Always have a phased approach with Phase 2 and 3 already planned in Phase 1. This gives the stakeholder the ability to still feel they're going to get a change at some point but perhaps can live with launching without it.
Be vigilant and communicate often about what's currently in and out of scope. I'm not sure whether stakeholders intentionally play dumb or whether people simply don't hear bad news but I've experienced several times the case where three meetings in a row we hear questions about a feature that we've already ruled out of scope and there is genuine surprise from the stakeholder that they're not getting a particular feature even if its been already discussed, documented, etc.
Have really strong leaders in QA. QA Leads, especially in the latter stages of projects, become crucial at communicating and organizing priorities. A great QA Lead can manage stakeholders on a daily bug review meeting and can handle the constant re-negotiation of last minute feature requests, low priority bug fixes, etc. and keep the process moving towards launch. If the project loses velocity during the stage it opens the door to ever increasing scope change. If the project has high velocity then everyone on the team feels like they better prioritize because time is running out and the project is going to launch without them. This helps crystallize priorities and puts good pressure on stakeholders to commit to the minimum work required to launch.

I've been in projects where we sat at 90% for 5 months because we kept stalling as we closed in on the finish line. In addition, I've been on projects with internal teams where the stakeholders simply got used to having a software team at their disposal week after week and their was never any pressure to launch anything or to stop working.

As a project manager, your goal is to launch - 90% complete is really 0% complete especially if it's still just running in development or still buggy in test.

Can You Trust Second Life Charity?

2007-07-18T17:32:00.000-05:00

At the foundation, we are interested in Second Life. For those who are not familiar, Second Life is a virtual 3D world that allows for users to create custom content, communicate with others in a virtual world and buy and sell items in a virtual currency. What makes it unique is that the virtual currency can be converted into real cash, allow for the conversion between the virtual and the real.

Commercial activities have evolved over the past year or so in Second Life to include things like virtual real estate, virtual products, etc. In the past little while, charities have started doing virtual fundraising by having virtual walkathons, virtual campaigns, etc. in Second Life. The virtual currency they raise they then convert to real dollars to help real people.

So here is the basic question - how do you ensure a trusted conversion between the fantasy and the real? In the case where a Second Life user is buying a virtual good, then there is no real issue - they pay virtual money and instantly get a virtual hat, a virtual house, etc. But if you donate virtual money to a virtual charity, how do you know that its really going to the charity? This is true for commercial conversions as well, e.g. if you pay in virtual currency for a physical good, how do ensure that the vendor is legitimate?

The number one objective for a fundraiser is raising money for their respective cause. But a very close number two is ensuring the maintenance of its brand, reputation and trust with its donors. Its critically important because its the only long term asset a fundraiser has at their disposal (in the same way as Nike, Sony, or Coca Cola).

There are a couple potential ways you can verify that a charity asking for money is legitimate and sanctioned:

Check the charities web site. For example, the American Cancer Society publicizes its Second Life Relay for Life and provides instructions on how to get involved.
Call the charity. If the person asking for money is legitimate, they'll be registered in some way as an event organizer.
Ask the person in Second Life basic questions like: do you have a contact I can talk to, how do I verify that you work for the charity, do you have a tax number, etc. The person should have a way to give you information that can be verified - at least that shows the person has some basic knowledge of the charity they are promoting.

There are some people in Second Life who may be asking for donations and legitimately giving the money to a charity. However, without some way to verify it I would suggest you're better off donating directly to the charity.

If you cannot verify, I would not donate a single L$ as you have no way to ensure the money is actually going to charity.

Creative use of Delegates in .NET

2007-07-18T07:49:00.000-05:00

I normally don't use delegates, since I don't do a lot of events programming where they are commonly used. But I was building a class library the other day and found a good use for them.

Here is the scenario: I have a series of methods that allow for merging of records in our main CRM. In order to connect to the CRM, there is a series of setup and tear down calls that have to be made such as connecting, logging in, etc.

So I found myself creating a dozen methods that all looked something like this:

public void mergeAddress(String masterID, String mergeID)
{
try
{
bunch of set up code
meat of the method
bunch of tear down code
}
catch (Exception e)
{
throw new MergeException("Unexpected error in method mergeAddress", e);
}
}

I found myself copying and pasting this chunk of code over and over again, which is always a good sign that some refactoring is required.

My solution was to use a delegate to represent the meat of the method in the middle. Therefore, instead of having a bunch of methods that repeat the same code over and over again, I can now rewrite the method to look something like this:

protected delegate void mergeDelegate(String MasterID, String MergeID);

public void mergeByDelegate(del method, String MasterID, String MergeID)
{
try
{
bunch of set up code
del.invoke();
bunch of tear down code
}
catch (Exception e)
{
throw new MergeException("Unexpected error in method mergeAddress", e);
}

}

Now the method becomes like a prototype - I can keep the set up and tear down in one place and simply sub in the method call in the middle.

(FYI, You could do the same thing by splitting the method into three chunks, moving them into a class and sub-classing the method in the middle, but I think using a simple delegate is easier.)

To implement the prototype now looks something like this:

public void mergeAddress(String MasterID, String MergeID)
{
mergeDelegate del = this.mergeAddress();
mergeByDelegate(del, MasterID, String MergeID);
}

public void mergeAddress(String MasterID, String MergeID)
{
meat of the method
}

All the set up and tear down code is now centralized into one method. The part that changes in the middle can now vary in a variety of ways.

Hope this helps everyone with another way of using delegates...I've always struggled to find a use for them.

Low Tech Approaches to Powerpoint Presentations

2007-07-10T07:25:00.000-05:00

Take a look at these videos as a low tech presentation format. Think about how you can use these techniques for selling ideas, explaining technical concepts, brainstorming, etc. For more ideas on low-tech requirements gathering, see my previous post as well.

LiveEarth not in the top 10?!

2007-07-07T07:37:00.000-05:00

Whoever organized LiveEarth really missed the blogosphere. A few points of evidence:

1. There are only 2049 blog posts on LiveEarth?
2. LiveEarth as a search term isn't hitting the Technorati top 10.
3. The LiveEarth facebook group only has 678 members.

Whoever is responsible for creating Buzz on the blogsophere for LiveEarth is doing a poor job - Paris Hilton and the IPhone are doing much better than LiveEarth, a supposedly global change event.

I'm going back to enjoying the music on the old fashion technology called the Television! :)

Problems with Live Earth Broadcast

2007-07-07T07:24:00.000-05:00

Compared to the Live8 broadcast that AOL did, the LiveEarth broadcast is a disappointment. The resolution of the video is significantly poorer and yet I'm finding that the stream is having a lot of problems handling the load. There are a lot of blackouts, frequent rebuffering of content and there have already been complaints that the site only works in Internet Explorer.

This could have been a fantastic showcase for Silverlight - with the quick download of the plugin Microsoft could have showcased its new technology including some fantastic video streaming.

But instead we see choppy video, disconnected audio and very slow response rate on the interface.

What a missed opportunity!

Creating a Single Sign-On Solution for .NET Part 2

2007-07-06T08:11:00.000-05:00

For the past couple days I've been building my single sign-on solution based on my previously posted design.

I built some simple web services that allow for login against a membership profile database using the supplied ASP.NET membership architecture. By registering the membership and logging in, we can provide a token that can be used to represent the logged in user. This token can then be distributed on the URL and passed to partner sites. The partner site can the validate the token and retrieve the membership information through a web service method.

Here is the code:

using System;
using System.Data;
using System.Web;
using System.Web.Caching;
using System.Web.Security;
using System.Collections;
using System.Web.Services;
using System.Web.Services.Protocols;
using System.ComponentModel;
using System.Web.Profile;
using System.Configuration;
using System.Security.Principal;

namespace SingleSignOn.WebService
{

[WebService(Namespace = "http://localhost/")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
[ToolboxItem(false)]
public class MemberWebService : System.Web.Services.WebService
{
///

/// Logs in a user and provides back a token. Tokens can then be used to regrab the profile data.
///

///
///
///
[WebMethod]
public string registerMember(string userName, string password)
{
if (Membership.ValidateUser(userName, password))
{
Guid token = Guid.NewGuid();
HttpContext.Current.Cache.Insert(token.ToString(), userName, null, Cache.NoAbsoluteExpiration, TimeSpan.FromMinutes(20));
return token.ToString();
}
else
return null;
}

///

/// Validates a token and provides back a membership user in return. If the member isn't found then null is returned.
///

/// Token to validate.
/// MembershipUser object for the mapped user. If token isn't found then null is returned.
[WebMethod]
public MembershipUser validateMember(string token)
{
string userName = (string) HttpContext.Current.Cache[token];
MembershipUser member = Membership.GetUser(userName);
return member;
}

///

/// Returns the current profile object.
///

/// Token to map to a currently logged in user.
/// Array of name value pairs.
[WebMethod]
public object[][] getProfile(string token)
{
string userName = (string)HttpContext.Current.Cache[token];
Hashtable profileHash = new Hashtable();
ProfileBase profile = WebProfile.Create(userName, true);
foreach (SettingsProperty property in WebProfile.Properties)
{
profileHash.Add(property.Name, profile[property.Name]);
}
return toJaggedArray(profileHash);
}

///

/// This doesn't work. I have not yet found a way to update profiles properly in ASP.NET.
///

///
///
///
[WebMethod]
public void updateProfileProperty(string token, string propertyName, string value)
{
try
{
}
catch (Exception e)
{
System.Diagnostics.Debug.Print(e.Message);
}
}

///

/// Invalidate the cache entry containing the token.
///

/// Token representing user
[WebMethod]
public void logout(string token)
{
HttpContext.Current.Cache.Remove(token);
}

private object[][] toJaggedArray(Hashtable ht)
{
object[][] oo = new object[ht.Count][];
int i = 0;
foreach (object key in ht.Keys)
{
oo[i] = new object[] { key, ht[key] };
i++;
}
return oo;
}

private Hashtable toHashtable(object[][] oo)
{
Hashtable ht = new Hashtable(oo.Length);
foreach (object[] pair in oo)
{
object key = pair[0];
object value = pair[1];
ht[key] = value;
}
return ht;
}
}
}

One small technical note - because web services don't support Hashtables, you need to convert these to a more basic array in order to return them. See the getProfile method as an example.

There is one fundamental problem - trying to manage other people's profiles. It seems that the ASP.NET profile management is based on the assumption that you are reading and writing a current logged in user's profile context. However, in this case, we want to be able to proxy the profile data out to the world which means being able to read and write any user's profile data. There doesn't seem to be an easy way to do this in the current profile architecture. You can get a Profile object from the HttpContext, but this is the current logged in user. You can use the ProfileManager class to grab profile objects but it doesn't provide you access to individual properties of the profile. You can load up a profile object using the GetProfile method, but then there is no way to save the profile.

So it seems that if I'm going to use a distributed global profile management system, I'm going to have to build my own. This isn't really a huge problem as the built in profile management system is fairly weak (see my previous post on general problems with the ASP.NET profile) and building a basic profile management system to store name value pairs or serialized objects isn't that difficult.

But the good news is the basic idea of proxying a token across a domain to store a logged in user seems to work in principle. The next challenge is figuring out the best way to secure it so that only valid partner sites can use the service.

Creating a Single Sign-On Solution for .NET

2007-07-04T10:35:00.000-05:00

We have a basic single sign-on requirement. We need to be able to allow users to login and pass that login to other distributed systems.

Here are the basic requirements:

1. We want to control the master user profile so that we can have access to the data and integrate it into our offline CRM systems.

2. We have a number of partners who provide hosted applications for us that require authentication. We need to provide a mechanism to have a single sign-on for users that can then be passed to to these partners.

3. The partners may have their own profile requirements that are independent of our needs. For example, let's assume we have a partner that provides community forums and saves the user's favourite forums in their profile. Only the partner cares about this information and centralizing it doesn't add any real value.

4. The solution has to work in a distributed, multi-platform environment. The only access to and from the partners and us is through port 80 using either REST or Web Services calls.

5. The domain may not be the same and therefore you cannot share cookies (see http://www.codeproject.com/aspnet/aspnetsinglesignon.asp and www.411asp.net/func/goto?id=5998410 for a solution based on the assumption that you can simply poke the cookie created by ASP.NET)

6. There needs to be a way to transfer state from one server to another in cases where the user is logged in already. If user logs in Site A and then goes to Site B, they should appear to be still logged in.

So here is the basic design:

1. User logs in by calling a login web service method (see this example for a web services method implementation of login). User is now authenticated on the main site.
2. User links to partner site which passes a token representation of the state (we could use the SessionID itself in .NET as a token).
3. The partner site receives the token and calls a ValidateToken web service method. This validates the passed in token and provides back the membership information for the current user.
4. If the partner site collects user information, the site can call the Update web service method to update the master profile.
5. If the user wants to link back from the partner site to the main site, the partner site provides a link back to the main site with the token embedded in the query string.
6. The main site receives the token and calls a ValidateToken web service method. This validates the passed in token and provides back the membership information for the current user.

This basic architecture should work with a couple basic caveats:

1. State needs to be managed centrally with the usual challenges of managing session state in a load balanced environment.
2. There needs to be some security around the token validation service to prevent malicious sites from getting access to the service. There are lots of solutions for this including putting the web services under VPN, including a site authentication mechanism in the web service method, filtering by IP, etc.
3. Each link needs to be dynamically re-written to include the token. If there are lots of existing cross-over points then this could be a challenge.
4. The parsing of the token needs to happen on every page request. In .NET this could be done using an HTTP Module as a central parsing code. Alternatively, you could have a redirect page that all pages link to first with the original target url embedded in a query parameter (this is how typical login pages work for example).
5. There needs to be some mechanism for keeping the state alive so that it doesn't time out. This could again done using an HTTP Module that on every page request does a web service request to validate token to simply keep the state alive.
6. The solution assumes that there is a single backing master profile that distributes profile information. A "peer to peer" type model could be developed but it would be more complicated.

From a look and feel perspective, the solution doesn't require a single login or registration experience but it probably would make sense. Keep the login and registration forms on the main web site and have them redirect back to the partner site if the request is coming from the partner.

Implementing an Agile Change Control Process

2007-07-04T08:22:00.000-05:00

What is Change Control?

Change Control traditionally is a project management process for managing change. The typical method for managing change in a classic PMP project is a Change Control form that is filled out containing the scope of the change, impact to the project, etc. This is then approved and incorporated into the master project plan, charter, etc.

In addition, Change Control processes are used by operations departments to manage the deployment and promotion of code from one environment to another. The Change Control Form provides the documentation for an operations team to deploy, operate and monitor the change.

Change Control and Agile

This traditional view of Change Control is at odds with Agile methodologies for the following reasons:

1. Change is viewed as a non-typical event, e.g. the project was running smoothly until someone threw a change into the project. Agile projects by definition are based on the assumption that changes are typical events and aren't special.

2. Change requests are simply Waterfall designs in miniature. Requirements for the change must be documented up front and analyzed. Stakeholders must approve of changes before they are made with the assumption that requirements, scope, risk, and design can be formally documented ahead of making a code change.

3. Traditional Change Control processes are document heavy. The process is designed as much to capture and document change as it is to make effective decisions. Change control forms are kept around forever with dates and identifiers so that someone could find out who approved a change. In many cases, these are also printed out, resulting in very large binders of paper with signatures for every change.

4. In an agile project, one of the objectives is to flatten the team and reduce hierarchies so that people can more effectively communicate. However, traditional change control processes require specialized knowledge, special powers of approval and a hierarchical command and control structure. Project Managers are usually the writers of change control forms, but then they must get approval from "Stakeholders" who will bless the change.

Here is a good example of a traditional Change Control process - have a look at Bell South's Change Control Process Guide. At 120 pages, it is a great example of how bloated a large formal change control process can get when Project Managers are working in a bureaucratic environment. Just managing the process documentation alone takes a massive investment - just look at the 20 pages of version numbers of this document and you'll see that someone's full-time job must be simply keeping the guide up to date!

Change Control isn't a bad thing - the original objectives behind formal Change Control are actually in line with the objectives of agile methodologies:

1. Define change and anticipate change early instead of leaving it to the end.
2. Providing a process for incorporating change during development of a project.
3. Communicate scope, risks and impacts of a change.
4. Provide the appropriate review and approval process to authorize a change.

It isn't that change control is bad - it's that the evolution of formal project methodologies has led to very bureaucratic, documentation heavy processes. In addition, keep in mind that most of these processes were invented before the incorporation of digital collaboration tools such as email, Wikis, sharepoint, JIRA, Word, etc. and so the typical change control process has artifacts from a non-digital era.

Implementing an Agile Change Control Process

Change Control can be used effectively in an Agile environment if it is sufficiently light weight, non-bureaucratic and collaborative. Here are my recommendations based on what I have seen work effectively on Agile and non-Agile projects:

1. Use an agile methodology such as XP, Crystal, Scrum, etc. that has change management built into its fundamental process. In Scrum, for example, all work is effectively a change and is logged as a "change request" in a work backlog. In XP the changes are logged as User Stories. Iterations allow for changes to happen on a regular basis - they're not special change events.

2. Use a generic task tracking system such as JIRA to log any change. In JIRA, you don't need a separate "bug tracking" system that is different than your change tracking system. In addition, you can take a change all the way from requirements to deployment without re-entering information. You could do the same thing with low tech tools such as Excel or Word simply by having a work log that is updated regularly.

3. If you have specific legal or regulatory reasons to track changes then try to understand the specific minimum requirements. I've worked in a government environment for example where all changes needed to be documented so that they could be audited. But the level of detail that was incorporated into each Change Control form became ridiculous - an auditor probably doesn't really need to see a change in an individual SQL script, a deployment checklist that includes changing ODBC connection strings, etc.!

4. Eliminate as many approvals as possible. We had a change control form that had 10 signatures on it - I was one of them! I used to get these forms across my desk documenting some change to an application that I had never even seen, had no idea how to measure the risk of the change and yet I was expected to "approve" it. Even worse, the signature had to be on paper so there was a constant stream of people running around with pieces of paper trying to find me to get my signature. Eliminate all this process and identify the minimum number of people who need to be involved in any change.

5. If you're going to have a Change Control Form/Process, keep in as light as possible. Your form should be no more than 2 pages and should be as automated as possible. We currently use JIRA to automate all our tickets. In previous environments, we've used Outlook forms. I've also seen this done use Word with Macros enabled to provide drop downs, automatic fill-in, etc. Keep your process light as well - if you need 2 hours of training to teach someone how to make a code change, then something is wrong!

6. Keep Operations teams involved from day one. One of the primary reasons why change control forms are deployed is because Operations teams get tired of having code dumped on them without documentation or involvement. The typical solution is to put in a gate keeping process to keep the developers from deploying something unsafe to production. This is a legitimate role, but change control processes typically provide artificial security - simply because a developer has filled in the proper form doesn't mean their code works as advertised. Instead, have an Operations Lead that is an integral part of the project from day one who will have a specific understanding of the project and create the right level of documentation, deployment guides, test cases, etc. and be testing these along the way. This is far more effective than a large form being written at the end of the cycle.

7. Keep stakeholders involved throughout the project. The reason why Change Control forms need to be so heavily documented is because the key stakeholders have not been involved all along the way. If stakeholders are seeing working software using Show and Tell techniques, then Change Control simply becomes regular involvement in the project.

8. Use multiple models of communication. Instead of having a single Change Control Form that forces people to write in a specific way, allow for multiple models of communication that could include meetings, power point presentations, white boards, Visio diagrams, etc.

Based on the above principles, here is my current favorite flavour of Change Control:

1. We build our requirements using use case documents that living documents. We currently use Word because its dumb simple and stakeholders can easily open it and change it. We use power point for wire frames and Visio for almost everything else.

2. Revisions of documents get stored in source code control for safe keeping and history. However, we're not dogmatic about it and in my experience the value of these minor revisions is usually not significant - most stakeholders want to see the current version. We keep them in our repository so that everything is stored together and document revisions are tied to code revisions.

3. We have iterations every 2 weeks that include review meetings with the entire team including stakeholders. We do a show and tell of all working code and we use the meeting to prioritize and change. We typically do this by opening up documents on the spot, but sometimes we use pen and paper and transcribe the changes right after the meeting.

4. We use JIRA for day to day tracking of tasks. We treat all changes, bugs, feature requests and tickets the same way. We have a very simple work flow that moves all tasks from open to resolved to closed. Our list of changes can be pulled directly from JIRA as a list of changes. The list of outstanding bugs can be also pulled at any time so we don't need a specific final bug review meeting - we will have been reviewing them on a daily basis.

5. When we're getting close to deployment, we use white board based checklists to capture the minute details (e.g. make sure the ODBC driver is configured properly) that need to be done to move to production. We check these off by simply crossing them off or wiping them from the list. We have 15-20 minute stand-up meetings that includes the Operations Lead so that we're communicating effectively with Operations and they know exactly where things are at as we move into production.

6. Just before doing a deployment to production, I have a meeting with executive stakeholders with a very high level presentation in power point. It presents the current status, any risks that they need to understand, confidence in going, and high level operations plan. Executives typically don't care about whether the SQL is right - they want to know that the team has confidence that it can deploy safely, has followed the appropriate testing processes and that the team is on standby with a contingency plan in case things go wrong.

There are lots of variations to this process and many different tool combinations. You can scale the process up or down depending on the size and formality of the project. In addition, if you're working in a specific regulatory context you may need to be producing more formal documents, approvals, audit trails, etc. My suggestion is always prioritize these below communication and if you need to do this for regulatory purposes then do so but keep the work as minimized as possible. Never substitute change control processes for good communication, team collaboration, stakeholder involvement, etc. In addition, remember that Change Control is a means and not an end - working software is the objective and should be focus for the team.

Communication of Software Risks

2007-06-24T11:37:00.000-05:00

One of the interesting challenges in managing a project is estimating technical risk.

Here is an example of a classic technology risk scenario:

The development team is building an online transaction system that sends orders from an e-commerce system to a warehouse. The e-commerce system is internal, the warehouse is external. The warehouse works on an AS400 system and no one on the team has ever worked with AS400 platforms before. There is a defined web services layer between the two systems which should hopefully hide the workings of the AS400 system behind the layer.

This is a pretty typical high level description of requirements from a manager, a request for proposal, or an email.

Most developers look at this and start analyzing all the things that could go wrong...there are dozens of risks just in this one paragraph.

A project manager might realize that there are some risks involved and if the PM has a technical background might be starting to ask some risk based questions that show some understanding of the potential problems. The worst case scenario is a non-technical PM who thinks that this project is the equivalent of ordering pizza from Domino's - you simply call and expect the order delivered by the guaranteed delivery window.

Let's assume that the entire team is on the ball, everyone's working together and the team comes together to review the project plan. A risk matrix gets created and the team comes up with the following risks:

Risk: Web Service might not work
Probability: Medium
Impact: High

Risk: AS400 might have performance problems
Probability: Medium
Impact: Medium

Risk: Transactions might not work across the two distributed systems.
Probably: High
Impact: Low/Medium

and so on....

This is the critical point where there is a massive communication gap between what a typical PM thinks a risk impact is and what a developer think of a risk impact.

A PM may think, "Well we have some risks so we should allocated some contingency in the budget. This is a high risk project, so we'll add in 30% contingency to our estimate and a month onto the time line."

A Developer may think: "If this server cannot move data fast enough, we're dead in the water. It won't matter how much time or money you throw at it because short of hauling out the entire AS400 out the warehouse we're toast."

Big difference in risk!

I've seen projects where there were mature team members and did all the traditional risk analysis that is recommended on software projects. But the project still failed because there were fundamental differences in assumptions in what "High Risk" actually meant to cost, time line and project success.

Here are some common reasons why there are differences in assumptions around risks.

1. Team members don't all have the same assumptions on impact of a risk. For example, a task is estimated at 3 days but the impact of the risk is "High". So what does that mean to the estimate? Add 10%? Add 50%? What about 10X? In my experience, project managers in particular do not understand the sheer magnitude of software risks - they'll see a risk as "High" and put in contingency of an additional 30% and think they're good to go.

A 10X risk impact is not unheard of - I've seen it myself on many projects. We had one project where one web service call was supposed to take 3 days and took 30 days because of technical complications.

2. There are some risks that result in project failure because in some cases requirements are not variable. For example, financial data cannot be more or less correct - its either right or its wrong. So getting to 80% right isn't really valuable - if you cannot hit the 100% mark then your software project fails. So if you have a risk that the data isn't going to be right then it isn't so much that your project went over budget or took too long to deliver - its a total failure because it cannot meet the basic requirements. Same thing with external software dependencies - for example we had a case where we were integrating with a web services framework and there was a bug in one of the supplied web services from the vendor. It wasn't a matter of "let's just find a work around" - the project was effectively dead in the water until that bug was solved.

Project managers tend to be solution oriented and optimistic, so while they like the idea of contingency, they tend to shy away from the concept that a risk might actually kill their project with no way to get around it.

3. The "I don't know" estimate will get quantified too early on in the process or not be listened to at all. If your team is saying, "I don't know" as a project manager, what is your approach? Many PMs are forced into providing estimates too early by management, so they simply fudge the "I don't know" into a contingency or pad the estimate. They think that by taking the developer's estimate and doubling it they've been given themselves a massive breathing space to worry about the problem later.

The cone of uncertainty on typical projects is a lot bigger than most PMs realize - Steve McConnell's work on estimation pegs the variance at 1600%!

If the team doesn't have a common understanding of how to quantify the impact of risks, then everyone can be communicating using a traditional risk process but the shared understanding of the impact of any risk will tend to be different.

Here are some recommendations for Project Managers for creating an improved risk communication culture:

1. Vary the risk based on expected uncertainty level and the phase of the project. For example, if you're in the planning phase the expected uncertainty level is high. If someone on the team says, "Well we don't really know how that's going to work but we have a plan" that's probably OK. If you hear the same thing in the middle of QA, then you should be very concerned and adjust accordingly.

2. Have the team translated risks from labels ("High", "Medium", "Low") to numbers where possible. In addition, don't assume that one set of labels means the same across the project. For example, a "High Risk" HTML issue might be something very different than a "High Risk" web services integration issue or performance issue.

3. Identify the requirements that are absolute and unmovable and pay particular attention to risks that are around them. For example, in some cases performance is an absolute requirement (e.g. it must serve 30 pages per second) and sometimes it isn't (it must be "fast"). If your requirements are absolute then any risk that hits those requirements will not just cause a delay, an increase in cost, etc. - it will kill your project.

4. Pay close attention to areas of the project where there is limited flexibility or options. Compare the flexibility of a creative front end vs. a bought software system. If someone doesn't like the the front-end, you can rebuild it, change it, etc. It might cost more, create a time impact, etc. but you've got lots of flexibility for change. If you buy Microsoft Content Management Server and it has a bug in it your options are pretty limited.

5. "I don't know" must be a viable option in any discussion. In addition, the response to "I don't know" should be to go find out - be careful about arbitrary padding of estimates simply to factor it into your plan.

6. Pay close attention to body language, talent and historical experience. Relationships matter in software teams and a team that has worked together before has the advantage that the PM starts to recognize the subtleties of individual communication styles and to understand how each person reacts to different questions or risks. Is John an optimist or a pessimist? When someone says, "I don't know" are they being cagey or honest? Teams who have worked together start to intuitive find out these non-verbal cues.

7. Review risks on a regular basis so that you can refine your strategies to mitigate them.

Hopefully that helps you in creating a more pro-active risk based approach and to be aware that just because people the team are calling somet "high risk" they may not actually mean the same thing.

Developer Weasel Words

2007-06-21T17:57:00.000-05:00

As a development manager or project manager, you here a lot of weasel words and excuses from your staff or from external consultants who are trying to hose you into believing things are better than they are. In many cases, developers use these phrases to even convince themselves that things are better than they are, resulting in chronic late delivery and poor quality.

So beware the following phrases from your teams:

It should work: this usually means that it doesn't. It also means that it was probably not tested properly as the result is current undetermined. The word "should" should be taken out every developer's vocabulary - it either does or it doesn't.
I just need: beware of that word "just". Its a belittling word meant to make things smaller than they are. Just take the phrase "I just need to write this component" to be "I need to write this component" and already the magnitude of the work involved grows. Developers tend to be chronic under-estimators and the use of the word "just" is a sign of that mentality.
Almost done: this is also a weasel word. When a developer tells you things are "almost done" ask for the specific tasks that are left over immediately. In addition, keep in mind that projects do not progress linearly - the last 10% is always about 40-50% of the work of the total project. I've seen projects that are chronically late be "almost done" for 3 months.
It was tested: this also usually means it that wasn't tested properly. Ask for a test plan and the specific tests that were done. If the developer cannot produce these with sufficient evidence that PROVES that it was tested, then it wasn't tested.
It must be an environment/configuration/deployment problem: this may be actually true, but it usually points to a larger stability problem. If you cannot build and deploy reliably then why would you have confidence that the code works?
If things go smoothly: this I hear a lot, e.g. if we don't hit any snags then we can be done by Friday. Guess what - you're likelihood of hitting a "snag" by next Friday is probably high and given the lack of risk based management, the team has probably got no mitigation or contingency strategy. Then next week, you'll hear the next phrase in our list, "Yes, it could have been done if it weren't for that Snag we had".
Yes, but, as in "Yes it can be done, but": this means it cannot be done. Tell your staff to just come clean and say, "No it cannot be done". Another variant on this is, "Yes it could have been done, if it weren't for marketing, requirements, technical risk, etc." This simply shows that your developers work in an idealistic world where things never go wrong.
We'll make up the time at the end: if you're already late by the end of requirements, you're likely going to be even later by the end if you simply keep going on the same track. In my experience, teams don't dramatically faster as they hit their stride. Even if there is some efficiency, its nowhere enough to make up for lost time.
I've got it done, but I need to build a few more components for you to be able to see it: then its not done! Encourage show and tells, code reviews, unit tests, etc. so that code is visible as soon as possible. Use slicing models so that you can see pieces of the application in weeks, not months. In addition, code that isn't checked in should never be counted - that means that someone cannot build it sufficiently to share it. It only counts if you can verify it. Ideally, its not "done" unless there are sufficient unit tests, a build script, and a document that someone walking into the code repository could check out the project run a build and have all the unit tests pass. Then the code is done - anything less is mythical.
I've got it done - I just need to integrate it: The word "Integrate" is a big weasel word. Think of a web service that adds two numbers together. The algorithm is one line of code. But the integration work is huge. In addition, integration usually means the first time that disparate teams are bringing code together which is always cause for issues. Don't under-estimate the integration, especially in today's world of distributing computing, web services, etc.
It Worked on my Machine!: programmers use this excuse to downplay a bug. The reality is actually the opposite - it means that you have an intermittent bug which is by far the worst kind of bug to have in your application. You want bugs to fail quickly and consistently - any variant such as "That's Weird", "That didn't happen yesterday", "That must be a data problem", etc. is admitting you have a bug that cannot be easily duplicated.

My recommendations to reduce the amount of excuse making from your team:

Encourage a culture of honesty and team work: You get these excuses when developers are hiding things, and sometimes this is because you've created a culture that encourages hiding because you don't want an honest answer.
Be ruthless with your quality and talent standards: don't excuse poor talent, bad management or chronic late delivery. If you create a culture where talent isn't rewarded the bar isn't kept high then you'll be excusing the team to continually strive for excellence.
Expect more than just code: measure performance based on estimation, quality, delivery and team work as well as pure code quality. If you have a developer who produces great code but cannot deliver on time then that's not a great developer.
Increase visibility and shrink delivery cycles: if you have to show your work on a constant basis and deliver on 2 week iteration cycles, your excuses tend to go away. You either deliver or you get found out pretty quickly. Use show and tells, code reviews and continuous integration to see what people are doing on a constant basis.
Don't give half credit for 50% done - its either done or not done: If your tasks cannot be managed this way, then you should split up your tasks until you can work this way.
Establish what "Done" really means: for example, at a minimum "Done" should mean checked into source code control and able to build into the current branch. If you're doing Test Driven Development, it should also mean all tests run successfully. If you have specific performance criteria, then its not "Done" until the performance tests pass.
Use counting techniques to measure wherever possible: this is a great suggestion from McConnell's book on estimation. The more you can count in units, the more accurate you estimation. So if you can count the number of pages, web services, objects, databases, tables, stored procedures, tasks, etc. that are left to accomplish then you can measure them more easily than if its a big blob of work. If your requirements aren't well defined enough to count objects, e.g. you don't know how many web pages you're building in your web site, then you're really not in a position to estimate your ship date.
Don't sucker, manipulate or bully your team: If as a project manager, you resort to traditional management tactics such as playing games, being political, establishing a blame culture, or bullying your team you'll lose your credibility and simply encourage lying. A tortured prisoner will tell you anything you want to hear - the same goes with development teams.

If you have a project that operates in the open, has a culture of honesty and establishes a high performance bar, you'll find that peer pressure as well as some overall guidance will get risks, problems and bugs out in the open. If when you discover these problems people work as a team to fix them instead of blaming each other then every problem solved becomes a victory and not a blame opportunity. You'll get better answers and improved morale on the team as you set clearer performance expectations.

How to be an Effective Stakeholder - Part 2

2007-06-18T20:13:00.001-05:00

The Role of the Stakeholder
Part 1 - Introduction

Stakeholders who are effective on projects understand their valuable role. This post articulates what effective Stakeholders bring to the table.

There are a few different types of Stakeholders:

Direct Customers: if you're lucky, you get end-users as stakeholders. These are great to have because you know that these users represent actual end users who will use the application on a daily basis.
Domain Owners: some stakeholders are involved in software projects because of their domain expertise. Software developers are good at building software, but they're usually not domain experts as well (although some will claim to be). So if you're building a financial application, its important to have someone on the project who knows the minute rules on accounting. Not only is it important to have general expertise, but it's equally important to have domain experts who know the specifics of the business. There are dozens if not hundreds of hidden business rules in any business and in many cases they are only learned through oral tradition. Domain owners provide this expertise.
Customer Group Representatives: if you cannot get direct customers or if your user base is large, then you need to have Stakeholders who can represent the needs of the customer. Be careful about this in that it's important that the Stakeholder is an accurate voice for the customer and doesn't have their own agenda.
Business Representatives: general Stakeholders that represent the actual business. If the business is selling soft drinks, then it's a good idea to have an understanding of how the actual business works and how your application will fit into and support the business of selling soft drinks. In any business, there are also specific experts who own global business areas such as marketing, security, privacy, branding, and customer service. These horizontal service areas are not specific to any vertical project but cross all projects being delivered.

Regardless of the type of Stakeholder, here is what I'm looking for them:

Decision Making: a Stakeholder's primary responsibility is to make a decision. Should we charge the credit card before we ship the product or after? Should the name field be 100 characters long? Should the colour of the button be purple or blue? There are hundreds of decisions that a software team must make and your role as a Stakeholder is to make the call when asked.
Approvals: similar to Decision Making, your job as a Stakeholder is to approve documents, decisions, milestones, etc. When I was a Stakeholder on a project with Accenture, they would send me test cases to review. I would simply reply with a single word, "Approved." Especially with external vendors, teams need a formal mechanism for sign-off before they can move forward.
Opinion and Feedback: in building web sites in particular, there are lots of subjective decisions on what a web site should look like, how it feels, whether it represents the brand, etc. As a stakeholder, your primary job is to have an opinion that can be translated into actions, revisions and improvements. An example of a poor opinion: I saw a graphic design present a screen design to the Creative Director. His opinion was, "I want exactly what you have, but just bigger and bolder!" How is someone supposed to action that opinion? A good opinion is, "I hate that text - its boring and its too long." It's just as subjective but at least you can action it.
Solicitation of Feedback Amongst the Broader Community: Most Stakeholders represent their department, a user community or a business unit. Key deliverables should be brought back to this broader community for feedback. As a Stakeholder, your job is to help explain, sell the concepts being presented and capture feedback that can be actioned by the team.
Be Supportive Externally and Demanding Internally: Stakeholders should be evangelists for the team as it helps boost confidence for the team, helps provide buy-in from the great organization, etc. Internally, Stakeholders should be demanding like a good customer. Stakeholders should be keeping the bar high and making sure that the software team feels that they have to meet a high bar for quality and performance.

What frustrates software teams to no end is when Stakeholders are not doing their job. Here are some examples of where Stakeholders drop the ball:

Showing up to Meetings Unprepared: if you cannot bother to take 15 minutes to read through a technical document that someone spent 2 weeks writing, what does this say about your level of respect for the team?
Being Late with Approvals or Feedback: when I manage projects, I put review and feedback periods in the project plan. Stakeholders have deadlines for turn around of feedback. Yet time and time again, I've had Stakeholders miss their deadlines with really lame excuses. If you miss your deadlines for reviewing a document, what kind of commitment can you expect for higher risk activities such as building new algorithms or leveraging the latest technologies?
Abusing the Team: this symptom seems particularly bad with external teams being managed by internal Stakeholders. There must be something in management training that tells middle managers to treat external vendors abusively in order to squeeze them for another 5-10% in performance. Unfortunately for many smaller and less mature vendors, the practice works which only reinforces the practice. However, in my experience a mature software team won't take abuse and understand the difference between having high expectations and simple bullying.
Poor requirements: if you're supposed to be a Domain Expert, then you need to be able to know your domain. Ideally, your domain is well articulated through existing documents, manuals or practices. If you're supposed to be providing requirements to the software team, then these requirements should be well defined and articulated. Here is an example of poor requirements: we were supposed to build an online store for a client, but no one had given any thought to basic requirements such as tax rules, catalogue structure, content requirements, fulfillment, inventory rules, etc. When we asked basic requirements such as, "What are you current inventory rules?" the answer we got back was, "Well, we don't really have any rules - what do you suggest?". Argh!
Us and Them: I would highly recommend for Stakeholders to embed themselves into the team as much as possible. The more that Stakeholders are part of the team, the better the communication. If you find yourself viewing yourself as the "customer" and not part of the team, figure out how to get yourself better integrated. Take the team for drinks, start coming to meetings, and support the team by doing your job. When Stakeholders build credibility with the delivery team, it means that when they actually need to put some pressure on them to deliver they'll have credibility to spend instead of simply crying wolf.

I've been on projects as a Stakeholder, a project manager, a technical architect and a sponsor. When the Stakeholder brings their value and integrates highly into the team, it makes a massive difference in the success of a project. If the Stakeholder is flying in and out, using politics and manipulation to get what they want, or being abusive in order to squeeze the team, it tends to put the entire project at risk. As a Stakeholder, I feel responsible for the entire success of the project and I'm not just there to provide input and walk away from the scene. In addition, I don't view software projects as ordering McDonald's whereby you simply throw in your requirements and drive up to the pick-up window to get your order!

Problem with Shoutwire Tag

2007-06-17T14:23:00.000-05:00

I've posted bugs previously relating to the little Digg and Shoutwire tags that allow you to vote. The generated CSS tends to be quite buggy and these tags don't play nice together, e.g. having a Digg and a Shoutwire button next to each other is impossible.

I discovered yesterday that having the Shoutwire Vote for Me tag breaks my blog template completely now. I'm not sure why it does, but having the Shoutwire TAG means the rest of the page gets mangled with pictures not showing properly, ad words not showing up, etc. Taking it out instantly restores the blog template to its proper format, so it must the generated HTML. It fails in both IE 7 and Firefox.

I've gone through my blog postings and stripped out all the Shoutwire buttons as a result. You can still go to Shoutwire.com and if you see a posting of mine there you can vote for it on their main site.

How to be an Effective "Stake Holder"

2007-06-17T10:33:00.000-05:00

Part 1- Introduction

Every software project has "Stakeholders". There are other terms used for this role such as "Internal Customer", "Steering Committee Member", "Department Head", "Boss", "Sponsor", "Champion", etc. Sometimes stakeholders are formally identified (usually by assigning them to a project directly) but there are also lots of informal stakeholders as well.

While the context for these articles is software development projects, in my experience being an effective Stakeholder is consistent across all projects and the same guidelines apply consistently for other projects as well, e.g. marketing projects, facilities projects, etc. As a Director of Information Technology, I play the role of Stakeholder on some projects and been on the delivery side on other projects. I've worked on projects that are internally delivered as well as worked as an external consultant. So I have seen the Stakeholder role from a number of different angles.

This introductory post gives you an introduction to the role of the Stakeholder and what is expected from you if you have been assigned as one. In future posts, I'll provide some specific techniques, guidelines and suggestions for managing and influencing project teams for Stakeholders.

Your Boss Comes to You with a New Project...

Here are some really bad but unfortunately typical ways that Stakeholders get assigned:

Your boss has been assigned the role and doesn't want to do it. So he or she delegates it to you. You can continue to delegate down if you choose to do so. I've actually had this happen to the point where the original stakeholder was the VP of Marketing and by the time the project had started a junior copy editor showed up saying, "I was told that I'm to represent marketing on this project"!
You're in a role of authority and someone on the project thinks you should be consulted. For example, you're the Director of Marketing and someone in HR decides that they want to change the web site to have job postings. You get roped in because you've issued the policy that no one changes the web site without your input. You're now a Stakeholder!
The project is half way through and someone realizes they need your input. This happens to Operations teams a lot - 80% of the way through development, someone on the team realizes that they need 10 more servers! As the person owning this non-budgeted purchase you're now a Stakeholder!
You're asked to join or are already part of a committee that regularly meets to discuss general issues. For example, many organizations have branding committees, privacy committees, project review committees, etc. Sometimes projects will make the entire committee a Stakeholder. I had this experience once working on a project building a government web site - there was a permanent "web committee" with 7 members on it who had to meet on every issue and come to consensus on every change. The entire committee was a Stakeholder.

If you find yourself getting labelled as a Stakeholder in any of these scenarios, beware! They are high risk and put increased risk on the project.

Recommendations

Here are some recommendations on when to sign-up to be a Stakeholder.

Be empowered and qualified to be a stakeholder. The primary role of a Stakeholder is to make decisions and provide input. If the project is expecting a VP level Stakeholder and they get the VP's secretary, they're not getting a stakeholder who is qualified or empowered to make decisions.
Avoid Stakeholder proxies. Stakeholder proxies are people who show up to all the meetings, collect a lot of notes but then cannot make decisions on their own. If the answer to questions asked of you is typically, "Well I need to ask my boss whether we're allowed to do that" then tell your boss that the project really needs the boss to be the Stakeholder, not you.
Be prepared to commit a minimum of 2-3 hours a week to the project. If you cannot commit to the time, then having you in the loop just creates additional communications overhead for the project team.
If you are primary stakeholder or if you are providing extensive documentation, business expertise, or testing support then expect this time commitment to be considerably higher. For example, we had a project to deliver a new B2B order entry system. One of our stakeholders was the person who ran the B2B business. During requirements gathering, she was involved in the project about 20-30 hours a week. During development, she was involved about 10 hours a week to clarify requirements, approve changes, etc. During testing, she was involved 40-50 hours a week to help test key functionality, provide clarification on missing requirements, facilitate testing with other people in her department, etc.
Stakeholders should be established up front and given ample warning of expectations. You should know you're on a project from the beginning even if you won't be needed until further down the line. In addition, you should have a context for the entire project and not just be isolated to your area.
Beware of people randomly assigning you as a Stakeholder through conversations in the hall, informal emails, etc. It may come back at you if the project turns south.
The inverse of the above is don't assign yourself as a Stakeholder simply because you're in a position of power. Simply inserting yourself into a project is not an effective way to be a Stakeholder. In addition, don't take credit for projects where you just went to a single meeting - that's not being a Stakeholder!
If you work in a traditional departmental structure, switching to a project based model will require radical changes in how department heads interact with project teams. Many Directors assume that if a single resource of their department is on any project, they automatically get to have a say in every project decision. Even worse, some Directors simply reserve the right to sit on the sidelines and snipe at the project teams at any time at random. You'll see this in emails coming from Directors that start with, "I noticed this project is trying to use MY resource (equipment, people, brains, etc.) and before I ALLOW YOU to do so, I must REVIEW AND APPROVE this project". Don't assume that just because you're a Director you're automatically a Stakeholder on every project!
Avoid projects where there are many Stakeholders, especially if there is no Stakeholder hierarchy or specific areas of ownership. Here is a good sign of too many stakeholders - if there are more stakeholders than project team members!
Try to be on projects where you have a personal interest in their success. This creates investment, accountability and passion. Part of your role as a Stakeholder is to be a champion, so having a personal stake in the success of the project makes a huge difference for you and the project team. If your career is on the line because of a project, do you think you're going to be a great Stakeholder and do what it takes to be responsive? Lesson to executives - make your Directors bonuses tied to project targets, goals and successes and not just the running of the department.

Depending on where you work, you will more or less control over the projects in which you are responsible. However, all this means is that the techniques you use are more or less political There are lots of ways to get on and off a project - some of them explicit and some of them implicit (More on this in future posts). In addition, if you have no choice but to be the Stakeholder for a project then there are things you can do to improve your contribution to the project and ensure a greater chance of success.

Stay tuned for future posts for more recommendations on how to be a better Stakeholder...

Why Agile Software Projects are Good for Customers

2007-06-12T21:40:00.000-05:00

We have two software projects that are going on at the Foundation. Project #1 is just finishing. It was not managed by me and it was done using a traditional waterfall death march type methodology. Project #2 is being managed by me and we're using a traditional Agile project management approach.

Comparing the two projects really shows the different in the benefits and obligations of the customer in an agile project:

Project #1 is 100% over original estimate for no apparent reason. The customer gave up trying to even get a date from because they haven't seen any deliverables, don't even really understand the scope of work and just want it done.
In Project #1, there was about 1 month just spent trying to define requirements. There were dozens of meetings trying to nail down field mappings with no code being written in the meantime (which starts to explain why the project is late). The right people were never in the room and the developers got different answers from different people.
In Project #1, despite the investment in time, the output in terms of requirements documentation and certainty of scope was quite poor. Despite the investment, the requirements still changed up until the last week before testing including major business rule corrections.
In Project #2, we've taken a traditional Agile PM approach. We include the customer in every meeting (usually once a week) and we are using a combination of use cases and wire frames to document requirements. This is the first time in the organization that the customers have seen either format so its a bit of a learning curve in terms of how to read them. But it took about an hour of reading through use cases before the customer fundamentally got the value of such a document.
In Project #2, I got my lead technical architect to start prototyping on day one. By week 2 of the project, we already have a prototype writing in .NET using SQL Server 2005. The data model is about 50% done. But before even the documentation is done we have a working model.
In Project #2 for project management we've taken an Iteration Cycle of 2 weeks as the basis for development. We have an architecture/planning/requirements spike for the first 2-3 weeks but have already started coding. Every Iteration has a plan to deliver a list of use case implementations and related wire frames as fully functional code. We have formal show and tell sessions at the end of each cycle.
In Project #2, we have a proper project plan with real estimation. Estimation is done at the wire frame and use case level and tasks are broken down to about 1-3 days in duration. In my experience anything more granular is a waste of time and anything more becomes unpredictable and subject to error.
In Project #2, we will be using Test Driven Development and Customer Testing all the way through the project as well as Continuous Integration.

The experience so far from a customer perspective has been really interesting so far:

Customer participation on Project #2 has been extremely high and involved. The customer role has been really well defined and our stakeholders have been good at providing the right level of input.
There is a good trust relationship and a sense of team - it isn't a US vs. Them approach. We're not delivering software for a customer - we're all as a team delivering on a project.
Our ability to shift priorities and change the schedule is highly responsive. For example, our marketing department is working with the team to figure out the content for the application and its taking a lot more time than expected. So we've started on the back end logistics sections and left the front-end until the end in order to give the marketing folks the most leeway. By having check ins every two weeks we can easily adjust.
In addition, because the application is built in vertical slices instead of horizontal ones, we don't have to have 100% of one tier done before moving to the next one. We can simply move from front end to back and hook together some disconnected pieces as they come available.
The project is simply more fun and more of a team based approach.

Some recommendations to software project teams in treatment of their customers (whether internal or external) in an agile context:

Don't patronize your customer because they've never done software projects before. Don't treat them like idiots because they've never read use cases or UML diagrams or database models before. If you take an hour to introduce the concepts and travel light, you'll find that customers "get it" rather quickly and then see the value of these requirements tools.
Always remember that the customer is the end user and not a stakeholder representing an end user. For example, many Directors of Marketing act as the "customer" in web project teams. They're not. They're representing a customer. Everyone on the team needs to always focus on the end customer and stakeholders always have to be able to filter out their own biases and represent the end user successfully.
Customers will buy into an agile project management approach if it produces results - period. If you cannot deliver software on time or its poor quality, an Agile approach won't save you and will quickly degenerate. Shorter iteration cycles means there is no where to hide, so talent/experience and ability to deliver are critical early success factors.
Show and Tells are critical to the Agile Project. We use to have them once a week on Fridays. Make Show and Tells Fun! We use to dim the lights and use the projector and have popcorn and snacks. Even if its some boring application, hype it up and celebrate it.
Customers should be the leading evangelists as well - some customers have been taught to be the evil task masters and try to crush development teams because they think it gets the team to work harder. I can tell you from the other side of the team that it doesn't work - developers work less, produce poorer quality and flip the bozo bit on you in short order.
One of they values of an Agile project team is transparency and team work. Transparency is a double edged sword - customers have to be prepared to hear the bad news with the good and to help the team prioritize effectively. Good customers develop a relationship with the team that allows them to be trusted - its critical in order to encourage honest dialogue.

Overall, so far Project #2 is doing much better than Project #1. It has lots of risks and we may not be able to deliver all the functions asked for in the time line. But the requirements are well understood, communication is open and use cases are prioritized. We've already had conversations about what parts of the scope could drop if needed and contingency plans if we go late. Customers are in the loop at all times and a core member of the team.

Agile Just Feels Natural

2007-06-12T21:31:00.000-05:00

Have done agile development for most of my career, I find it interesting when I first introduce the concepts to new developers. We're just starting to introduce it to people in my department, and we have got started with reading through the usual standard books by Ambler, Fowler, Beck, et al and I've start acting as the Agile PM on some key software projects.

What I find interesting is when I introduce it to developers, the universal reaction seems to be the following:

1. What a breath of fresh air - this is how we should have doing software all along.
2. I've already been using these techniques but didn't have a methodology to explain it and wasn't rewarded for it. I find that developers who have been raised in the classical SDLC have secretly rebelled against it and agile methodologies simply validate the rebellion.
3. The biggest flaw as most software developers experience with the traditional waterfall model (or worse the traditional death march just get it done by a fixed deadline approach) is that they now have some confidence in their work and ability to deliver against timelines.
4. When I find developers revert back to their old bad habits, they almost always run back into trouble and then slap themselves in the head of going back to hiding their risks, not being iterative and letting projects slip into death marches.

See my next post on why Agile is Great for Customers...

How to Use Microsoft Project Efficiently

2007-06-07T20:03:00.000-05:00

I've worked with a lot of project managers who don't really know how to use MS Project efficiently. I was trained well on MS Project because in the consulting business, you have to be able to build million dollar project plans in days for sales pitches. Our PMO figured out a number of ways of using Microsoft Project more efficiently.

Here are my recommendations:

1. Get rid of the GANTT chart by dragging it to the right. Use the extra space for some additional columns.

2. Go to Window and click on Split. This will bring up the Task Form dialogue box at the bottom. You will find that entering data in the task form is much more efficient and visible than any other method.

3. Go to Tools->Options and click on Schedule. Change the Default Task Type to Fixed Duration and click off the box marked "New Tasks are Effort Driven". The reason is that when you're estimating, you'll find in most cases its easier to think in duration and allocating a % of a persons time across that duration than thinking about person hours. For example, instead of thinking of a task taking 25 hours, its easier to think about a task taking 3 days at 100%. Its also easier to think about a task as taking 2 people x 3 days, especially if you're days are 7.5 hours (do you think of a task as taking 50% of a week or 18.75 hours?)

4. Don't put in any absolute dates in your plans. Make all your dates based on dependencies except for the basic project start date.

5. For tasks that you want to go in parallel, use the SS predecessor type to make sure that multiple tasks start at the same time.

6. Go to Format->Text Styles and click on Item to Change. Select Critical Path and change the colour to something obvious (I use BOLD RED). This will help you in making sure that every task has the appropriate predecessor set - if it doesn't, you'll notice that the critical path stops half way through your project plan. It should highlight tasks from the beginning to the very end of your project.

7. Lets assume you have a project plan that is way past your target timeline and you want to now parallel your effort by adding more people. Many project managers will start creating streams of tasks in parallel with complex dependencies. But here is a simpler way if all you want to know is how many people you need to complete the work on time - simply take your tasks, change them to fixed work and increase the percentage to 400%, 500%, etc. until you get your timeline in the ball park. If its 500%, that means you need 5 full time developers to get the work done.

NOTE: the one draw back to this approach is that it doesn't tell you when each task will be done. But in my experience, this is not that useful as development projects tend to have 50 tasks that can largely be done at any time over a longer period and you simply spread the load across a pool of developers until the work is done. If there are specific tasks that are dependent, then set them up as a seperate stream. But if its simply a pool of tasks that need to get done this technique is much faster and simpler to manage.

8. Don't allocate overhead resources to tasks (e.g. your PM, your TA, your QA lead, your stakeholders, etc.) unless you have very specific requirements. In most cases, you can estimate an overhead resource at a % allocation across an entire project phase. For example, I'll allocate a PM at 30% for the duration of the project, a TA for 20%, etc. This again is a bit of a cheat - if you want to plan that a PM is going to be busy at 100% on Wednesday and 20% on Thursday and 40% on Friday, then this technique fails. In my experience, this is not the way overhead resources work and in the end having your project plan reflect this micro-detail is not worth the work to maintain it.

9. Here are a few ways to calculate contingency:

Click on Insert Column and pick Cost1. Right click on it and select customize field. Then click on Formula and type in Cost*1.10. This will calculate contingency at 10%. If you want 25% contingency then use Cost1*1.25. Make sure that Calculation for Task and Summary Rows has Rollup selected and Sum.

NOTE: There is a bug in Project 2003 where it sometimes does recalculate fields properly. If you notice this, then go to Tools->Options->Calculation and click Calculate Now. This will recalculate the field.

Create a task called Contingency and set a price for it. This is good for setting an arbitrary contingency amount.

Create a task called Contingency and assign a resource called Contingency at sufficient hours to drive up the price. The advantage of this approach over the one above is that it means your total hours assigned also has contingency built it and your hourly rate stays consistent with what it would be without the contingency assigned.

Create a task called Schedule Contingency and set the duration to an amount of time with no resources attached. Bury the task where you think the schedule risk is going to occur - I usually put it into the QA phase of the project as this is the mostly likely area where you'll need the contingency.

10. Save often. I've had too many cases where Project crashes and there is no automatic recovery. In addition, there is only one level of Undo in Project 2003 (this is fixed in 2007) so if you make some significant changes its tough to revert them.

11. Use the keyboard as much as possible. Learn keyboard shortcuts. Don't drag things around when you can copy and paste. Don't use your mouse to indent when you can click on Alt-Shift-Right Arrow. Typing in predecessors manually instead of clicking on the link task button in the toolbar generally is faster.

Hopefully, that will help speed up your MS Project work dramatically. If you work this way, you'll find that planning with MS Project is feasible. If you're doing it the traditional way by dragging grantt bars around you'll find a major speed improvement working this way.

Low Tech Approaches to Requirements Gathering - Part 2

2007-06-07T07:23:00.000-05:00

For those who missed Part 1, I wrote on how to use low tech tools such as paper, Post-IT Notes, whiteboards, etc. to gather requirements.

In this post, I'll describe how to better use tools such as Office, Project, Visio, etc. to gather requirements.

Here are my recommendations about approaching tools such as these in an agile context:

1. Use "Just Enough" tools. Don't use Rational Rose for UML diagramming if Visio will do quite nicely.

2. Buy the absolutely best tools for the job. In theory, you can write .NET applications using Notepad and command lines tools and save a few bucks on licenses. But the productivity will be poor and learning curve will be worse. Don't allow your organization to scrimp on tool costs - in my experience if its the right tool for job, paying the $500-1000 for a particular tool is worth the cost.

3. Invest in tools that are standardized. If you're working in Java, use Eclipse. If you're working in .NET, using Visual Studio. If you're an Oracle DBA, get Toad in your office. If you want a source control tool, get VSS, Subversion or CVS and not Seapine. That way when you go hiring for new developers, they've probably used your tools before.

4. Get really good at using the tools. It sounds like a basic suggestion, but I don't know how many project managers I've worked with don't really understand Microsoft Project. If you set up project in particular ways such as exposing the dependency columns, splitting the window to have the resource allocation on the bottom instead on the side, getting rid of the GANTT chart altogether, etc. productivity goes up considerably. Same thing with Word - the first thing I do with any document sent to me is look at how many Styles are being used and typically I find hundreds of them. Whomever wrote the document didn't understand how styles work, why they improve productivity, etc.

5. Buy tools that are easy to learn. Again, another dumb suggestion but I look for tools that a junior developer or programmer can understand. In many cases, this means buying more tools as the simplest ones can be single purpose. I've rarely come across a monster tool that works as a really good Swiss Army Knife for all occasions. The profiler I use though is stupid simple - its from RedGate and you load it up, point it at your project and start profiling in about 30 seconds. Its dumb simple to learn and cheap as well.

6. Have everyone have access to all the tools and have everyone knowledgeable about them. For example, I've worked in organization where only the Senior Project Managers were "allowed" to have Microsoft Project. If you weren't a Senior PM, then you had to either use Excel or find a Senior Project Manager to help you build a project plan. Technical resources should be able to open up a project plan as well - if you want your team to be accountable for deliverables, telling them that they're not important enough to see the plan sends the wrong message!

7. Automate work as much as possible. Use Continuous Integration, Test Driven Development, build scripts, O/R mapping layers, code generators, documentation generators, etc. where appropriate to automate drudgery work. Invest in time sheet systems, collaboration spaces, source code control systems, etc. where appropriate instead of tracking things manually.

8. Remember that every document, artifact and tool costs money to buy and to maintain. Every software license has maintenance costs. Every document takes up space on a file server. Artifacts have to be managed, secured, backed-up, etc. So travel light as product just enough documentation. Use just enough tools to do the job which also usually means use the ones that are the most productive. Remember that emails are documents as well - if you're finding that you're getting hundreds of emails a day, figure out how to get your team to be more concise, more action oriented and spend the majority of their time producing deliverables instead of documents!

9. Get the fastest computers you can afford. Get two monitors minimum for every developer. Get a good video card. If you have to wait 20 minutes to do a build, trying to do agile development is going to be tough. If paying an extra $500-1000 will cut that build time to 5 minutes then I would pay the money as it will result in massive productivity improvements on a daily basis.

Low Tech Approaches to Requirements Gathering: Part 1

2007-06-05T18:30:00.000-05:00

One of the tenents of agile development is just enough documentation. I would recommend adding another suggestion: just enough tooling.

The boss comes to you and says, we need a project plan for this large software project. Budget is about $1 million and I need it done in 6 months. Figure out what is required, how many resources you need and what kind of scope can be delivered.

So how many of you have already reached for Microsoft Project?

Forget it for now - come back to it after you've tried some more low tech approaches. Here are some suggestions in order of really low to medium tech approach:

1. Talking: before you start visualizing, diagraming, or white boarding, start talking! Some people talk and diagram at the same time (I do) but I've been to too many meetings where an architect gets up to the front of the room and the team sits there for 30 minutes waiting for the person to finish his fancy whiteboard picture.

2. Piece of paper: I love white boards, but they're hard to carry to a bar, a coffee shop, an airplane, etc. So keep a sketchpad, a napkin, etc. and use it as your white board.

3. White board: the number one tool for an agile PM. If you want to record your white board, take a digital picture of it!

Note: Make your white boards portable. Get the white boards that are on wheels or make them small enough that you can move them from desk to desk. We used to have our release checklist on white boards and it worked great - we could leave it up permanently (without the PLO in a meeting room) and then we would erase things off the board as they got done. This also stops the need for dedicated planning rooms - simply bring the tools with you.

4. Big sheets of paper and Post-IT Notes: Ever done a web site map with a big sheet of paper, some markers and post-it notes? Try it and you'll never go back to using Visio to do site maps again.

Note: If you want to be really advanced, use coloured Post-IT Notes. Get Red, Yellow, Green and you can use them to connote priority, level of difficulty, etc.

5. Paper templates: if you find yourself redrawing, create some templates in word and then print them off. For example, don't draw yourself a calendar each time - simply print off one in outlook and then start writing on it.

All of these techniques mean no use of a computer (except for create a couple templates). You can use these techniques in any environment that has access to an Office Depot.

Low tech approaches to planning, requirements gathering, design, etc. have some specific benefits:

1. No license costs: while this isn't such an issue with Project, if your development tool standard is Rational Rose, Together Soft or some other expensive IDE where every user must fork out thousands to build a UML diagram, a project plan or a document then using lower tech approaches will have dramatic cost savings.

2. Encourage collaboration: if you get people to talk, to collaborate and to physically draw, use construction paper, or write the collaboration happens naturally.

3. Take away bias: If you're a technical architect, you think in objects. If you're a database designer, you think in tables, stored procedures, queries, etc. If you're a project manager, you think in gantt charts, dependencies and resources. So which toolset do you want to work with together? Post-it notes makes everything give up their biases and focus on real requirements.

4. Speed: How fast are you really with Project, Visio, Word, etc. I'm a power-user on all these tools but I still find whiteboards are faster.

5. Freedom to reboot: People get attached to their work. In addition, if you've spent the past 2 hours making sure the lines all connect properly to the boxes in Visio or making sure that you've leveled your resources in Project you really don't want to relive that experience in the same afternoon. But whiteboards are really easy to reboot and people don't get as attached to Post-IT notes.

6. Multiple models: If you use computer based tools, its more difficult to use multiple models. You need Visio for your wireframes, you need Word for some documents, you need Project for your gantt, etc. The amount of tools required to support a multiple-model approach. Alternatively, you get sucked into the idea of using one BIG GRANDIOSE TOOL - enter in the Rational Rose, MDL, Case Tools, Heavy IDEs, etc. that try to be all things to all people and cost a fortune in the process.

One of the challenges with low-tech approaches is how to translate them into digital, cleaned-up format for posterity. Here are some basic suggestions:

1. Don't bother: every document requires upkeep, so don't bother capturing every design decision in a fancy document unless its really necessary. Low-tech documentation is more about the process of thinking through drawing then creating fancy documents. Keep the top 10% of what comes out and dump the rest.

2. Take a picture: take a picture of your white board - you've now got documentation! Put it on your WIKI, store in Word documents with notes, etc. and you're good to go.

3. Data-entry: sometimes you want to capture the notes, diagrams, etc. in a digital file so it can be further manipulated. So at the end of the meeting, take the 20 minutes it requires in most cases to data-enter it using the right tool. Better yet, get your junior or admin assistant to do it for you - why have a $100K technical architect creating circles in Visio when a $40K secretary could do it for you?

Note: this is probably an out-sourcing opportunity as well. Do some white boarding, take a picture and then send it overnight to diagrammed in India. Walk in the next morning with your fancy new diagram complete with colour coding and code stubs created!

Once you've done some design work in a low-tech manner, then use a computer based tool if you need it. See tomorrow for part 2 where I discuss how to use the computer in a low tech way and also how to use the existing tools more efficiently.

I'm Somehow Ranked 6 for the Google Phrase "Buy Books"

2007-06-04T21:15:00.000-05:00

I have an Amazon affiliate store (see the link to the right). It has a few books in it, but not much more. I have never received a sale through this link.

Yet my friend Jordan pointed out today that if you search on "Buy Books" the link to my store shows up as ranked number 6 in Google's list!

How is this possible? What does this really say about Google's ability to filter results? Is it simply because I'm a random Amazon store?

Check out Funny or Die

2007-06-03T14:00:00.000-05:00

Check out the new site from Will Farrell, Funny or Die.

The Landlord

While YouTube has volume, the sheer amount of it means that it suffers from mediocrity. If you search for something specific you'll find some content that is good, but just browing for entertainment is pretty limited given the sheer amount of garbage content.

Funny or Die has some distinct advantages that give it benefits of YouTube and traditional broadcast television:

1. Star Power - lots of videos from Will Farrell, Jimmy Fallon, etc.
2. Original Content - YouTube has lots of content that is just copies of other media e.g. videos, music, etc.
3. Cutting Edge Humour - this sight makes Saturday Night Live look pedestrian by comparison.
4. Some selective choosing - the site has a rating setting of "Immortal" which basically means that Will Farrell put it up and so no one can rate above him. This isn't a bad thing - this means that there is some real selective editing on this site and its driven through a particular perspective. This makes it different than a generic search engine.

Check it out - some great content worth viewing...

Pandora's IP Blocking for Canadians isn't 100% Effective

2007-05-28T07:34:00.000-05:00

As has been posted widely, Pandora was shut down in Canada due to license restrictions.

But Pandora's IP blocking mechanism isn't 100% - I just tried from my office computere here in Toronto and I could access it without any problems. I'm not sure what their criteria is for blocking but I'm just happy I can get through and still access the service.

The Best Old Computer Commercials

2007-05-27T20:31:00.001-05:00

Here are some great old computer commercials.

One thing that is noting is how amazing sexist they are - Dad always is the one who buys the computer and the Son is always the recipient because he has poor grades. The most Mom ever uses a computer for is to put in a recipe or two and sister is almost never seen. The worst case is the last ADAM commercial below where the sister helps brother get a computer so that she can get a Cabbage Patch Kid - heaven forbid she might want to play Donkey Kong herself!

William Shatner got as fat as bloatware!

Apparently, only boys played with computers in the 80's!

Computers don't lie - they're honest...huh?

Balmer still sounds like that 20 years later...and when did Reversi become a feature?!

I had this computer myself - for the price it was actually a great computer, but it got smoked thanks to poor marketing and support from TI by Commodore in the mid-80s.

I had this computer as well. This was one of the last generation of personal computers before the PC takeover in the mid-80s which accounts for its everything but the kitchen sink approach. It had video games, a super speed tape drive, a daisy wheel printer, a build in word processor, etc. It was an odd computer, and it didn't sell very well. But I had one for about 2-3 years and it worked quite nicely for my needs when I was in junior high school.

Another strange note on the ADAM: the version of BASIC was a complete rip-off of Apple II BASIC. I noticed the syntax similarities when I was over at a friends place and then went to the library and borrowed all the Apple II BASIC programming books I could find and had a blast!

New Feature Request for Facebook - Add Support for Maiden Names

2007-05-27T19:40:00.000-05:00

I've been on Facebook for a few months now and am always searching up old colleagues, highschool mates, etc.

But I've found one general problem - searching for people who have changed their last names.

What about supporting searching by maiden names. For those people looking for people they knew when they were much younger, that would help immensely. I've had several people I couldn't track down because they married and changed their name.

Bug of the Week: Blackberry Bug

2007-05-27T19:36:00.000-05:00

For those who have Blackerries, Brickerbreaker is the Breakout clone that comes with the Blackberry OS.

In the latest version, I have found a mystery bug - for some reason every once in a while, the ball hits the top of the screen and instead of bouncing downwards, it instantly moves to the bottom corner of the screen and starts going upwards! Its not that common, but given enough play testing you would definately experience it.

The strange thing is that I cannot figure out the pattern as to why it happens - it doesn't seem to be any particular spot, ball position, level, etc. that causes the bug.

Web 2.0 Bug of the Day - Facebook Bugs

2007-05-21T18:58:00.000-05:00

I found a couple Facebook bugs today...

1. If you click on "My Friends Posted Items" it should give you an RSS feed. However, instead it prompts you to save a PHP file with the XML inside. MIME type isn't right I'm assuming?

2. The paging in Facebook isn't quite right. For example, if I do a search for classmates from my highschool (Woodlands School, Mississauga, Ontario, 1991 Graduation Year) I get a list of 23 people. If you go to page 2, you get 2 people repeated from the previous page.

3. Facebook security check routinely now returns an image that is unreadable. Sometimes you have to hit the refresh 4-5 times to get it to come up.

New People Search Engine - Zoom Info

2007-05-21T18:45:00.000-05:00

I'm always interested in networking sites, especially those with good people search engines. Since I've been on the Internet a very long time (pre-WWW was even invented) I have a very long trail of posts, press articles, and teenage writings from years and years ago.

As I've written before, finding people is not difficult if they have contributed to facebook, MySpace, etc. and there will definitely be a problem in 10 years where people want to remove their facebook pages after they have grown out of their teenage years. See my article on this previously.

For those who are in my generation (I'm 33 years old), many never were as adventurous and so finding people becomes a big more difficult, but not impossible. Almost everyone has some kind of Internet presence. Its a matter of digging in the right places to track it down.

A new tool called ZoomInfo helps in this regard. Type in any name and it will find many old press references for that name. What's smart about it is that it seems to be good at parsing out references to relationships between people, companies and other colleagues. It also seems to be able to collate references together into a profile, e.g. it can recognize the same press article in multiple places and summarize them together.

If you look up yourself and find your own profile, you can claim your profile either by verifying against your credit card. They don't charge the card, they just use it to authenticate your name.