Key IT Challenges for 2009

Here are some of the key IT challenges I see for 2009 based on feedback from our customers, the media, and my own understanding of key technologies coming on the scene. In addition, these challenges reflect the broader economy which is going to go through a global recession in 2009, impacting IT budgets in a number of different ways (see my previous post on this issue).


Challenge #1: Maximizing Existing IT Investments

In the past 2 years, there have been a significant amount of investments made in new technology platforms that were supposed to empower office workers to higher productivity and less dependance on IT. Products like content management systems, portals, data warehouses, analytics, CRM, ERP, etc. all have become increasingly mainstream over the past couple years as the economy boomed and CIOs were looking to make long term investments.

This year, in this time of restraint, I see a key challenge for all IT departments in maximizing existing investments that were made over the past two years. For example, how many companies have invested in SharePoint and are only using it to power their intranet? I have many customers who are looking to see how can they can justify their SharePoint platform investment through more advanced applications. For consultants, this is an opportunity to sell more vertically integrated, business centric solutions and move away from selling the latest technology and this will continue for at least another year.

Challenge #2: The Proliferation of Virtual Machines

The virtual machine as a server platform has really hit the IT mainstream. Virtualization has been around for years but thanks to a very mature platform with VMWare ESX and an upcoming push from Microsoft Hyper-V, the use of virtual machines is expanding rapidly.

Virtual servers have started replacing physical ones, first with non-core systems such as development servers to now in some cases full production environments.

One of the key challenges with Virtualization is that because it becomes so easy to spin up a new server, the number of servers proliferates, leading to VM sprawl. Management of virtual servers doesn't easily map from the physical world especially when doing tasks such as asset management (how do you manage a virtual asset). IT managers will spend a lot of time in 2009 coming to terms with these challenges.

Challenge #3: Web 2.0 Goes Mainstream

As technologies go mainstream, the language, terms and concepts that were reserved for leading edge technologists land in the hands of business managers, stakeholders and consumers. Unfortunately, they get them typically without the deep technical or even conceptual understanding.

This happened a couple years ago with the concept of "Portals". Everyone wanted a portal. Portals went from being these esoteric technologies to jargon words in marketing magazines. The same is happening now with concepts like "Blogs", "Wikis", "Web 2.0", etc.

I'm now receiving RFPs from government agencies with requests to "use Web 2.0 technologies" with no rationale, purpose or objectives. Some manager has now heard of Facebook and so now wants it on their web site. As technology consultants, the challenge will be seperating the myth from the fact, selling the real effort it takes to use these technologies effectively and trying to guide stakeholders to realize relevance from what has now become a Fad.

Challenge #4: Storage

Even a few years ago, it was a big deal to have a Storage Area Network (SAN). SANs were reserved for large organizations with lots of data. With the proliferation of media files, volumes of emails and documents, and no discipline when it comes to archiving in most organizations, managing the ever increasing demand for more storage is becoming a major headache. With hard drives getting increasingly cheaper the perception that storage is nearly free is pervasive (I can go buy a terrabyte backup drive now at BestBuy for about $100). Unfortunately, corporate storage is nowhere near as cheap when the requirement includes high performance, highly redundant and low power consuming. For example, at my last organization we spent almost $50,000 implementing a SAN in order to gain faster I/O access for our core databases. The SAN has 28 drives in it, is highly redundant, etc. but even with its several terrabytes of storage it will fill up rather quickly if not managed well.

Challenge #5: Search and Document Management

I mean search here in the broadest sense - finding stuff. Go try and find a document on your corporate intranet. Try and locate a file on your file server. Try and understand the taxonomy or meta-data structure for your document repository (assuming your organization has even defined one).

Creating documents is now so easy that they have proliferated. In addition, there is typically in most organizations very poor meta-data strategies for indexing data and immature archiving strategies to remove old documents. For example, at my last position we had a file server with about 4 terrabytes of documents. We did a scan of the thousands of documents on that server and found that over 80% had not been touched in the past 2 years. Even if they are not being accessed they act as an information clog when trying to find important documents.

Document management used to require very expensive and proprietary software such as OpenText, Documentum, HummingBird, etc. and was reserved for companies such as law firms who had very regulated and structured document repository. But with lower cost tools such as SharePoint you can quite easily now implement some decent meta-data taxonomies and start to catalogue, version control and archive documents in a methodical way.

The biggest challenge isn't the technology - its getting commitment from the business to organize the thousands of old documents and migrate them into the new structure. This is a massive effort and requires significant involvement from the business. While there are some tools to help with this effort, the clean-up effort of document migration is still a continuing challenge.

Online Trends for 2009

Here are some online trends to look out for in 2009.

Advertising Takes a Beating

With the sluggishness of the US economy, advertising will decline significantly and online advertising in particular will be hit hard. Online advertising will still continue to grow but at a much slower pace.

This slowdown in growth will put pressure on web 2.0 applications with lots of content but lower conversion rates. Applications such as Facebook, Flickr, YouTube, etc. will need to focus more heavily on monetizing all those collaboration interactions that millions of people now do for free every day.

There will also be a continued emphasis on Cost Per Action instead of Cost Per Click to link advertised costs to conversion. As the cost of online advertising increases (for example the cost of buying keywords) there will be increased pressure on Google and other advertising platforms to show direct linkage between advertising and conversion to revenue. This is where online advertising as a channel has had an advantage over traditional media channels.

Internet TV is Finally a Reality

The "watch video on your computer" revolution started about 10 years ago, but until this year watching the equivalent of television via the Internet really wasn't possible. Instead, users have sacrificed the video quality that even low definition television provides for increased collaboration, crowd created content, etc. In addition, the selection for mainstream content has been limited online until the last year or so.

In 2009, the compromise on quality and selection will fade away. Video sites such as YouTube, Facebook, etc. are all upgrading their video streaming so that the videos uploaded by users are streamed at much higher resolution. In addition, sites such as Hulu now stream hundreds of mainstream television programs in almost broadcast quality. While not able to yet compete with HDTV, video on the Internet will now be viable as a full-screen, sit in the living room type experience.

Part of the challenge has been the source video - consumer digital cameras tended to be lower resolution video than television. However, the latest digital cameras produce very good video quality including my Panasonic camera that can shoot HD video in wide screen format.

Web 2.0 will Become Mainstream

Don Tapscott's new book on the Net Generation points out that when technology becomes boring and we stop talking about it then the next level of innovation is possible. Web 2.0 will simply become The Internet as usual and we'll stop talking about it as this new found technology revolution (until we think of the next one of course).

Applications like YouTube, Facebook, etc. are now so mainstream that your mother uses them. Applications such as Facebook, MySpace, YouTube, etc. have all already demographically shifted to the mainstream from being in the hands of the power users, the young and the affluent.

Mobile Applications Will See Massive Growth

With the launch of the IPhone and its inevitable competitors, the phone now becomes a viable application platform. IPhone is as significant for mobile applications as IPod was for downloadable music.

Another mobile device seeing large sales is the Amazon Kindle. It allows users to purchase books and view them in a mobile device. The idea of e-books has been around for over 10 years but it takes a device like the Kindle with its complete vertical integration to drive the adoption.

Geotagging and Live/Google Earth

As the amount of content increases, one of the simplest ways to filter content is through geography. In addition, with the maturity and increased use of mapping interfaces such as Live Earth, Google Earth, Google Maps, etc. both as destination sites and as mash-ups, geotagging of content, pictures, videos, and people will be a vital way of filtering content in 2009.

One of the simplest examples is the integration of Panaramio with Google Maps. In Google Maps if you look up any address it will show you related photos from Panaramio, all geotagged on upload by location.

Change of Strategy From The RIAA/MPAA

For the past several years, the global culture industry has been taking people to court over copyright infringement. Approximately 30,000 people have been sued in the US for copyright infringement over the past 5 years. The impact on file sharing has been no real impact on file sharing but has made the RIAA one of the most hated organizations worldwide.

It appears that after 5 years of harassment law suits, the RIAA has decided to change its strategy and use a softer approach. In addition, new technology and business partnerships will allow the media conglomerates to start making money off their content no matter who uploads it. For example, Universal now stands to make over $100 million in ad revenue through a deal inked with YouTube that uses search terms to identify user contributed music videos that map to Universal's artists.

Casual Entertainment Over Hard Core Gaming

The success of the Nintendo Wii (sold out again this Christmas!) as well as other casual games such as the Sims and Spore will continue to provide online entertainment for the masses. Collaboration, casual games and customization will continue to push the mainstream entertainment market. Games like RockStar, Wii Fit, etc. will continue to provide mainstream entertainment while leaving more hard core games to cater to the hard core 18-30 male demographic.

See the above note as well on platforms like IPhone and Facebook - these are perfect jumping points for casual gaming as they cater to an audience that might play a game of solitaire but isn't going to be spending hours with Call of Duty.

The casual gaming market is still in massive growth mode and this will continue in 2009.

Difference, Story Telling and Community

This isn't a trend, but it will continue to be true in 2009 as it always has been: story telling, difference and community building continue to drive the content creation process.

The revolutionary part of the Internet is the ease at which these tools can be employed for someone to tell a story, create some innovative content and enable a community. The growth of blogs shows that everyone has something to say and the size of the audience it generates is how well it connects to its community of interest. Determining "success" depends on the objective - it may simply be distributing family pictures or it may be making your opinion heard by millions as Barak Obama did in 2008.

The Key Challenge in 2009: Conversion

I have a friend who is very passionate about informing parents about Toy Recalls. She has an active blog, she uses Twitter and Facebook and she is able to get her message out to several hundred viewers a day. When we discussed her "success criteria", she seemed pleased that she had grown this community from zero but her next question was, "ok so now what?". This is I think the key question for all this collaboration activity going on - how do we convert the energy of all this discussion, file sharing, photo sharing, tagging, blogging, etc. into some sort of concrete action so that its simply not just hanging out (although that is not a bad goal in itself).

It's as if we have got people off the couch and started them talking again. So now the community is sharing, talking, collaborating. So now what happens? How do the tools available provide methods to convert this community to action?

This is the future growth opportunity for application platforms like Facebook and you can see the evolution of these tools going in this direction already - applications like the Causes application for example are a first attempt to translate a community of action into a concrete action (in this case donating money to charity).

Corporate IT Departments Trends for 2009

Corporate IT departments are, at the best of times, the internal consulting organization for their organizations and provide strategic technology services to help propel the overall success of the business. The role and objectives of the corporate IT department will change in 2009 as the economy continues to lag, technologies mature and organizations that saw "Web 2.0" as bleeding edge now start to adopt some of more mainstream applications of online technologies.

Here are some of the key trends I see in 2009 for the corporate IT department. For consultants such as myself, this will have a major influence on how we can continue to provide high value to our internal IT partners by providing services and solutions that contribute to the value proposition for the corporate IT department in 2009.

Corporate IT Departments Will Lag Behind in Upgrades

With budgets tightening, corporate IT departments will take the hit in upgrades for both hardware and software. IT departments will be asked to do more with less and those departments who may have planned on an Office 2007 upgrade, a SharePoint 2003 to 2007 conversion project, a .NET framework 1.1 to 2.0/3.0/3.5 upgrade project, server refreshes, etc. will be asked to defer them to save costs.

Scrutiny on Capital and Cash Flow

Typically, large scale expenditures can be amortized over a 3-5 year period. For example, if a Director IT needs to purchase $500K in new servers, they can typically budget these over a 3-5 year amortization period so they don't get hit with the entire cost in year one. However, from a cash flow perspective, the money still gets spent and in cases where cash is tight (e.g. credit crunches) these types of cash outlays will be dramatically curtailed.

As a result, this will lead to arrangements with providers where costs can be paid on a monthly basis, e.g. leases, software as a service, outsourcing, financing, fee for service, etc. where cash flows out of the organization in a more even way than a large capital expenditure.

Long Term Consulting Under Attack, Short Term Consulting Will Increase

As budgets tighten, the following tends to happen during the budget planning cycle:

1. Full time hires are deferred
2. High cost consultants are targeted
3. Projects have to show very quick ROI

If you are a $200 / hr consultant that has been working on a project for 2 years and the project will continue for another 2 years, then expect that cost to be reviewed and for department managers to try and squeeze out consultants and replace them with cheaper internal resources.

However, because full-time hires become blocked, the only way to get new work done in the short term tends to be through direct consulting costs. In addition, because the business does not have long term visibility they will be hesitant to take on a full-time resource as a long term commitment. If the business stakeholder can justify the ROI in the short term, they will tend to outsource the work simply because the Corporate IT department cannot hire any full time resources to do the work. In addition, outsourcing the work as a fixed cost to a consulting firm is less risk and the cost is directly attached to the cost of the project, making it more tangible for a business stakeholder than a resource hire.

Incremental Optimization

For both revenue and profit, the goal for 2009 will be to optimize as much as possible. For example, technologies such as SEO, analytics, data mining, reporting, CRM (especially for those who already have an existing platform), channel management, marketing optimization, etc. will sell very well to organizations who have over the past 2-3 years have been growing their customer base but now need to focus on retention, optimization of marketing dollars and efficiency. Technologies that can help do this will continue to get the green light as they produce ROI faster than brand new initiatives and they are less risk for a CIO who is under budget pressure.

Layoffs for Many, Indispensable Positions for a Few

In tough times, there are typically layoffs. Depending on the speed of action required, layoffs are many times done without proper planning and project/application coverage (the worst case is where you see companies having to hire back the people they just laid off on contract!). For those who are laid off, it means that they are now looking for a job. But for those who are left standing, they can find themselves now indespensable where there used to be three people who knew how to fix an application and now there is only one. In some cases, the one left is a consultant!

For those who are in this position, it provides leverage to those left behind. If you are the only developer who now has access to source code and can run a build, you were kept because they couldn't afford to dump you. If you can stretch to grab the work that was being done by other developers and demonstrate the flexibility and tenacity to help clean up the mess inevitably left behind by these changes then this will be looked at favourably by the company.

For consultants, there are opportunities to fill the gaps left behind in the rush to lay off staff through short term and long term consulting arrangements, especially if the consultant has had previous experience with the applications in the organization. There are also opportunities to pitch alternatives to lay-offs without adequate transition through outsourcing arrangements. A service company can provide flexibility in resource ramp-up or ramp down that is difficult for an internal department to manage especially in a time of crisis. In addition, a consulting company has the advantage of being able to work in stealth, allowing for a Corporate IT Director for example to quietly outsource key functions to an outsource provider while planning a lay off so as to avoid disruption and continue service levels.

Opportunity for Entrepreneurialism

There is an opportunity for IT departments to re-invigorate through a focus on entrepreneurialism. For those IT leaders who can show the business a clear plan to short term revenue, increased profit, etc. this will tend to get more attention than more themes such as scalability, efficiency, etc. Entrepreneurs focus on delivering high value services on a shoe-string budget - this is what is required in an economic down turn.

Business stakeholders will also be more apt to listen when they are also hungrier for more revenue. For example, many sales businesses will find their traditional sources of revenue tapped out and will be looking around for new revenue sources. If the Corporate IT Department has some creativity and can partner effectively, they can help provide some great new ideas for using Web 2.0 technologies, creating online campaigns, optimizing CRM data, using data mining techniques, etc. to squeeze more revenue out of existing channels or to open up new ones.

For consultants, this is the ONLY way to sell business in 2009 - either demonstrate how to squeeze more revenue/profit out of existing channels or show how a very small investment in a new channel can lead to large growth potential. Be prepared to take risks and be entrepreneurial with your customers in how they pay for your services, how much risk you take on, etc. and this will also be attractive to customers who are hesitant to spend money.

Focus on Business Solutions that Make Revenue or Profit, Not Technology

This is always the case with business stakeholders, but in times of restraint this will be even more pronounced: Corporate IT Departments who are focused on generating concrete business centric solutions that create revenue or profit opportunities will be prioritized.

If you are a Corporate IT Department or one of its service providers and your focus is heavy technology buzzword projects like: SOA, Cloud Computing, Web 2.0, Architecture, Software Development Life Cycle, Enterprise Data Warehouse, ERP, etc. then you will be in trouble in 2009. For example, at one of the banks here in Canada for example, I know that their list of thirty major IT has been cut to less than ten for 2009.

Projects where existing horizontal technologies in place are now leveraged to create additional value will have significantly more priority. For example, we have many customers who have previously bought Microsoft SharePoint 2007 but aren't leveraging it effectively or maximizing its use. Incremental investments in adding verticalized and targeted applications on top of this already existing investment will show quicker pay off than investments in additional infrastructure technologies.

Business Intelligence is More Than Just a Fancy Graph

"Business Intelligence" is a hot topic these days. There are companies that produce business intelligence products such as Microsoft, Business Objects, Dundas Charts, and so on that focus on visualization of data with the approach that by providing a simplified view of complex data it will enable key stakeholders to make decisions.

A good example is Dundas Charts - the company produces an amazing set of widgets and controls for building dashboards, charts, and visualization of data.



The price is right too - you can buy these fancy BI controls for a license of about $1000.

However, in my experience with Business Intelligence applications, I have come to the following basic conclusion:

Visualizing data is relatively easy.
Obtaining the data in an enterprise environment is hard.
Defining what the data means is even harder.

Let me give you an example. Wait Times are calculated in Ontario and presented to the public in order to show accountability for the funds being invested into improving access to surgeries.

Here is what the data looks like:



As you can see, the indicators are quite simple. The data could have been visualized in a number of different ways such as a red-yellow-green style KPI, a guage, charts, etc. Implementing this would simply involve taking a control library like Dundas Charts and feeding it the above data to get a graphical representation. However, visualization is actually not that valuable - the average person does not need a graph to understand a basic set of numbers.

Obtaining this particular set of data is hard. It requires the synthesis of raw data coming in through automated feeds from about 150 hospitals into a central database that is then scrubbed to match patient records and wait times records together. The cost of building this application to collect the data was millions of dollars - the cost of producing the PDF file containing the numbers is significantly less.

Defining the original business rules, definitions and targets for how wait times data was to be collected was even harder. What does a "Wait" actually mean? There are in Ontario two different wait periods (called Wait One and Wait Two). Wait One is defined as the time it takes for you to get the appointment with the doctor who provide the diagnosis. Wait Two is defined as the time it takes between the point where the doctor provides the diagnosis and the point where you get the procedure. This is what the current Wait Times application tracks - Wait Two data. Different provinces have different definitions and track different wait intervals. In addition, you will notice that only certain types of procedures are represented. These were chosen through government priority - another period of business analysis that took several years to define, prioritize and fund. Similarly, the "targets" were defined through another complex process using clinical experts who invested significant time to analyze and provide recommendations on what was the clinically appropriate targets.

If you are looking at a report, a set of numbers, etc. you are looking at the tip of the iceberg - the amount of energy, work and thought that goes into defining the data and then obtaining it is enormous in comparison to building a fancy graph. In the case of Wait Times, producing the report now is done by running automated processes. The cost of the underlying system to obtain the data cost millions of dollars and took years to implement.

So the next time you look at a report, appreciate the amount of work that has gone into it. And as an IT professional, the value to the customer is in helping them to define the data and then obtain it from a variety of data sources. Produce the actual report is simple in comparison.

Top 10 List of Bad Movie Computer Moments

I watched Eagle Eye over the weekend, and it was fun but just terrible in its use of the old boogie man plot device of the computer taking over the world. In appreciation, here are some of my all time favourite bad computer movie moments.

1. Eagle Eye (2008): an all powerfull, all seeing surveillance computer tries to take out the entire US government but can be defeated by poking it in the eye with an metal rod.

2. The Net (1995): Sandra Bullock plays a notorious hacker who throughout the entire movie mispronounces the word MODEM (as if it rhymes with bottom) and uses a Macintosh and a drag and drop interface to hack computers! If only it were that simple.

3. Jurassic Park (1993): the computer that controls the park is apparently UNIX (identified by the super genius girl who apparently learned it in grade school). She then proceeds to operate it using this 3D flying interface!

Jurassic Park - Unix System Scene
by unixguru

4. Independence Day (1996): Jeff Goldblum and Will Smith design a computer virus that can infect an ALIEN COMPUTER via TCP/IP using a MacBook! Apparently alien invaders use the Internet but without the McAfee subscription.

5. WarGames (1983): Matthew Broderick and Ally Sheedy hack into the government using a 300 baud modem (with accoustic coupler for visual impact). The super computer is a room full of tape drives and flashing lights that is defeated by forcing it to play chess until it figures out the concept of a stalemate. "The only way to win is not to play".

6. Mission Impossible (1996): Standard computer animations like file uploads, logins, etc. are really boring to look at. In Mission Impossible, a standard macbook was used but the OS was suped up with some fancy animations. Why have a progress bar when you can have the screen whiz around and scream at you (even though its supposed to be top secret)?

7. You've Got Mail (1998): Email clients are also very boring. Who wants to watch someone reading email. This movie is a great example of how special effects people take great liberty to make computer applications much more interesting with animations, sound, etc.

8. Star Wars (1977): The computers in Star Wars are not bad actually, although the amount of blinking lights in the room that the Droids hang out in for most of the movie are a bit much. The demo on how to destroy the the death star is quite primitive (they look a bit like the original asteroids video game) but at least looks somewhat plausible.

9. Golden Eye (1995): Thanks to some product placement, all the computers are running OS 2/Warp. If the Russians were running Windows, maybe they could have had a chance against 007.

10. Swordfish (2001): Hugh Jackman is a hacker who puts together viruses using some sort of 3D visualization software. The one thing he does get right is the repeated swearing at his computer while he works.