I just attended a session by Polita Paulus and Mahesh Prakriya on Dynamic Data Controls in ASP.NET.
Here is the basic concept behind these controls - take the standard grid controls and instead of configuring them, point the controls instead at a database and they configure themselves.
Polita demonstrated simply taking a page, calling it the name of the table you want to display, and up comes all the controls you need for editing, filtering, deleting, inserting and linking to child/parent tables. Pretty interesting stuff especially for those creating quick one-off web apps. You can extend the controls pretty easily as the dynamic data controls can extend any of the standard ASP.NET data driven controls such as DataGrid, DataList, etc. You can then extend them as normal using data binding, custom formatting, extension through event binding, etc.
The current CTP only supports the SQL Datasource through connection strings. Forget the idea of using these dynamic data controls to build off of objects, web services, XML, etc.
The bad news is that the only recommendation for these types of heterogeneous (e.g. any serious real world scenario these days) provided seems to be to support a LINQ data source, which is Microsoft's new way of doing O/R mapping and language integrated query. The answer to having heterogeneous data sources, classes, business rules, etc. from Polita was wait until you get the LINQ datasource and then use that as the basis for all your data needs.
I'm going to the LINQ data source presentation next, but this seems problematic and limiting as an overall one size fits all strategy. When we use ASP.NET data controls today, we rarely use the Wizard type approach - instead we're doing heavy duty business objects that generate very simply array lists of custom built value objects. This is especially true with distributed scenarios such as web services integration.
But I'm onto the LINQ presentation next, so maybe it's the panacea after all! :)
Monday, April 30, 2007
MIX07 Update: Feedback on Expression Web
I just attended a seminar on Expression Web 1.0 and while it seems to be definitely a 1.0 product (e.g. nothing too flashy, not that mature quite yet) it has some interesting features.
What makes it cool:
What makes it cool:
- Really strong CSS editing - ability to diagnose inheritance issues, see what styles are applied, etc.
- Refactoring support - easily rename styles, move styles from to page, promote styles from page level to an external style sheet, etc.
- Word like style support. Think of the Word 2003 Format Styles dialog box for HTML. This would make it really easy to apply pre-defined styles especially for non-technical users.
This product seems to be really an HTML editor with very little integration. There was some discussion that this developer/designer integration is coming in the 2.0 product, but I wonder why they couldn't include the same CSS/HTML editing tools in VS 2005 or the next version of it? In my experience, one of the biggest challenges is interface and back-end HTML integration, and having everything in one tool would certainly help on this front. I've had too many experiences where the interface developer creates a beautifully formatted static HTML template and then either one of two things happen: 1) the CSS is so complicated that the ASP.NET team cannot inject their code into it easily or 2) the ASP.NET team ends up creating subtle HTML/CSS bugs when the dynamic code is inserted. I'm not sure how ExpressionWeb really helps in this regard, but I've only so far seen the demo.
Saturday, April 28, 2007
Bringing You The Best from MIX07
I will be flying to the Mix 2007 Conference in Las Vegas on Monday. Stay tuned and I'll post highlights of what I see including pictures, videos, etc.
Finding People Across Multiple Social Networks
In the past year, we've had linked-in, facebook, myspace, twitter, etc. all register hundreds of thousands of user profiles.
I'm currently on linked-in and facebook heavily. I haven't tried myspace or twitter yet. In addition, there are dozens of other user profiles floating around the Internet in blogs, web pages, etc. And there are still old-school sites such as Classmates.com which provide people finding services through a local service.
Finding people on the Internet is actually reasonably hard to do. Simply googling a person's name might find the person, but if your name is "John Smith" then your out of luck. In addition, let's say you want to find everyone in your high school class (a typical facebook or classmates use case) - doing this using Google is really really tough because of the need for very structured meta-data against a person which google doesn't index.
So what about creating a search engine that uses metadata based searching to find people? For example, in facebook you have employment information, high school information, etc.
Here are a couple mock-ups to illustrate what I'm thinking of here...
In addition, what about linking profiles between networks so you could see an aggregated profile of a person's blog, facebook page, myspace page, every video they've posted to YouTube, etc.
A couple of issues:
1. Unique identifiers - there would to be a way to uniquely identify someone across networks. The most obvious identifier to use would be email address since this is commonly included in most user profiles.
This is a common problem in health care and there are already good solutions that use probabilistic scoring algorithms to analyze meta-data based on linkage business rules determine that two profiles are really the same person. A simple obvious example - if two profiles have the same email address, we can probably assume they're the same person. It gets more interesting when trying to match without a unique identifier and you're dealing with scoring of a variety of attributes such as gender, date of birth, first name, last name, postal code, etc. Given the rich profile detail on most of these network sites the ability to match profiles should be very strong - if you two profiles with the same gender, date of birth, last name and geographic indicator such as postal code, its pretty safe to assume that they are pointing to the same individual.
2. What information gets aggregated - this is a trickier problem. One solution would be to simply require participating networks to have an opt-in/opt-out ability to syndicate their profile information to the aggregator. This is similar to what Facebook or LinkedIn does today - the user gets to decide which information is public. Another option would be to enable any user to see their own aggregated profile and at least diagnose where the information is coming from. Then at least I can easily tell in a single view what is being syndicated and change it.
3. How I get contacted - all networking sites have different rules on how they allow contacts in order to prevent spammers, stalkers, etc. Again, one simple solution would be to simply delegate the contact rules back to the local network. If I find a facebook profile and want to contact that person (adding them as a friend) then I would follow the facebook process to do so (which is quite different than LinkedIn). This would also allow networking sites to compete on features but still provide a common meeting point.
4. Indexing additional information - it would be great to see additional general information from Google but optimized for people searching. There must be some interesting algorithms that could be developed for analyzing any web page for common profile meta-data. For example, take a standard web based discussion board. Each message might have at least a name, an email and the message content. If you can make a match on this, then you could aggregate this content together to see what people have posted over the past 6 months.
I think there are tonnes of possibilities here, and with the increase in meta-data from these networks, we should be able to do a lot better than standard google searching and having to go from network to network to search for people.
I'm currently on linked-in and facebook heavily. I haven't tried myspace or twitter yet. In addition, there are dozens of other user profiles floating around the Internet in blogs, web pages, etc. And there are still old-school sites such as Classmates.com which provide people finding services through a local service.
Finding people on the Internet is actually reasonably hard to do. Simply googling a person's name might find the person, but if your name is "John Smith" then your out of luck. In addition, let's say you want to find everyone in your high school class (a typical facebook or classmates use case) - doing this using Google is really really tough because of the need for very structured meta-data against a person which google doesn't index.
So what about creating a search engine that uses metadata based searching to find people? For example, in facebook you have employment information, high school information, etc.
Here are a couple mock-ups to illustrate what I'm thinking of here...
In addition, what about linking profiles between networks so you could see an aggregated profile of a person's blog, facebook page, myspace page, every video they've posted to YouTube, etc.
A couple of issues:
1. Unique identifiers - there would to be a way to uniquely identify someone across networks. The most obvious identifier to use would be email address since this is commonly included in most user profiles.
This is a common problem in health care and there are already good solutions that use probabilistic scoring algorithms to analyze meta-data based on linkage business rules determine that two profiles are really the same person. A simple obvious example - if two profiles have the same email address, we can probably assume they're the same person. It gets more interesting when trying to match without a unique identifier and you're dealing with scoring of a variety of attributes such as gender, date of birth, first name, last name, postal code, etc. Given the rich profile detail on most of these network sites the ability to match profiles should be very strong - if you two profiles with the same gender, date of birth, last name and geographic indicator such as postal code, its pretty safe to assume that they are pointing to the same individual.
2. What information gets aggregated - this is a trickier problem. One solution would be to simply require participating networks to have an opt-in/opt-out ability to syndicate their profile information to the aggregator. This is similar to what Facebook or LinkedIn does today - the user gets to decide which information is public. Another option would be to enable any user to see their own aggregated profile and at least diagnose where the information is coming from. Then at least I can easily tell in a single view what is being syndicated and change it.
3. How I get contacted - all networking sites have different rules on how they allow contacts in order to prevent spammers, stalkers, etc. Again, one simple solution would be to simply delegate the contact rules back to the local network. If I find a facebook profile and want to contact that person (adding them as a friend) then I would follow the facebook process to do so (which is quite different than LinkedIn). This would also allow networking sites to compete on features but still provide a common meeting point.
4. Indexing additional information - it would be great to see additional general information from Google but optimized for people searching. There must be some interesting algorithms that could be developed for analyzing any web page for common profile meta-data. For example, take a standard web based discussion board. Each message might have at least a name, an email and the message content. If you can make a match on this, then you could aggregate this content together to see what people have posted over the past 6 months.
I think there are tonnes of possibilities here, and with the increase in meta-data from these networks, we should be able to do a lot better than standard google searching and having to go from network to network to search for people.
Tuesday, April 24, 2007
The Growing Phenomenon of Internet
This brings me back - I found this old CBC new video circa 1993 on this new fangled "Internet".
In 1993, I was in 3rd year university and had one of the few accounts thanks to some connections through my parents who were academics. In those days, only grad students were allowed Internet accounts. This was pre-WWW where unix terminals were using protocols such as email, ftp, gopher and veronica. Mosaic (which then became Netscape) wouldn't come out for another year or so.
An another interesting historical note - "CRS" standard for Canada Remote Systems which was one of the leading Bulletin Board Systems in Toronto. They were a commercial BBS with hundreds of phone lines allowing for local chat. They figured out and tried to hook up to the new Internet in order to avoid extinction - unsuccessfully. CRS got wiped out with the rest of BBS's as the Internet took over.
In 1993, I was in 3rd year university and had one of the few accounts thanks to some connections through my parents who were academics. In those days, only grad students were allowed Internet accounts. This was pre-WWW where unix terminals were using protocols such as email, ftp, gopher and veronica. Mosaic (which then became Netscape) wouldn't come out for another year or so.
An another interesting historical note - "CRS" standard for Canada Remote Systems which was one of the leading Bulletin Board Systems in Toronto. They were a commercial BBS with hundreds of phone lines allowing for local chat. They figured out and tried to hook up to the new Internet in order to avoid extinction - unsuccessfully. CRS got wiped out with the rest of BBS's as the Internet took over.
Classic Presentation from Uncle Bob Martin
I saw Bob Martin at a Software Development conference a couple years ago and loved his presentation. He's one of the founders of the Agile Manifesto and one of the smartest old-school architects in the business.
He's just posted his classic presentation on The Principles of Agile Design. This is a must see for any Technical Architect.
He's just posted his classic presentation on The Principles of Agile Design. This is a must see for any Technical Architect.
Service Oriented Architecture - "I think we spend way too much on the 'A' and not enough on the 'SO.'"
There is a really neat article in today's CIO.com. The article describes some of the trends in IT over the next few years, and the general argument from Gartner points to a vision of an IT department that is customer focused.
I showed this cartoon to my IT team and described the "traditional" view of IT - this is how a lot of customers see us as IT professionals:
For my entire career, I've been in Information Technology but have never really been a technologist. I can certainly code, architect solutions, but if you pressured me to come up with a fast algorithm I'd be hard pressed to do so. I've never worked with mainframes, punch cards, assembler, or hardware other than my PC.
What I do know how to do and what is increasingly valued in the new vision of Information Technology is to come up with creative technology solutions and I can span from end to end of the technology development life cycle. I can sell solutions, evangelize change and think strategically about a problem and create an applied technology solution.
When I meet young programmers, I actively encourage this type of thinking because as the article points out, this is really the differentiators in my line of business. If you want to make yourself outsourcable, concentrate on algorithms, data, and optimization of technology solutions. If you want to stop being outsourced, start thinking about the solution, the customer and how you can take most of your components from cheap off the components and bring them to life to solve a real business problem. That's how the modern IT department creates value - the rest will be quickly outsourced or commoditized.
I showed this cartoon to my IT team and described the "traditional" view of IT - this is how a lot of customers see us as IT professionals:
For my entire career, I've been in Information Technology but have never really been a technologist. I can certainly code, architect solutions, but if you pressured me to come up with a fast algorithm I'd be hard pressed to do so. I've never worked with mainframes, punch cards, assembler, or hardware other than my PC.
What I do know how to do and what is increasingly valued in the new vision of Information Technology is to come up with creative technology solutions and I can span from end to end of the technology development life cycle. I can sell solutions, evangelize change and think strategically about a problem and create an applied technology solution.
When I meet young programmers, I actively encourage this type of thinking because as the article points out, this is really the differentiators in my line of business. If you want to make yourself outsourcable, concentrate on algorithms, data, and optimization of technology solutions. If you want to stop being outsourced, start thinking about the solution, the customer and how you can take most of your components from cheap off the components and bring them to life to solve a real business problem. That's how the modern IT department creates value - the rest will be quickly outsourced or commoditized.
Monday, April 23, 2007
Where Will The Cool Kids Be Hanging Out Next?
Friendster and MySpace is no longer a youth hang-out. Over 50% of the demographic for MySpace is now over 35 years old. No cool teenager wants to be hanging out in the same space as their mother, so most have moved onto cooler pastures.
Facebook now seems to be going through the same transition. I can measure it quite easily because I'm on the cusp (I'm 33 years old) and over the past 3-4 months the number of old high school contacts has been increasing steadily. This shows that people of my age group are invading what used to be the cool youth hang-out.
How will this impact those of us who are interested in marketing to the online audience? From my perspective, it depends on what demographic you're interested in. If you're interested in the youth market it means an increasing shift from site to site as a fickle audience moves from one trend site to another (as with any other youth oriented product). However, if you're interested in an older demographic, it means that social networking sites should be viewed as finally completely mainstream. Facebook, YouTube, MySpace are now longer fringe cool clubs but are the equivalent of the suburban mall. This presents a great opportunity to market to those adults now using these technologies to network.
A good example of this is LinkedIn. There are 6 million+ users in LinkedIn, and the demographic is largely urban professionals with a bias towards technology users. For a recruiter, this presents an opportunity. For $145US a posting, you can post your job posting to all 6 million users. In addition, those users that are connected to you via the LinkedIn network can get to you through your network connections and help further the conversation. We tried our first LinkedIn posting for a couple positions at the Foundation and were quite successful - I received about 15-20 resumes for 2 positions and they generally ranged from decent to excellent.
Facebook now seems to be going through the same transition. I can measure it quite easily because I'm on the cusp (I'm 33 years old) and over the past 3-4 months the number of old high school contacts has been increasing steadily. This shows that people of my age group are invading what used to be the cool youth hang-out.
How will this impact those of us who are interested in marketing to the online audience? From my perspective, it depends on what demographic you're interested in. If you're interested in the youth market it means an increasing shift from site to site as a fickle audience moves from one trend site to another (as with any other youth oriented product). However, if you're interested in an older demographic, it means that social networking sites should be viewed as finally completely mainstream. Facebook, YouTube, MySpace are now longer fringe cool clubs but are the equivalent of the suburban mall. This presents a great opportunity to market to those adults now using these technologies to network.
A good example of this is LinkedIn. There are 6 million+ users in LinkedIn, and the demographic is largely urban professionals with a bias towards technology users. For a recruiter, this presents an opportunity. For $145US a posting, you can post your job posting to all 6 million users. In addition, those users that are connected to you via the LinkedIn network can get to you through your network connections and help further the conversation. We tried our first LinkedIn posting for a couple positions at the Foundation and were quite successful - I received about 15-20 resumes for 2 positions and they generally ranged from decent to excellent.
Do We Really Need an AJAX Conference?
"The AJAX Experience" is coming to Boston and San Francisco if anyone is interested. Personally, I'm really not sure why you need 3 days to learn about AJAX.
Another strange note - the conference web site doesn't use AJAX! They have these tab like controls that refresh the whole page when you click on them.
Another strange note - the conference web site doesn't use AJAX! They have these tab like controls that refresh the whole page when you click on them.
Microsoft .NET is Hit with Patent Suit
Microsoft has just been hit with a patent suit by Vertical Computer Systems Inc. alleging that it violates Patent No. 6,826,744.
I went to Vertical's web site, and I found this diagram that describes their "patented" solution:
The definition of the patent is for the following:
A system and method for generating computer applications in an arbitrary object framework. The method separates content, form, and function of the computer application so that each may be accessed or modified separately. The method includes creating arbitrary objects, managing the arbitrary objects throughout their life cycle in an object library, and deploying the arbitrary objects in a design framework for use in complex computer applications.
And of course, while the going was good they made sure to make the patent as broad as possible so they included a claim on text file pointers, binary file pointers, compiled executables, shell commands, remote procedure calls, global variables, local variables, local objects, global parent objects, cached objects, database queries, etc.
Huh? Doesn't this describe any modern OO application in existence? In 2004 when the patent was granted, there were certainly applications that separated content, form and functionality - we were building java based search engine in 1996 and they would have have separated content, form and functionality. The .NET framework has been around since 2000 and also provides similar functionality.
Is the problem with these technology patents that the US patent office has no understanding of technology? Surely if this went through a decent technical review someone would have decided this was a) too broad to be patent worthy and b) already existed.
Have a look at this company's latest SEC filing and you'll see a lot of legal proceedings (mainly between the company's owners and shareholders suing each other), claims of lack of profitability and no real hope in sight other than trying to protect their IP by suing other companies.
Our success is dependent, in part, upon our ability to protect and leverage the value of our original SiteFlash™ and Emily® technology products and Internet content, as well as our trade secrets, trade names, trademarks, service marks, domain names and other proprietary rights we either currently have or may have in the future.
there can be no assurance that we will generate significant revenues or that we will attain a level of profitability in the future.
Let's hope that Microsoft defends this one without mercy for all of us who are working in the technology space who believe that innovation is more profitable than bureaucracy and legal precedent.
I went to Vertical's web site, and I found this diagram that describes their "patented" solution:
The definition of the patent is for the following:
A system and method for generating computer applications in an arbitrary object framework. The method separates content, form, and function of the computer application so that each may be accessed or modified separately. The method includes creating arbitrary objects, managing the arbitrary objects throughout their life cycle in an object library, and deploying the arbitrary objects in a design framework for use in complex computer applications.
And of course, while the going was good they made sure to make the patent as broad as possible so they included a claim on text file pointers, binary file pointers, compiled executables, shell commands, remote procedure calls, global variables, local variables, local objects, global parent objects, cached objects, database queries, etc.
Huh? Doesn't this describe any modern OO application in existence? In 2004 when the patent was granted, there were certainly applications that separated content, form and functionality - we were building java based search engine in 1996 and they would have have separated content, form and functionality. The .NET framework has been around since 2000 and also provides similar functionality.
Is the problem with these technology patents that the US patent office has no understanding of technology? Surely if this went through a decent technical review someone would have decided this was a) too broad to be patent worthy and b) already existed.
Have a look at this company's latest SEC filing and you'll see a lot of legal proceedings (mainly between the company's owners and shareholders suing each other), claims of lack of profitability and no real hope in sight other than trying to protect their IP by suing other companies.
Our success is dependent, in part, upon our ability to protect and leverage the value of our original SiteFlash™ and Emily® technology products and Internet content, as well as our trade secrets, trade names, trademarks, service marks, domain names and other proprietary rights we either currently have or may have in the future.
there can be no assurance that we will generate significant revenues or that we will attain a level of profitability in the future.
Let's hope that Microsoft defends this one without mercy for all of us who are working in the technology space who believe that innovation is more profitable than bureaucracy and legal precedent.
Saturday, April 21, 2007
Amazing Use of Google Earth - Crisis in Darfur
The United States Holocaust Memorial Museum has released an amazing KML layer for Google Earth which allows you to see a massive amount of information on the crisis in the Darfur region of Sudan. It's an amazing use of Google Earth and really shows the untapped power of Google Earth as a new way of "browsing". As I have posted previously, the whole method of going to a web site and getting information in "web pages" is outdated. In the next few years, the power of applications such as Google Earth, Microsoft Vista, Google software, AJAX and Flash based applications, etc. will change the whole view of a web site from being a set of web pages to a complete and three dimensional application.
Instead of doing the typical site seeing tour in Google Earth by looking at Las Vegas, Grand Canyon, etc. you should spend some time with this layer and see in highly visible detail the destruction in this region of the world. In addition, each village is marked with additional information and links to how you can help out.
Instead of doing the typical site seeing tour in Google Earth by looking at Las Vegas, Grand Canyon, etc. you should spend some time with this layer and see in highly visible detail the destruction in this region of the world. In addition, each village is marked with additional information and links to how you can help out.
Is Posting on Facebook Enough Action To Ease Your Conscience?
Last week, as most of you know there was a horrible school shooting at Virginia Tech.
As a result, there has been an outpouring of blog, facebook and myspace activity of mourners around the world. There are currently 500+ facebook groups dedicated to posting nice notes about Virginia Tech.
My question is a simple one - how much of that virtual action can be translated into real action that has impact in the world? This is a critical question for us who are associated with organizations seeking to change the world through some real action. In my case, I want to be able to translate lots of virtual activity such as blog posts, message boards, etc. into tangible donations for SickKids Foundation. In Amnesty International's case, they want to translate web activity to letter writing campaigns, activism around the world, etc.
Does having 500 groups devoted to Virginia Tech on facebook accomplish anything other than communication? And does the virtual activity encourage real action or does it encourage complacency, e.g. it eases people's conscience enough to avoid having to actually DO something in real life about the world's problems?
I see the same problem with the current hype over the Greenhouse effect. This week in Toronto the city tried to ban leaf blowers as they create pollution and release a lot of carbon dioxide into the atmosphere. People would be encouraged to a push mower instead. What's interesting is that despite all the good will, positive communication and media attention when there is a little bit of real action required, the commitment evaporates. The leaf blower ban was defeated easily thanks to the "leaf blower lobby" of people with large lawns, the landscaping industry, etc.
So I think the next wave of these social networking sites is the ability to drive the hype to conversion - if all this social activity doesn't actually drive itself to a transaction (a petition, a protest, an event, a donation, a purchase, etc.) then it's really just hot air...
The Cause for RIM Going Down is a Software Failure
RIM has finally published a press release explaining why their entire network went down this week.
It turns out that it was a software failure, not a hardware failure. As our phone technology is more and more software based (IP Phones, Skype, Blackberries, IPhone, etc.) the risk of these types of failures either at the client level (your phone crasehs) or at the network level (the network fails) is going to increase dramatically.
Ordinary old-school phones have amazing uptime because they're almost 100% hardware based - the software in them is almost non-existent. The technology also hasn't changed in about a hundred years.
But with the evolution of phones, you'll start to see that reliability go down even as the dependency on our phones as they become mobile data appliances goes up.
I'm not sure this is such a great thing, but your car, your television, and soon your appliances are all on the same trajectory. Your car is already there - the number of "computers" in the average car is growing every year and the more luxurious the car, the more micro-processors are controlling the driving experience.
I had a problem with my car where the on-board sensor (e.g. micro-processor) was failing to regulate the amount of fuel going into the engine. The car worked fine but the engine light would come because the sensor couldn't see that it was working. Cost of repair was $1100!
It turns out that it was a software failure, not a hardware failure. As our phone technology is more and more software based (IP Phones, Skype, Blackberries, IPhone, etc.) the risk of these types of failures either at the client level (your phone crasehs) or at the network level (the network fails) is going to increase dramatically.
Ordinary old-school phones have amazing uptime because they're almost 100% hardware based - the software in them is almost non-existent. The technology also hasn't changed in about a hundred years.
But with the evolution of phones, you'll start to see that reliability go down even as the dependency on our phones as they become mobile data appliances goes up.
I'm not sure this is such a great thing, but your car, your television, and soon your appliances are all on the same trajectory. Your car is already there - the number of "computers" in the average car is growing every year and the more luxurious the car, the more micro-processors are controlling the driving experience.
I had a problem with my car where the on-board sensor (e.g. micro-processor) was failing to regulate the amount of fuel going into the engine. The car worked fine but the engine light would come because the sensor couldn't see that it was working. Cost of repair was $1100!
Tuesday, April 17, 2007
We Need a New Paradigm for Web Analytics
I have been studying how to make our web analytics better for reporting dynamic campaign data. We have WebTrends as our analytics, but and on this blog I use Google Analytics for my personal analytics solution.
From an architecture perspective, the basic way an analytics engine gets its data is two ways:
1. A javascript that sends a message to the engine with some tracking data (this is how Google Analytics works for example)
2. A standard web log file is injested (this is how WebTrends typically works)
Both tend to log the static URL plus any additional HTTP information available, e.g. the referrer, query parameters, geographic information, browser information, etc.
All of this works fine for a site built in 1999 when static URLs were the driving force behind a web site. But in an ASP.NET or any other server side generated web site, trying to store all this traffic data in a client side URL is pretty limiting. Consider the following scenarios:
1. You have an ASP.NET forms based application which gets posted back to the same URL over and over again as the form gets edited. The URL doesn't change but lots of information is changing in the background.
2. You have an ASP.NET page which is dynamically built out page fragments such as a portal page.
3. You want to associate a non-page related event to a web session along with the page data you are tracking. For example, suppose that you have a checkout process and you want to register that an order was created and link that to your web page analytics so that you can see which preceding pages link to an order.
In all these cases, the traditional web analytics approach doesn't work. If you look at how WebTrends suggests to do campaign tracking, this gives you an idea on how to fake out the software to track these events. You can store query parameters to identify variables, events, etc. and then get WebTrends to report against them. However, this isn't really a feasible solution for the above scenarios in that a) you don't necessarily want to expose data back on the URL; b) you don't want to have to constantly dynamically re-write every URL in your application; c) if you have a complicated set of objects your tracking then writing out a serialized objected isn't feasible either.
To me this is a real opportunity for Microsoft and the .NET framework in the future. It would be a snap for them to implement it - they already most of the pieces in place and could offer a very open, developer friendly approach to web analytics. This would allow for some very funky tracking in the background and provide the developer the flexibility to log a whole bunch of information as needed. The trick in building the solution is performance - the reason why WebTrends works on web logs is because its fast and means no performance hit on your web traffic. But at least in our case, we would sacrifice a minor performance hit to get more powerful data - we get relatively small amounts of traffic but understanding the patterns in high amounts of detail is very powerful in improve conversion, analyze campaign effectiveness, etc.
Has anyone come across a product that approaches web analytics this way?
From an architecture perspective, the basic way an analytics engine gets its data is two ways:
1. A javascript that sends a message to the engine with some tracking data (this is how Google Analytics works for example)
2. A standard web log file is injested (this is how WebTrends typically works)
Both tend to log the static URL plus any additional HTTP information available, e.g. the referrer, query parameters, geographic information, browser information, etc.
All of this works fine for a site built in 1999 when static URLs were the driving force behind a web site. But in an ASP.NET or any other server side generated web site, trying to store all this traffic data in a client side URL is pretty limiting. Consider the following scenarios:
1. You have an ASP.NET forms based application which gets posted back to the same URL over and over again as the form gets edited. The URL doesn't change but lots of information is changing in the background.
2. You have an ASP.NET page which is dynamically built out page fragments such as a portal page.
3. You want to associate a non-page related event to a web session along with the page data you are tracking. For example, suppose that you have a checkout process and you want to register that an order was created and link that to your web page analytics so that you can see which preceding pages link to an order.
In all these cases, the traditional web analytics approach doesn't work. If you look at how WebTrends suggests to do campaign tracking, this gives you an idea on how to fake out the software to track these events. You can store query parameters to identify variables, events, etc. and then get WebTrends to report against them. However, this isn't really a feasible solution for the above scenarios in that a) you don't necessarily want to expose data back on the URL; b) you don't want to have to constantly dynamically re-write every URL in your application; c) if you have a complicated set of objects your tracking then writing out a serialized objected isn't feasible either.
To me this is a real opportunity for Microsoft and the .NET framework in the future. It would be a snap for them to implement it - they already most of the pieces in place and could offer a very open, developer friendly approach to web analytics. This would allow for some very funky tracking in the background and provide the developer the flexibility to log a whole bunch of information as needed. The trick in building the solution is performance - the reason why WebTrends works on web logs is because its fast and means no performance hit on your web traffic. But at least in our case, we would sacrifice a minor performance hit to get more powerful data - we get relatively small amounts of traffic but understanding the patterns in high amounts of detail is very powerful in improve conversion, analyze campaign effectiveness, etc.
Has anyone come across a product that approaches web analytics this way?
Sunday, April 15, 2007
Are You A Guerrilla Project Manager?
Most Project Managers if they have been groomed through the PMP process are like generals in the Army or foreman on a construction site. Their approach tends to favour:
1. Command and control
2. Constant monitoring
3. Process over results
4. Risk aversion and mitigation
5. Dependence on Tools (Project Plans, Status Reports, Risk Matrices, etc.)
6. Waterfall methodologies
Tom Peters in his latest book, Re-Imagine! Business Excellence in a Disruptive Age,
uses the example of terrorist cells as the model for the new virtual organization.
I like the term "Guerrilla" as it has less negative connotations. If you want to feel patriotic, use the analogy of being in the Special Forces.
Frankly, I think this is going to be a tough sell to most traditional Project Managers as their entire training is based on high amounts of process, change management and tooling to avoid adaptability, gut instinct and autonomy.
So if you are a project manager, ask yourself: can you manage like a terrorist?
1. Could you survive without your trusted Microsoft Project Plan? Can you create a schedule using only a white board, a notepad, or a sticki-note?
2. Could you build a multi-million $ project plan in a day and stick to it? Or would you snivel back to your boss saying, "Uhm I don't have the requirements, deliverables, scope, resource, allocation, etc."
3. Have you built your team so that if you disappeared for a week it could survive?
4. Can you really change on a dime? Can you do it without filling out process forms, change management requests, etc.?
5. Is the goal for your project simple? If you had to explain your project in 10-15 minutes, could you do it?
6. Do you have vertically specialized knowledge or are you just overhead? Could you take over one of your team mates workload? If you're on a software project, do you know how to code? Do you understand the architecture of the project? Could you do a build? If you're running a health care project can you talk HL7? Or is your expertise simply pumping out status reports on a weekly basis?
7. Are you a passionate defender of your project and project team? Are you politically strong enough to jump into the fray for your team? Are you afraid of hierarchy? Can you take out an opponent in your way?
8. Could you make a decision by gut feel in 30 seconds? Can you plan in your head? Can you make decisions with patchy information?
9. Are you a PM that likes to keep everything in control? Do you play it safe? Do you follow the traditional PMP methodology of trying to avoid risks at all costs through mitigation, documentation and forecasting? If you were thrown into a crisis, could you manage?
10. Can you jump into a project that's already in crisis and fix it? Can you do it in days, not months?
11. Can you manage fall-out? Can you live with an imperfect solution in order to make time-to-market targets (I once had a project where we launched an e-commerce application that for the first three months took a room full of temps to do data-entry in the background because we couldn't automate the whole solution)?
If you're designing a building, a space shuttle or a health care application, this isn't the type of management style you want from your project manager. The risks are too high - if you take the risk and someone dies as a result, then you need a formal approach and PMBOK will work quite nicely.
But the vast majority of applications in business are far more slippery, agile and time-to-market sensitive. If as a PM you are not entrepreneurial, agile and a gut feel leader then you're going to be toast if any small crisis hits your project.
When I hire project managers, I tell them the following: I expect you to be a competent project manager with good process. That's a given, but at the end of the day, that's not your real value proposition.
Your real value proposition is this: at some point during your project, you will hit a crisis which requires strength of character, real leadership and the ability to make a judgement call. You're going to have to stand in the middle of that crisis and keep your project together, lead your team and be able to make judgements with little to no support.
Example of what I mean: I was running a project where I was launching a software application. I was working for a program manager who was quite process oriented and very sensitive to "adoption", e.g. how are we going to manage our customers. We were supposed to launch at 3pm, and I was still getting performance test reports at 2pm that said were weren't going to make the target performance requirements. We ran one last test, and at 2:45 I got the last results. We were good to go!
So I had a decision to make - we could rush the launch (which was quite time sensitive) and make the time line or slow down and scrub the launch giving us some more time to prepare documentation, change request orders, deployment guides and communication materials.
My Program Manager freaked and said, "But where are your PowerPoint slides?! How are we going to communicate this to our stakeholders at our 3pm con call?!". Instant panic.
I walked into the CIO's office who had to make the ultimate decision to launch or not and this Program Manager actually started apologizing for not being prepared and started trying to paraphrase what I had just told her to the CIO! Knowing that this wasn't instilling confidence, I quickly interupted her and simply said, "Look, here's the deal. We just got off the floor and we're ready to go. We can either scrub this launch or you can go ahead. I can tell you that from my perspective we're ready to launch, but if you would like to review it further we can delay the launch date."
The CIO took one look at me, trusted me and said, "Ok let's go."
But this just opened up another "adoption" problem - we had a conference call scheduled with all the customers and typically agenda items and materials were send 24 hours before the meeting. I was typically not involved in these calls as I was too far down the hierarchy (command and control!) to be talking directly to customers. The program manager was anxious, "How am I supposed to communicate this to our stakeholders?"
I suggested a simple solution - I got on the con call myself and simply talked to all the stakeholders. I got on the call, told the group what the status of the project was and without hesitation told them that we were ready to launch and that I had 100% confidence that we were ready to go.
At the end of day, we launched on that day. It was a bit messy on the back-end and we had lots of people in IT nervously monitoring the system for any hiccups, but we made the date and launched successfully. That's what happens in the real world...
Could you do that? Do you have the guts as a PM to walk into a meeting without materials, communicate to your stakeholders about a launch that you just verified could work 15 minutes ago, and then with confidence and poise communicate that to your stakeholders in a way that gives them confidence that you know what you're talking about?
That's what a Guerrilla PM does...
1. Command and control
2. Constant monitoring
3. Process over results
4. Risk aversion and mitigation
5. Dependence on Tools (Project Plans, Status Reports, Risk Matrices, etc.)
6. Waterfall methodologies
Tom Peters in his latest book, Re-Imagine! Business Excellence in a Disruptive Age,
uses the example of terrorist cells as the model for the new virtual organization.I like the term "Guerrilla" as it has less negative connotations. If you want to feel patriotic, use the analogy of being in the Special Forces.
Frankly, I think this is going to be a tough sell to most traditional Project Managers as their entire training is based on high amounts of process, change management and tooling to avoid adaptability, gut instinct and autonomy.
So if you are a project manager, ask yourself: can you manage like a terrorist?
1. Could you survive without your trusted Microsoft Project Plan? Can you create a schedule using only a white board, a notepad, or a sticki-note?
2. Could you build a multi-million $ project plan in a day and stick to it? Or would you snivel back to your boss saying, "Uhm I don't have the requirements, deliverables, scope, resource, allocation, etc."
3. Have you built your team so that if you disappeared for a week it could survive?
4. Can you really change on a dime? Can you do it without filling out process forms, change management requests, etc.?
5. Is the goal for your project simple? If you had to explain your project in 10-15 minutes, could you do it?
6. Do you have vertically specialized knowledge or are you just overhead? Could you take over one of your team mates workload? If you're on a software project, do you know how to code? Do you understand the architecture of the project? Could you do a build? If you're running a health care project can you talk HL7? Or is your expertise simply pumping out status reports on a weekly basis?
7. Are you a passionate defender of your project and project team? Are you politically strong enough to jump into the fray for your team? Are you afraid of hierarchy? Can you take out an opponent in your way?
8. Could you make a decision by gut feel in 30 seconds? Can you plan in your head? Can you make decisions with patchy information?
9. Are you a PM that likes to keep everything in control? Do you play it safe? Do you follow the traditional PMP methodology of trying to avoid risks at all costs through mitigation, documentation and forecasting? If you were thrown into a crisis, could you manage?
10. Can you jump into a project that's already in crisis and fix it? Can you do it in days, not months?
11. Can you manage fall-out? Can you live with an imperfect solution in order to make time-to-market targets (I once had a project where we launched an e-commerce application that for the first three months took a room full of temps to do data-entry in the background because we couldn't automate the whole solution)?
If you're designing a building, a space shuttle or a health care application, this isn't the type of management style you want from your project manager. The risks are too high - if you take the risk and someone dies as a result, then you need a formal approach and PMBOK will work quite nicely.
But the vast majority of applications in business are far more slippery, agile and time-to-market sensitive. If as a PM you are not entrepreneurial, agile and a gut feel leader then you're going to be toast if any small crisis hits your project.
When I hire project managers, I tell them the following: I expect you to be a competent project manager with good process. That's a given, but at the end of the day, that's not your real value proposition.
Your real value proposition is this: at some point during your project, you will hit a crisis which requires strength of character, real leadership and the ability to make a judgement call. You're going to have to stand in the middle of that crisis and keep your project together, lead your team and be able to make judgements with little to no support.
Example of what I mean: I was running a project where I was launching a software application. I was working for a program manager who was quite process oriented and very sensitive to "adoption", e.g. how are we going to manage our customers. We were supposed to launch at 3pm, and I was still getting performance test reports at 2pm that said were weren't going to make the target performance requirements. We ran one last test, and at 2:45 I got the last results. We were good to go!
So I had a decision to make - we could rush the launch (which was quite time sensitive) and make the time line or slow down and scrub the launch giving us some more time to prepare documentation, change request orders, deployment guides and communication materials.
My Program Manager freaked and said, "But where are your PowerPoint slides?! How are we going to communicate this to our stakeholders at our 3pm con call?!". Instant panic.
I walked into the CIO's office who had to make the ultimate decision to launch or not and this Program Manager actually started apologizing for not being prepared and started trying to paraphrase what I had just told her to the CIO! Knowing that this wasn't instilling confidence, I quickly interupted her and simply said, "Look, here's the deal. We just got off the floor and we're ready to go. We can either scrub this launch or you can go ahead. I can tell you that from my perspective we're ready to launch, but if you would like to review it further we can delay the launch date."
The CIO took one look at me, trusted me and said, "Ok let's go."
But this just opened up another "adoption" problem - we had a conference call scheduled with all the customers and typically agenda items and materials were send 24 hours before the meeting. I was typically not involved in these calls as I was too far down the hierarchy (command and control!) to be talking directly to customers. The program manager was anxious, "How am I supposed to communicate this to our stakeholders?"
I suggested a simple solution - I got on the con call myself and simply talked to all the stakeholders. I got on the call, told the group what the status of the project was and without hesitation told them that we were ready to launch and that I had 100% confidence that we were ready to go.
At the end of day, we launched on that day. It was a bit messy on the back-end and we had lots of people in IT nervously monitoring the system for any hiccups, but we made the date and launched successfully. That's what happens in the real world...
Could you do that? Do you have the guts as a PM to walk into a meeting without materials, communicate to your stakeholders about a launch that you just verified could work 15 minutes ago, and then with confidence and poise communicate that to your stakeholders in a way that gives them confidence that you know what you're talking about?
That's what a Guerrilla PM does...
Saturday, April 14, 2007
The Biggest Security Threat - Paper and the Front Door
In Information Technology, most of us digital professionals tend to think about security. We spend out energy on making sure that hackers cannot penetrate from the outside, we look at virus/trojan protection and we clamp down on the firewall to make sure data gets in and out. In addition, if we're really effective, we might start to look at internal security such as who has access to data, vendor management, working from home arrangements, etc.
Because we spend most of the time working with technology, many IT folks will forget the most insecure form of data - Paper!
If you want to see what I'm talking about, go in on a Sunday afternoon and walk around the office and snoop on people's desks. See what paper is sitting around their office out in the open. You may be suprised at what's sitting there. There isn't much point in protecting your database if someone in accounting has printed off the data and left it on their desk.
Another basic security problem that no one in IT tends to worry about - the Front Door!
Most companies have a reception area, and if they are really well secured the reception area will act as a gatekeeper to the other floors. However, most receptionists tend to work from 8:30-4:30. In addition, they tend to take lunch breaks and may not have someone to cover them.
So if you want to test your front door, simply go into the office at 8:30, 4:30 or noon and see if you can walk right in. If you can, then you can probably also walk right out with a printer, a laptop, etc. without anyone questioning it. Most workers if they see someone in the office will assume they've been authorized through reception. If you can crack that reception authorization mechanism, you're likely going to get very little resistance from anyone.
So try it out...and if its a problem, shut it down immediately. Laptop thieves for example are in and out in about 5-10 minutes - if they can get through the front door at any time then you're asking for trouble.
Because we spend most of the time working with technology, many IT folks will forget the most insecure form of data - Paper!
If you want to see what I'm talking about, go in on a Sunday afternoon and walk around the office and snoop on people's desks. See what paper is sitting around their office out in the open. You may be suprised at what's sitting there. There isn't much point in protecting your database if someone in accounting has printed off the data and left it on their desk.
Another basic security problem that no one in IT tends to worry about - the Front Door!
Most companies have a reception area, and if they are really well secured the reception area will act as a gatekeeper to the other floors. However, most receptionists tend to work from 8:30-4:30. In addition, they tend to take lunch breaks and may not have someone to cover them.
So if you want to test your front door, simply go into the office at 8:30, 4:30 or noon and see if you can walk right in. If you can, then you can probably also walk right out with a printer, a laptop, etc. without anyone questioning it. Most workers if they see someone in the office will assume they've been authorized through reception. If you can crack that reception authorization mechanism, you're likely going to get very little resistance from anyone.
So try it out...and if its a problem, shut it down immediately. Laptop thieves for example are in and out in about 5-10 minutes - if they can get through the front door at any time then you're asking for trouble.
Friday, April 13, 2007
Fun with Naming Conventions
Back in the days of Visual Basic 6, they way you did error checking was something like this:
On Error Goto ErrHandler:
N = 1 / 0 ' cause an error
'
' more code
'
Exit Sub
ErrHandler:
' error handling code
Resume Next
End Sub
When I worked on one project, I actually found this code running in production:
On Error Goto Hell:
'
' code was here
'
Exit Sub
Hell:
' error handling code
Resume Next
End Sub
So remember, don't drink and code...
On Error Goto ErrHandler:
N = 1 / 0 ' cause an error
'
' more code
'
Exit Sub
ErrHandler:
' error handling code
Resume Next
End Sub
When I worked on one project, I actually found this code running in production:
On Error Goto Hell:
'
' code was here
'
Exit Sub
Hell:
' error handling code
Resume Next
End Sub
So remember, don't drink and code...
E-File Follow-up
My tax refund arrived in the mail today...so I guess that means that the e-file system is fully operational.
Of course now that its working, no one in the media is asking any questions about what actually happened.
See my original article for more information on our government's IT practices...
Of course now that its working, no one in the media is asking any questions about what actually happened.
See my original article for more information on our government's IT practices...
Wednesday, April 11, 2007
Best Tool for Task and Bug Tracking is JIRA!
I've been using a product from Atlassian called JIRA for the past couple years to track tasks, bugs, and requests.
I highly highly recommend it - its an amazing product and the team behind it provides great support and rapid response. They're also highly supportive of charity and open-source organizations - if you ask nicely, they'll give you an enterprise license to their products for free!
What makes JIRA great?
I highly highly recommend it - its an amazing product and the team behind it provides great support and rapid response. They're also highly supportive of charity and open-source organizations - if you ask nicely, they'll give you an enterprise license to their products for free!
What makes JIRA great?
- Completely web based
- Highly customizable, including allowing for custom fields for each task type
- Configuration notifications via email on changes to tasks. We currently use this to notify the assigned user and the reporter of a task when changes are made to their task.
- Stupid simple to use - it takes about 30 seconds to create a task and about a minute to train an end user on how to use it.
- Everything is permission based, so you can allow people to view status of issues but not allow them to change them for example.
- Java based, runs in Tomcat on just about any kind of machine.
- The database is flexible. We use SQL Server, but it will support MySQL, Oracle, Postgres, etc.
- You can use it for everything throughout the development life cycle. We start using JIRA right at the requirements gathering stage to collect user stories and then translate them into functional tasks during development. Then as we transition to QA, we mark each task as resolved and QA tests and closes them. QA can then add bugs as tasks and re-assign them to the developers.
- From a project management perspective, the tool provides a centralized view of the entire team - its very easy to see all the tasks assigned to a single person, group, component, project, etc. and then re-assign with a few clicks.
- Its simple enough that non-technical users can benefit from it. We're currently using it to track marketing tasks, HR requests, IT helpdesk tasks, etc. as well as software changes. It's that flexible.
I've been using JIRA on multiple software projects for a few years now and would highly recommend it as a solution for managing tasks, bugs, features, etc. in a complex environment.
Even if you're a commercial company, the price for JIRA is totally worth it - licenses are $1200-$4,800US depending on the version you need.
Vintage Material on YouTube
Along with my love of technology, I also love early American music, especially older Jazz and Blues circa 1920's - 1940's.
So I did some searches on YouTube, and I found that along with the usual America's Funniest Stupid Video type clips, there are quite a few videos of vintage music.
See the following URLs for some good examples, but if you search for any of the Jazz greats such as Ella Fitzgerald, Duke Ellington, Django Reinhart, etc. you'll find some great material. There are lots of old blues artists there as well.
Ella...
Django...
Leadbelly...
So I did some searches on YouTube, and I found that along with the usual America's Funniest Stupid Video type clips, there are quite a few videos of vintage music.
See the following URLs for some good examples, but if you search for any of the Jazz greats such as Ella Fitzgerald, Duke Ellington, Django Reinhart, etc. you'll find some great material. There are lots of old blues artists there as well.
Ella...
Django...
Leadbelly...
Sunday, April 8, 2007
Desktop R.I.P? Don't Count on It!
There is an interesting article in this month's Wired with the title, Desktop R.I.P. The argument goes that as more people use applications like Google, Yahoo Mail, Flickr, SalesForce, etc. the need for a desktop will go away and that we'll all live in this Internet-based world where we can have a virtual desktop sitting on some hosted server or servers.
Personally, I don't think this is going to happen for a number of reasons:
1. Hosted applications are fundamentally rented software. In most cases, the rent also scales with the load, e.g. number of users, number of transactions, etc. While this is also true with desktop/server software (maintenance fees, upgrade fees, etc.) there is typically more flexibility in options.
2. Hosted cloud based applications are a return to the "dumb-terminal" concept that has been thrown around and largely rejected for the past 30 years. Why have a $1500 machine on your desk when all it does is act like a green-screen terminal? The PC revolution over the past 20-30 years has shown that having some local hardware makes a huge impact on the experience of software.
3. There are large software segments that will never go cloud, for example games. Do you think that Electronic Arts is going to be creating the equivalent of Google Desktop anytime soon?
4. Desktops are cheap. Servers are expensive. I can buy a desktop for $1199 at MDG. The equivalent Dell server is about $3600, albeit with better power management, smaller footprint, etc. However, in general the economic models don't make much sense - why give up a cheap, non-redundant client and push all that computing to highly redundant servers that are double or triple the price for the equivalent power?
In addition, desktops are getting cheaper. If the economic engine was that desktops were getting more expensive, then a hosted model could come in as a low-cost alternative. But desktops get cheaper every year and in the case of mobile devices, video game consoles, etc. they're subsidized. So why throw away all that local computing power?
5. The desktop is no longer a single desktop - a desktop is now a TV appliance, a mobile phone, a traditional desktop, an XBOX 360, a kiosk, etc. While it is possible to boil all these down to a browser based experience, the experience is pretty lack-luster and compromises the power of these devices in their local context.
Forget desktops - just look at your blackberry and compare your blackberry email built into the client vs. going to mobile.msn.com on your WAP browser. Which one would you choose?
6. Your data sitting on someone else's servers - for the majority of killer apps, this is a non-starter, especially if the data is in the United States. If you host your CRM data in a US data centre, you've now exposed your entire customer database to the reach of the Patriot Act.
So I think the idea of the desktop death watch is premature at best. In fact, it was supposed to die a while ago with the network computer. The Network Computer was supposed to kill the desktop in the mid-90s. The exact same vision as Google's - why have that expensive desktop running local apps when you could replace with a terminal and run everything in the cloud (and of course pay Oracle/Sun a fee for it). Oracle and Sun in the mid 90's were the Google of today - they had publicly declared war on Microsoft and used the exact same arguments - Windows is too expensive, too flaky, too hard to maintain, etc. so just give up on desktops and move to the cloud where everything will be secure, cheap, etc.
How many Network Computers do you see kicking around today?
Personally, I don't think this is going to happen for a number of reasons:
1. Hosted applications are fundamentally rented software. In most cases, the rent also scales with the load, e.g. number of users, number of transactions, etc. While this is also true with desktop/server software (maintenance fees, upgrade fees, etc.) there is typically more flexibility in options.
2. Hosted cloud based applications are a return to the "dumb-terminal" concept that has been thrown around and largely rejected for the past 30 years. Why have a $1500 machine on your desk when all it does is act like a green-screen terminal? The PC revolution over the past 20-30 years has shown that having some local hardware makes a huge impact on the experience of software.
3. There are large software segments that will never go cloud, for example games. Do you think that Electronic Arts is going to be creating the equivalent of Google Desktop anytime soon?
4. Desktops are cheap. Servers are expensive. I can buy a desktop for $1199 at MDG. The equivalent Dell server is about $3600, albeit with better power management, smaller footprint, etc. However, in general the economic models don't make much sense - why give up a cheap, non-redundant client and push all that computing to highly redundant servers that are double or triple the price for the equivalent power?
In addition, desktops are getting cheaper. If the economic engine was that desktops were getting more expensive, then a hosted model could come in as a low-cost alternative. But desktops get cheaper every year and in the case of mobile devices, video game consoles, etc. they're subsidized. So why throw away all that local computing power?
5. The desktop is no longer a single desktop - a desktop is now a TV appliance, a mobile phone, a traditional desktop, an XBOX 360, a kiosk, etc. While it is possible to boil all these down to a browser based experience, the experience is pretty lack-luster and compromises the power of these devices in their local context.
Forget desktops - just look at your blackberry and compare your blackberry email built into the client vs. going to mobile.msn.com on your WAP browser. Which one would you choose?
6. Your data sitting on someone else's servers - for the majority of killer apps, this is a non-starter, especially if the data is in the United States. If you host your CRM data in a US data centre, you've now exposed your entire customer database to the reach of the Patriot Act.
So I think the idea of the desktop death watch is premature at best. In fact, it was supposed to die a while ago with the network computer. The Network Computer was supposed to kill the desktop in the mid-90s. The exact same vision as Google's - why have that expensive desktop running local apps when you could replace with a terminal and run everything in the cloud (and of course pay Oracle/Sun a fee for it). Oracle and Sun in the mid 90's were the Google of today - they had publicly declared war on Microsoft and used the exact same arguments - Windows is too expensive, too flaky, too hard to maintain, etc. so just give up on desktops and move to the cloud where everything will be secure, cheap, etc.
How many Network Computers do you see kicking around today?
Friday, April 6, 2007
SSIS and Visual Studio 2005 are very slow...
We've been using the Business Intelligence Studio as part of Visual Studio 2005. In general, we find its a well designed ETL tool, but for one problem - its too SLOW!
From what I can see, there are two basic problems: 1) the need to repeatedly validate packages against the database; and 2) the assumption that each package gets its own connection to the database.
Problem #1: Validation
The basic problem with the speed issue seems to be its constant need to validate against the underlying database. For example by default:
I have a project where we're doing replication of tables from a SYBASE database to a SQL Server 2005 database. For each table, we've created a package. We now have 50-60 packages.
Every time you open up the project, it re-validates each package against both the SYBASE database and the SQL database. Each build re-validates. Each execute in Debug mode re-validates. Each run in production re-validates. All this validation on a slow database connection makes basic development tasks very slow.
The same thing happens when running the package in DTExec or through the SQL Agent - it pre-validates every package. This is both good and bad - its good in that it means it will fail your package before trying to execute some bad SQL or badly mapped columns, but given that you're going to run the same package over and over again without a change, there should be a way to turn off this step for packages that haven't changed.
I've looked around and haven't seen any method to change this behaviour - it would be great to have some more lazy validation, e.g. once I have validated it once, don't re validate unless I actually make a change to the package.
Problem #2: Data Sources
In an SSIS Project, there are data sources that you can define once as a common data source. However, there seems to be two problems with the data source model:
1. You cannot store an ADO.NET data source configuration as a shared data source - these have to be re-created in each package. Given how common these of connections are, this seems to me to be a big oversight.
2. Each package when its loaded creates its own connection to the database. Its very slow to create a connection and then tear it down especially with either a slow network or a slow database (e.g. an old legacy connection).
This causes problems for us with so many packages. When I open my project, Visual Studio opens each package and validates it. This causes a connection to created to every database in the package in parallel! With our old SYBASE database, it simply gets overwhelmed when Studio tries to validate 50 packages all in parallel.
My only solution so far is to consolidate packages together - I've been taking 20-30 packages and combining them into a single monster package. This reduces the connections created and speeds up validation. However, this strikes me as unreasonable from a re-usability perspective as it reduces package modularity.
I've done a bit of searching around these settings and I haven't found any way to configure the way that packages are validated - if anyone knows of a way, put a comment up so I can be corrected!
From what I can see, there are two basic problems: 1) the need to repeatedly validate packages against the database; and 2) the assumption that each package gets its own connection to the database.
Problem #1: Validation
The basic problem with the speed issue seems to be its constant need to validate against the underlying database. For example by default:
I have a project where we're doing replication of tables from a SYBASE database to a SQL Server 2005 database. For each table, we've created a package. We now have 50-60 packages.
Every time you open up the project, it re-validates each package against both the SYBASE database and the SQL database. Each build re-validates. Each execute in Debug mode re-validates. Each run in production re-validates. All this validation on a slow database connection makes basic development tasks very slow.
The same thing happens when running the package in DTExec or through the SQL Agent - it pre-validates every package. This is both good and bad - its good in that it means it will fail your package before trying to execute some bad SQL or badly mapped columns, but given that you're going to run the same package over and over again without a change, there should be a way to turn off this step for packages that haven't changed.
I've looked around and haven't seen any method to change this behaviour - it would be great to have some more lazy validation, e.g. once I have validated it once, don't re validate unless I actually make a change to the package.
Problem #2: Data Sources
In an SSIS Project, there are data sources that you can define once as a common data source. However, there seems to be two problems with the data source model:
1. You cannot store an ADO.NET data source configuration as a shared data source - these have to be re-created in each package. Given how common these of connections are, this seems to me to be a big oversight.
2. Each package when its loaded creates its own connection to the database. Its very slow to create a connection and then tear it down especially with either a slow network or a slow database (e.g. an old legacy connection).
This causes problems for us with so many packages. When I open my project, Visual Studio opens each package and validates it. This causes a connection to created to every database in the package in parallel! With our old SYBASE database, it simply gets overwhelmed when Studio tries to validate 50 packages all in parallel.
My only solution so far is to consolidate packages together - I've been taking 20-30 packages and combining them into a single monster package. This reduces the connections created and speeds up validation. However, this strikes me as unreasonable from a re-usability perspective as it reduces package modularity.
I've done a bit of searching around these settings and I haven't found any way to configure the way that packages are validated - if anyone knows of a way, put a comment up so I can be corrected!
Thursday, April 5, 2007
Is it a Web Site or Blog?
Tecnorati claims they are monitoring 57 million+ blogs. In the past 30 days, I've seen news.com, nationalgeographic.com, wired.com, yahoo.com, cnn.com all show up on blog search engines.
Is a blog simply now a web site with postings? Isn't that any web site with update-able content?
How do we define a blog if everyone is now "blogging" including large corporate editorial boards?
Even wikipedia's definition of a blog seems really out of date now that corporate blogs have jumped on the bandwagon:
A blog is a user-generated website where entries are made in journal style and displayed in a reverse chronological order.
Isn't this just a web site these days? Is a newspaper site a blog? News.com and Wired certainly seem to want to claim this and they do show up in most blog search engines.
My prediction is that in a year from now the whole idea of a "Blog" will be ubiquitous and therefore an irrelevant distinction.
Is a blog simply now a web site with postings? Isn't that any web site with update-able content?
How do we define a blog if everyone is now "blogging" including large corporate editorial boards?
Even wikipedia's definition of a blog seems really out of date now that corporate blogs have jumped on the bandwagon:
A blog is a user-generated website where entries are made in journal style and displayed in a reverse chronological order.
Isn't this just a web site these days? Is a newspaper site a blog? News.com and Wired certainly seem to want to claim this and they do show up in most blog search engines.
My prediction is that in a year from now the whole idea of a "Blog" will be ubiquitous and therefore an irrelevant distinction.
Expression Web and Blend will be available for MSDN subscribers
In a smart move, Microsoft has announced that it will provide Express Web and Blend products as part of the premium MSDN subscription.
See my last post on Microsoft Expression - from the first look of it, this is a great new addition to the .NET Studio Family and will help to extend the reach of the average .NET developer into the semi-creative space, e.g. user interfaces, rich media, web pages, etc.
Let's just hope it doesn't mangle XAML, HTML and CSS code like the good ol' days of FrontPage. :) Otherwise, I'm going back to Dreamweaver.
See my last post on Microsoft Expression - from the first look of it, this is a great new addition to the .NET Studio Family and will help to extend the reach of the average .NET developer into the semi-creative space, e.g. user interfaces, rich media, web pages, etc.
Let's just hope it doesn't mangle XAML, HTML and CSS code like the good ol' days of FrontPage. :) Otherwise, I'm going back to Dreamweaver.
Wednesday, April 4, 2007
Microsoft should let Apple have the Toy Market
I was in Best Buy and saw some computers running vista with all the bells and whistles turned on. From what I could see, it was more annoying than cool. The whole presentation layer strikes me as something from a bad PowerPoint presentations - too much overuse of transitions, sparkly effects that provide no re all value, etc.
Microsoft, let Apple have the toy market. Give them the 1-5% of the consumer market that wants a dumb terminal with a fancy TV like interface. Concentrate on your strengths and where the money is long term. It's also where Apple will not be able to compete since they don't have the developer tools and programming language support that you get with Microsoft developer tools (.NET, IIS, SQL Server, etc.)
We've been getting exposure to Microsoft's new expression products and I can tell you first hand that its VERY COOL technology. And its cool not in the gee-whiz bang kind of way, its cool in how seamless and effective a tool it is. If you think about how difficult 3D applications used to be and how developer-centric these applications ended up being as a result, this changes the game.
So when you see Vista as a consumer OS, ignore it. Its important for Microsoft to compete in this space just maintain control over the desktop, but the real power of the newest Microsoft technologies and more importantly developer tools is in the use of 3D and rich media technologies to power money making applications.
So let Apple have the hip-cool market and concentrate on building applications that have been powering the Microsoft business market since VB 3.0 struck in big almost 15 years ago - medical applications, financial applications, business work flow and processes, logistics, medium sized businesses, etc.
VB 3.0, Access, ASP.NET, Visual Studio, etc. all were successful because an average business could get some instant productivity boost by hiring some decent intermediate programmers and get them rapidly developing business applications. Now with Expression, Microsoft can now add graphic designers, HTML coders, and digital media experts into the mix and power some very cool applications that have up to this point been out of reach of most average development shops.
Here are a few examples:
http://www.microsoft.com/products/expression/en/community/spotlight/Thirteen23.mspx
http://channel9.msdn.com/Showpost.aspx?postid=109413
Microsoft, let Apple have the toy market. Give them the 1-5% of the consumer market that wants a dumb terminal with a fancy TV like interface. Concentrate on your strengths and where the money is long term. It's also where Apple will not be able to compete since they don't have the developer tools and programming language support that you get with Microsoft developer tools (.NET, IIS, SQL Server, etc.)
We've been getting exposure to Microsoft's new expression products and I can tell you first hand that its VERY COOL technology. And its cool not in the gee-whiz bang kind of way, its cool in how seamless and effective a tool it is. If you think about how difficult 3D applications used to be and how developer-centric these applications ended up being as a result, this changes the game.
So when you see Vista as a consumer OS, ignore it. Its important for Microsoft to compete in this space just maintain control over the desktop, but the real power of the newest Microsoft technologies and more importantly developer tools is in the use of 3D and rich media technologies to power money making applications.
So let Apple have the hip-cool market and concentrate on building applications that have been powering the Microsoft business market since VB 3.0 struck in big almost 15 years ago - medical applications, financial applications, business work flow and processes, logistics, medium sized businesses, etc.
VB 3.0, Access, ASP.NET, Visual Studio, etc. all were successful because an average business could get some instant productivity boost by hiring some decent intermediate programmers and get them rapidly developing business applications. Now with Expression, Microsoft can now add graphic designers, HTML coders, and digital media experts into the mix and power some very cool applications that have up to this point been out of reach of most average development shops.
Here are a few examples:
http://www.microsoft.com/products/expression/en/community/spotlight/Thirteen23.mspx
http://channel9.msdn.com/Showpost.aspx?postid=109413
Countdown to Mix 07 in Las Vegas
Microsoft is hosting its Mix 07 in the Venetian Hotel in Las Vegas from April 30-May 2.
I will be attending and probably doing some press as well. Please feel free to send me a note and I'll try to find you there.
I will be attending and probably doing some press as well. Please feel free to send me a note and I'll try to find you there.
Sunday, April 1, 2007
Test Driven Development within Visual Studio 2005
I found a little product that makes test driven development within the VS 2005 environment a breeze - check it out at www.testdriven.net.
It allows you to run any unit tests you have under the debugger, through the command line or launched in NUnit by simply right clicking on the unit test. This makes it much easier to integrate your tests into your development environment and allows for debugging driven through your NUnit tests.
It allows you to run any unit tests you have under the debugger, through the command line or launched in NUnit by simply right clicking on the unit test. This makes it much easier to integrate your tests into your development environment and allows for debugging driven through your NUnit tests.
