I have been studying how to make our web analytics better for reporting dynamic campaign data. We have WebTrends as our analytics, but and on this blog I use Google Analytics for my personal analytics solution.
From an architecture perspective, the basic way an analytics engine gets its data is two ways:
1. A javascript that sends a message to the engine with some tracking data (this is how Google Analytics works for example)
2. A standard web log file is injested (this is how WebTrends typically works)
Both tend to log the static URL plus any additional HTTP information available, e.g. the referrer, query parameters, geographic information, browser information, etc.
All of this works fine for a site built in 1999 when static URLs were the driving force behind a web site. But in an ASP.NET or any other server side generated web site, trying to store all this traffic data in a client side URL is pretty limiting. Consider the following scenarios:
1. You have an ASP.NET forms based application which gets posted back to the same URL over and over again as the form gets edited. The URL doesn't change but lots of information is changing in the background.
2. You have an ASP.NET page which is dynamically built out page fragments such as a portal page.
3. You want to associate a non-page related event to a web session along with the page data you are tracking. For example, suppose that you have a checkout process and you want to register that an order was created and link that to your web page analytics so that you can see which preceding pages link to an order.
In all these cases, the traditional web analytics approach doesn't work. If you look at how WebTrends suggests to do campaign tracking, this gives you an idea on how to fake out the software to track these events. You can store query parameters to identify variables, events, etc. and then get WebTrends to report against them. However, this isn't really a feasible solution for the above scenarios in that a) you don't necessarily want to expose data back on the URL; b) you don't want to have to constantly dynamically re-write every URL in your application; c) if you have a complicated set of objects your tracking then writing out a serialized objected isn't feasible either.
To me this is a real opportunity for Microsoft and the .NET framework in the future. It would be a snap for them to implement it - they already most of the pieces in place and could offer a very open, developer friendly approach to web analytics. This would allow for some very funky tracking in the background and provide the developer the flexibility to log a whole bunch of information as needed. The trick in building the solution is performance - the reason why WebTrends works on web logs is because its fast and means no performance hit on your web traffic. But at least in our case, we would sacrifice a minor performance hit to get more powerful data - we get relatively small amounts of traffic but understanding the patterns in high amounts of detail is very powerful in improve conversion, analyze campaign effectiveness, etc.
Has anyone come across a product that approaches web analytics this way?
Tuesday, April 17, 2007
Sunday, April 15, 2007
Are You A Guerrilla Project Manager?
Most Project Managers if they have been groomed through the PMP process are like generals in the Army or foreman on a construction site. Their approach tends to favour:
1. Command and control
2. Constant monitoring
3. Process over results
4. Risk aversion and mitigation
5. Dependence on Tools (Project Plans, Status Reports, Risk Matrices, etc.)
6. Waterfall methodologies
Tom Peters in his latest book, Re-Imagine! Business Excellence in a Disruptive Age,
uses the example of terrorist cells as the model for the new virtual organization.
I like the term "Guerrilla" as it has less negative connotations. If you want to feel patriotic, use the analogy of being in the Special Forces.
Frankly, I think this is going to be a tough sell to most traditional Project Managers as their entire training is based on high amounts of process, change management and tooling to avoid adaptability, gut instinct and autonomy.
So if you are a project manager, ask yourself: can you manage like a terrorist?
1. Could you survive without your trusted Microsoft Project Plan? Can you create a schedule using only a white board, a notepad, or a sticki-note?
2. Could you build a multi-million $ project plan in a day and stick to it? Or would you snivel back to your boss saying, "Uhm I don't have the requirements, deliverables, scope, resource, allocation, etc."
3. Have you built your team so that if you disappeared for a week it could survive?
4. Can you really change on a dime? Can you do it without filling out process forms, change management requests, etc.?
5. Is the goal for your project simple? If you had to explain your project in 10-15 minutes, could you do it?
6. Do you have vertically specialized knowledge or are you just overhead? Could you take over one of your team mates workload? If you're on a software project, do you know how to code? Do you understand the architecture of the project? Could you do a build? If you're running a health care project can you talk HL7? Or is your expertise simply pumping out status reports on a weekly basis?
7. Are you a passionate defender of your project and project team? Are you politically strong enough to jump into the fray for your team? Are you afraid of hierarchy? Can you take out an opponent in your way?
8. Could you make a decision by gut feel in 30 seconds? Can you plan in your head? Can you make decisions with patchy information?
9. Are you a PM that likes to keep everything in control? Do you play it safe? Do you follow the traditional PMP methodology of trying to avoid risks at all costs through mitigation, documentation and forecasting? If you were thrown into a crisis, could you manage?
10. Can you jump into a project that's already in crisis and fix it? Can you do it in days, not months?
11. Can you manage fall-out? Can you live with an imperfect solution in order to make time-to-market targets (I once had a project where we launched an e-commerce application that for the first three months took a room full of temps to do data-entry in the background because we couldn't automate the whole solution)?
If you're designing a building, a space shuttle or a health care application, this isn't the type of management style you want from your project manager. The risks are too high - if you take the risk and someone dies as a result, then you need a formal approach and PMBOK will work quite nicely.
But the vast majority of applications in business are far more slippery, agile and time-to-market sensitive. If as a PM you are not entrepreneurial, agile and a gut feel leader then you're going to be toast if any small crisis hits your project.
When I hire project managers, I tell them the following: I expect you to be a competent project manager with good process. That's a given, but at the end of the day, that's not your real value proposition.
Your real value proposition is this: at some point during your project, you will hit a crisis which requires strength of character, real leadership and the ability to make a judgement call. You're going to have to stand in the middle of that crisis and keep your project together, lead your team and be able to make judgements with little to no support.
Example of what I mean: I was running a project where I was launching a software application. I was working for a program manager who was quite process oriented and very sensitive to "adoption", e.g. how are we going to manage our customers. We were supposed to launch at 3pm, and I was still getting performance test reports at 2pm that said were weren't going to make the target performance requirements. We ran one last test, and at 2:45 I got the last results. We were good to go!
So I had a decision to make - we could rush the launch (which was quite time sensitive) and make the time line or slow down and scrub the launch giving us some more time to prepare documentation, change request orders, deployment guides and communication materials.
My Program Manager freaked and said, "But where are your PowerPoint slides?! How are we going to communicate this to our stakeholders at our 3pm con call?!". Instant panic.
I walked into the CIO's office who had to make the ultimate decision to launch or not and this Program Manager actually started apologizing for not being prepared and started trying to paraphrase what I had just told her to the CIO! Knowing that this wasn't instilling confidence, I quickly interupted her and simply said, "Look, here's the deal. We just got off the floor and we're ready to go. We can either scrub this launch or you can go ahead. I can tell you that from my perspective we're ready to launch, but if you would like to review it further we can delay the launch date."
The CIO took one look at me, trusted me and said, "Ok let's go."
But this just opened up another "adoption" problem - we had a conference call scheduled with all the customers and typically agenda items and materials were send 24 hours before the meeting. I was typically not involved in these calls as I was too far down the hierarchy (command and control!) to be talking directly to customers. The program manager was anxious, "How am I supposed to communicate this to our stakeholders?"
I suggested a simple solution - I got on the con call myself and simply talked to all the stakeholders. I got on the call, told the group what the status of the project was and without hesitation told them that we were ready to launch and that I had 100% confidence that we were ready to go.
At the end of day, we launched on that day. It was a bit messy on the back-end and we had lots of people in IT nervously monitoring the system for any hiccups, but we made the date and launched successfully. That's what happens in the real world...
Could you do that? Do you have the guts as a PM to walk into a meeting without materials, communicate to your stakeholders about a launch that you just verified could work 15 minutes ago, and then with confidence and poise communicate that to your stakeholders in a way that gives them confidence that you know what you're talking about?
That's what a Guerrilla PM does...
1. Command and control
2. Constant monitoring
3. Process over results
4. Risk aversion and mitigation
5. Dependence on Tools (Project Plans, Status Reports, Risk Matrices, etc.)
6. Waterfall methodologies
Tom Peters in his latest book, Re-Imagine! Business Excellence in a Disruptive Age,
uses the example of terrorist cells as the model for the new virtual organization.I like the term "Guerrilla" as it has less negative connotations. If you want to feel patriotic, use the analogy of being in the Special Forces.
Frankly, I think this is going to be a tough sell to most traditional Project Managers as their entire training is based on high amounts of process, change management and tooling to avoid adaptability, gut instinct and autonomy.
So if you are a project manager, ask yourself: can you manage like a terrorist?
1. Could you survive without your trusted Microsoft Project Plan? Can you create a schedule using only a white board, a notepad, or a sticki-note?
2. Could you build a multi-million $ project plan in a day and stick to it? Or would you snivel back to your boss saying, "Uhm I don't have the requirements, deliverables, scope, resource, allocation, etc."
3. Have you built your team so that if you disappeared for a week it could survive?
4. Can you really change on a dime? Can you do it without filling out process forms, change management requests, etc.?
5. Is the goal for your project simple? If you had to explain your project in 10-15 minutes, could you do it?
6. Do you have vertically specialized knowledge or are you just overhead? Could you take over one of your team mates workload? If you're on a software project, do you know how to code? Do you understand the architecture of the project? Could you do a build? If you're running a health care project can you talk HL7? Or is your expertise simply pumping out status reports on a weekly basis?
7. Are you a passionate defender of your project and project team? Are you politically strong enough to jump into the fray for your team? Are you afraid of hierarchy? Can you take out an opponent in your way?
8. Could you make a decision by gut feel in 30 seconds? Can you plan in your head? Can you make decisions with patchy information?
9. Are you a PM that likes to keep everything in control? Do you play it safe? Do you follow the traditional PMP methodology of trying to avoid risks at all costs through mitigation, documentation and forecasting? If you were thrown into a crisis, could you manage?
10. Can you jump into a project that's already in crisis and fix it? Can you do it in days, not months?
11. Can you manage fall-out? Can you live with an imperfect solution in order to make time-to-market targets (I once had a project where we launched an e-commerce application that for the first three months took a room full of temps to do data-entry in the background because we couldn't automate the whole solution)?
If you're designing a building, a space shuttle or a health care application, this isn't the type of management style you want from your project manager. The risks are too high - if you take the risk and someone dies as a result, then you need a formal approach and PMBOK will work quite nicely.
But the vast majority of applications in business are far more slippery, agile and time-to-market sensitive. If as a PM you are not entrepreneurial, agile and a gut feel leader then you're going to be toast if any small crisis hits your project.
When I hire project managers, I tell them the following: I expect you to be a competent project manager with good process. That's a given, but at the end of the day, that's not your real value proposition.
Your real value proposition is this: at some point during your project, you will hit a crisis which requires strength of character, real leadership and the ability to make a judgement call. You're going to have to stand in the middle of that crisis and keep your project together, lead your team and be able to make judgements with little to no support.
Example of what I mean: I was running a project where I was launching a software application. I was working for a program manager who was quite process oriented and very sensitive to "adoption", e.g. how are we going to manage our customers. We were supposed to launch at 3pm, and I was still getting performance test reports at 2pm that said were weren't going to make the target performance requirements. We ran one last test, and at 2:45 I got the last results. We were good to go!
So I had a decision to make - we could rush the launch (which was quite time sensitive) and make the time line or slow down and scrub the launch giving us some more time to prepare documentation, change request orders, deployment guides and communication materials.
My Program Manager freaked and said, "But where are your PowerPoint slides?! How are we going to communicate this to our stakeholders at our 3pm con call?!". Instant panic.
I walked into the CIO's office who had to make the ultimate decision to launch or not and this Program Manager actually started apologizing for not being prepared and started trying to paraphrase what I had just told her to the CIO! Knowing that this wasn't instilling confidence, I quickly interupted her and simply said, "Look, here's the deal. We just got off the floor and we're ready to go. We can either scrub this launch or you can go ahead. I can tell you that from my perspective we're ready to launch, but if you would like to review it further we can delay the launch date."
The CIO took one look at me, trusted me and said, "Ok let's go."
But this just opened up another "adoption" problem - we had a conference call scheduled with all the customers and typically agenda items and materials were send 24 hours before the meeting. I was typically not involved in these calls as I was too far down the hierarchy (command and control!) to be talking directly to customers. The program manager was anxious, "How am I supposed to communicate this to our stakeholders?"
I suggested a simple solution - I got on the con call myself and simply talked to all the stakeholders. I got on the call, told the group what the status of the project was and without hesitation told them that we were ready to launch and that I had 100% confidence that we were ready to go.
At the end of day, we launched on that day. It was a bit messy on the back-end and we had lots of people in IT nervously monitoring the system for any hiccups, but we made the date and launched successfully. That's what happens in the real world...
Could you do that? Do you have the guts as a PM to walk into a meeting without materials, communicate to your stakeholders about a launch that you just verified could work 15 minutes ago, and then with confidence and poise communicate that to your stakeholders in a way that gives them confidence that you know what you're talking about?
That's what a Guerrilla PM does...
Saturday, April 14, 2007
The Biggest Security Threat - Paper and the Front Door
In Information Technology, most of us digital professionals tend to think about security. We spend out energy on making sure that hackers cannot penetrate from the outside, we look at virus/trojan protection and we clamp down on the firewall to make sure data gets in and out. In addition, if we're really effective, we might start to look at internal security such as who has access to data, vendor management, working from home arrangements, etc.
Because we spend most of the time working with technology, many IT folks will forget the most insecure form of data - Paper!
If you want to see what I'm talking about, go in on a Sunday afternoon and walk around the office and snoop on people's desks. See what paper is sitting around their office out in the open. You may be suprised at what's sitting there. There isn't much point in protecting your database if someone in accounting has printed off the data and left it on their desk.
Another basic security problem that no one in IT tends to worry about - the Front Door!
Most companies have a reception area, and if they are really well secured the reception area will act as a gatekeeper to the other floors. However, most receptionists tend to work from 8:30-4:30. In addition, they tend to take lunch breaks and may not have someone to cover them.
So if you want to test your front door, simply go into the office at 8:30, 4:30 or noon and see if you can walk right in. If you can, then you can probably also walk right out with a printer, a laptop, etc. without anyone questioning it. Most workers if they see someone in the office will assume they've been authorized through reception. If you can crack that reception authorization mechanism, you're likely going to get very little resistance from anyone.
So try it out...and if its a problem, shut it down immediately. Laptop thieves for example are in and out in about 5-10 minutes - if they can get through the front door at any time then you're asking for trouble.
Because we spend most of the time working with technology, many IT folks will forget the most insecure form of data - Paper!
If you want to see what I'm talking about, go in on a Sunday afternoon and walk around the office and snoop on people's desks. See what paper is sitting around their office out in the open. You may be suprised at what's sitting there. There isn't much point in protecting your database if someone in accounting has printed off the data and left it on their desk.
Another basic security problem that no one in IT tends to worry about - the Front Door!
Most companies have a reception area, and if they are really well secured the reception area will act as a gatekeeper to the other floors. However, most receptionists tend to work from 8:30-4:30. In addition, they tend to take lunch breaks and may not have someone to cover them.
So if you want to test your front door, simply go into the office at 8:30, 4:30 or noon and see if you can walk right in. If you can, then you can probably also walk right out with a printer, a laptop, etc. without anyone questioning it. Most workers if they see someone in the office will assume they've been authorized through reception. If you can crack that reception authorization mechanism, you're likely going to get very little resistance from anyone.
So try it out...and if its a problem, shut it down immediately. Laptop thieves for example are in and out in about 5-10 minutes - if they can get through the front door at any time then you're asking for trouble.
Friday, April 13, 2007
Fun with Naming Conventions
Back in the days of Visual Basic 6, they way you did error checking was something like this:
On Error Goto ErrHandler:
N = 1 / 0 ' cause an error
'
' more code
'
Exit Sub
ErrHandler:
' error handling code
Resume Next
End Sub
When I worked on one project, I actually found this code running in production:
On Error Goto Hell:
'
' code was here
'
Exit Sub
Hell:
' error handling code
Resume Next
End Sub
So remember, don't drink and code...
On Error Goto ErrHandler:
N = 1 / 0 ' cause an error
'
' more code
'
Exit Sub
ErrHandler:
' error handling code
Resume Next
End Sub
When I worked on one project, I actually found this code running in production:
On Error Goto Hell:
'
' code was here
'
Exit Sub
Hell:
' error handling code
Resume Next
End Sub
So remember, don't drink and code...
E-File Follow-up
My tax refund arrived in the mail today...so I guess that means that the e-file system is fully operational.
Of course now that its working, no one in the media is asking any questions about what actually happened.
See my original article for more information on our government's IT practices...
Of course now that its working, no one in the media is asking any questions about what actually happened.
See my original article for more information on our government's IT practices...
Wednesday, April 11, 2007
Best Tool for Task and Bug Tracking is JIRA!
I've been using a product from Atlassian called JIRA for the past couple years to track tasks, bugs, and requests.
I highly highly recommend it - its an amazing product and the team behind it provides great support and rapid response. They're also highly supportive of charity and open-source organizations - if you ask nicely, they'll give you an enterprise license to their products for free!
What makes JIRA great?
I highly highly recommend it - its an amazing product and the team behind it provides great support and rapid response. They're also highly supportive of charity and open-source organizations - if you ask nicely, they'll give you an enterprise license to their products for free!
What makes JIRA great?
- Completely web based
- Highly customizable, including allowing for custom fields for each task type
- Configuration notifications via email on changes to tasks. We currently use this to notify the assigned user and the reporter of a task when changes are made to their task.
- Stupid simple to use - it takes about 30 seconds to create a task and about a minute to train an end user on how to use it.
- Everything is permission based, so you can allow people to view status of issues but not allow them to change them for example.
- Java based, runs in Tomcat on just about any kind of machine.
- The database is flexible. We use SQL Server, but it will support MySQL, Oracle, Postgres, etc.
- You can use it for everything throughout the development life cycle. We start using JIRA right at the requirements gathering stage to collect user stories and then translate them into functional tasks during development. Then as we transition to QA, we mark each task as resolved and QA tests and closes them. QA can then add bugs as tasks and re-assign them to the developers.
- From a project management perspective, the tool provides a centralized view of the entire team - its very easy to see all the tasks assigned to a single person, group, component, project, etc. and then re-assign with a few clicks.
- Its simple enough that non-technical users can benefit from it. We're currently using it to track marketing tasks, HR requests, IT helpdesk tasks, etc. as well as software changes. It's that flexible.
I've been using JIRA on multiple software projects for a few years now and would highly recommend it as a solution for managing tasks, bugs, features, etc. in a complex environment.
Even if you're a commercial company, the price for JIRA is totally worth it - licenses are $1200-$4,800US depending on the version you need.
Vintage Material on YouTube
Along with my love of technology, I also love early American music, especially older Jazz and Blues circa 1920's - 1940's.
So I did some searches on YouTube, and I found that along with the usual America's Funniest Stupid Video type clips, there are quite a few videos of vintage music.
See the following URLs for some good examples, but if you search for any of the Jazz greats such as Ella Fitzgerald, Duke Ellington, Django Reinhart, etc. you'll find some great material. There are lots of old blues artists there as well.
Ella...
Django...
Leadbelly...
So I did some searches on YouTube, and I found that along with the usual America's Funniest Stupid Video type clips, there are quite a few videos of vintage music.
See the following URLs for some good examples, but if you search for any of the Jazz greats such as Ella Fitzgerald, Duke Ellington, Django Reinhart, etc. you'll find some great material. There are lots of old blues artists there as well.
Ella...
Django...
Leadbelly...
