I normally don't use delegates, since I don't do a lot of events programming where they are commonly used. But I was building a class library the other day and found a good use for them.
Here is the scenario: I have a series of methods that allow for merging of records in our main CRM. In order to connect to the CRM, there is a series of setup and tear down calls that have to be made such as connecting, logging in, etc.
So I found myself creating a dozen methods that all looked something like this:
public void mergeAddress(String masterID, String mergeID)
{
try
{
bunch of set up code
meat of the method
bunch of tear down code
}
catch (Exception e)
{
throw new MergeException("Unexpected error in method mergeAddress", e);
}
}
I found myself copying and pasting this chunk of code over and over again, which is always a good sign that some refactoring is required.
My solution was to use a delegate to represent the meat of the method in the middle. Therefore, instead of having a bunch of methods that repeat the same code over and over again, I can now rewrite the method to look something like this:
protected delegate void mergeDelegate(String MasterID, String MergeID);
public void mergeByDelegate(del method, String MasterID, String MergeID)
{
try
{
bunch of set up code
del.invoke();
bunch of tear down code
}
catch (Exception e)
{
throw new MergeException("Unexpected error in method mergeAddress", e);
}
}
Now the method becomes like a prototype - I can keep the set up and tear down in one place and simply sub in the method call in the middle.
(FYI, You could do the same thing by splitting the method into three chunks, moving them into a class and sub-classing the method in the middle, but I think using a simple delegate is easier.)
To implement the prototype now looks something like this:
public void mergeAddress(String MasterID, String MergeID)
{
mergeDelegate del = this.mergeAddress();
mergeByDelegate(del, MasterID, String MergeID);
}
public void mergeAddress(String MasterID, String MergeID)
{
meat of the method
}
All the set up and tear down code is now centralized into one method. The part that changes in the middle can now vary in a variety of ways.
Hope this helps everyone with another way of using delegates...I've always struggled to find a use for them.
Wednesday, July 18, 2007
Tuesday, July 10, 2007
Low Tech Approaches to Powerpoint Presentations
Take a look at these videos as a low tech presentation format. Think about how you can use these techniques for selling ideas, explaining technical concepts, brainstorming, etc. For more ideas on low-tech requirements gathering, see my previous post as well.
Saturday, July 7, 2007
LiveEarth not in the top 10?!
Whoever organized LiveEarth really missed the blogosphere. A few points of evidence:
1. There are only 2049 blog posts on LiveEarth?
2. LiveEarth as a search term isn't hitting the Technorati top 10.
3. The LiveEarth facebook group only has 678 members.
Whoever is responsible for creating Buzz on the blogsophere for LiveEarth is doing a poor job - Paris Hilton and the IPhone are doing much better than LiveEarth, a supposedly global change event.
I'm going back to enjoying the music on the old fashion technology called the Television! :)
1. There are only 2049 blog posts on LiveEarth?
2. LiveEarth as a search term isn't hitting the Technorati top 10.
3. The LiveEarth facebook group only has 678 members.
Whoever is responsible for creating Buzz on the blogsophere for LiveEarth is doing a poor job - Paris Hilton and the IPhone are doing much better than LiveEarth, a supposedly global change event.
I'm going back to enjoying the music on the old fashion technology called the Television! :)
Problems with Live Earth Broadcast
Compared to the Live8 broadcast that AOL did, the LiveEarth broadcast is a disappointment. The resolution of the video is significantly poorer and yet I'm finding that the stream is having a lot of problems handling the load. There are a lot of blackouts, frequent rebuffering of content and there have already been complaints that the site only works in Internet Explorer.
This could have been a fantastic showcase for Silverlight - with the quick download of the plugin Microsoft could have showcased its new technology including some fantastic video streaming.
But instead we see choppy video, disconnected audio and very slow response rate on the interface.
What a missed opportunity!
This could have been a fantastic showcase for Silverlight - with the quick download of the plugin Microsoft could have showcased its new technology including some fantastic video streaming.
But instead we see choppy video, disconnected audio and very slow response rate on the interface.
What a missed opportunity!
Friday, July 6, 2007
Creating a Single Sign-On Solution for .NET Part 2
For the past couple days I've been building my single sign-on solution based on my previously posted design.
I built some simple web services that allow for login against a membership profile database using the supplied ASP.NET membership architecture. By registering the membership and logging in, we can provide a token that can be used to represent the logged in user. This token can then be distributed on the URL and passed to partner sites. The partner site can the validate the token and retrieve the membership information through a web service method.
Here is the code:
using System;
using System.Data;
using System.Web;
using System.Web.Caching;
using System.Web.Security;
using System.Collections;
using System.Web.Services;
using System.Web.Services.Protocols;
using System.ComponentModel;
using System.Web.Profile;
using System.Configuration;
using System.Security.Principal;
namespace SingleSignOn.WebService
{
[WebService(Namespace = "http://localhost/")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
[ToolboxItem(false)]
public class MemberWebService : System.Web.Services.WebService
{
///
/// Logs in a user and provides back a token. Tokens can then be used to regrab the profile data.
///
///
///
///
[WebMethod]
public string registerMember(string userName, string password)
{
if (Membership.ValidateUser(userName, password))
{
Guid token = Guid.NewGuid();
HttpContext.Current.Cache.Insert(token.ToString(), userName, null, Cache.NoAbsoluteExpiration, TimeSpan.FromMinutes(20));
return token.ToString();
}
else
return null;
}
///
/// Validates a token and provides back a membership user in return. If the member isn't found then null is returned.
///
/// Token to validate.
///MembershipUser object for the mapped user. If token isn't found then null is returned.
[WebMethod]
public MembershipUser validateMember(string token)
{
string userName = (string) HttpContext.Current.Cache[token];
MembershipUser member = Membership.GetUser(userName);
return member;
}
///
/// Returns the current profile object.
///
/// Token to map to a currently logged in user.
///Array of name value pairs.
[WebMethod]
public object[][] getProfile(string token)
{
string userName = (string)HttpContext.Current.Cache[token];
Hashtable profileHash = new Hashtable();
ProfileBase profile = WebProfile.Create(userName, true);
foreach (SettingsProperty property in WebProfile.Properties)
{
profileHash.Add(property.Name, profile[property.Name]);
}
return toJaggedArray(profileHash);
}
///
/// This doesn't work. I have not yet found a way to update profiles properly in ASP.NET.
///
///
///
///
[WebMethod]
public void updateProfileProperty(string token, string propertyName, string value)
{
try
{
}
catch (Exception e)
{
System.Diagnostics.Debug.Print(e.Message);
}
}
///
/// Invalidate the cache entry containing the token.
///
/// Token representing user
[WebMethod]
public void logout(string token)
{
HttpContext.Current.Cache.Remove(token);
}
private object[][] toJaggedArray(Hashtable ht)
{
object[][] oo = new object[ht.Count][];
int i = 0;
foreach (object key in ht.Keys)
{
oo[i] = new object[] { key, ht[key] };
i++;
}
return oo;
}
private Hashtable toHashtable(object[][] oo)
{
Hashtable ht = new Hashtable(oo.Length);
foreach (object[] pair in oo)
{
object key = pair[0];
object value = pair[1];
ht[key] = value;
}
return ht;
}
}
}
One small technical note - because web services don't support Hashtables, you need to convert these to a more basic array in order to return them. See the getProfile method as an example.
There is one fundamental problem - trying to manage other people's profiles. It seems that the ASP.NET profile management is based on the assumption that you are reading and writing a current logged in user's profile context. However, in this case, we want to be able to proxy the profile data out to the world which means being able to read and write any user's profile data. There doesn't seem to be an easy way to do this in the current profile architecture. You can get a Profile object from the HttpContext, but this is the current logged in user. You can use the ProfileManager class to grab profile objects but it doesn't provide you access to individual properties of the profile. You can load up a profile object using the GetProfile method, but then there is no way to save the profile.
So it seems that if I'm going to use a distributed global profile management system, I'm going to have to build my own. This isn't really a huge problem as the built in profile management system is fairly weak (see my previous post on general problems with the ASP.NET profile) and building a basic profile management system to store name value pairs or serialized objects isn't that difficult.
But the good news is the basic idea of proxying a token across a domain to store a logged in user seems to work in principle. The next challenge is figuring out the best way to secure it so that only valid partner sites can use the service.
I built some simple web services that allow for login against a membership profile database using the supplied ASP.NET membership architecture. By registering the membership and logging in, we can provide a token that can be used to represent the logged in user. This token can then be distributed on the URL and passed to partner sites. The partner site can the validate the token and retrieve the membership information through a web service method.
Here is the code:
using System;
using System.Data;
using System.Web;
using System.Web.Caching;
using System.Web.Security;
using System.Collections;
using System.Web.Services;
using System.Web.Services.Protocols;
using System.ComponentModel;
using System.Web.Profile;
using System.Configuration;
using System.Security.Principal;
namespace SingleSignOn.WebService
{
[WebService(Namespace = "http://localhost/")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
[ToolboxItem(false)]
public class MemberWebService : System.Web.Services.WebService
{
///
/// Logs in a user and provides back a token. Tokens can then be used to regrab the profile data.
///
///
///
///
[WebMethod]
public string registerMember(string userName, string password)
{
if (Membership.ValidateUser(userName, password))
{
Guid token = Guid.NewGuid();
HttpContext.Current.Cache.Insert(token.ToString(), userName, null, Cache.NoAbsoluteExpiration, TimeSpan.FromMinutes(20));
return token.ToString();
}
else
return null;
}
///
/// Validates a token and provides back a membership user in return. If the member isn't found then null is returned.
///
/// Token to validate.
///
[WebMethod]
public MembershipUser validateMember(string token)
{
string userName = (string) HttpContext.Current.Cache[token];
MembershipUser member = Membership.GetUser(userName);
return member;
}
///
/// Returns the current profile object.
///
/// Token to map to a currently logged in user.
///
[WebMethod]
public object[][] getProfile(string token)
{
string userName = (string)HttpContext.Current.Cache[token];
Hashtable profileHash = new Hashtable();
ProfileBase profile = WebProfile.Create(userName, true);
foreach (SettingsProperty property in WebProfile.Properties)
{
profileHash.Add(property.Name, profile[property.Name]);
}
return toJaggedArray(profileHash);
}
///
/// This doesn't work. I have not yet found a way to update profiles properly in ASP.NET.
///
///
///
///
[WebMethod]
public void updateProfileProperty(string token, string propertyName, string value)
{
try
{
}
catch (Exception e)
{
System.Diagnostics.Debug.Print(e.Message);
}
}
///
/// Invalidate the cache entry containing the token.
///
/// Token representing user
[WebMethod]
public void logout(string token)
{
HttpContext.Current.Cache.Remove(token);
}
private object[][] toJaggedArray(Hashtable ht)
{
object[][] oo = new object[ht.Count][];
int i = 0;
foreach (object key in ht.Keys)
{
oo[i] = new object[] { key, ht[key] };
i++;
}
return oo;
}
private Hashtable toHashtable(object[][] oo)
{
Hashtable ht = new Hashtable(oo.Length);
foreach (object[] pair in oo)
{
object key = pair[0];
object value = pair[1];
ht[key] = value;
}
return ht;
}
}
}
One small technical note - because web services don't support Hashtables, you need to convert these to a more basic array in order to return them. See the getProfile method as an example.
There is one fundamental problem - trying to manage other people's profiles. It seems that the ASP.NET profile management is based on the assumption that you are reading and writing a current logged in user's profile context. However, in this case, we want to be able to proxy the profile data out to the world which means being able to read and write any user's profile data. There doesn't seem to be an easy way to do this in the current profile architecture. You can get a Profile object from the HttpContext, but this is the current logged in user. You can use the ProfileManager class to grab profile objects but it doesn't provide you access to individual properties of the profile. You can load up a profile object using the GetProfile method, but then there is no way to save the profile.
So it seems that if I'm going to use a distributed global profile management system, I'm going to have to build my own. This isn't really a huge problem as the built in profile management system is fairly weak (see my previous post on general problems with the ASP.NET profile) and building a basic profile management system to store name value pairs or serialized objects isn't that difficult.
But the good news is the basic idea of proxying a token across a domain to store a logged in user seems to work in principle. The next challenge is figuring out the best way to secure it so that only valid partner sites can use the service.
Wednesday, July 4, 2007
Creating a Single Sign-On Solution for .NET
We have a basic single sign-on requirement. We need to be able to allow users to login and pass that login to other distributed systems.
Here are the basic requirements:
1. We want to control the master user profile so that we can have access to the data and integrate it into our offline CRM systems.
2. We have a number of partners who provide hosted applications for us that require authentication. We need to provide a mechanism to have a single sign-on for users that can then be passed to to these partners.
3. The partners may have their own profile requirements that are independent of our needs. For example, let's assume we have a partner that provides community forums and saves the user's favourite forums in their profile. Only the partner cares about this information and centralizing it doesn't add any real value.
4. The solution has to work in a distributed, multi-platform environment. The only access to and from the partners and us is through port 80 using either REST or Web Services calls.
5. The domain may not be the same and therefore you cannot share cookies (see http://www.codeproject.com/aspnet/aspnetsinglesignon.asp and www.411asp.net/func/goto?id=5998410 for a solution based on the assumption that you can simply poke the cookie created by ASP.NET)
6. There needs to be a way to transfer state from one server to another in cases where the user is logged in already. If user logs in Site A and then goes to Site B, they should appear to be still logged in.
So here is the basic design:
1. User logs in by calling a login web service method (see this example for a web services method implementation of login). User is now authenticated on the main site.
2. User links to partner site which passes a token representation of the state (we could use the SessionID itself in .NET as a token).
3. The partner site receives the token and calls a ValidateToken web service method. This validates the passed in token and provides back the membership information for the current user.
4. If the partner site collects user information, the site can call the Update web service method to update the master profile.
5. If the user wants to link back from the partner site to the main site, the partner site provides a link back to the main site with the token embedded in the query string.
6. The main site receives the token and calls a ValidateToken web service method. This validates the passed in token and provides back the membership information for the current user.
This basic architecture should work with a couple basic caveats:
1. State needs to be managed centrally with the usual challenges of managing session state in a load balanced environment.
2. There needs to be some security around the token validation service to prevent malicious sites from getting access to the service. There are lots of solutions for this including putting the web services under VPN, including a site authentication mechanism in the web service method, filtering by IP, etc.
3. Each link needs to be dynamically re-written to include the token. If there are lots of existing cross-over points then this could be a challenge.
4. The parsing of the token needs to happen on every page request. In .NET this could be done using an HTTP Module as a central parsing code. Alternatively, you could have a redirect page that all pages link to first with the original target url embedded in a query parameter (this is how typical login pages work for example).
5. There needs to be some mechanism for keeping the state alive so that it doesn't time out. This could again done using an HTTP Module that on every page request does a web service request to validate token to simply keep the state alive.
6. The solution assumes that there is a single backing master profile that distributes profile information. A "peer to peer" type model could be developed but it would be more complicated.
From a look and feel perspective, the solution doesn't require a single login or registration experience but it probably would make sense. Keep the login and registration forms on the main web site and have them redirect back to the partner site if the request is coming from the partner.
Here are the basic requirements:
1. We want to control the master user profile so that we can have access to the data and integrate it into our offline CRM systems.
2. We have a number of partners who provide hosted applications for us that require authentication. We need to provide a mechanism to have a single sign-on for users that can then be passed to to these partners.
3. The partners may have their own profile requirements that are independent of our needs. For example, let's assume we have a partner that provides community forums and saves the user's favourite forums in their profile. Only the partner cares about this information and centralizing it doesn't add any real value.
4. The solution has to work in a distributed, multi-platform environment. The only access to and from the partners and us is through port 80 using either REST or Web Services calls.
5. The domain may not be the same and therefore you cannot share cookies (see http://www.codeproject.com/aspnet/aspnetsinglesignon.asp and www.411asp.net/func/goto?id=5998410 for a solution based on the assumption that you can simply poke the cookie created by ASP.NET)
6. There needs to be a way to transfer state from one server to another in cases where the user is logged in already. If user logs in Site A and then goes to Site B, they should appear to be still logged in.
So here is the basic design:
1. User logs in by calling a login web service method (see this example for a web services method implementation of login). User is now authenticated on the main site.
2. User links to partner site which passes a token representation of the state (we could use the SessionID itself in .NET as a token).
3. The partner site receives the token and calls a ValidateToken web service method. This validates the passed in token and provides back the membership information for the current user.
4. If the partner site collects user information, the site can call the Update web service method to update the master profile.
5. If the user wants to link back from the partner site to the main site, the partner site provides a link back to the main site with the token embedded in the query string.
6. The main site receives the token and calls a ValidateToken web service method. This validates the passed in token and provides back the membership information for the current user.
This basic architecture should work with a couple basic caveats:
1. State needs to be managed centrally with the usual challenges of managing session state in a load balanced environment.
2. There needs to be some security around the token validation service to prevent malicious sites from getting access to the service. There are lots of solutions for this including putting the web services under VPN, including a site authentication mechanism in the web service method, filtering by IP, etc.
3. Each link needs to be dynamically re-written to include the token. If there are lots of existing cross-over points then this could be a challenge.
4. The parsing of the token needs to happen on every page request. In .NET this could be done using an HTTP Module as a central parsing code. Alternatively, you could have a redirect page that all pages link to first with the original target url embedded in a query parameter (this is how typical login pages work for example).
5. There needs to be some mechanism for keeping the state alive so that it doesn't time out. This could again done using an HTTP Module that on every page request does a web service request to validate token to simply keep the state alive.
6. The solution assumes that there is a single backing master profile that distributes profile information. A "peer to peer" type model could be developed but it would be more complicated.
From a look and feel perspective, the solution doesn't require a single login or registration experience but it probably would make sense. Keep the login and registration forms on the main web site and have them redirect back to the partner site if the request is coming from the partner.
Implementing an Agile Change Control Process
What is Change Control?
Change Control traditionally is a project management process for managing change. The typical method for managing change in a classic PMP project is a Change Control form that is filled out containing the scope of the change, impact to the project, etc. This is then approved and incorporated into the master project plan, charter, etc.
In addition, Change Control processes are used by operations departments to manage the deployment and promotion of code from one environment to another. The Change Control Form provides the documentation for an operations team to deploy, operate and monitor the change.
Change Control and Agile
This traditional view of Change Control is at odds with Agile methodologies for the following reasons:
1. Change is viewed as a non-typical event, e.g. the project was running smoothly until someone threw a change into the project. Agile projects by definition are based on the assumption that changes are typical events and aren't special.
2. Change requests are simply Waterfall designs in miniature. Requirements for the change must be documented up front and analyzed. Stakeholders must approve of changes before they are made with the assumption that requirements, scope, risk, and design can be formally documented ahead of making a code change.
3. Traditional Change Control processes are document heavy. The process is designed as much to capture and document change as it is to make effective decisions. Change control forms are kept around forever with dates and identifiers so that someone could find out who approved a change. In many cases, these are also printed out, resulting in very large binders of paper with signatures for every change.
4. In an agile project, one of the objectives is to flatten the team and reduce hierarchies so that people can more effectively communicate. However, traditional change control processes require specialized knowledge, special powers of approval and a hierarchical command and control structure. Project Managers are usually the writers of change control forms, but then they must get approval from "Stakeholders" who will bless the change.
Here is a good example of a traditional Change Control process - have a look at Bell South's Change Control Process Guide. At 120 pages, it is a great example of how bloated a large formal change control process can get when Project Managers are working in a bureaucratic environment. Just managing the process documentation alone takes a massive investment - just look at the 20 pages of version numbers of this document and you'll see that someone's full-time job must be simply keeping the guide up to date!
Change Control isn't a bad thing - the original objectives behind formal Change Control are actually in line with the objectives of agile methodologies:
1. Define change and anticipate change early instead of leaving it to the end.
2. Providing a process for incorporating change during development of a project.
3. Communicate scope, risks and impacts of a change.
4. Provide the appropriate review and approval process to authorize a change.
It isn't that change control is bad - it's that the evolution of formal project methodologies has led to very bureaucratic, documentation heavy processes. In addition, keep in mind that most of these processes were invented before the incorporation of digital collaboration tools such as email, Wikis, sharepoint, JIRA, Word, etc. and so the typical change control process has artifacts from a non-digital era.
Implementing an Agile Change Control Process
Change Control can be used effectively in an Agile environment if it is sufficiently light weight, non-bureaucratic and collaborative. Here are my recommendations based on what I have seen work effectively on Agile and non-Agile projects:
1. Use an agile methodology such as XP, Crystal, Scrum, etc. that has change management built into its fundamental process. In Scrum, for example, all work is effectively a change and is logged as a "change request" in a work backlog. In XP the changes are logged as User Stories. Iterations allow for changes to happen on a regular basis - they're not special change events.
2. Use a generic task tracking system such as JIRA to log any change. In JIRA, you don't need a separate "bug tracking" system that is different than your change tracking system. In addition, you can take a change all the way from requirements to deployment without re-entering information. You could do the same thing with low tech tools such as Excel or Word simply by having a work log that is updated regularly.
3. If you have specific legal or regulatory reasons to track changes then try to understand the specific minimum requirements. I've worked in a government environment for example where all changes needed to be documented so that they could be audited. But the level of detail that was incorporated into each Change Control form became ridiculous - an auditor probably doesn't really need to see a change in an individual SQL script, a deployment checklist that includes changing ODBC connection strings, etc.!
4. Eliminate as many approvals as possible. We had a change control form that had 10 signatures on it - I was one of them! I used to get these forms across my desk documenting some change to an application that I had never even seen, had no idea how to measure the risk of the change and yet I was expected to "approve" it. Even worse, the signature had to be on paper so there was a constant stream of people running around with pieces of paper trying to find me to get my signature. Eliminate all this process and identify the minimum number of people who need to be involved in any change.
5. If you're going to have a Change Control Form/Process, keep in as light as possible. Your form should be no more than 2 pages and should be as automated as possible. We currently use JIRA to automate all our tickets. In previous environments, we've used Outlook forms. I've also seen this done use Word with Macros enabled to provide drop downs, automatic fill-in, etc. Keep your process light as well - if you need 2 hours of training to teach someone how to make a code change, then something is wrong!
6. Keep Operations teams involved from day one. One of the primary reasons why change control forms are deployed is because Operations teams get tired of having code dumped on them without documentation or involvement. The typical solution is to put in a gate keeping process to keep the developers from deploying something unsafe to production. This is a legitimate role, but change control processes typically provide artificial security - simply because a developer has filled in the proper form doesn't mean their code works as advertised. Instead, have an Operations Lead that is an integral part of the project from day one who will have a specific understanding of the project and create the right level of documentation, deployment guides, test cases, etc. and be testing these along the way. This is far more effective than a large form being written at the end of the cycle.
7. Keep stakeholders involved throughout the project. The reason why Change Control forms need to be so heavily documented is because the key stakeholders have not been involved all along the way. If stakeholders are seeing working software using Show and Tell techniques, then Change Control simply becomes regular involvement in the project.
8. Use multiple models of communication. Instead of having a single Change Control Form that forces people to write in a specific way, allow for multiple models of communication that could include meetings, power point presentations, white boards, Visio diagrams, etc.
Based on the above principles, here is my current favorite flavour of Change Control:
1. We build our requirements using use case documents that living documents. We currently use Word because its dumb simple and stakeholders can easily open it and change it. We use power point for wire frames and Visio for almost everything else.
2. Revisions of documents get stored in source code control for safe keeping and history. However, we're not dogmatic about it and in my experience the value of these minor revisions is usually not significant - most stakeholders want to see the current version. We keep them in our repository so that everything is stored together and document revisions are tied to code revisions.
3. We have iterations every 2 weeks that include review meetings with the entire team including stakeholders. We do a show and tell of all working code and we use the meeting to prioritize and change. We typically do this by opening up documents on the spot, but sometimes we use pen and paper and transcribe the changes right after the meeting.
4. We use JIRA for day to day tracking of tasks. We treat all changes, bugs, feature requests and tickets the same way. We have a very simple work flow that moves all tasks from open to resolved to closed. Our list of changes can be pulled directly from JIRA as a list of changes. The list of outstanding bugs can be also pulled at any time so we don't need a specific final bug review meeting - we will have been reviewing them on a daily basis.
5. When we're getting close to deployment, we use white board based checklists to capture the minute details (e.g. make sure the ODBC driver is configured properly) that need to be done to move to production. We check these off by simply crossing them off or wiping them from the list. We have 15-20 minute stand-up meetings that includes the Operations Lead so that we're communicating effectively with Operations and they know exactly where things are at as we move into production.
6. Just before doing a deployment to production, I have a meeting with executive stakeholders with a very high level presentation in power point. It presents the current status, any risks that they need to understand, confidence in going, and high level operations plan. Executives typically don't care about whether the SQL is right - they want to know that the team has confidence that it can deploy safely, has followed the appropriate testing processes and that the team is on standby with a contingency plan in case things go wrong.
There are lots of variations to this process and many different tool combinations. You can scale the process up or down depending on the size and formality of the project. In addition, if you're working in a specific regulatory context you may need to be producing more formal documents, approvals, audit trails, etc. My suggestion is always prioritize these below communication and if you need to do this for regulatory purposes then do so but keep the work as minimized as possible. Never substitute change control processes for good communication, team collaboration, stakeholder involvement, etc. In addition, remember that Change Control is a means and not an end - working software is the objective and should be focus for the team.
Change Control traditionally is a project management process for managing change. The typical method for managing change in a classic PMP project is a Change Control form that is filled out containing the scope of the change, impact to the project, etc. This is then approved and incorporated into the master project plan, charter, etc.
In addition, Change Control processes are used by operations departments to manage the deployment and promotion of code from one environment to another. The Change Control Form provides the documentation for an operations team to deploy, operate and monitor the change.
Change Control and Agile
This traditional view of Change Control is at odds with Agile methodologies for the following reasons:
1. Change is viewed as a non-typical event, e.g. the project was running smoothly until someone threw a change into the project. Agile projects by definition are based on the assumption that changes are typical events and aren't special.
2. Change requests are simply Waterfall designs in miniature. Requirements for the change must be documented up front and analyzed. Stakeholders must approve of changes before they are made with the assumption that requirements, scope, risk, and design can be formally documented ahead of making a code change.
3. Traditional Change Control processes are document heavy. The process is designed as much to capture and document change as it is to make effective decisions. Change control forms are kept around forever with dates and identifiers so that someone could find out who approved a change. In many cases, these are also printed out, resulting in very large binders of paper with signatures for every change.
4. In an agile project, one of the objectives is to flatten the team and reduce hierarchies so that people can more effectively communicate. However, traditional change control processes require specialized knowledge, special powers of approval and a hierarchical command and control structure. Project Managers are usually the writers of change control forms, but then they must get approval from "Stakeholders" who will bless the change.
Here is a good example of a traditional Change Control process - have a look at Bell South's Change Control Process Guide. At 120 pages, it is a great example of how bloated a large formal change control process can get when Project Managers are working in a bureaucratic environment. Just managing the process documentation alone takes a massive investment - just look at the 20 pages of version numbers of this document and you'll see that someone's full-time job must be simply keeping the guide up to date!
Change Control isn't a bad thing - the original objectives behind formal Change Control are actually in line with the objectives of agile methodologies:
1. Define change and anticipate change early instead of leaving it to the end.
2. Providing a process for incorporating change during development of a project.
3. Communicate scope, risks and impacts of a change.
4. Provide the appropriate review and approval process to authorize a change.
It isn't that change control is bad - it's that the evolution of formal project methodologies has led to very bureaucratic, documentation heavy processes. In addition, keep in mind that most of these processes were invented before the incorporation of digital collaboration tools such as email, Wikis, sharepoint, JIRA, Word, etc. and so the typical change control process has artifacts from a non-digital era.
Implementing an Agile Change Control Process
Change Control can be used effectively in an Agile environment if it is sufficiently light weight, non-bureaucratic and collaborative. Here are my recommendations based on what I have seen work effectively on Agile and non-Agile projects:
1. Use an agile methodology such as XP, Crystal, Scrum, etc. that has change management built into its fundamental process. In Scrum, for example, all work is effectively a change and is logged as a "change request" in a work backlog. In XP the changes are logged as User Stories. Iterations allow for changes to happen on a regular basis - they're not special change events.
2. Use a generic task tracking system such as JIRA to log any change. In JIRA, you don't need a separate "bug tracking" system that is different than your change tracking system. In addition, you can take a change all the way from requirements to deployment without re-entering information. You could do the same thing with low tech tools such as Excel or Word simply by having a work log that is updated regularly.
3. If you have specific legal or regulatory reasons to track changes then try to understand the specific minimum requirements. I've worked in a government environment for example where all changes needed to be documented so that they could be audited. But the level of detail that was incorporated into each Change Control form became ridiculous - an auditor probably doesn't really need to see a change in an individual SQL script, a deployment checklist that includes changing ODBC connection strings, etc.!
4. Eliminate as many approvals as possible. We had a change control form that had 10 signatures on it - I was one of them! I used to get these forms across my desk documenting some change to an application that I had never even seen, had no idea how to measure the risk of the change and yet I was expected to "approve" it. Even worse, the signature had to be on paper so there was a constant stream of people running around with pieces of paper trying to find me to get my signature. Eliminate all this process and identify the minimum number of people who need to be involved in any change.
5. If you're going to have a Change Control Form/Process, keep in as light as possible. Your form should be no more than 2 pages and should be as automated as possible. We currently use JIRA to automate all our tickets. In previous environments, we've used Outlook forms. I've also seen this done use Word with Macros enabled to provide drop downs, automatic fill-in, etc. Keep your process light as well - if you need 2 hours of training to teach someone how to make a code change, then something is wrong!
6. Keep Operations teams involved from day one. One of the primary reasons why change control forms are deployed is because Operations teams get tired of having code dumped on them without documentation or involvement. The typical solution is to put in a gate keeping process to keep the developers from deploying something unsafe to production. This is a legitimate role, but change control processes typically provide artificial security - simply because a developer has filled in the proper form doesn't mean their code works as advertised. Instead, have an Operations Lead that is an integral part of the project from day one who will have a specific understanding of the project and create the right level of documentation, deployment guides, test cases, etc. and be testing these along the way. This is far more effective than a large form being written at the end of the cycle.
7. Keep stakeholders involved throughout the project. The reason why Change Control forms need to be so heavily documented is because the key stakeholders have not been involved all along the way. If stakeholders are seeing working software using Show and Tell techniques, then Change Control simply becomes regular involvement in the project.
8. Use multiple models of communication. Instead of having a single Change Control Form that forces people to write in a specific way, allow for multiple models of communication that could include meetings, power point presentations, white boards, Visio diagrams, etc.
Based on the above principles, here is my current favorite flavour of Change Control:
1. We build our requirements using use case documents that living documents. We currently use Word because its dumb simple and stakeholders can easily open it and change it. We use power point for wire frames and Visio for almost everything else.
2. Revisions of documents get stored in source code control for safe keeping and history. However, we're not dogmatic about it and in my experience the value of these minor revisions is usually not significant - most stakeholders want to see the current version. We keep them in our repository so that everything is stored together and document revisions are tied to code revisions.
3. We have iterations every 2 weeks that include review meetings with the entire team including stakeholders. We do a show and tell of all working code and we use the meeting to prioritize and change. We typically do this by opening up documents on the spot, but sometimes we use pen and paper and transcribe the changes right after the meeting.
4. We use JIRA for day to day tracking of tasks. We treat all changes, bugs, feature requests and tickets the same way. We have a very simple work flow that moves all tasks from open to resolved to closed. Our list of changes can be pulled directly from JIRA as a list of changes. The list of outstanding bugs can be also pulled at any time so we don't need a specific final bug review meeting - we will have been reviewing them on a daily basis.
5. When we're getting close to deployment, we use white board based checklists to capture the minute details (e.g. make sure the ODBC driver is configured properly) that need to be done to move to production. We check these off by simply crossing them off or wiping them from the list. We have 15-20 minute stand-up meetings that includes the Operations Lead so that we're communicating effectively with Operations and they know exactly where things are at as we move into production.
6. Just before doing a deployment to production, I have a meeting with executive stakeholders with a very high level presentation in power point. It presents the current status, any risks that they need to understand, confidence in going, and high level operations plan. Executives typically don't care about whether the SQL is right - they want to know that the team has confidence that it can deploy safely, has followed the appropriate testing processes and that the team is on standby with a contingency plan in case things go wrong.
There are lots of variations to this process and many different tool combinations. You can scale the process up or down depending on the size and formality of the project. In addition, if you're working in a specific regulatory context you may need to be producing more formal documents, approvals, audit trails, etc. My suggestion is always prioritize these below communication and if you need to do this for regulatory purposes then do so but keep the work as minimized as possible. Never substitute change control processes for good communication, team collaboration, stakeholder involvement, etc. In addition, remember that Change Control is a means and not an end - working software is the objective and should be focus for the team.
