Managing Configuration Files and .NET

I've been using the .NET config files lately, and its been rather frustrating lately because of the confusion that arises when running a bunch of different ways to access the same code.

In my scenario, I have a class library called DBAccess.dll and I call it in the following ways:

1. From a web site using the ASP.NET built in web server launched from VS 2005.
2. From a web site running in IIS.
3. From a forms application
4. By running TestDriven.NET using a seperate test assembly.
5. From a windows service.

In all these cases, I'm expecting to get configuration values through the main configuration file for connection string so that it can be used by Microsoft Enterprise Library.

With so many options, .NET doesn't make it easy for you because of the differences in naming conventions on the default configuration file, e.g. in the case of a web app it expects a web.config, a forms app expects an app.config and if you run the NUnit application it wants the assembly config file, e.g. DBAccess.dll.config. Maintaining all these configuration files is a nightmare!

So in sorting through this, I came up with a few basic strategies to combat this configuration file overload:

• If you add an App.Config file to your class library project, it will automatically translate this into Assembly.Dll.Config for you.
• For NUnit, it expects a config file that is rooted to your test assembly. It does not read any dependent assemblies config files, app.config, web.config, etc. even if its the same directory of your assembly. Alternatively, you can add configuration information directly into the app.config file of the NUnit executable (either the GUI or the Console) but this isn't a very portable approach.
• If you have an App.Config file and an assembly config file, .NET will always read the App.Config file. It does not by design read your assembly specific config files. There is some code floating around that can do this for you if you want to have more distributed configuration mechanisms that are attached to assemblies instead of executables. This is one way to mitigate the problem of re-using an assembly across multiple projects where some are web projects and some are forms projects.
• Another great alternative is to use proper build files instead of the default studio builds. Either MSBuild or NANT will allow you the flexibility to store a single configuration file and rename it and then deploy it to the appropriate destination. The tricky part would be that you may need a configuration file named Assembly.Dll.config for your unit tests if you've incorporated them into your build process and a final App.config or Web.config for running the executable.
• Another great idea that I have used before is to use the Microsoft Enterprise Library to specifiy an external configuration file. This removes the naming problem with your configuration files in that you can have a seperate external file that represents the master configuration file and simply have Web.Config, App.Config, Assembly.Dll.Config point to your external file. This works quite nicely in my experience as long as you remember to add the external file to your project and to configure studio to always copy it into your build directory.
• You can build your own configuration loader. I have done this with custom defined configuration sections by having code in the custom section that contains a path to an external configuration file. You can arbitrarily load up any configuration file in code and grab whatever sections out of it. This is really useful for custom configuration sections where you want to provide code that saves and loads the configuration file with an arbitrary name. One important note with this strategy - there are cases where the Enterprise Library does the configuration loading for you, e.g. the DatabaseFactory will load connection strings for you automatically. In these cases, you cannot provide a configuration section or source as an object so you're stuck with the App.Config, Web.Config, etc. as a starting point.
• The ConfigurationManager in .NET allows you to open the Machine.config file. You might want to peer into this file directory to look at global values instead of your local App.config values.
• You can use a complete different configuration framework - no is forcing you to use App.config just because it's there (although most people do so). For example, look at Nini as an option (although I haven't tried it yet!)

I hope that provides some ideas on how to manage configuration files in a way that will allow you to have more re-usable configuration schemes than a basic App.config or Web.config file so that it can be used across multiple projects and different application contexts.

Test Driven Development When Writing New Code

I have been doing TDD for several years now, and I can still not get around the recommendation to test first to the point where it breaks the code, write enough code to pass the test, and then go to the next test. In my experience this tends to work with mature code bases where there is a chunk of existing code that is being changed. However, I'm not sure this approach works for new applications that are being built from scratch. In my experience, a less purist approach seems to work for me:

• Write enough code to get something at least working minimally. If you think of a use case, I'm not sure there is much point in writing gobs of test cases before you even have a primary path through the use case. The XP methodology talks about an architectural or design spike, but in my experience there is also a coding spike. For example, it might be the first iteration that instead of writing code + tests we just write as much code as we can to push through the primary path as widely as possible instead of trying to be exhaustive in our testing.
  
• I use at least a basic unit test in NUnit as an entry point in all code debugging. That enforces the practice of having tests but also provides a useful way to gain entry into the code without web pages, windows forms, or a global main method. Using TestDriven.NET, you can simply right-click a test and run it right in studio using the debugger, NUnit, NCover, etc. and I find this higly efficient when writing new code.
  
• There does seem to be a drop off point where writing more and more tests provides less value, as in any testing effort. I'm seen some overly puritanical approaches to TDD where it was insisted that every boundary condition was to be coded up as a unit test no matter how unlikely the scenario or risk. In my experience there is a point where coding these scenarios is not worth the benefit of the test. In addition, I have found that in some cases it is still easier and better to manually test something than to try and code it up as a test.
  
• I definately like the idea of using build scripts and NUnit tests when making modifications to code. However, In my experience the tests actually don't change as much as the underlying code. In some cases there are new requirements but in many cases they are implementation changes. As long as the existing tests are still green after the change, then no new tests are required.
  
• There seems to be a trade-off between quick progress and the feeling of "being on a roll" and the pace of TDD which tends to be plodding but regular. Again, I think a less purist approach is best - at some points you feel your creative juices flowing and you want to plow through 2-3 pages of code in one go without writing exhaustive unit tests for each little method. Other times if you're being more cautious you want to test the living hell out of every line of code. As a mature developer, you have to feel the right pattern depending on the code you're writing.
  
• Lastly, I think its critical that at a minimum unit tests are written before code is checked into source control. That way if you have continuous integration the unit tests will run on every build. It seems also just a courtesy - you might have insight on why you left out some tests but your team mates might not. So before checking in code make sure there are tests to go with it. I find also that the unit tests are really a good description in code of the requirements so for new code its really helpful to have them as a guide.
  

For code changes in a larger mature application, I like the traditional TDD process of making new tests so that they fail, writing code to fix the tests and then regression. But for coding applications from scratch or starting a new major code project, I find a looser style tends to produce better results.

So with any agile technique, don't get to ideological about the approach - in my experience, TDD is best used as an approach and not a rigid perscription.

Example of Fat Code

Why exactly does the Google Toolbar need to be 47 megabytes?! It's a tool bar for heaven's sake...

Agile Tools for .NET Part 2

This is a follow up to my previous post on Agile Tools for .NET. I got to the office and I noticed a few more tools that I missed in my previous post.

Confluence

Atlassian's professional WIKI solution, it works great with JIRA and provides a really nice WIKI solution. For those who want to travel light with documentation, WIKI's provide a nice fit for technical documentation, build documentation, internal designs, etc. as they easily accomodate rapid changes, are completely web based and allow for automatic version tracking.

Data Dictionary Creator

One of the principles of Agile is make your code self-documenting. What about making your database self-documenting? You can do this in SQL 2005 in T-SQL by adding meta-data and descriptions to each field in your database. The Data Dictionary Creator tool does the same thing through a nice GUI and allows for export and import of the documentation. You can then document your database and store the meta-data in source safe for version control and safe keeping. In addition, you can export the descriptions to an excel file for instant documentation!

Wisdom Screen Hunter

Not really a development tool, but how many times have you been creating documents and wanted a quick screen capture? Forget using the PrntScrn and fiddling in paint - use this tool. Its free and allows you to capture any arbitrary rectangular area as well as any active window. Simply hit F6 and off you go.

Agile Tools for .NET

The following is a list of tools for doing agile based development with Microsoft .NET. There are quite a few tools out there that will accelerate your use of continuous integration, test driven development, source control integration, etc.

I have used all these tools - most I use on a daily basis. They are highly recommend and most are either low cost or free.

JIRA

Best tool for task management. For more details see my review of JIRA from a previous post. It's bugzilla on steroids and is much better and simpler than most windows based tools. It's java based, but there is still no reason not to use it for .NET based development.

NUNIT

NUnit is an implementation of the XUNIT framework for .NET. It allows you to create unit test classes that can be run by either a GUI or console application. NUnit will also integrate into your build process so that on each build, you run through your automated tests to make sure they still pass.

NANT

NANT is a port of the ANT build tool from Java. NANT allows for build scripts based on simple XML files. NANT can do things such as compile code, deploy assemblies, manage directories, start and stop processes, run executables, etc.

NCover

NCover is a code coverage tool for .NET. It will instrument your assemblies when they are built to analyze each line of code in your assembly and whether it was actually executed (exercised). If you combine this with NUnit, you can then tell how much your unit tests actually go through every path of code.

dotTrace or Ants Profiler

Either tool will do, but personally I like Ants Profiler better and you can get it as a bundle with Ants Loader for cheaper. Both tools do the same thing - they profile either memory or performance data for your executing code. You can profile an ASP.NET web site, a Winform application or a Windows Service. What you will get in the case of memory profiling is the ability to snapshot at any point of execution a picture of what is in memory. This can help with objects not getting garbage collected properly or where you're running into memory issues. For performance profiling, the tool will tell you the amount of time each line of code took to execute. This can be useful for performance testing in that you can see the breakdown of time for a method to isolate the particular lines of code that are taking the longest.

Cruise Control .NET

Cruise Control is a port of the java application of the same name. It is a continuous integration server which will fire a build on every check into your source code repository. It can run either a NANT script or an MSBuild script or simply tell Visual Studio 2005 to do a build. It then stores the status of each build in a little intranet site that you can monitor and it can email you notifications as well. This provides an automated way to integrate code and test to make sure it builds and passes all your unit tests.

My Generation

My Generation is a code generator. It ships with a number of templates for generating O/R mapping code for frameworks such as Gentle, NHibernate, DAAB, etc.

TestDriven .NET

This is one of my favourite tools. It allows you to integrate your unit testing right into studio. Want to run one of your NUnit Tests? Just right click on the test code and up comes a Run with option that you can then run with the debugger, NCover, NUnit, etc. The one I use most is run with Debugger as it means you can step through the code hit by each of your unit tests.

ANKH

Ankh is a subversion add-in for Visual Studio 2005. For those who are using subversion as your source code control system (and I would recommend subversion in general over Visual Source Safe) it allows you to update and commit code from the solution explorer window. My only problem with this tool is that its quite slow especially if you have large projects, so you may find it easier to check in with external tools after a while. But for those who are just getting used to subversion, its a great first step.

Getting to the Finish Line in an Agile Project

One of the great potentials of Agile software development is flexibility in changing requirements. We currently use agile methodologies and have scheduled 2 week iterations. We have show and tells with stakeholders every two weeks where we show them where things are at and gather new user stories, functionality changes and other feedback to plan for the next iteration.

One of the challenges we have found with this approach is that the project quickly turns into perpetual development. One of the advantages of a traditional waterfall model is a solid arc in development and a very specific deliverable - you have a good definition of when you're done. In an environment where the requirements evolve, the time lines are fluid and the stakeholders are not that experienced at software projects we are finding that this means that the project never seems to end! Every two week iteration, stakeholders simply pile on new requirements and the launch date drifts away.

In a consulting environment, this isn't such a bad thing if you're on a time and materials engagement - the client simply gets to spend their money as long as they want. But in an internal development team, this poses a challenge because most stakeholders aren't really aware of how much money they are spending and they simply keep monopolizing the team. In the meantime, there are other projects or priorities queueing up in the background.

Based on what we have found works, here are some practical recommendations on how to combat this problem:

• Establish clear business objectives and as many solid functional requirements as possible. There is a happy medium between XP style iterative programming and a rigid waterfall model. Try to put some acceptance criteria that can be measured so that the entire team has a common understanding of the target.
• Remember the law of triple constraints - of cost, resources, and time changing any of these impacts the others. Make sure that stakeholders are aware of the real cost (even if it is an internal cost) through an internal budgeting model. In our organization, we have a shared cost model in IT but we can allocated the shared cost to any department we choose. In cases where we see departments monopolizing our time we can allocate more cost to them. This gives internal stakeholders an incentive to prioritize and conserve resources.
• Make stakeholders compete with each other. This is pure politics but it works. The easiest way to stop a project is to simply tell another stakeholder that they're waiting in the queue behind someone else. There will be a pretty quick meeting between stakeholders where one will start to put a lot of pressure on the other to get their project done quickly or to get out of line altogether.
• Time box and resource constraint the project explicitly. We do this quite a bit with reporting needs which seem to continually evolve ad nauseum. When we start these types of projects, we establish a budget of 2-3 iterations and that's the budget. The requirements can evolve but there continues to be pressure on the stakeholder to get it right the first time or else lose their window.
• Always have a phased approach with Phase 2 and 3 already planned in Phase 1. This gives the stakeholder the ability to still feel they're going to get a change at some point but perhaps can live with launching without it.
• Be vigilant and communicate often about what's currently in and out of scope. I'm not sure whether stakeholders intentionally play dumb or whether people simply don't hear bad news but I've experienced several times the case where three meetings in a row we hear questions about a feature that we've already ruled out of scope and there is genuine surprise from the stakeholder that they're not getting a particular feature even if its been already discussed, documented, etc.
• Have really strong leaders in QA. QA Leads, especially in the latter stages of projects, become crucial at communicating and organizing priorities. A great QA Lead can manage stakeholders on a daily bug review meeting and can handle the constant re-negotiation of last minute feature requests, low priority bug fixes, etc. and keep the process moving towards launch. If the project loses velocity during the stage it opens the door to ever increasing scope change. If the project has high velocity then everyone on the team feels like they better prioritize because time is running out and the project is going to launch without them. This helps crystallize priorities and puts good pressure on stakeholders to commit to the minimum work required to launch.

I've been in projects where we sat at 90% for 5 months because we kept stalling as we closed in on the finish line. In addition, I've been on projects with internal teams where the stakeholders simply got used to having a software team at their disposal week after week and their was never any pressure to launch anything or to stop working.

As a project manager, your goal is to launch - 90% complete is really 0% complete especially if it's still just running in development or still buggy in test.

Can You Trust Second Life Charity?

At the foundation, we are interested in Second Life. For those who are not familiar, Second Life is a virtual 3D world that allows for users to create custom content, communicate with others in a virtual world and buy and sell items in a virtual currency. What makes it unique is that the virtual currency can be converted into real cash, allow for the conversion between the virtual and the real.

Commercial activities have evolved over the past year or so in Second Life to include things like virtual real estate, virtual products, etc. In the past little while, charities have started doing virtual fundraising by having virtual walkathons, virtual campaigns, etc. in Second Life. The virtual currency they raise they then convert to real dollars to help real people.

So here is the basic question - how do you ensure a trusted conversion between the fantasy and the real? In the case where a Second Life user is buying a virtual good, then there is no real issue - they pay virtual money and instantly get a virtual hat, a virtual house, etc. But if you donate virtual money to a virtual charity, how do you know that its really going to the charity? This is true for commercial conversions as well, e.g. if you pay in virtual currency for a physical good, how do ensure that the vendor is legitimate?

The number one objective for a fundraiser is raising money for their respective cause. But a very close number two is ensuring the maintenance of its brand, reputation and trust with its donors. Its critically important because its the only long term asset a fundraiser has at their disposal (in the same way as Nike, Sony, or Coca Cola).

There are a couple potential ways you can verify that a charity asking for money is legitimate and sanctioned:

• Check the charities web site. For example, the American Cancer Society publicizes its Second Life Relay for Life and provides instructions on how to get involved.
  
• Call the charity. If the person asking for money is legitimate, they'll be registered in some way as an event organizer.
  
• Ask the person in Second Life basic questions like: do you have a contact I can talk to, how do I verify that you work for the charity, do you have a tax number, etc. The person should have a way to give you information that can be verified - at least that shows the person has some basic knowledge of the charity they are promoting.

There are some people in Second Life who may be asking for donations and legitimately giving the money to a charity. However, without some way to verify it I would suggest you're better off donating directly to the charity.

If you cannot verify, I would not donate a single L$ as you have no way to ensure the money is actually going to charity.