Continuous Integration is extremely important part of software development process, but is also the least used at the same time. Even at relatively big software shops, the continuous integration usually is something, that everyone heard of, but that’s it.
Among all the things you can do to shift the focus of your software developers to what they really have to do instead of what they have to do now, the continuous integration is perhaps the most rewarding.
What is Continuous Integration, anyway?
Continuous Integration is - going through the full cycle of building, deploying and testing of your software product after each change made to the codebase. It makes more sense, when it’s automated by TFS. It is especially useful, when developers and testers are working simultaneously (but asynchronously), preparing product for release.
Basically, it means that testers are working with your web application while it’s auto-updating itself to a newer build, and so the cycle of debugging turns very fast from discovery to fix. Also, the whole development process becomes way more agile, because when requirements change, it's easy to trace which tests should be updated.
We were already talking about automated builds and testing, but we didn’t touch one important topic – the automated deployment. It’s very simple thing to do, once you know how to build and test. So here are the links to previous articles on Team Foundation Server to get you started.
As from the hardware and software perspectives, we are not going to introduce anything extra – the automatic deployment can be hosted by TFS build controller, usually the same machine that is hosting the TFS web application.
The only thing you’ll need to master is the continuous integration mindset. And not just in form of manifesto (“we build every time the change is made, blah blah”), but it should affect everything you do and emphasize the best practices of collaborative software development. What would be the most important ones? Here is my shortlist:
- Listen up!
You have your daily scrum meetings of about 15 minutes every morning, when everyone is sharing their plan for today. That’s where you learn that someone is about to affect your code, by either changing or consuming it. So why not to listen? Team up, collaborate, guide and learn. This will help to reduce the pain of merging the changes together.
- Green Code Only
Do not submit (check-in) the code, which doesn’t build on your machine. Fix it first, then check-in.
- Pull before the push
When you are about to build your code before checking-in, make sure you fetch the latest sources. Your code should resemble the current state of the project, together with all changes from other developers.
- Comment your code
When you introduce the changes in someone else’s code – put an inline comment to signal that it’s been modified. Of course, everyone could see your changes in the history of changes, but this log doesn’t answer all the why’s.
Since it’s a teamwork, it’s in your best interests to make sure, that people understand not only your code, but also your intent. A short comment is an insurance against the pain of reverting someone’s changes in your algorithm.
- Don’t be afraid
to change the code. Refactor, analyze, comment the code of juniors when you see the problem. Make sure you can argument your changes, but don’t hesitate just because it will lead to repeating the tests covering this piece of code. With continuous integration, the effects of your changes will be illuminated within minutes.
- Check-in often
Once you’ve finished the task, added or removed the file, changed the properties of the project or solution (such as added references) – fetch the latest sources, build and check-in. This way you’ll ensure that everyone works with latest sources and that in case of merging the changes of different developers you will have to deal with lesser and more focused (as opposed to wide-spread across the project) scope of changes.
- Check the results
In continuous integration, the server rebuilds the solution every time you introduce the change. Check that build doesn’t fail after you’ve made your changes.
There is a special policy in TFS (“gated check-ins”) to ensure that code builds before allowing the change to be integrated into the code base. During the later stage of the project it might be a good idea to switch it on, to reduce the chance of red code and alerting the team to not grab the latest damaged sources, as they don’t build. Gated check-ins may require you to install additional build agent. One of the practices is to install them on virtual machines, which could run on developer workstations, if resources permit. And since software developer’s machines, by definition, should be the most powerful in the company, most likely you won’t have problems here. If you have only one, default, build agent, you may get a queue of builds waiting for gated check-ins.
In TFS 2012, gated check-ins can be batched (“merged”) together, to reduce the queue. If the whole batch fails, then TFS will build each piece of the batch to figure out what change caused the build to fail.
Another way to reduce the queue is to change the gated check-in build definition to not include the tests and code analysis. You will, however, run the unit tests and perhaps perform the code analysis in other CI builds.
Don’t rely on your self-discipline, if it can be scripted then just do so. The reason to not rely on your personal discipline is that we are talking about the teamwork, the collaborative project, where your skills are both diluted and enhanced by others.
I will extend some of these rules later, for now I only wanted to highlight the necessity of “soft” changes everyone needs to accept at the personal level.
Two Ways of Automated Deployment
There are basically 2 ways to ensure your binaries are deployed to your test environment. One of them – to modify the build script. And mastering the MSBuild skills will surely take some time, and we’ll talk about it later, but today we’ll focus on a simple, GUI-based way to configure the automatic deployment of successful build. Using the Continuous Integration Agent utility.
Requirements of Continuous Integration Agent
This application is very simple to set up and run. It requires Windows Server 2008 SP2 and up (due to .NET 4.5), and it doesn’t have to be installed on the same machine as your TFS server.
If you want to be notified about every integration, make sure you have an SMTP mail account set up.
Setting up the continuous deployment
The continuous integration is a chain of continuous integration of changes, continuous build, continuous testing and continuous deployment. I’ve already highlighted in other articles and videos how we can set up the automated build, and TFS takes care of it together with automatic unit testing and UI testing. The only thing that’s left behind is deployment.
That’s where Continuous Integration Agent (CIA) comes into play.
Here is the screen shot of CIA in action. As you can see, it’s a typical window for an agent, with log and some stats at the main screen. The only drawback is that it’s not a Windows service yet, so you have to have it running in user session.
One agent can work with multiple assignments simultaneously, taking care of continuous integration and nightly builds for many different projects.
To create the assignment, you can either use Ctrl+N key combination, or go to Tools \ Create New Task menu. You’ll see this dialogue:
The Monitoring Task Name field is a short description of the task.
Directory to watch – the TFS build directory where new directories should appear for each build of a particular project. Usually this directory resides in TFS build directory, and the name is the same as of the project name
Target directory name mask – is the simple mask of the directory for each build. This is a new child in the previously discussed directory. Usually it is created by TFS for each build, and you specify the prefix (“CI – “ in this case) for each type of the build. Here, the asterisk (“*”) defines that the name of the new directory should begin with “CI – “ with anything after it.
Wait for X minutes – this is important, as the agent is unable to define when the build is over. As a good practice, take the usual build time for the project and multiply it by 2 - 3. So, if it builds in 2 minutes when triggered manually, set the waiting time to 5 minutes.
Ignore these files – put the file masks of the files that you don’t want to copy to the target directory. By default, it has web.config, *.obj and *.pdb files selected. Important to note, that these masks are XCOPY masks, i.e. to exclude all files with extension “obj”, the file mask is not the “*.obj”, but simply “.obj”.
Ignore these directories – the same as files, but you put only directory names here.
Ensure this directory exists – you can tell the successful build from a failure, if a specific directory exists. In case of web applications, it should be “_PublishedWebsites\name of your application”, but can be different in other scenarios.
And use that directory as the source of files (root) – once that directory found, only files from that directory will be considered as the build, and the parent directory will be ignored. This is the right thing to do for web application.
Copy files from root directory to – this is the path to the web root of your lab server. Files will be copied there using the xcopy utility.
Do not replace files which didn’t change – for some files, like 3rd party libraries, this option might increase the speed of deployment.
Compress results and copy to – if you want to archive the build in form of 7z file – this option is for you.
Send notification e-mail to – if you would like to be notified by e-mail each time the continuous build is processed, use this option. You can always switch off notifications in the main window without having to modify the rules.
Attach compressed file – the mentioned compressed archive of your build will be attached to the e-mail. Use this option with caution, or don’t use it at all.
Once you fill all these fields, you are ready to monitor the directory for new builds. The dialogue will ask you where you want to set the task definition (which is an XML file), and that’s it. You can always modify resulting XML file later in any editor.
Loading and running the tasks
When you are starting the agent, it doesn’t load any task definitions. You have to go to the File menu and open task definition (Ctrl + O) in order to start monitoring the directory for new builds. Agent starts monitoring immediately after the task is loaded.
There is another option in the File menu – “Pause Agent”, which will pause all loaded tasks.
Tools menu has two useful options – to disable e-mail alerts (useful Friday option) and unload all tasks. Currently agent doesn’t allow to unload specific task.
Further customization is possible using the app.config file, which is a simple XML file at the same directory as the agents exe file.
It’s only beta!
Although it works perfectly for me, this application is still under development. It was tested for weeks and didn’t produce any errors, but still it is not intended for production environment (i.e. automatic deployment to production servers). It is good for deployment of latest changes to the lab server, but it’s better to prepare production deployment under human supervision.
If you have any question regarding this utility, you can always contact me and I’ll be glad to help.
This archive contains executable file and configuration file. You have to install .NET Framework 4.5 and 7-zip archive manager.
ContinuousIntegrationAgent.rar (86.58 kb)