Generating semantic version build numbers in Teamcity

Background

This is what we want to achieve, build numbers with minor, major and build counter parts together with the branch name.

This is what we want to achieve, build numbers with minor, major and build counter parts together with the branch name.

What is a good version number? That depends on who you ask I suppose, but one of the most popular versioning schemes is semantic versioning. A version number following that scheme can look like the following:

1.2.34-beta

In this post I will show how an implementation for generating this type of version numbers in Teamcity with the following key aspects:

  1. The major and minor version numbers (“1” and “2” in the example above) will be coming for the AssemblyInfo.cs file.
    • It is important that these numbers come from files in the VCS repository (I’m using Git) where the source code is stored so that different VCS branches can have different version numbers.
  2. The third group (“34”) is the build counter in Teamcity and is used to separate different builds with the same major and minor versions from each other.
  3. The last part is the pre-release version tag and we use the branch name from the Git repository for this.
    • We will apply some filtering and formatting for our final version since the branch name may contain unsuitable characters.

In the implementation I’m using a Teamcity feature called File Content Replacer which was introduced in Teamcity 9.1. Before this we we had to use another feature called Assembly Info Patcher but that method had several disadvantages, mainly that there was no easy way of using the generated version number in other build steps.

In the solution described below, we will replace the default Teamcity build number (which is equal to the build counter) with a custom one following the version outlined above. This makes it possible to reuse the version number everywhere, e.g. in file names, Octopus Release names, etc. This is a great advantage since it helps make connect everything produced in a build chain together (assemblies, deployments,  build monitors, etc).

Solution outline

The steps involved to achieve this are the following:

  1. Assert that the VCS root used is properly configured
  2. Use the File Content Replacer to update AssemblyInfo.cs
  3. Use a custom build step with a Powershell script to extract the generated version number from AssemblyInfo.cs, format it if needed, and then tell Teamcity to use this as the build number rather than the default one
  4. After the build is complete, we use the VCS Labeling build feature to add a build number tag into the VCS

VCS root

The first step is to make sure we get proper branch names when we use the Teamcity %teamcity.build.branch% variable. This will contain the branch name from the version control system (Git in our case) but there is a special detail worth mentioning, namely that the branch specification’s wildcard part should be surrounded with parentheses:

VCS root branch specification with parenthesis.

VCS root branch specification with parenthesis.

If we don’t do this, then the default branch (“develop” in this example) will be shown as <default> which is not what we want. The default branch should be the name branch name just like every other branch, and adding the parentheses ensures that.

Updating AssemblyInfo.cs using the File Content Replacer build feature

In order for the generated assembly to have the correct version number we update the AssemblyInfo.cs before building it. We want to update the following two lines:


[assembly: AssemblyVersion("1.2.0.0")]
[assembly: AssemblyFileVersion("1.2.0.0")]

The AssemblyVersion attribute is used to generate the File version property of the file and the AssemblyFileVersion attribute is used for the Product version.

File version and product version of the assembly.

File version and product version of the assembly.

We keep the first two integers to use them as major and minor versions in the version number. Note that the AssemblyVersion attribute has restriction on it so that it must be four integers separated by dots, while AssemblyFileVersion does not have this restriction and can contain our branch name as well.

To accomplish the update, we use the File Content Replacer build feature in Teamcity two times (one for each attribute) with the following settings:

AssemblyVersion

  • Find what:
    (^\s*\[\s*assembly\s*:\s*((System\s*\.)?\s*Reflection\s*\.)?\s*AssemblyVersion(Attribute)?\(")([0-9\*]+\.[0-9\*]+\.)([0-9\*]+\.[0-9\*]+)("\)\])$
  • Replace with:
    $1$5\%build.counter%$7

AssemblyFileVersion

  • Find what:
    (^\s*\[\s*assembly\s*:\s*((System\s*\.)?\s*Reflection\s*\.)?\s*AssemblyFileVersion(Attribute)?\(")([0-9\*]+\.[0-9\*]+\.)([0-9\*]+\.[0-9\*]+)("\)\])$
  • Replace with:
    $1$5\%build.counter%-%teamcity.build.branch%$7

As you can see, the “Find what” parts are regular expressions that finds the part of AssemblyInfo.cs that we want to update and “Replace with” are replacement expressions in which we can reference the matching groups of the regex and also use Teamcity variables. The latter is used to insert the Teamcity build counter and the branch name.

In our case we keep the first two numbers but if the patch number (the third integer) should also be included, then these two expressions can be adjusted to accomodate for this.

The FIle Content Replacer build feature in Teamcity.

The FIle Content Replacer build feature in Teamcity.

When we have done the above, the build will produce an assembly with proper version numbers, similarly to what we could accomplish with the old Assembly Info Patcher, but the difference with the old method is that we now have an patched AssemblyInfo.cs file whereas with the old method it was unchanged as only the generated assembly DLL file was patched. This allows us to extract the generated version number in the next step.

Setting the Teamcity build number

Up to now, the Teamcity build number has been unchanged from the default of being equal to the build counter (a single integer, increased after every build). The format of the build number is set in the General Settings tab of the build configuration.

The General Settings tab for a build configuration.

The General Settings tab for a build configuration.

The build number is just a string uniquely identifying a build and it’s displayed in the Teamcity build pages and everywhere else where builds are displayed, so it would be useful to include our full version number in it. Doing that also makes it easy to use the version number in other build steps and configurations since the build number is always accessible with the Teamcity variable %system.build.number%.

To update the Teamcity build number, we rely on a Teamcity service message for setting the build number. The only thing we have to do is to make sure that our build process outputs a string like the following to the standard output stream:

##teamcity[buildNumber '1.2.34-beta']

When Teamcity sees this string, it will update the build number with the supplied new value.

To output this string, we’re using a separate Powershell script build step that extracts the version string from the AssemblyInfo.cs file and does some filtering and truncates it. The latter is not strictly necessary but in our case we want the build number to be usable as the name of a release in Octopus Deploy so we format it to be correct in that regard, and truncate it if it grows beyond 20 characters in length.

Build step for setting the Teamcity build number

Build step for setting the Teamcity build number

The actual script looks like this (%MainAssemblyInfoFilePath% is a variable pointing to the relative location of the AssemblyInfo.cs file):

function TruncateString([string] $s, [int] $maxLength)
{
	return $s.substring(0, [System.Math]::Min($maxLength, $s.Length))
}

# Fetch AssemblyFileVersion from AssemblyInfo.cs for use as the base for the build number. Example of what
# we can expect: "1.1.82.88-releases/v1.1"
# We need to filter out some invalid characters and possibly truncate the result and then we're good to go.  
$info = (Get-Content %MainAssemblyInfoFilePath%)

Write-Host $info

$matches = ([regex]'AssemblyFileVersion\(\"([^\"]+)\"\)').Matches($info)
$newBuildNumber = $matches[0].Groups[1].Value

# Split in two parts:  "1.1.82.88" and "releases/v1.1"
$newBuildNumber -match '^([^-]*)-(.*)$'
$baseNumber = $Matches[1]
$branch = $Matches[2]

# Remove "parent" folders from branch name.
# Example "1.0.119-bug/XENA-5834" =&amp;amp;amp;gt; "1.0.119-XENA-5834"
$branch = ($branch -replace '^([^/]+/)*(.+)$','$2' )

# Filter out illegal characters, replace with '-'
$branch = ($branch -replace '[/\\ _\.]','-')

$newBuildNumber = "$baseNumber-$branch"

# Limit build number to 20 characters to make it work with Octopack
$newBuildNumber = (TruncateString $newBuildNumber 20)

Write-Host "##teamcity[buildNumber '$newBuildNumber']"

(The script is based on a script in a blog post by the Octopus team.)

When starting a new build with this build step in it, the build number will at first be the one set in the General Settings tab, but when Teamcity sees the output service message, it will be updated to our version number pattern. Pretty nifty.

Using the Teamcity build numbers

To wrap this post up, here’s an example of how to use the updated build number to create an Octopus release.

Creating an Octopus release with proper version number from within Teamcity.

Creating an Octopus release with proper version number from within Teamcity.

 

In Octopus, the releases will now have the same names as the build numbers in Teamcity, making it easy to know what code is part of the different releases.

The releases show up in Octopus with the same names as the build number in Teamcity.

The releases show up in Octopus with the same names as the build number in Teamcity.

Tagging the VCS repository with the build number

The final step for adding full tracability to our build pipeline is to make sure that successful builds adds a tag to the last commit included in the build. This makes it easy in the VCS to know exactly what code is part of a given version, all the way out to deployed Octopus releases. This is very easy to accomplish using the Teamcity VCS labeling build feature. Just add it to the build configuration with values like in the image below, and tags will created automatically everytime a build succeeds.

The VCS Labeling build feature in Teamcity.

The VCS Labeling build feature in Teamcity.

The tags show up in Git like this:

Tags in the Git repository connects the versions of successful builds with the commit included in the build.

Tags in the Git repository connects the versions of successful builds with the commits included in the build.

Mission accomplished.

/Emil

Setting the font of a PowerShell console to Lucida Console won’t work

Ever tried changing the font of a PowerShell console to Lucida Console, only to see the setting gone the next time you open the console? In that case, you’re not alone! I’ve been pulling my hair over this problem many times but today I decided to investigate it further.

There are several different solutions and none of them worked for me. For some people it helps if you set the font size to something other than 12 points, but not for me. For others it helps to start the console as administrator, but not for me. And here’s a strange thing: In a good old Command Prompt (CMD.EXE) Lucida Console works as a default font with no problem at all. It’s only in PowerShell console I can’t set it as default.A few of these tricks are discussed at superuser.com.

My problem turned out to be different as it was related to the regional settings of my Windows installations. The problem is described very briefly by a Microsoft support engineer here. Apparently Lucide Console “is not properly supported on CJK languages and other languages that are not localized by PowerShell (e.g. Arabic, Hebrew, etc.)”. It seems that Swedish belongs to the same category of languages that for some reason is not deemed compatible with Lucida Console. Strange thing is that it works perfectly when setting it on the console instance…

Anyway, to fix the problem all I had to do was to change my system locale to “English (United States)”:

Setting system locale to a language that is "supported" for Lucida Console solves the problem...

Setting system locale to a language that is “supported” for Lucida Console solves the problem…

Voila, my PowerShell prompt is pretty everytime I open it, instead of using the ugly “Raster Fonts” which it falled back to before.

The problem description that Lucida Console is not compatible with all languages makes very little sense to me, but at least my problem is solved.

/Emil

Retrieving the message count for MSMQ queues

Sometimes it can be useful to retrieve the number of messages in an MSMQ queue, for example for monitoring. However, it’s not immediately apparent how to do it if you google it, so here are my thoughts on the subject.

Other blog posts suggest iterating over messages (e.g. Counting Messages in an MSMQ MessageQueue from C#) or doing it using WMI. WMI is the best alternative in my opinion and if you want a quick way of doing it then PowerShell is the easiest:

$queues = Get-WmiObject Win32_PerfFormattedData_msmq_MSMQQueue
$queues | ft -property Name,MessagesInQueue

The result will be something similar to this:

Name                                                         MessagesInQueue
----                                                         ---------------
active714\private$\notify_queue$                                           0
active714\private$\deltagarloggservice\deltagarloggservi...                0
active714\private$\order_queue$                                            0
active714\private$\admin_queue$                                            0
Computer Queues                                                           27

This can also be done on remote machines:

$host = ...
$cred = get-credential
$queues = Get-WmiObject Win32_PerfFormattedData_msmq_MSMQQueue -computer $host -credential $cred
$queues | ft -property Name,MessagesInQueue

The Get-Credential Cmdlet will display a login dialog which is fine in interactive sessions but if you need to set the credentials in a non-interactive script, then the tip in this blog post might help: PowerShell – How to create a PSCredential object.

Retrieving message counts from code takes a little more coding but here’s an example in C# that searches for a given queue and returns its message count:

private static int GetMsmqMessageCount(string queuePath, string machine,
  string username, string password)
{
  var options = new ConnectionOptions
    {Username = username, Password = password};
  var path = string.Format(@"\\{0}\root\CIMv2", machine);
  var scope = new ManagementScope(path, options);
  scope.Connect();

  string queryString = 
    String.Format("SELECT * FROM Win32_PerfFormattedData_msmq_MSMQQueue WHERE Name = '{0}'",
	  queuePath);
  var query = new ObjectQuery(queryString);

  var searcher = new ManagementObjectSearcher(scope, query);

  IEnumerable<int> messageCountEnumerable = 
    from ManagementObject queue in searcher.Get()
    select (int) (UInt64) queue.GetPropertyValue("MessagesInQueue");

  return messageCountEnumerable.First();
}

This code also uses WMI but this time we do a query rather than enumerate all queues.

The above snippets has worked well for us so I felt it would be useful to post them here. Please leave a comment if you have issues with them or suggestions for improvement!

/Emil

Log file searching using PowerShell

I recently had to do some IIS log file parsing to solve a performance problem and it turned out that PowerShell was a very useful tool for doing this. I thought it might be a good idea to share a few one-liners for the benefit of others (as well as myself the next time I need to do it).

Here we go:

Get-Item *.log | Where-Object { $_.LastWriteTime -gt "2011-09-01" } | Select-String 'foobar'

The above line retrieves a collection of files filtered so that only files with the “.log” extension is matched. It then checks that the last modification time is later than a given date and finally it selects and displays all content lines containing the string “foobar”.

To make it search for files recursively in sub-folder, switch the Get-Item cmdlet for something like Get-ChildItem . -recurse. This can look like this:

Get-ChildItem . -recurse -filter *.log | Select-String 'foobar' | Select-Object -property Path, Line

(Note that is that example we also limit the display to include the properties (file) Path and (matched) Line to make the result easier to read.)

If you want to analyze the results in detail it’s probably a good idea to output the results to a file:

Get-Item *.log | Where-Object { $_.LastWriteTime -gt "2011-09-01" } | Select-String 'foobar' | Add-Content foobarsearchresult.txt

Note that we use the Add-Content cmdlet instead of piping (using the > operator) in order to avoid newlines at the console width position in the file.

To display the number of matched lines in the files (assuming we created several search result files above), we can do things like this:

Get-Item *.txt | % { $_.Name; get-content $_ | measure-object }

The result will be similar to this:

searchresult_foo.txt
Count    : 10
Average  :
Sum      :
Maximum  :
Minimum  :
Property :

searchresult_bar.txt
Count    : 122
Average  :
Sum      :
Maximum  :
Minimum  :
Property :

Apparently we had 10 lines in the ‘searchresult_foo.txt’ file and 122 in ‘searchresult_bar.txt’. This was enough to find my problem (which happened to be an overambitious SharePoint search crawl, but that’s another story…).

Before I stopped, I couldn’t resist creating a slightly prettier line count report:

Get-Item  *.txt | % { Write-Host ("{0,-50} " -f $_.Name) -NoNewLine; get-content $_ | measure-object | % { "{0,10} lines" -f $_.Count } }

The result is now:

searchresult_foo.txt                                       10 lines
searchresult_bar.txt                                      122 lines

Besides giving prettier results, this script also prints the name of each file before actually counting the lines in it. This gives nice feedback to the user when the files are large and the counting takes time.

PowerShell is really becoming an increasingly important tool in my toolbox…

/Emil

Retrieving domain information in Powershell

Here’s how to retrieve the current domain:

[reflection.assembly]::loadwithpartialname("System.DirectoryServices")
[System.DirectoryServices.ActiveDirectory.Domain]::GetComputerDomain()

The result is a System.DirectoryServices.ActiveDirectory.Domain object:

Forest                  : i.activesolution.se
DomainControllers       : {ACTIVESERVER101.i.activesolution.se, ACTIVESERVER102.i.activesolution.se}
Children                : {}
DomainMode              : Windows2008R2Domain
Parent                  :
PdcRoleOwner            : ACTIVESERVER102.i.activesolution.se
RidRoleOwner            : ACTIVESERVER102.i.activesolution.se
InfrastructureRoleOwner : ACTIVESERVER102.i.activesolution.se
Name                    : i.activesolution.se

/Emil

Array or no array in Powershell? Why using @(…) is a good idea.

Here’s a funny thing in Powershell:

PS C:\> cd C:\WINDOWS\system32
PS C:\WINDOWS\system32> (dir *.dll).GetType()

IsPublic IsSerial Name                           BaseType
-------- -------- ----                           --------
True     True     Object[]                       System.Array

PS C:\WINDOWS\system32> (dir kernel32.dll).GetType()

IsPublic IsSerial Name                           BaseType
-------- -------- ----                           --------
True     True     FileInfo                       System.IO.FileSystemInfo

PS C:\WINDOWS\system32>

See what’s happening? If dir (or actually Get-ChildItem) finds several items matching our criteria then it returns an array, which is what you’d expect. But if the cmdlet finds just a single item then that item is returned directly instead of being encapsulated into an array. If you’re unexperienced with Powershell then that’s probably not what you’re expecting.

For example, this script will not always work as expected:

if ((dir *.dll).Length -gt 1) { 'many items' }

If there is more than 1 matching item then all will be fine, but if exactly one item is found then that script will break. And worse, it will break without the user noticing since no exception is thrown. It will simply print out ‘many items’ in this case, even though there is just one item. To understand why, try this:

PS C:\WINDOWS\system32> (dir kernel32.dll).Length
989696

What happens is that dir returns one item, of type FileInfo, which happens to have a property called Length which contains the size of the file. Thus we’re calling System.IO.FileInfo.Length instead of System.Array.Length… Very confusing!

Fortunately, there’s an easy way to make sure that you always get an array, namely the @(…) construct which does nothing if its expression evaluates to an array and if it doesn’t, encapsulates it into one.

Examples:

PS C:\WINDOWS\system32> @(dir kernel32.dll).Length
1
PS C:\WINDOWS\system32> if (@(dir *.dll).Length -gt 1) { 'many items' }

To sum it up: Always use @(…) if you want to work with arrays! Note that this applies to all cmdlets, not just Get-Childitem!

/Emil

Listing and resuming workflow instances

We recently had a need to list suspended Workflow Foundation 4 instances hosted in Windows Server AppFabric from code, and here’s how to do it:

SqlInstanceQueryProvider provider = new SqlInstanceQueryProvider();
var providerArgs = new NameValueCollection { { "connectionString", _instanceStoreConnectionString } };
provider.Initialize("WfInstanceProviderA", providerArgs);

InstanceQuery query = provider.CreateInstanceQuery();
var args = new InstanceQueryExecuteArgs { InstanceStatus = InstanceStatus.Suspended };

IAsyncResult asyncResult = query.BeginExecuteQuery(args, new TimeSpan(0, 0, 0, 30), null, null);
IEnumerable instances = query.EndExecuteQuery(asyncResult);

By the way, this requires a reference to Microsoft.ApplicationServer.StoreManagement.dll which is installed with AppFabric.

Resuming the workflow instances can be done like this:

SqlInstanceControlProvider provider = new SqlInstanceControlProvider();
var providerArgs = new NameValueCollection { { "connectionString", _instanceStoreConnectionString } };
provider.Initialize("WfInstanceProviderA", providerArgs);

InstanceControl control = provider.CreateInstanceControl();

foreach (InstanceInfo instanceInfo in instances)
{
    InstanceCommand cmd = new InstanceCommand
    {
        CommandType = CommandType.Resume,
        InstanceId = instanceInfo.InstanceId,
        ServiceIdentifier = instanceInfo.HostInfo.HostMetadata
    };

    // Not included in instanceInfo.HostInfo.HostMetadata...
    cmd.ServiceIdentifier["VirtualPath"] = 
         "/EducationWorkflowServices/AktivitetService.xamlx";

    IAsyncResult asyncResult = control.CommandSend.BeginSend(cmd,
        new TimeSpan(0, 0, 0, 30), null, null);
    control.CommandSend.EndSend(asyncResult);
}

AppFabric also has some great PowerShell cmdlets for administration that can be used for the same thing. Here are a few examples:

Import-Module ApplicationServer
Get-ASAppServiceInstance -status 'suspended'
Get-ASAppServiceInstance -status 'suspended' | Resume-ASAppServiceInstance
Get-ASAppServiceInstance -status 'suspended' | Stop-ASAppServiceInstance -Terminate
Get-ASAppServiceInstance -status 'suspended' | Remove-ASAppServiceInstance
Get-ASAppServiceInstance -InstanceId 95a25419-0d71-42c4-ab70-aa523ba603fc
Get-ASAppServiceInstance -InstanceId 95a25419-0d71-42c4-ab70-aa523ba603fc | Suspend-ASAppServiceInstance

Yet another alternative is to query the AppFabric persistance store database directly using SQL:

SELECT *
FROM [AppFabric_Application_Extensions].[System.Activities.DurableInstancing].[InstancesTable]
WHERE IsSuspended=1

Choose the alternative most to your liking 🙂

/Emil

Working with cultureinfo in PowerShell

As a simple example of working with culture information in PowerShell, here’s how to list Swedish monthnames:

(new-object system.globalization.cultureinfo("sv-SE")).DateTimeFormat.MonthNames

The result is an array:

januari
februari
mars
april
maj
juni
juli
augusti
september
oktober
november
december

Try doing that in a BAT file… 🙂

/Emil

Powershell script to kill long-running processes

Here’s a small script to kill all Word or Excel-processes that have been running for more than 5 minutes. This can be useful on a server that launches those applications to generate documents. You probably shouldn’t do it that way but if you do you’re likely to discover that the processes don’t always terminate as they should. In those situations, a script like this can come in handy.

$startTimeLimit = (get-date) - (new-timespan -minutes 5)
$processes_to_kill = get-process | 
    where {$_.StartTime -lt $startTimeLimit -and ($_.path -like "*winword.exe" -or $_.path -like "*excel.exe") }
if ($processes_to_kill -ne $null)
{
    $processes_to_kill | foreach { $_.Kill() }
}

/Emil