0 Comments

Part 1 – General Overview & Account Structure.
Part 2 – Setting up the Azure and Visual Studio Online (VSO) structure and maintenance.
Part 3 – Setting up the Azure Environments with ARM and PowerShell.
Part 4 – Setting up Build Servers.
Part 5 – Setting up Continuous delivery (CD) on CI Development builds.
Part 6 - Setting up Release Manager for Dev nightly, QA, Stage and Prod deployments.

Part 5 - Setting up Continuous delivery (CD) on CI Development builds.

Now we have a build server (see part 4) we can now setup our CI build and our Continuous deployment build to the development server. We have several options on how to do this:

- use Azure websites inbuilt “Continuous Delivery” from VSO. This is great for simply deployments, but as we start adding more complex features to our enterprise apps this is unlikely to suffice going forward.

- use VSO Resource Group deployment functionality from the vNext CI build. Unfortunately this is not an option available to all of us, see Abrish's comments on this MSDN article, hence until this is fully released we need something else.

- User VNext for CI and use XAML build and Release Manager for CD on a rolling build basis. We will be using Release Manager throughout the rest of our pipeline to production, although not ideal due to the performance overhead, especially for CD, this is the only real option available.

We chose to implement the last point as we felt it offered us a gentle way into the new build system while still giving us full compatibility with the functionality of Release Manager we required. To do this we first require to setup a vNext CI build from VSO. To do this:

1). First open up the VSO web interface and navigate to the “Build” tab. Click on the green plus button to create a new build template. Choose visual studio and click ok.

image

2). You should now see the New Build definition page. As this is for CI purposes only, we can delete the “Index sources…” and “Publish Build Artifacts” steps from the default template.

image

image

3). In the “Visual Studio Build” step select the “…” button next to the Solution and navigate to your Dev branch and in turn select the .sln (solution file). I also prefer my builds to be clean for every build, so I enabled the checkbox “Clean”

image

4). Now select the Visual Studio Test step and expand Advanced options. To better encourage best practice check the checkbox “Code Coverage Enabled”. If your unit test projects are in the same solution as your other projects you should not need to change the default “Test Assembly” location.

image

5). Now click on the “Repository” tab and change the cloak location for your build folder. In my case this is a folder inside the dev branch.

image

6). Now select the Triggers tab and enable “Continuous Integration (CI)”

image

7). Now select the “General” tab and change the Default Queue to the Build Server group we setup in blog post Part 4.

image

8). We are now good to go, click the Save button, give it a name, and check in some changes to test. Any issues please make sure you do not have NuGet packages checked in with your solution and that the “Restore NuGet Packages” checkbox is enabled on the Visual Studio Build step.

image

Summary:

We have now setup our CI build for our project, but still need a way of Continuously Deploying it to the given Azure Environment. In the next article I will discuss how this can be achieved currently with Release Manager 2015 on VSO while we wait for Release vNext to be released by Microsoft.

0 Comments

Part 1 – General Overview & Account Structure.
Part 2 – Setting up the Azure and Visual Studio Online (VSO) structure and maintenance.
Part 3 – Setting up the Azure Environments with ARM and PowerShell.
Part 4 – Setting up Build Servers.
Part 5 – Setting up Continuous delivery (CD) on CI Development builds.
Part 6 - Setting up Release Manager for Dev nightly, QA, Stage and Prod deployments.

Part 4 – Setting up Build Servers.

Now we have an azure environment, we need some way to build our assets before deploying them out there. With VSO we currently have 2x2 options. 2x2? yes, see the below grid and note a mix and match from each column.

TemplatesControllers
VNextHosted
XamlOur own VM

VSO and TFS2015 introduced new build templates that are night and day better than the old xaml ones, for an overview of why see Colin Dembovsky blog post. While Xaml builds are still currently supported it would appear they finally seem to be headed for the retirement home. The other 2 options we have with VSO is whether to use the hosted build controllers provided by VSO or provision our own VMs for building what we have defined in our templates (whether that is xaml or vNext). After some extensive testing, that is a blog post on its own, we came to the conclusion that at present the hosted build controllers are simply too slow for anything more than a couple of developers (20 times slower is some cases) – since this is an article on moving Enterprise to the cloud, our dev numbers are likely to be in the 10’s to 100’s. Due to the pricing model of “build minutes”, and just how dramatically slow the hosted builds are, the hosted build controllers on any non trivial project are equally as expensive as provisioning the VMs yourself.

Setting up the Build Server:

Luckily a Microsoft engineer, Thiago Almeida, has made setting up a build server and attaching it to VSO incredibly straightforward by providing an ARM template on his GitHub repository. Before using this though we have to setup Alternate Authentication Credentials on our VSO account so our build agent can access our VSO account.

1). Go over to your VSO account and sign in as a service account (a generic user you have setup for this purpose) with admin rights across the full VSO collection.

image

1.1). Click your name /the service accounts name and select my profile

image

1.2). In my profile select the security tab, then “Alternate Authentication Credentials”.

image

1.3). At the present time the build controller setup will not work with Personal Access Tokens, hence override this warning below and check the “Enable Alternate Authentication Credentials”. Now enter a username and password and click “Save”.

image

Setup Agent Pool (optional)

2). You can create a custom group that holds your build agents together. This can be useful if you plan on creating build agents in azure and on premise.

2.1). To do this go to the Agent Pool Admin panel at https://VSOUSERNAME.visualstudio.com/_admin/_AgentPool and select “New Pool”

image

2.2). Enter a pool name and click ok, note down this name for later.

Deploying the Build VM

3). Open up the PowerShell IDE on your desktop and run the following commands. This will first prompt you to log in, then display a list of all subscriptions your account has access to, and then selects the appropriate subscriptions that we wish to run our script in, before finally switching PowerShell over to the new mode.


Add-AzureAccount
Get-AzureSubscription
Select-AzureSubscription -SubscriptionName "Developer Tools"
Switch-AzureMode AzureResourceManager

3.1). We can now execute the below line of PowerShell which will use the given template provided by Thiago and deploy the assets into a resource group named “BuildServers”.


New-AzureResourceGroupDeployment -Name BuildServerSetup -ResourceGroupName BuildServers -TemplateUri https://raw.githubusercontent.com/azure/azure-quickstart-templates/master/visual-studio-vsobuildagent-vm/azuredeploy.json -vmVisualStudioVersion VS-2015-Enterprise-AzureSDK-2.7-WS2012R2

PowerShell will now prompt you for all parameters required by the script. Note: it will take many minutes for Azure to spin this environment up. Pay careful attention to the parameter guidelinesover here

image

Here is what I provided:

ParamValue
StorageNamecompanybuildserverstorage
deployLocation“North Europe”
vmNameCompanyBuildServer01
vmAdminUserNameadmin
vmAdminPassword***********
vmIPPublicDnsNamecompanybuildserver01
vsoAccountcompany
vsoUserThe user created above
vsoPass*********
poolName“Default”

Tip: use the following to clear down the resource group if you require multiple runs of the above.

Get-AzureResourceGroup BuildServers | ForEach-Object { Remove-AzureResource -ResourceId $_.ResourceId -Force}

3.2). An alternative to 3.1 above, is to hit the button below which will take you into the azure portal to deploy the ARM script manually. I would recommend 3.1 though; getting used to PowerShell, if you are not already, will make your life on Azure much easier.

3.3). You can now go back over to the Agent Pool admin page and expand the Agent pool you created, or the default pool if you did not. If the script was successful you should now see your new Build Server under the agent pool. If you don’t see this make sure you are part of the Team Foundation Server administrators group and try again.

Deploying a XAML Build Agent

Unfortunately in order for us to use Release Manager online, as of Aug/Sep 2015, we will require our builds to be XAML based and as such we can’t fully switch over to the nice new “Team Build vNExt” system. As such we must configure our VM we created at point 3 with a XAML build agent.

4). Firstly remote desktop (RDP) onto the build server you just created and download TFS 2015 from MSDN here.

4.1). Now run the TFS 2015 setup and when prompted select “Configure XAML Build Service” and select “Start Wizard”.

image

4.2). Select your feed back participation options on the next screen and click next.

image

4.3). You will now be asked to configure which TFS instance (VSO in our case). Select Browse, then servers, then add and enter your VSO address in the format https://YOURID.visualstudio.com

image

4.4). Click ok and you should be asked to sign into your live account, use and id with admin rights, and click login. If you are successful you should see the below screen.

image

4.5). Click on Close and then “Connect”.

4.6). You will now be taken back to the build service configuration screen, now click next.

image

4.7). On the next screen you will be prompted to select the amount of “Build Agents” you wish to place on this server. TFS best practice says 1 agent per core available on the CPU, but since this is azure and we can scale when we need, override the recommended and select “4” and click next.

image

4.8). You will now be prompted to select the account to run the agents as. You may wish to setup a custom user in your AD, but using the system account Local service should suffice in most cases. Select this and click on next.

image

4.9). Click on next again on the “review” screen, and the installer will run some checks. If all is successful you will be displayed with the below screen, click “Configure”.

image

4.10). Everything being ok you should now be presented with the below screen. You can now exit the wizard, a XAML build server should now be visible to you through the XAML Build templates created through Visual Studio.

image

Summary:

In this post we have setup a VM in Azure, setup a vNext build controller on that VM, and then setup a XAML build agent on that same VM and made them all available inside VSO to accept builds. In the next post I will show how we setup the build templates for the project.

0 Comments

Part 1 – General Overview & Account Structure.
Part 2 – Setting up the Azure and Visual Studio Online (VSO) structure and maintenance.
Part 3 – Setting up the Azure Environments with ARM and PowerShell.
Part 4 – Setting up Build Servers.
Part 5 – Setting up Continuous delivery (CD) on CI Development builds.
Part 6 - Setting up Release Manager for Dev nightly, QA, Stage and Prod deployments.

Part 3 – Setting up the Azure Environments with ARM and PowerShell for a single project.

A key component of continuous deployment is automating the creation and configuration of infrastructure. Having a reliable state to deploy to, in every environment, and removing manual work between builds that change infrastructure configuration is key to having short release cycles to production.

At the current time of writing, Azure is going through a gradual migration from the “Old” Azure portal to the new “preview” portal. As such, the way of administering different asset types varies greatly meaning you will ultimately be jumping between portals often. Powershell has been, until now, the preferred way of automating asset creation and tear down in Azure. However, as Microsoft have moved assets to the new “preview” portal they have opened up a new way of continuously deploying and configuring assets to Azure in the form of Azure Resource Manager (ARM) scripts. ARM script are a declarative JSON description of what assets and configuration you want to deploy to a given resource group. The scripts we build below will be reused later when we come to setup Release Manager.

1). Open the PowerShell IDE, and run the below commands 1 by 1 to switch to the correct azure mode (the PowerShell equivalent of new vs. old portal), log into your account, list all subscriptions in your account and select the correct subscription

Switch-AzureMode AzureResourceManager

Add-AzureAccount

Get-AzureSubscription

Select-AzureSubscription -SubscriptionName "Department1 Development"

2). You now need to build your ARM script. This will vary project by project as to which azure asset types you require, but for now we will script a simple Web app. See the MSDN documentation over here on authoring ARM scripts from scratch. Alternatively there is a bunch of pre-made templates that you may find useful over here on Github. Visual Studio 2015 now has some inbuilt tooling for the creation of ARM scripts, but the tooling is early. Alternatively there is a web based ARM explorer over here. None the less the VS inbuilt tooling is a good starting point, so we will make use of it. See this blog post on MSDN for more details.

Tip: The preview portal uses ARM itself by default.If you hit problems with the ARM Script, open the Preview portal and pull up the dev tools in your browser (or Fiddler). If you create the assets manually you will see, after hitting the create button, the ARM request going out. This can be handy for reverse engineering.

2.1). Open VS2015 and open the project you checked into VSO earlier. Now add a new project to the solution, selecting Cloud then “Azure Resource Group”. Give you project a name and select OK.

image

2.2). On the next prompt select “Web App”

image

That will create the project structure below. Note the 3 folders, scripts, templates and tools. Open the templates folder and you will see 2 files, WebSite.Json and WebSite.param.dev.json. The first file contains the json description of our assets, while the param file contains input parameters that will be passed in at runtime (it is this file we will replace later when we use Release Manager). Now open the WebSite.json file, and don’t panic, it’s not as bad as it first looks. Note the toolbox on the left hand side, this will help you edit the json structure by adding in and building up the template you require; for now keep this simple with the default configuration. By default the WebApp has application insights added and performance alters, feel free to delete these if you do not require them. Personally I leave these in. Application insights is a cross of elmah error logging and Google analytics on steroids for all cross platform things in web, mobile app and cloud.

image

2.3). Open the param file and add some values to the siteName, hostingPlanName and siteLocation. These values will be replaced later when we configure Release Manager but this will get us up and running for now. SiteName and hosting plan name can be called anything you like (as long as it is unique and meets the naming restrictions for azure assets), while siteLocation should be the Azure datacenter closes to you – in my case North Europe. Note I have also added the SKU, at the top of our json file this is declared as an optional param with a default value.

image

2.4). Open the Deploy-AzureResourceGroup.ps1 file contained in the Scripts folder. Right Click the file and select “open with PowerShell IDE”. Currently, Aug 2015, there is a bug with Release Manager 2015 that means we are unable to run PowerShell Strict mode 2.0. You will receive the following error in your release manager build log if you do not perform this step “System.Management.Automation.PropertyNotFoundException: The property 'Length' cannot be found on this object. Verify that the property exists.”

You will hence need to update the PowerShell strict mode as below.

Set-StrictMode -Version 1.0

Tip: Release manager will always pass $UploadArtifacts as false, if you wish you can remove all the upload logic from the default script. Release Manager takes care of the upload for us so no need to have this as part of our script.

2.5). Execute this script by running the below command in powershell. Again it is this we will use later in Release Manager, it is worth sticking with PowerShell and not being tempted to run the JSON script through the Azure portal.

**PathToFile**\Deploy-AzureResourceGroup.ps1 -ResourceGroupLocation "North Europe"

Tip: if you require to clear down the resource group run Get-AzureResourceGroup _ResourceGroup_Dev_name | ForEach-Object { Remove-AzureResource -ResourceId $_.ResourceId -Force}

And that’s it, you should now have a dev environment in Azure… yes this is a simple example, but starting simple gives us a platform to build from.

Next up is configuring the Build servers so we can get our builds out onto the environment.