While Azure Resource Manager is not a unique tool to serverless, its ability to deploy a full environment at the click of a button and ensure that your environment stays true to the definition makes it a great addition to the toolbox. While also reducing the manual effort required to set-up your serverless application.
It is not without its fiddly parts, but it helps bring the notion of infrastructure much closer to the rest of the software development lifecycle than before. It also reduces the amount of time spent hunting down why one environment works when another doesn’t.
Compared with the days of waiting three months for a new server to be ordered, built, tested and then handed over before you could even think about getting going, it still feels like a welcome relief.
What Is Azure Resource Manager?
Azure Resource Manager (or ARM) is ultimately an API provided by Azure that allows you to create Azure-based resources through templates. This is commonly used by uploading the templates through the Azure Portal, running command line scripts using PowerShell or Bash or via a CI/CD process. The same templates can be used regardless of the method of executing and you can create parameters so that a single template can be used to create multiple environments, whether for a dev/test/live structure or to have separate environments for different clients.
The main benefits of this approach are: to ensure consistency between environments and reduce the manual effort involved in creating a new environment. Solution architects can define the environment which can then be deployed automatically or manually by support/Dev Ops teams without requiring access to live environments.
How Does It Work?
The templates themselves are JSON files broken down into various areas:
- Parameters – these can have default values but also receive a separate parameter file with environment specific details
- Variables – these are calculations on the parameters themselves e.g. if the environment is called BCDev then your web app may be bcdev.azurewebsites.net and the Cosmos database could be bcdev.documents.azure.com, then the parameter could just be BCDev and variables for the others can be used to allow re-use
- Functions – any user-defined functions can be created in this section although these are not required
- Resources – these are the items created in Azure themselves such as the web app, function, SQL database, app insights, storage etc
- Outputs – should you need to use some of the values created in the build process, (e.g. the app insights instrumentation key) then the value can be output and used in the next step
Building up the templates themselves is a pretty complex process to do from scratch but there are two things that make this much easier. There is a wide range of sample templates supplied by Microsoft that you can use as the basis for your application. They are great for seeing how they can be used and what options are available.
If you already have your environment set up, then another way of starting is to export your current environment as a template. From either a specific resource or a whole resource group (as in the image below), you can select Export Template (formerly called Automation Script) which provides an extract of the Template for that environment. This is similar to using Excel and recording a macro and the same risks should be considered. Much of the structure is not how you would necessarily use it and it will include every single possible parameter, not just those you have set. Many of these will be null so there will be plenty of tidying up needed.
Viewing the template by clicking on Export Template will show:
- The template itself as JSON
- A navigation pane for the template to view the resources
- Tab to define parameters
- Various tabs for applying the template to Azure using PowerShell, .Net, Ruby and Azure CLI
At the top, you may have noticed that there is a warning about resource types that cannot yet be exported. There are still some elements that cannot be included in the ARM templates and either need to be scripted with PowerShell/Azure CLI or even manually set up through the portal. In this case, it was around Azure B2C (which cannot be fully scripted at the time of writing) and the Bing Maps API.
The template itself is broken down into four main sections:
- Parameters – set of values to be passed to the template that include things like the name of resources or client keys and secrets
- Variables – these are calculated values that can then be used like parameters in the rest of the template, allowing you to supply the environment as a parameter and then have consistent names using that environment name e.g. BCServerlessDemoVaultDev, BCServerlessDemoVaultTest etc
- Resources – this is the meat of the template that creates the required resources
- Outputs – any properties that you need the value for, to be outputted to the scripts (useful for debugging or passing to subsequent scripts)
The resources allow plenty of options for creating the different services needed. As mentioned above, if you are new to ARM templates, create the service you need and then export that item itself, for example, below is the Azure Storage Account:
You can see that it has been defined as Standard LRS, it only supports Https traffic and has the Hot access tier. There are further details such as the network ACLs (access control lists) that are not as obvious and also some values such as scale that are set as null. You can either toggle these values or keep them as the exported version. You can even export each service individually so that you can make changes to one service and copy out the required config.
How Do I Run the Template?
There are several options for executing the templates.
Through the portal
You can select a new resource from the Azure Portal and search for Template Deployment. From here, you can either use one of the predefined templates or paste your own template created as above. You can also pass a separate parameters file for that environment.
Using command line and scripting tools
In the Export Template window mentioned above, there are details for running the templates using .Net, PowerShell, Ruby and the Azure CLI. These will connect using your credentials and apply the exported JSON templates and so you can build them into any existing processes you have.
Using Visual Studio
The final way, and my preferred method for integrating into an Azure DevOps pipeline is to use Visual Studio. This is included in the Azure deployment extension and again allows you to select from a set of predefined templates or your own by using a blank template. This provides intellisense for the resources and validation prior to running. It also includes the same navigation pane that you see in the Export Template screen in the Azure Portal.
From Visual Studio itself, you can choose to deploy from the solution explorer directly, which is great for testing. Once you have confirmed your template, you can check this into your Source Control and run the templates using a task in Azure DevOps, meaning that your full environment can be created and code deployed from a single deployment pipeline.
You can then configure the deployment, connecting to the required Resource Group, defining the template from your built artifacts and then either specifying a parameters file or setting the parameters directly in the Azure DevOps portal. This final option is ideal where you are separating the set-up of the configuration from the developers and ensuring that connection details and keys are never held in source control.
Is There Any Additional Cost?
To use the templates, there is no additional cost. The only cost will be for the utilisation of the services created. As many of the services are utilisation based, it means that you can recreate environments for testing with minimal cost as long as you delete them promptly afterwards.
What Issues Will I Face With It?
Using the exported templates is messy
One of the first issues you are likely to find is that the exported templates are not always that intelligent and create a lot of duplicates. For example, in the export I took earlier, it has included all the deployments as well!
This does not remove the usefulness of exporting templates but you will need to budget time to clean up, especially for larger Resource Groups.
Some services are still not included
You will still find services like Azure B2C and some of the Cognitive Services are not included in the export and others that cannot be created at all. The exported template is very clear on what these are, allowing you to define workarounds either with a script or with manual processes.
Linked resources can be fiddly
There are scenarios where you need to get keys or details from one service and include in another. For example, where you create an App Insights instance and need to pass the instance ID into Azure Functions for monitoring. It is possible using the ARM functions (an example is shown below) and using the output fields helps so that you can see what values are used. However, be prepared for lots of trial and error.
Not a huge amount of resources to solve issues
Initially, there was limited documentation, especially around some of the functions and retrieving properties for linked resources but this has improved greatly. The main site has plenty of examples and now the reference pages have details on the functions that you can use. However, there is not a large community using ARM templates and so make the most of the code samples available and useful people like Sam Cogan who have examples that they have shared.
Hard to set authentication for App Services
One specific issue that we faced was setting the authentication for an app service. The exported template captured the configuration but would not set the Azure AD authentication as the one to use. I found a useful page that included the setting for unauthenticated client action instead but there was a lot of sweat involved with getting to that point. This also leads back to the point above about the exported template not always being ideal so you do need to ensure that you test your templates well, not just for incremental runs.
Summary
ARM Templates are ideal for ensuring consistency across your environments and treating your infrastructure in the same way as you treat your code, building it into Continuous Integration/Continuous Deployment pipelines. There are some pitfalls to watch out for and you should be prepared to include scripted and occasionally manual workarounds but your overall application quality will be greatly improved.
About the author
Kevin McDonnell is a respected Senior Technical Architect at Ballard Chalmers. With a Master of Engineering (MEng), Engineering Science degree from the University of Oxford he specialises in .NET & Azure development and has a broad understanding of the wider Microsoft stack. He listens to what clients are looking to achieve and helps identify the best platform and solution to deliver on that. Kevin regularly blogs on Digital Workplace topics and is a regular contributor to the monthly #CollabTalk discussions on Twitter.
