skip to Main Content

Protect your Connection Strings using Azure Key Vault

For as long as they have been around Connection strings to the database have needed protection from being used by someone who shouldn’t have access, be that hackers or a disgruntled employee who just left the business on bad terms. However, there are also other valid reasons for protecting them and that is to ensure only the services you want to connect to the data store actually connect and not some test POC App which accidentally drops a table or two.

We are careful to not store out connection strings in files that we commit to Git by storing them in a config file and adding that file to the git.ignore but that’s not really securing them so let’s look at a great way to secure those secrets.

Is there a better way?

Well actually there is and it’s the subject of this blog post and that is to store the secrets, and this includes things that are not just connection strings but any secret that you need to store and programmatically recover in Azure Key Vault.

Azure Key Vault which is, as its name suggests, a secure vault that can hold your secrets, it’s very simple to set up and is super cheap to use with me hosting this demo instance in UK South it’s going to cost per 10,000 transactions £0.023 / €0.023 / $0.023 (Notice Azure billing just change the currency symbol for the pricing here!). A transaction for Key Vault is any interaction with the Vault so any CRUD operation etc but you can read more here as well as confirm the pricing for your region.

Configuring Key Vault

This set-up could not be easier, head out to your Azure Portal and in the menu select ‘Create a Resource’ and search for ‘Key Vault’ and click create.

Create Key Vault

Fill in the details as highlighted and click ‘Create’, as a side note you can add Access Policy and Networking etc, but we will come back and deal with those later.

Regarding the recovery options, you just need to stop and think about your situation For me, as I am building this as a demo, I am disabling the recovery but if it was Prod, I would click Enable Purge Protection.

You can read the Docs to find out more.

Adding Something to Protect

Now we have an Azure Key Vault we need something to protect that we can call from our WebAPI, so how about an Azure Cosmos DB instance Connection String?

It’s a managed NoSQL database that has super-fast read/write from anywhere in the world and 99.999% availability. Best of all, it’s super cheap with a free amount of requests per month as well, but you can read the full detail on pricing at azure.microsoft.com/pricing/details/cosmos-db.

To add a Cosmos instance let’s head back to the Azure Portal and again select ‘Create a Resource’ and search for ‘Azure Cosmos DB’ and click create. You will then be shown a page that asks what API best suits, and this is fairly new but means you can choose what you know and love when dealing with your NoSQL Data. For me, I will pick Core(SQL) as I’m an SQL fan so can create my Queries using that syntax even though it’s a NoSQL DB.

What API best suits?

Once you have selected your API Option, fill in the form as you would normally do for a new resource with a Resource Group, an Account name and Location and then if you want a Provisioned throughput or if you prefer the serverless model for charges. This is outside the scope of this blog as we are dealing with secrets and Azure Key Vault but read the docs to find out which method will be best and most cost-effective for your setup docs.microsoft.com/azure/cosmos-db/throughput-serverless.

We don’t really need to change any of the other settings in the other tabs but feel free to have a look at them and then click the ‘Review + Create’ button at the bottom.

Azure Cosmos DB Account

Where is the secret key?

Now we have a data store. If we were to connect to this from say a WebAPI, mobile device or even some job that runs overnight processing, we would have many places that would need the Connection string. Now if we wished to regenerate the keys and thus change the connection string, we would have multiple places to update and then test which is not ideal, add in the Dev/QA/Prod teams and the list grows even more.

So let’s take the connection string and, like mentioned before, secure it by storing it in Azure Key Vault.

So, in the Cosmos DB blade of the Azure Portal, select ‘Keys’ in the menu under ‘Settings’ and then copy the ‘Primary Connection String’.Cosmos DB - Keys

Adding the Secret to the Vault

Now we have the secret we can add it to the Azure Key Vault. From there we can secure access to the secret so only the application we approve, and even the users we approve, can have access and view the secret. Again, we are storing a Connection string here, but it could be a secret password or setting value even a certificate we wish to secure access to, basically any value we deem a secret can be stored.

Head back to the portal and your Azure Key Vault and select ‘Secrets’ from the menu under ‘Settings’ and then click ‘Generate/Import’.

Azure Key Vault - Secrets

We can give our secret a name and enter the Connection String into the ‘Value’ entry box and ensure ‘Enabled’ is set to yes and we are good to go. However, while we are here on this page notice that we can also set an Activation and Expiration date, and this allows us to control the date and time the secret can be accessed which allows you to roll between secrets on say a monthly basis which helps with security.

Create a secret

Accessing the Secret

We have now created a secure secret in the Azure Key Vault which is great but how do we access this from our WebAPI? Well, let’s create a WebAPI and prove how easy this process is.

As before in the Azure portal ‘Create a Resource’ and search for ‘Web App’ (Yes, it’s a different name but we will create an API with it rather than a full App) and complete the details, I have opted for the new .NET 6 as the runtime and leaving it to auto-create me a new App Service Plan. Review the other Tabs as you might want to turn on the deployment settings to connect to your GitHub for CI/CD and you really do want to review the Monitoring Tab.

WARNING!

Unlike before where you can ignore the other tabs and click ‘Review + Create’ you might want to check the ‘Monitoring’ tab as it defaults to Application Insights being enabled and there is a cost for this that isn’t trivial and could blow through your Azure credits really quickly. You can read more about Pricing and the docs for App Insights and Azure Monitor so you can make that decision. It’s a fantastic tool for monitoring your application but I have seen many people caught out, with the default as on.

Create web app

Allow the WebAPI to connect to Azure Key Vault

Next, we need to tell Azure Key Vault to allow the WebAPI to access our secret. So to enable this, we first need to give our WebAPI an identity within Azure AD for your subscription, and we do this by opening the WebAPI App Service and selecting ‘Identity’ under Settings and for the ‘System Assigned’ turn on the status and click save.

WebAPI App Service - Identity

The WebApp now has a system-assigned identity which will mean the Application will be added to the Azure AD for you and you can now return to the Azure Key Vault blade in the portal and under the Settings menu select ‘Access Policies’ followed by ‘+ Add Access Policy’.

Access Policies

The new screen will show, and you can use the first dropdown to select ‘Secret Management’ as that is what we are doing and then I always check the ‘Secret Permissions’ drop down as it tends to add more permissions than I am happy to allow. If we think about this, the WebAPI is only ever going to read the Connection String so we can remove all the other permissions as this will reduce the attack vector even more so adding more security.

Add Access policy

Once you’re happy with the permissions, you need to select the ‘Principle’ you are giving the permission to, so clicking this will show you a selection box to the right of the screen and in here you can type the name you gave to your Web API/Web App. As this is for a demo it’s ‘BloggingWebApi’ as you start typing this, the list will filter so when you see it you can select it and click ‘Select’ at the bottom.

Principal

You will be taken back to the Access Policies page, and you will see that the new Principal of your WebAPI will be shown, and you will think like I have done way too many times that you are done and move on but until you click save at the top these changes will not take effect. BE WARNED!!! Click Save.

Access Policies

Creating a WebAPI

We have now set up Azure ready to host our WebAPI so we need to finally write some code within a new WebAPI that we can publish to Azure. I am a Windows and Visual Studio user so will focus on that, but this process is very similar on other IDE’s and also OS.

Open Visual Studio and ‘File New -> New Project’ to create a new ‘ASP.NET Core Web API’ and follow your normal flow when creating a new project with respect to folder location etc and I have selected .NET6 (LTS).

Once your project is created, we can add the required NuGet’s to the project and they are:

Microsoft.Azure.Services.AppAuthentication
Microsoft.Azure.KeyVault
Microsoft.Azure.Cosmos

Adding a Data Service

As is good practice, let’s add a Data Service that will connect to Cosmos DB using our secret from Azure Key Vault. I prefer to create services as a Class in a services folder and in this class, add the following code.

public static class DataService
{
    private static CosmosClient cosmosClient;

    private static async Task InitDBConnection()
    {
        var secretUri = “<< Insert SecretURI From KeyVault >>”;
        var keyVaultTokenProvider = new AzureServiceTokenProvider().KeyVaultTokenCallback;
        var keyVaultClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(keyVaultTokenProvider));

        var mySecret = await keyVaultClient.GetSecretAsync(secretUri);
        cosmosClient = new CosmosClient(mySecret.Value);
    }

    public static async Task<List<T>> Get<T>(string Query = “”)
    {
        try
        {
            var allItems = new List<T>();

            if (cosmosClient is null)
            {
                await InitDBConnection();
            }

            QueryRequestOptions options = new QueryRequestOptions() { MaxBufferedItemCount = 100 };

            var database = cosmosClient.GetDatabase(“ToDoList”);
            var container = database.GetContainer(“Items”);
            var queryText = $”SELECT * FROM c”;
            if (!string.IsNullOrEmpty(Query))
            {

                queryText += Query;
            }

            using (FeedIterator<EmptyDocument<T>> query = container.GetItemQueryIterator<EmptyDocument<T>>(

                queryText,
                requestOptions: options))
            {
                while (query.HasMoreResults)
                {
                    foreach (var item in await query.ReadNextAsync())
                    {
                        allItems.Add(item.Document);
                    }
                }
            }

            return allItems;

        }
        catch (Exception ex)
        {
            Debug.WriteLine($”ERROR – {ex.Message}”);
            return null;
        }
    }
}

You will notice that this Generic Service will take in a Query string but if one isn’t supplied, it will just return a complete list of the ‘Items’ in the ‘ToDoList’ database. However, you will notice that it returns a Generic List rather than a list of our ToDo Model (See below) and there is a very good reason.

Next, we need a model for our data and like all good demo projects, I’m going with a ToDo List. So, create a new Class in a ‘Models’ folder and add the following which is a simple class that will hold the Name of our ToDo item and some text along with a Due DateTime, super simple just what we need.

 public class ToDo
{
    public string ID { get; set; }
    public string TodoName { get; set; }
    public string TodoText { get; set; }
    public DateTime DueDate { get; set; }
}

Now that we have our Model, rather than just using this directly and then when the business comes back and asks us to add a Date/Time when the ToDO was created we have to modify the Model and then the database to add this extra column which could be a lot of work!

With Cosmos DB we can use a Document Model that will take our ToDo Model and add this as the Payload to an EmptyDocument, wrapping it with the required ID and PartitionKey such that we can store anything we wish in this document. We could even swap out the TODO Model live in A/B Tests etc.

This means we need another Class in our ‘Models’ folder called ‘EmptyDocument’, it has just a few lines of code:

public class EmptyDocument<T>
{
    [JsonProperty(“id”)]
    public string Id { get; set; } = “”;
    [JsonProperty(“document”)]
    public T Document { get; set; }
    public string PartitionKey { get; set; } = “”;
}

Simple Controller

With a DataService that will handle the calls out to Azure Key Vault to retrieve the Secret connection string and then make the calls out to Cosmos DB to collect the data we want, we just need a Controller in our API to make use of this service.

You can either add this to the default Weather API Controller or add a completely new controller, the choice is yours, but the code could not be simpler.

 // GET: api/<TodoController>
    [HttpGet]
    public async Task<string> Get()
    {
        return System.Text.Json.JsonSerializer.Serialize(await DataService.Get<ToDo>());
    }

Testing

Now as this post is about the Securing of the secrets and not about WebAPI’ s and Cosmos DB I am not going to labour the WebAPI parts anymore but just show it working.

You may have noticed that at the top of the DataService there is the ‘SercretURI’ placeholder, and you may be thinking to yourself we will be exposing a secret in our code. But as we have told Azure Key Vault to only allow access to the Secret from the WebAPI, this link is safe. Let’s prove that by first grabbing the link from Azure Key Vault to put into our code.

Head out to the Azure Key Vault blade of your Azure Portal and click the ‘Secrets’ under settings and then the name you gave your secret.So for me ‘OurCosmosDBConnectionString’

Secrets

You will then be shown the versions of this secret. Again, this shows the power of Azure Key Vault as not only can you have a Secret, but you can store multiple versions of this secret which means you can have Dev/Test/Prod versions set up ready to use. We only have one version here so click that.

Secret versions

Now you will see a box with the ‘Secret Identifier’ shown and it is this we need in our code, so copy that to the clipboard and paste it into the placeholder in our Data Service code.

Secret Identifier

While you still have that link in your clipboard try pasting it into a new browser tab and see what you get back…

We get an ‘Unauthorized’ message as we are not the WebAPI making the request, and this proves that we are safe to add this to our code as the access is controlled by Azure Key Vault.

We now have all the parts and we can test that we can access the Cosmos DB data by clicking F5 to run the code in Visual Studio, this will start a Swagger instance and from there we can test the Get method. If you haven’t used Swagger before then I strongly suggest you give it a try as it’s a great way to test API’s for all the standard CRUD operations and you can read more about it here.

As you can see, I added a ToDo Controller with all the CRUD operations, and we can select the ‘GET’ Operation ‘Try It Out’ and then ‘Execute’

Swagger

It will reach out to Azure Key Vault to grab the CosmosDB Secret Connection string and then connect to grab a list of the ToDo Entries which for this demo I have just the one… But Success it worked…

ToDo

Wait we are running locally

I know what you’re thinking, we are running locally so why can we get the secret from Key Vault? Well, there is a very clever system built into the KeyVaultClient we call in our DataService that will try using the Credentials of the Logged In user and as long as that User has the Owner or Service Admin for the Key Vault this will work. If for you this isn’t working, all you need to do is publish your WebAPI to the Azure App Service, we created earlier, and it will work or add yourself to the Azure Key Vault Principal list for this secret.

Conclusion

This is a rather long post with lots of Images and code blocks, but I hope you agree that for what could be a very confusing set-up until you have a play, this blog post walks you through the set-up showing you every step along the way, assuming you have no knowledge of Azure Key Vault. So I hope it helps you understand Azure Key Vault.

On all my client projects I use Azure Key Vault to secure anything I consider a secret, even items that need to be shared among team members for a project but are not needed within the code, this means we can control who has access and look at the logs for when they were last accessed etc. If a team member moves on, we can revoke access in Azure AD, and they can no longer view the secrets, so even if they take a copy of the code base with them the connection strings and other secrets are still safe.

I hope you enjoyed this walkthrough and feel free to ask questions or suggest other topics to the Ballard Chalmers team. And if you need further help, here at Ballard Chalmers, our Azure specialists are available for full custom software development, short or long term resourcing engagements or expert consultancy.

Post Terms: Azure Key Vault | connection strings | Security

About the Author

BC Technical Team

Our technical team contribute with blogs from their respective specialities, be that Azure, SQL, BizTalk, SharePoint, Xamarin and more. From the lead architect to developers and testers, each person provides content straight from their experience.

Back To Top