This is Part Eight of an eight-part blog series looking into a serverless application built using Azure components. The earlier parts of the series are available here: Part One | Part Two | Part Three | Part Four | Part Five | Part Six | Part Seven
After many months, we are at the end of the modern serverless development series. As with any good continuous agile development project, this is a timely place to look at how the application is working, so that the usage can be fed into the requirements for future work.
So this post will cover the use of App Insights and Log Analytics to monitor across the stack of services that we have mentioned previously, such as Azure Functions, App Services with Angular and authentication with Azure B2C.
What is Azure Monitor (And Wasn’t It Called App Insights)?
As the world is changing faster than these posts are being written and released, the functionality of App Insights has now actually rolled into a broader service called Azure Monitor. This includes:
- Application Insights – an extensible platform for developers to be able to monitor and analyse their applications.
- Log Analytics – workspace for analysing data from across many sources.
Azure Monitor provides easy ways for developers to integrate hooks into their applications, capturing events and issues and then allowing them to be easily searchable through the logs.
For many core technologies such as NodeJS, .Net Core or Azure Functions and Logic Apps among others, there are built-in performance monitors so you can track how your application performs over time. As well as configure alerts when these performance metrics go above or below certain levels.
For example, our sample application should send an alert if Azure Function calls are taking more than 10 seconds to complete over a period of 10 seconds as that is a pretty good indicator that something is going wrong. We shouldn’t need to have users of the application contact us to say that something is wrong. This way we can be proactive and deal with it before it impacts the users.
One of the other great benefits is that Microsoft is using their experience with Machine Learning to recommend and send alerts when the performance is different from its usual values. So, if the Azure Function calls usually take 0.5 seconds and they start completing in 0.01 or 10 seconds then it is quite likely that something is wrong and Azure Monitor will automatically send an alert to the defined administrators.
This area is constantly evolving and improving with Microsoft aiming this addition to cause it to stand out from other similar services.
How Does It Work?
For App Insights, once the service has been created you need to set up the applications that need monitoring. This will depend on the service that you are using and I will focus on the services that are covered in the rest of the Modern Serverless Development series.
Creating an App Insights instance
To create a fresh App Insights instance, it is a simple case of navigating to the Azure Portal, clicking on New Resource and then searching for App Insights.
The details required are very simple with just the type of application (this sets-up some predetermined insights), the subscription, the Resource Group and data centre location needed.
Once the service itself is set up, you need to configure your application. You could create an Insights instance for each service in your application but there is a lot of benefit to keeping in a single instance so that you can write queries across all elements, e.g. see the total time taken by a click on the UI and then break it down to the time taken for the Azure Function and Cosmos DB data alongside the UI itself.
For all the services, you will need to copy out the Instrumentation Key that you can find from the Overview screen for App Insights.
Azure Functions
These are very easy to configure as the portal itself allows you to pick your App Insights instance. Alternately you can set the Instrumentation Key using the APPINSIGHTS_INSTRUMENTATIONKEY app setting. This latter method is particularly useful when deploying the application via ARM.
It is worth noting that this value will also work locally so you will need to either have separate instances and update the Instrumentation Key on deployment to each environment (my recommended approach) or have a set of filters in your App Insights to identify each environment.
Cosmos DB
Cosmos DB has no connection to App Insights but does work well with Log Analytics. You can set up the Diagnostic Settings so that Data Plane Requests and Query information is sent to Log Analytics where you can query the information and set up custom dashboards. There is more info on setting this up and using the logs to review the number of Resource Units (RUs) used by your queries in one of my earlier blogs.
Angular
For the Angular client, you can integrate using a third-party open source package which will capture:
- Page views – these will be defined by the routes since this is a Single Page Application
- Logged events – these are custom events defined in the code to allow diagnosis of issues
The set-up which has now been added to the Github project for the client was all derived from a great post by Cyprien from Geek Learning. Ultimately, I added the ApplicationInsights-js and @types/applicationinsights-js packages from npm and then included the MonitoringService.
Rather than changing the LogService throughout the project to use the Monitoring Service, I changed the LogService to log to the Monitoring Service as well as the console. This was also a good time to change any instances of console.log outside of the log service to make sure that all useful messages were captured.
There is certainly more that can be done with the client-side logging but this was enough to get useful diagnostics initially.
How Is It priced?
Further details can be found on the Azure Monitor Pricing page.
Log Analytics
The pricing is per GB processed and you will receive 5GB free at time of writing and then £2.229 per GB after that for Western Europe at the time of writing.
App Insights
The data is priced at the same value as Log Analytics with no further costs unless you require alerting.
What Sort of Thing Can I Do With It?
The first place to start is the default dashboard where you can see a summary of failed requests, response time, server requests and availability for the period selected (the last hour below). This gives you an overview of what is happening during that period. For each of these graphs, you can click to see further details.
If you click on Server Requests, you can see further details on the Requests made.
As you can see above, it will show the number over time but also the number made over a duration range so you can see any requests which are taking a much larger amount of time.
The Operation list shows you how many of each Function is being called and the average duration. You can also see the common properties on the right and click to view details on those specific calls.
Here you can see the real detail that can be essential for diagnosing issues. This will show the logs from the Azure Functions so you can capture the state within your code or surface exception easily.
These messages are useful but what if you wanted to see how often a specific message occurred for a specific Function?
From the main screen, you can navigate to Analytics and start writing your own queries using the Kusto Query Language. I do have a slightly mixed relationship with Kusto as it is very powerful but also a slightly strange mix between C# and SQL so you often get a little confused as to the right syntax. The good news is that it includes plenty of sample queries that you can use, either as they are or updated to meet your exact needs.
These queries can also be joined allowing you to get queries like the number of page views vs server response over time.
One of the other areas that I found useful during testing and handling issues is the Live Metrics that you can access from the main App Insights screen. This shows a snapshot of the live performance and also the live traces so you can easily get an instant view of the logs to diagnose error messages.
There is a huge amount to App Insights that you can spend hours delving into so I couldn’t cover all the areas here.
The browser and user sessions data is also useful to analyse how the application is being used from the end user perspective and to dig further into specific journeys. It is not quite up to the Google Analytics level but it’s very capable when used across the entire application.
What Issues Will I Face?
Time taken to get data
As a developer, I can often be impatient to get the data I need to diagnose issues. While the Live Metrics view is great for this, the other areas can take 5-10 minutes to get the latest data which can prove frustrating. For general analytics and reviewing the performance of the application, this is not an issue.
Good old GDPR
This needn’t be an issue but it should be a consideration.
Depending on what you are logging, you may need to declare the logs as a data repository as it may hold sensitive information. You will need to be aware of the data that is being logged and who has access. You may have strict access controls to segregate access to data in your application but if a wider number of users have access to the App Insights data then that will be pointless. Consider what information you have captured, who can access it and what they can do with it and you won’t have any issues.
Too many tools
While it is great to have so many things that you can do, it can be a little daunting to find exactly what you need. Keep your custom dashboard simple and just showing what you need, not every single thing that you can measure. Also, don’t panic when you see a slight blip. It may have little impact so investigate further before hitting panic stations.
Summary
Azure Monitor gives you a great, integrated set of metrics and log analytics that will help teams to keep the application running but also identify how the application is being used.
You can use the data to refine your application and put focus on areas that are not performing well. As well as those that are being heavily used and would offer better benefits to the users.
This also brings to a close the Modern Serverless Development series where I have covered the services and techniques that we have been using with many of our clients and sharing some of the pitfalls that we have encountered along the way.
It would be great to hear any feedback you have via Twitter. So please let us know if these have helped and if there are any other similar areas that you would like to hear more about. The source for the demo applications will remain on Github under our main page at https://github.com/ballardchalmers.
About the author
Kevin McDonnell is a respected Senior Technical Architect at Ballard Chalmers. With a Master of Engineering (MEng), Engineering Science degree from the University of Oxford he specialises in .NET & Azure development and has a broad understanding of the wider Microsoft stack. He listens to what clients are looking to achieve and helps identify the best platform and solution to deliver on that. Kevin regularly blogs on Digital Workplace topics and is a regular contributor to the monthly #CollabTalk discussions on Twitter.
