In this blog, we are going to take an introductory look at Azure Cognitive Search and then give you a real-world scenario of speeding up searches using this Azure capability at Ballard Chalmers.
What is Azure Cognitive Search?
Azure Cognitive Search is a cloud search service that gives developers infrastructure, APIs, and tools for building a rich search experience over private, heterogeneous content in web, mobile, and enterprise applications.
Architecturally, a search service sits between the external data stores that contain your un-indexed data, and your client app that sends query requests to a search index and handles the response.
In this article, we are going to create a service, ingest and perform a search from a simple Azure Cognitive Search and I’m going to show you a real-world scenario from our work at Ballard Chalmers.
Creating a simple Azure Cognitive Search Service
Creating your service in Azure
In the Azure portal go to create resources and look for “Azure Cognitive Search” then press “Create”.
You will need to select your subscription, Resource Group, Service Name, Location and Pricing tier (note the Pricing Tier I selected is “Free”).
Review and create your service.
Ingesting Data
To access our service we will need the API Key which we can get from the Azure portal. Note that we have two types of keys, admin keys have more privileges and should be used only to create and delete, and the query key should be used to read data.
To ingest data in our service we are going to use Visual Studio. Create a console app with .NET 5 and import the NuGet Package SDK for Azure Cognitive search called “Azure.Search.Documents”.
We will need to define our Model first, I am using a Hotel as an example.
public class Hotel
{
[SimpleField(IsFilterable = true, IsKey = true)]
public string HotelId { get; set; }
[SearchableField(IsSortable = true)]
public string HotelName { get; set; }
[SearchableField(IsFilterable = true, IsSortable = true, IsFacetable = true)]
public string Category { get; set; }
[SimpleField(IsFilterable = true, IsSortable = true, IsFacetable = true)]
public bool? ParkingIncluded { get; set; }
}
We will need to create an index in the Azure Search to ingest our data:
static async Task Main(string[] args)
{
string serviceName = "search-sample"; //the name you have given on azure
string indexName = "sample-index";
string adminAPIKey = "XXXXXXXXXXXXXXXX"; //Your Admin API Key goes here
Uri serviceEndpoint = new Uri($"https://{serviceName}.search.windows.net/");
AzureKeyCredential credential = new AzureKeyCredential(adminAPIKey);
SearchIndexClient adminClient = new SearchIndexClient(serviceEndpoint,credential);
await CreateIndexAsync(indexName, adminClient);
}
private static async Task CreateIndexAsync(string indexName, SearchIndexClient adminClient)
{
var fieldBuilder = new FieldBuilder();
var searchFields = fieldBuilder.Build(typeof(Hotel));
var definition = new SearchIndex(indexName, searchFields);
await adminClient.CreateOrUpdateIndexAsync(definition);
}
Now we are going to ingest data in our service:
static async Task Main(string[] args)
{
string serviceName = "search-sample"; //the name you have given on azure
string indexName = "sample-index";
string adminAPIKey = "XXXXXXXXXXXXXXXX"; //Your Admin API Key goes here
Uri serviceEndpoint = new Uri($"https://{serviceName}.search.windows.net/");
AzureKeyCredential credential = new AzureKeyCredential(adminAPIKey);
SearchIndexClient adminClient = new SearchIndexClient(serviceEndpoint,credential);
SearchClient ingestClient = adminClient.GetSearchClient(indexName);
await UploadDocumentsAsync(ingestClient);
}
private static async Task UploadDocumentsAsync(SearchClient ingestClient)
{
IndexDocumentsBatch<Hotel> batch = IndexDocumentsBatch.Create(
IndexDocumentsAction.Upload(
new Hotel
{
HotelId = "1",
HotelName = "The Grand Hotel NY",
Category = "5 Stars",
ParkingIncluded = true
}),
IndexDocumentsAction.Upload(
new Hotel
{
HotelId = "2",
HotelName = "Prince of Walles Hotel 5",
Category = "2 Stars",
ParkingIncluded = false
}),
IndexDocumentsAction.Upload(
new Hotel
{
HotelId = "3",
HotelName = "Boulevar Hotel",
Category = "4 Stars",
ParkingIncluded = true
}),
IndexDocumentsAction.Upload(
new Hotel
{
HotelId = "4",
HotelName = "The Mayfair Townhouse",
Category = "3 Stars",
ParkingIncluded = true
})
);
await ingestClient.IndexDocumentsAsync(batch);
}
Searching Data
After ingesting our test data, we will be able to perform searches in our cognitive service. In this sample, we are going to demo a simple search, but Azure Cognitive Search provides us with a powerful tool where we can filter and do a lot more advanced stuff that you can look at in the documentation if you are interested.
In the following code, I am showing the simple search by text. I have requested to the service the text “5”, it runs a search for all properties in the hotel model and brings us the results. Note that we have “5” in just two hotels but one is in the Name Property and the other is in the Category Property.
static async Task Main(string[] args)
{
string serviceName = "search-sample"; //the name you have given on azure
string indexName = "sample-index";
Uri serviceEndpoint = new Uri($"https://{serviceName}.search.windows.net/");
string queryAPIKey = "XXXXXXXXXXXXXXXXXXXX"; //Your query API key goes here
AzureKeyCredential queryCredential = new AzureKeyCredential(queryAPIKey);
SearchIndexClient queryClient = new SearchIndexClient(serviceEndpoint, queryCredential);
SearchClient searchClient = queryClient.GetSearchClient(indexName);
await SearchDocumentsAsync(searchClient, "5");
}
public static async Task SearchDocumentsAsync(SearchClient searchClient, string searchText)
{
SearchResults<Hotel> results = await searchClient.SearchAsync<Hotel>(searchText);
foreach (SearchResult<Hotel> result in results.GetResults())
{
Console.WriteLine($"{result.Document.HotelName} - {result.Document.Category}");
}
}
Output
Azure Cognitive Search Real World Scenario
At Ballard Chalmers, we are using Azure Cognitive Search to solve problems with big data and slow responses. Using Azure Cognitive Search we can build a search much faster than a DB search. It is easier to implement and has features which are complex to implement.
Problem: Excel file with about 250k rows from Scottish Assessors Association with a valuation for rating.
Solution: We imported the file to Azure Cognitive Search and created a user-friendly page consuming all features from Azure.
Queries
Azure allows us to perform complex queries. In the sample, we are looking for ”Hospital Aberdeen”.
It looks for all properties containing both words and brings us the top 10 in less than 200 milliseconds. If we were to do this via a DB query it would take longer to implement, and the query would be slow due to the number of properties and rows we need to check.
Search Suggestions
Suggesters are typeahead or “search-as-you-type”, Azure Cognitive Search brings us the first items that match the start of the word being typed by the user.
We have implemented this feature to allow the user to quickly navigate through the record been searched.
Facets Navigation
Faceted navigation is used for self-directed drill-down filtering on query results in a search app, where your application offers form controls for scoping search to groups of documents. For example in the image below, Billing Authority, Current RV or Group VO, and Azure Cognitive Search provide the data structures and filters to back the experience.
When the user selects a Current RV range to filter, it sends it to Azure Cognitive search where it filters it out and returns the values matching the filter in less than 200 milliseconds.
Conclusion
Azure Cognitive Search is recommended when the client needs performance and has a large amount of data. When compared with a DB search, we can say it will improve performance in the client app because we don’t use the client app resources to perform the complex queries, it is all done by Azure Cognitive Search. Additionally, the development is easier to implement and the maintenance will be improved as well.
If you’d like advice on using Azure with your custom software development or are looking to outsource your development altogether, get in touch for a no-commitment initial consultation.
