This article shows how to implement a full text search in ASP.NET Core using Azure Cognitive Search. The search results are returned using paging and the search index can be created, deleted from an ASP.NET Core Razor Page UI.
Code: https://github.com/damienbod/AspNetCoreAzureSearch
Posts in this series
- Implement a full text search using Azure Cognitive Search in ASP.NET Core
- Using Azure Cognitive Search Suggesters in ASP.NET Core and Autocomplete
- Implement a Blazor full text search using Azure Cognitive Search
History
2021-08-20 Updated packages and search
Creating the Search in the Azure Portal
In the Azure Portal, search for Azure Cognitive Search and create a new search service. Create the search using the portal wizard and choose the correct pricing model as required. The free version supports three indexes but does not support managed identities. This is good for exploring, evaluating the service
If using the free version, you will need to use API keys to access the search service. This can be found in the Keys blade of the created cognitive search.
Of course the Azure Cognitive Search could also be created using Azure CLI, Arm templates or Powershell. The service can also be created direct from code.
Create an Azure Cognitive Search index
In the ASP.NET Core Razor page application, the Azure.Search.Documents nuget package is used to create and search the Azure Cognitive search service. Add this to your project.
The index and the document field definitions can be created in different ways. We will use attributes and add these to the document search class properties to define the fields of the documents.
public class PersonCity
{
[SimpleField(IsFilterable = true, IsKey = true)]
public string Id { get; set; }
[SearchableField(IsFilterable = true, IsSortable = true)]
public string Name { get; set; }
[SearchableField(IsFilterable = true, IsSortable = true)]
public string FamilyName { get; set; }
[SearchableField(IsFilterable = true, IsSortable = true)]
public string Info { get; set; }
[SearchableField(IsFilterable = true, IsSortable = true, IsFacetable = true)]
public string CityCountry { get; set; }
[SearchableField(IsFilterable = true, IsSortable = true, IsFacetable = true)]
public string Metadata { get; set; }
[SearchableField(IsFilterable = true, IsSortable = true)]
public string Web { get; set; }
[SearchableField(IsFilterable = true, IsSortable = true)]
public string Github { get; set; }
[SearchableField(IsFilterable = true, IsSortable = true)]
public string Twitter { get; set; }
[SearchableField(IsFilterable = true, IsSortable = true)]
public string Mvp { get; set; }
}
A SearchProvider class was created to managed and search the indexes and the documents. The SearchProvider was added to the DI in the startup class. The configuration secrets for the Azure search was added to the user secrets of the project. A SearchIndexClient and a SearchClient instance was created using the configurations for your service. The SearchIndexClient can the be used to create a new instance using the CreateIndexAsync method and the field builder which uses the attribute definitions.
public class SearchProviderIndex
{
private readonly SearchIndexClient _searchIndexClient;
private readonly SearchClient _searchClient;
private readonly IConfiguration _configuration;
private readonly IHttpClientFactory _httpClientFactory;
private readonly string _index;
public SearchProviderIndex(IConfiguration configuration, IHttpClientFactory httpClientFactory)
{
_configuration = configuration;
_httpClientFactory = httpClientFactory;
_index = configuration["PersonCitiesIndexName"];
Uri serviceEndpoint = new Uri(configuration["PersonCitiesSearchUri"]);
AzureKeyCredential credential = new AzureKeyCredential(configuration["PersonCitiesSearchApiKey"]);
_searchIndexClient = new SearchIndexClient(serviceEndpoint, credential);
_searchClient = new SearchClient(serviceEndpoint, _index, credential);
}
public async Task CreateIndex()
{
FieldBuilder bulder = new FieldBuilder();
var definition = new SearchIndex(_index, bulder.Build(typeof(PersonCity)));
definition.Suggesters.Add(new SearchSuggester(
"personSg", new string[] { "Name", "FamilyName", "Info", "CityCountry" }
));
await _searchIndexClient.CreateIndexAsync(definition).ConfigureAwait(false);
}
public async Task DeleteIndex()
{
await _searchIndexClient.DeleteIndexAsync(_index).ConfigureAwait(false);
}
public async Task<(bool Exists, long DocumentCount)> GetIndexStatus()
{
try
{
var httpClient = _httpClientFactory.CreateClient();
httpClient.DefaultRequestHeaders.CacheControl = new CacheControlHeaderValue
{
NoCache = true,
};
httpClient.DefaultRequestHeaders.Add("api-key", _configuration["PersonCitiesSearchApiKey"]);
var uri = $"{_configuration["PersonCitiesSearchUri"]}/indexes/{_index}/docs/$count?api-version=2020-06-30";
var data = await httpClient.GetAsync(uri).ConfigureAwait(false);
if (data.StatusCode == System.Net.HttpStatusCode.NotFound)
{
return (false, 0);
}
var payload = await data.Content.ReadAsStringAsync().ConfigureAwait(false);
return (true, int.Parse(payload));
}
catch
{
return (false, 0);
}
}
public async Task AddDocumentsToIndex(List<PersonCity> personCities)
{
var batch = IndexDocumentsBatch.Upload(personCities);
await _searchClient.IndexDocumentsAsync(batch).ConfigureAwait(false);
}
}
Once created, VS Code with the Azure Cognitive Search extension can be used to view the index with the created fields. This can also be viewed, managed in the Azure portal.
https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-azurecognitivesearch
Adding search documents
Now that the index exists, it needs some documents so that we can search. Azure Cognitive Search provides many powerful ways of importing data into the search indexes, this is one of its strengths. In this demo, we added some data from a data helper class and uploaded the documents in a batch. This could be the way to add data if using the search as a secondary search engine for your solution.
public async Task AddDocumentsToIndex(List<PersonCity> personCities)
{
var batch = IndexDocumentsBatch.Upload(personCities);
await _searchClient.IndexDocumentsAsync(batch).ConfigureAwait(false);
}
The ASP.NET Razor Search Admin page provides a post method OnPostAddDataAsync to add the index documents.
public async Task<ActionResult> OnPostAddDataAsync()
{
try
{
PersonCityData.CreateTestData();
await _searchProviderIndex.AddDocumentsToIndex(PersonCityData.Data).ConfigureAwait(false);
Messages = new[] {
new AlertViewModel("success", "Documented added", "The Azure Search documents were uploaded! The Document Count takes n seconds to update!"),
};
var indexStatus = await _searchProviderIndex.GetIndexStatus().ConfigureAwait(false);
IndexExists = indexStatus.Exists;
DocumentCount = indexStatus.DocumentCount;
return Page();
}
catch (Exception ex)
{
Messages = new[] {
new AlertViewModel("danger", "Error adding documents", ex.Message),
};
return Page();
}
}
The view uses a Bootstrap 4 card to display this and documents can be added to the index.
@page
@model SearchAdminModel
@{
ViewData["Title"] = "SearchAdmin";
}
<div class="jumbotron jumbotron-fluid">
<div class="container">
<h1 class="display-4">Index: @Model.IndexName</h1>
<p class="lead">Exists: <span class="badge badge-secondary">@Model.IndexExists</span> Documents Count: <span class="badge badge-light">@Model.DocumentCount</span> </p>
</div>
</div>
<div class="card-deck">
<div class="card">
<div class="card-body">
<h5 class="card-title">Create index: @Model.IndexName</h5>
<p class="card-text">Click to create a new index in Azure Cognitive search called @Model.IndexName.</p>
</div>
<div class="card-footer text-center">
<form asp-page="/SearchAdmin" asp-page-handler="CreateIndex">
<button type="submit" class="btn btn-primary col-sm-6">
Create
</button>
</form>
</div>
</div>
<div class="card">
<div class="card-body">
<h5 class="card-title">Add Documents to index: @Model.IndexName</h5>
<p class="card-text">Add documents to the Azure Cognitive search index: @Model.IndexName.</p>
</div>
<div class="card-footer text-center">
<form asp-page="/SearchAdmin" asp-page-handler="AddData">
<button type="submit" class="btn btn-primary col-sm-6">
Add
</button>
</form>
</div>
</div>
<div class="card">
<div class="card-body">
<h5 class="card-title">Delete index: @Model.IndexName</h5>
<p class="card-text">Delete Azure Cognitive search index: @Model.IndexName.</p>
</div>
<div class="card-footer text-center">
<form asp-page="/SearchAdmin" asp-page-handler="DeleteIndex">
<button type="submit" class="btn btn-danger col-sm-6">
Delete
</button>
</form>
</div>
</div>
</div>
<br />
@if (Model.Messages != null)
{
@foreach (var msg in Model.Messages)
{
<div class="alert alert-@msg.AlertType alert-dismissible fade show" role="alert">
<strong>@msg.AlertTitle</strong> @msg.AlertMessage
<button type="button" class="close" data-dismiss="alert" aria-label="Close">
<span aria-hidden="true">×</span>
</button>
</div>
}
}
Checking the status of the index
In the ASP.NET Core search administration Razor Page view, we would like to be able to see if the index exists and how many documents exist in the index. The easiest way to do this, is to use the REST API from the Azure search service. The HttpClient is used and the count is returned or a 404.
{
try
{
var httpClient = _httpClientFactory.CreateClient();
httpClient.DefaultRequestHeaders.CacheControl = new CacheControlHeaderValue
{
NoCache = true,
};
httpClient.DefaultRequestHeaders.Add("api-key", _configuration["PersonCitiesSearchApiKey"]);
var uri = $"{_configuration["PersonCitiesSearchUri"]}/indexes/{_index}/docs/$count?api-version=2020-06-30";
var data = await httpClient.GetAsync(uri).ConfigureAwait(false);
if (data.StatusCode == System.Net.HttpStatusCode.NotFound)
{
return (false, 0);
}
var payload = await data.Content.ReadAsStringAsync().ConfigureAwait(false);
return (true, int.Parse(payload));
}
catch
{
return (false, 0);
}
}
When the application is started, the search admin displays the amount of documents, can create or delete the index and add documents to the index.
Implementing a search with Paging
The search is implemented using the QueryPagingFull method which uses the SearchAsync method. The QueryType is set to SearchQueryType.Full in the options so that we can use a fuzzy search. The page size and the range for the paging is defined at the top. The SearchAsync returns a SearchResults object which contains the results. This can then be used as required.
public async Task QueryPagingFull(SearchData model, int page, int leftMostPage)
{
var pageSize = 4;
var maxPageRange = 7;
var pageRangeDelta = maxPageRange - pageSize;
var options = new SearchOptions
{
Skip = page * pageSize,
Size = pageSize,
IncludeTotalCount = true,
QueryType = SearchQueryType.Full
}; // options.Select.Add("Name"); // add this explicitly if all fields are not required
model.PersonCities = await _searchClient.SearchAsync<PersonCity>(model.SearchText, options).ConfigureAwait(false);
model.PageCount = ((int)model.PersonCities.TotalCount + pageSize - 1) / pageSize;
model.CurrentPage = page;
if (page == 0)
{
leftMostPage = 0;
}
else if (page <= leftMostPage)
{
leftMostPage = Math.Max(page - pageRangeDelta, 0);
}
else if (page >= leftMostPage + maxPageRange - 1)
{
leftMostPage = Math.Min(page - pageRangeDelta, model.PageCount - maxPageRange);
}
model.LeftMostPage = leftMostPage;
model.PageRange = Math.Min(model.PageCount - leftMostPage, maxPageRange);
}
The Razor Page uses the SearchProvider and sets up the models so the view can display the data and call the search APIs.
public class SearchModel : PageModel
{
private readonly SearchProviderPaging _searchProvider;
private readonly ILogger<IndexModel> _logger;
public string SearchText { get; set; }
public int CurrentPage { get; set; }
public int PageCount { get; set; }
public int LeftMostPage { get; set; }
public int PageRange { get; set; }
public string Paging { get; set; }
public int PageNo { get; set; }
public SearchResults<PersonCity> PersonCities;
public SearchModel(SearchProviderPaging searchProvider,
ILogger<IndexModel> logger)
{
_searchProvider = searchProvider;
_logger = logger;
}
public void OnGet()
{
}
public async Task<ActionResult> OnGetInitAsync(string searchText)
{
SearchData model = new SearchData
{
SearchText = searchText
};
await _searchProvider.QueryPagingFull(model, 0, 0).ConfigureAwait(false);
SearchText = model.SearchText;
CurrentPage = model.CurrentPage;
PageCount = model.PageCount;
LeftMostPage = model.LeftMostPage;
PageRange = model.PageRange;
Paging = model.Paging;
PersonCities = model.PersonCities;
return Page();
}
public async Task<ActionResult> OnGetPagingAsync(SearchData model)
{
int page;
switch (model.Paging)
{
case "prev":
page = PageNo - 1;
break;
case "next":
page = PageNo + 1;
break;
default:
page = int.Parse(model.Paging);
break;
}
int leftMostPage = LeftMostPage;
await _searchProvider.QueryPagingFull(model, page, leftMostPage).ConfigureAwait(false);
PageNo = page;
SearchText = model.SearchText;
CurrentPage = model.CurrentPage;
PageCount = model.PageCount;
LeftMostPage = model.LeftMostPage;
PageRange = model.PageRange;
Paging = model.Paging;
PersonCities = model.PersonCities;
return Page();
}
}
The view uses Bootstrap 4 and displays the results. All requests are sent using HTTP GET which are cached and can be navigated using the back button. The searchText is added to the query string and also the handler required for the Razor page.
@page "{handler?}"
@model SearchModel
@{
ViewData["Title"] = "Search with Paging";
}
<form asp-page="/Search" asp-page-handler="Init" method="get">
<div class="searchBoxForm">
@Html.TextBoxFor(m => m.SearchText, new { @class = "searchBox" })
<input class="searchBoxSubmit" type="submit" value="">
</div>
</form>
@if (Model.PersonCities != null)
{
<p class="sampleText">
Found @Model.PersonCities.TotalCount Documents
</p>
var results = Model.PersonCities.GetResults().ToList();
@for (var i = 0; i < results.Count; i++)
{
<div>
<b><span><a href="@results[i].Document.Web">@results[i].Document.Name @results[i].Document.FamilyName</a>: @results[i].Document.CityCountry</span></b>
@if (!string.IsNullOrEmpty(results[i].Document.Twitter))
{
<a href="@results[i].Document.Twitter"><img src="/images/socialTwitter.png" /></a>
}
@if (!string.IsNullOrEmpty(results[i].Document.Github))
{
<a href="@results[i].Document.Github"><img src="/images/github.png" /></a>
}
@if (!string.IsNullOrEmpty(results[i].Document.Mvp))
{
<a href="@results[i].Document.Mvp"><img src="/images/mvp.png" width="24" /></a>
}
<br />
<em><span>@results[i].Document.Metadata</span></em><br />
@Html.TextArea($"desc{1}", results[i].Document.Info, new { @class = "infotext" })
<br />
</div>
}
}
@if (Model != null && Model.PageCount > 1)
{
<table>
<tr>
<td>
@if (Model.CurrentPage > 0)
{
<p class="pageButton">
<a href="/Search?handler=Paging&paging=0&SearchText=@Model.SearchText">|<</a>
</p>
}
else
{
<p class="pageButtonDisabled">|<</p>
}
</td>
<td>
@if (Model.CurrentPage > 0)
{
<p class="pageButton">
<a href="/Search?handler=Paging&paging=prev&SearchText=@Model.SearchText"><</a>
</p>
}
else
{
<p class="pageButtonDisabled"><</p>
}
</td>
@for (var pn = Model.LeftMostPage; pn < Model.LeftMostPage + Model.PageRange; pn++)
{
<td>
@if (Model.CurrentPage == pn)
{
<p class="pageSelected">@(pn + 1)</p>
}
else
{
<p class="pageButton">
@{var p1 = Model.PageCount - 1;}
<a href="/Search?handler=Paging&paging=@pn&SearchText=@Model.SearchText">@(pn + 1)</a>
</p>
}
</td>
}
<td>
@if (Model.CurrentPage < Model.PageCount - 1)
{
<p class="pageButton">
@{var p1 = Model.PageCount - 1;}
<a href="/Search?handler=Paging&paging=next&SearchText=@Model.SearchText">></a>
</p>
}
else
{
<p class="pageButtonDisabled">></p>
}
</td>
<td>
@if (Model.CurrentPage < Model.PageCount - 1)
{
<p class="pageButton">
@{var p7 = Model.PageCount - 1;}
<a href="/Search?handler=Paging&paging=@p7&SearchText=@Model.SearchText">>|</a>
</p>
}
else
{
<p class="pageButtonDisabled">>|</p>
}
</td>
</tr>
</table>
}
Searching, Fuzzy Search
The search can be used by entering a text and clicking the search icon. The results and paging are returned as defined. Ten results were found for “Switzerland” using a full word match.
If you spell the required word incorrectly or leave out a letter, no results will be returned.
This can improved or allowed by using a fuzzy search. The “~” can be added to use a fuzzy search for Azure Cognitive Search. Then the ten results will be found again. Azure Cognitive search supports different types of search queries, search filters and the indexes can be created to support different search types.
Notes
The demo here was built using the Azure search samples found here.
Links
https://docs.microsoft.com/en-us/azure/search
https://docs.microsoft.com/en-us/azure/search/search-what-is-azure-search
https://docs.microsoft.com/en-us/rest/api/searchservice/
https://github.com/Azure-Samples/azure-search-dotnet-samples/
https://channel9.msdn.com/Shows/AI-Show/Azure-Cognitive-Search-Deep-Dive-with-Debug-Sessions
https://channel9.msdn.com/Shows/AI-Show/Azure-Cognitive-Search-Whats-new-in-security
Software Engineering 
[…] Implement a full text search using Azure Cognitive Search in ASP.NET Core (Damien Bowden) […]