Shazwazza

Letting AI agents loose on Examine

Fri, 03 Jul 2026 17:45:20 Z

I've been maintaining Examine for a very long time now. It's the search and indexing library that sits underneath every Umbraco site, and like any project that's been around long enough, it has plenty of hot paths that could be faster, a steady trickle of issues that need triaging, and a backlog of "I'll get to that eventually" performance work that, well… I never actually get to.

So a couple of months ago I thought I'd try an experiment - what if I stopped doing all of that myself and instead let a bunch of AI agents do it for me, on a schedule, while I got on with everything else? This post is about how that went. I've shipped a few releases off the back of it and some of the hot-path improvements are genuinely a bit ridiculous.

So what are these "agentic workflows"?

The thing I installed is GitHub Next's Agentic Workflows - a set of markdown-defined workflows that run in GitHub Actions and drive an AI agent to do actual maintenance work on your repo. They're not chat bots. They run on a schedule, they have persistent memory, they open real pull requests, and they leave the "should this ship?" decision to me.

I ended up with a few of them running on Examine:

Perf Improver - runs daily, hunts for performance bottlenecks, writes benchmarks to actually prove the improvement, and opens a draft PR with the before/after numbers.
Efficiency Improver - its scrappier sibling, focused on the smaller allocation-and-LINQ-state-machine wins that add up over time.
Daily Issue Triage - goes through untriaged issues, sets types, applies labels, spots duplicates, and leaves a tidy triage report for me.
Agentic Maintenance - keeps the whole setup ticking along.

Each one is just a markdown file in .github/workflows/ with a description, a schedule, some safe-outputs limits (things like "you may open at most 4 PRs per run, and they must be drafts") and a big prompt describing how to behave. That's it. The nice part is that the guardrails are declarative, so the agent can't merge its own PRs, it can't touch protected files, and it can only comment so many times per run. All of that is baked in.

How it actually works day to day

The bit that makes this more than a gimmick is the persistent memory. Every run, the Perf Improver reads its own notes - which build/test/benchmark commands it validated, what's on its optimisation backlog, what it worked on last time, and which suggestions I've already ticked off. Then it does a couple of tasks in a round-robin fashion so it's not endlessly poking at the same corner of the codebase.

It also keeps a single rolling "Monthly Activity" issue open with a checklist of what needs my attention. So my side of it is pretty simple: the agent opens a draft PR with measured before/after numbers, I read it and run CI, and if I'm happy with it I merge. That's the whole loop. I'm the reviewer, the agent is the workhorse doing the grind I never had time for.

So what did they actually get done?

Over roughly the last two months, here's what these workflows actually got up to on Examine:

124 successful Perf Improver runs and 9 successful Efficiency Improver runs
20 successful Daily Issue Triage runs quietly keeping the issue tracker tidy
24 performance/efficiency PRs reviewed and merged (13 from Perf Improver, 11 from Efficiency Improver) between late May and the end of June
And the bit that actually matters - three releases shipped off this work: v3.8.0, v3.9.0 and the v4.0.0-beta.7 pre-release

That last point is really the whole thing. This isn't a pile of speculative branches rotting in a fork somewhere - it's code that went through my review, passed CI, and is now sitting in NuGet packages that real Umbraco sites are running.

Show me the numbers

Right, this is the part I actually get excited about. One of my favourite things about the Perf Improver is that its prompt tells it to only attempt improvements it can actually measure - establish a baseline first, make the change, then measure again and document both numbers. For the algorithmic hot-path stuff that means benchmarks, so along the way it built out a proper BenchmarkDotNet suite that compares the current source against the published NuGet packages (3.0.1 through 3.3.0), which means I can show you real, reproducible deltas instead of hand waving.

The full-text search hot path

ManagedQuery is the primary full-text search entry point in Examine - it's what runs on basically every search. The agent noticed it had no dedicated benchmark, wrote one, and then stacked up a series of small, individually measured changes: a volatile factory cache in SearchContext.GetFieldValueType, an early return in the extract-terms check, and killing off some redundant ConcurrentDictionary lookups in AddDocument.

Here's the current source vs the most recent 3.3.0 release, on a 1,000-document index:

Method	Version	Mean	Allocated
ManagedQueryAllFields	3.3.0	11.42 ms	1,323 KB
ManagedQueryAllFields	Source	2.17 ms	371 KB

That's about 5.3x faster and roughly 3.6x less memory allocated on the single most travelled code path in the whole library. On the busiest thing Examine does. I'll happily take that.

Building queries

GroupedAnd / GroupedOr / GroupedNot are the workhorses of the query builder. A little string[] fast-path (skipping a defensive .ToArray() copy when the caller already handed it a string[]) plus some allocation trimming got this:

Method	Version	Mean	Allocated
CreateQueryOnly	3.3.0	3,995 ns	8.34 KB
CreateQueryOnly	Source	319 ns	2.20 KB
GroupedAndStringArray	3.3.0	21,377 ns	21.10 KB
GroupedAndStringArray	Source	16,659 ns	14.34 KB

The CreateQuery() baseline dropping from 8.34 KB down to 2.2 KB is a lovely little win, and the grouped clauses are about 25% faster with a third of the allocation shaved off.

Constructing a ValueSet

Every single document you index goes through a ValueSet constructor. The old path allocated an intermediate dictionary and a generator state machine per field, which is exactly the kind of thing you don't notice until you're bulk indexing a big site. The agent got rid of both:

Method	Version	Mean	Allocated
FromDictionary5Fields	3.3.0	1,183 ns	2,200 B
FromDictionary5Fields	Source	226 ns	592 B
FromDictionary20Fields	3.3.0	4,007 ns	6,544 B
FromDictionary20Fields	Source	654 ns	1,520 B

That's roughly 5-6x faster and about 4x less allocated on indexing, and when you're rebuilding the index on a large Umbraco site that adds up fast.

The really nice touch is that these benchmark result tables now live in <remarks> doc-comments right next to the benchmark code, so the numbers are versioned in the repo alongside the thing they measure. The agent did that bit too.

Was it actually worth it?

For me the thing that makes it work is the measurement. Everything comes to me as a small, focused, draft PR with the numbers attached, so I can look at it, sanity check it, run CI, and decide in a couple of minutes. It's not zero effort - I still read every change before it goes anywhere near main - but the ratio is fantastic. I'm getting a steady stream of well-measured, single-purpose performance PRs on a library I care about, on paths I'd genuinely never have found the time to optimise by hand, and I've shipped real releases because of it. For a project I maintain around everything else, that's a pretty great deal.

If you maintain a repo with a backlog you never get to, especially performance work that needs benchmarks to justify it, I'd recommend giving GitHub Next's agentics a go.

You can see all of it out in the open on the Examine repo - the [perf-improver] and [efficiency-improver] PRs, the benchmark suite, and the releases they fed into. And there's a nice bonus here for me too: ExamineX, my managed, cloud-hosted Examine search offering, runs on this exact same internal plumbing. So all this work the agents have been doing to tighten up Examine's query and indexing hot paths feeds straight through into ExamineX - the underlying engine gets faster and leaner, and every ExamineX site gets those wins for free without changing a thing. If you'd rather have your search running as a managed service instead of hosting Lucene indexes on your own servers, that's what ExamineX is there for. 🙂

An Examine fix for Umbraco index corruption

Wed, 31 Jul 2024 17:49:54 Z

A new Examine version 3.3.0 has been released to address a long awaited bug fix for Umbraco websites that use the SyncedFileSystemDirectoryFactory which is the default setting for Umbraco CMS.

The bug typically means that indexes cannot be used and log entries such as Lucene.Net.Index.CorruptIndexException: invalid deletion count: 2 vs docCount=1 are present.

Understanding the problem:

The SyncedFileSystemDirectoryFactory directory is there to avoid performance implications of rebuilding indexes on startup when a site is moved to another worker in Azure. The reason an index rebuild would occur is because in Azure, Lucene files need to work off of the local 'fast drive' (C:\%temp%), not the default/shared network 'slow drive' (D:), and whenever a site is moved, or spawned on a new worker in Azure, the local 'fast drive' is empty, meaning no indexes exist. The SyncedFileSystemDirectoryFactory attempts to work around this challenge by continually synchronizing a copy of the indexes from the 'fast drive' to the 'slow drive' so that when a site is moved to another worker, it can sync (restore) from the 'slow drive' back to the 'fast drive' in order to avoid the index rebuild overhead.

The problem with SyncedFileSystemDirectoryFactory is that this implementation doesn't take into account what happens if the index files in your main storage ('slow drive') become corrupted which can happen for a number of reasons - misconfiguration, network latency, process termination, etc...

Understanding the solution:

The SyncedFileSystemDirectoryFactory has been updated to:

Check the health of the main index if it exists ('slow drive').
Check the health of the local index if it exists ('fast drive').
If the main index is unhealthy or doesn't exist and the local index is healthy, it will synchronize the local index to the main index. This can occur only if a site hasn't moved to a new worker.
If the main index is unhealthy and the local index doesn't exist or is unhealthy, then it will delete the main (corrupted) index.
Once health checks are done, the index from main is always synced to local. If the main index was deleted due to corruption, this will mean that the local index is empty and an index rebuild will occur.

This change will attempt to keep any healthy index that is available (main vs local), but if nothing can be read, the indexes will be deleted and an index rebuild will occur.

There's also a new option to fix a corrupted index but this is not enabled by default since it can mean a loss of documents.

Understanding the rebuilding overhead

The performance overhead of index rebuilding is due to the Umbraco database queries that need to be executed in order to populate the indexes. The only reason SyncedFileSystemDirectoryFactory exists is to prevent this overhead when hosting in Azure App Service (which is what Umbraco Cloud uses), and it can only be used on your Umbraco primary node. It does not prevent index rebuilding overhead for non-primary nodes when load balancing or scaling out because the main network 'slow drive' is shared between all workers and an index can only be read/written to be a single process.

This means that it's only useful if you are hosting in Azure App Service without any load balancing while keeping in mind that it does not always prevent index rebuilds (see above).

The index rebuilding overhead can be dramatic when load balancing or scaling out, for example: If you scale out to +5 nodes in a load balancing setup, that means that 5x nodes will be performing index rebuilds around the same time, this means that your DB is going to get hammered by queries to build all of those new indexes. The performance hit isn't the index building - it is the DB queries and this can lead to DB locks and lead to the dreaded SQL Lock Timeout issue in the Umbraco back office. Plus, if search is critical to your front-end, than for a while after your site has started up, there won't be any index which means there won't be any search until the background processing is done.

... Many of these reasons is why ExamineX was created:

Reliably and centrally persisted indexes means nothing is out of sync between nodes.
No index rebuilding when your site is moved or scaled = no rebuilding overhead.
Prevents SQL Timeout locks due to DB rebuilding queries.
Very easy to setup and seamlessly changes your Lucene based indexes to Azure/Elastic search indexes.
Ideal when hosting Umbraco on Azure Web Apps (or Umbraco Cloud) and a perfect solution for load balancing and scaling.
Automatically index Umbraco media file content without the need for additional indexes with support for PDFs, Microsoft Office documents and more.
(coming soon) Automatically generate Umbraco media image descriptions, tags, locations, and more using AI allowing your editors to quickly find the media/images they need.

Configuring a Suggester with ExamineX and Azure AI Search

Wed, 01 May 2024 14:51:52 Z

We recently had a customer ask about integrating the Azure AI Search suggester API with ExamineX and this turns out to be quite straight forward :)

First thing is to ensure that the fields you want to use the Suggester for are configured correctly:

// Configuration for the External Index for the City and Country fields.
// The Azure AI Search Suggester requires fields to be 'string fields only'
// and using the Standard Analyzer (default on the External Index)
services.Configure<AzureSearchIndexOptions>(
    Constants.UmbracoIndexes.ExternalIndexName,
    options =>
    {
         options.FieldDefinitions = new FieldDefinitionCollection(
            new FieldDefinition("City", AzureSearchFieldDefinitionTypes.FullText),
            new FieldDefinition("Country", AzureSearchFieldDefinitionTypes.FulText));
    });

Next up, is to create the Suggester which is done as part of the CreatingOrUpdatingIndex event:

externalIndex.CreatingOrUpdatingIndex += AzureIndex_CreatingOrUpdatingIndex_ScoringProfile;

private void AzureIndex_CreatingOrUpdatingIndex_Suggester(object sender, CreatingOrUpdatingIndexEventArgs e)
{
    switch (e.EventType)
    {
        case IndexModifiedEventType.Creating:
        case IndexModifiedEventType.Rebuilding:
            var index = e.AzureSearchIndexDefinition;

            var suggester = new SearchSuggester(
                "sg",
                new[] { "Country", "City" });

            index.Suggesters.Add(suggester);

            break;
    }
}

When data has been indexed for these fields, using the Suggester API is simple. For example, lets assume that the following ValueSets were indexed:

externalIndex.IndexItems(new[]
{
    ValueSet.FromObject(id1, new { City = "Calgary", Country = "Canada" }),
    ValueSet.FromObject(id2, new { City = "Sydney", Country = "Australia" }),
    ValueSet.FromObject(id3, new { City = "Copenhagen", Country = "Denmark" }),
    ValueSet.FromObject(id4, new { City = "Vienna", Country = "Austria" }),
});

Then to get suggestions for "Au":

// cast to AzureSearchIndex to expose underlying Azure Search APIs
var azureSearchIndex = (AzureSearchIndex)externalIndex;
var result = await  azureSearchIndex.IndexClient.SuggestAsync<ExamineDocument>(
    "Au",
    "sg");

The result will contain 2x entries, one for the Australia and one for the Austria documents. Super easy!

One thing to watch out for is if your index is not configured to use the Standard Analyzer by default (i.e. like the Internal Index in Umbraco). In that case, you'll need to define a custom field type with that indexer:

services.Configure<AzureSearchIndexOptions>(
    Constants.UmbracoIndexes.ExternalIndexName,
    options =>
    {
        // This adds a custom field type that uses the Standard Analyzer
        options.IndexValueTypesFactory = new Dictionary<string, IAzureSearchFieldValueTypeFactory>
        {
            ["standard"] = new AzureSearchFieldValueTypeFactory(s =>
                new AzureSearchFieldValueType(s, SearchFieldDataType.String, LexicalAnalyzerName.StandardLucene.ToString()))
        };

        // Set these field types to the one defined above
        options.FieldDefinitions = new FieldDefinitionCollection(
            new FieldDefinition("City", "standard"),
            new FieldDefinition("Country", "standard"));
    });

Happy Searching!

Searching with IPublishedContentQuery in Umbraco

Thu, 23 Mar 2023 15:10:02 Z

I recently realized that I don’t think Umbraco’s APIs on IPublishedContentQuery are documented so hopefully this post may inspire some docs to be written or at least guide some folks on some functionality they may not know about.

A long while back even in Umbraco v7 UmbracoHelper was split into different components and UmbracoHelper just wrapped these. One of these components was called ITypedPublishedContentQuery and in v8 is now called IPublishedContentQuery, and this component is responsible for executing queries for content and media on the front-end in razor templates. In v8 a lot of methods were removed or obsoleted from UmbracoHelper so that it wasn’t one gigantic object and tries to steer developers to use these sub components directly instead. For example if you try to access UmbracoHelper.ContentQuery you’ll see that has been deprecated saying:

Inject and use an instance of IPublishedContentQuery in the constructor for using it in classes or get it from Current.PublishedContentQuery in views

and the UmbracoHelper.Search methods from v7 have been removed and now only exist on IPublishedContentQuery.

There are API docs for IPublishedContentQuery which are a bit helpful, at least will tell you what all available methods and parameters are. The main one’s I wanted to point out are the Search methods.

Strongly typed search responses

When you use Examine directly to search you will get an Examine ISearchResults object back which is more or less raw data. It’s possible to work with that data but most people want to work with some strongly typed data and at the very least in Umbraco with IPublishedContent. That is pretty much what IPublishedContentQuery.Search methods are solving. Each of these methods will return an IEnumerable<PublishedSearchResult> and each PublishedSearchResult contains an IPublishedContent instance along with a Score value. A quick example in razor:

@inherits Umbraco.Web.Mvc.UmbracoViewPage
@using Current = Umbraco.Web.Composing.Current;
@{
    var search = Current.PublishedContentQuery.Search(Request.QueryString["query"]);
}

<div>
    <h3>Search Results</h3>
    <ul>
        @foreach (var result in search)
        {
            <li>
                Id: @result.Content.Id
                <br/>
                Name: @result.Content.Name
                <br />
                Score: @result.Score
            </li>
        }
    </ul>
</div>

The ordering of this search is by Score so the highest score is first. This makes searching very easy while the underlying mechanism is still Examine. The IPublishedContentQuery.Search methods make working with the results a bit nicer.

Paging results

You may have noticed that there’s a few overloads and optional parameters to these search methods too. 2 of the overloads support paging parameters and these take care of all of the quirks with Lucene paging for you. I wrote a previous post about paging with Examine and you need to make sure you do that correctly else you’ll end up iterating over possibly tons of search results which can have performance problems. To expand on the above example with paging is super easy:

@inherits Umbraco.Web.Mvc.UmbracoViewPage
@using Current = Umbraco.Web.Composing.Current;
@{
    var pageSize = 10;
    var pageIndex = int.Parse(Request.QueryString["page"]);
    var search = Current.PublishedContentQuery.Search(
        Request.QueryString["query"],
        pageIndex * pageSize,   // skip
        pageSize,               // take
        out var totalRecords);
}

<div>
    <h3>Search Results</h3>
    <ul>
        @foreach (var result in search)
        {
            <li>
                Id: @result.Content.Id
                <br/>
                Name: @result.Content.Name
                <br />
                Score: @result.Score
            </li>
        }
    </ul>
</div>

Simple search with cultures

Another optional parameter you might have noticed is the culture parameter. The docs state this about the culture parameter:

When the culture is not specified or is *, all cultures are searched. To search for only invariant documents and fields use null. When searching on a specific culture, all culture specific fields are searched for the provided culture and all invariant fields for all documents. While enumerating results, the ambient culture is changed to be the searched culture.

What this is saying is that if you aren’t using culture variants in Umbraco then don’t worry about it. But if you are, you will also generally not have to worry about it either! What?! By default the simple Search method will use the “ambient” (aka ‘Current’) culture to search and return data. So if you are currently browsing your “fr-FR” culture site this method will automatically only search for your data in your French culture but will also search on any invariant (non-culture) data. And as a bonus, the IPublishedContent returned also uses this ambient culture so any values you retrieve from the content item without specifying the culture will just be the ambient/default culture.

So why is there a “culture” parameter? It’s just there in case you want to search on a specific culture instead of relying on the ambient/current one.

Search with IQueryExecutor

IQueryExecutor is the resulting object created when creating a query with the Examine fluent API. This means you can build up any complex Examine query you want, even with raw Lucene, and then pass this query to one of the IPublishedContentQuery.Search overloads and you’ll get all the goodness of the above queries. There’s also paging overloads with IQueryExecutor too. To further expand on the above example:

@inherits Umbraco.Web.Mvc.UmbracoViewPage
@using Current = Umbraco.Web.Composing.Current;
@{
    // Get the external index with error checking
    if (ExamineManager.Instance.TryGetIndex(
        Constants.UmbracoIndexes.ExternalIndexName, out var index))
    {
        throw new InvalidOperationException(
            $"No index found with name {Constants.UmbracoIndexes.ExternalIndexName}");
    }

    // build an Examine query
    var query = index.GetSearcher().CreateQuery()
        .GroupedOr(new [] { "pageTitle", "pageContent"},
            Request.QueryString["query"].MultipleCharacterWildcard());


    var pageSize = 10;
    var pageIndex = int.Parse(Request.QueryString["page"]);
    var search = Current.PublishedContentQuery.Search(
        query,                  // pass the examine query in!
        pageIndex * pageSize,   // skip
        pageSize,               // take
        out var totalRecords);
}

<div>
    <h3>Search Results</h3>
    <ul>
        @foreach (var result in search)
        {
            <li>
                Id: @result.Content.Id
                <br/>
                Name: @result.Content.Name
                <br />
                Score: @result.Score
            </li>
        }
    </ul>
</div>

The base interface of the fluent parts of Examine’s queries are IQueryExecutor so you can just pass in your query to the method and it will work.

Recap

The IPublishedContentQuery.Search overloads are listed in the API docs, they are:

Search(String term, String culture, String indexName)
Search(String term, Int32 skip, Int32 take, out Int64 totalRecords, String culture, String indexName)
Search(IQueryExecutor query)
Search(IQueryExecutor query, Int32 skip, Int32 take, out Int64 totalRecords)

Should you always use this instead of using Examine directly? As always it just depends on what you are doing. If you need a ton of flexibility with your search results than maybe you want to use Examine’s search results directly but if you want simple and quick access to IPublishedContent results, then these methods will work great.

Does this all work with ExamineX ? Absolutely!! One of the best parts of ExamineX is that it’s completely seamless. ExamineX is just an index implementation of Examine itself so all Examine APIs and therefore all Umbraco APIs that use Examine will ‘just work’.