Shazwazza

Letting AI agents loose on Examine

Fri, 03 Jul 2026 17:45:20 Z

I've been maintaining Examine for a very long time now. It's the search and indexing library that sits underneath every Umbraco site, and like any project that's been around long enough, it has plenty of hot paths that could be faster, a steady trickle of issues that need triaging, and a backlog of "I'll get to that eventually" performance work that, well… I never actually get to.

So a couple of months ago I thought I'd try an experiment - what if I stopped doing all of that myself and instead let a bunch of AI agents do it for me, on a schedule, while I got on with everything else? This post is about how that went. I've shipped a few releases off the back of it and some of the hot-path improvements are genuinely a bit ridiculous.

So what are these "agentic workflows"?

The thing I installed is GitHub Next's Agentic Workflows - a set of markdown-defined workflows that run in GitHub Actions and drive an AI agent to do actual maintenance work on your repo. They're not chat bots. They run on a schedule, they have persistent memory, they open real pull requests, and they leave the "should this ship?" decision to me.

I ended up with a few of them running on Examine:

Perf Improver - runs daily, hunts for performance bottlenecks, writes benchmarks to actually prove the improvement, and opens a draft PR with the before/after numbers.
Efficiency Improver - its scrappier sibling, focused on the smaller allocation-and-LINQ-state-machine wins that add up over time.
Daily Issue Triage - goes through untriaged issues, sets types, applies labels, spots duplicates, and leaves a tidy triage report for me.
Agentic Maintenance - keeps the whole setup ticking along.

Each one is just a markdown file in .github/workflows/ with a description, a schedule, some safe-outputs limits (things like "you may open at most 4 PRs per run, and they must be drafts") and a big prompt describing how to behave. That's it. The nice part is that the guardrails are declarative, so the agent can't merge its own PRs, it can't touch protected files, and it can only comment so many times per run. All of that is baked in.

How it actually works day to day

The bit that makes this more than a gimmick is the persistent memory. Every run, the Perf Improver reads its own notes - which build/test/benchmark commands it validated, what's on its optimisation backlog, what it worked on last time, and which suggestions I've already ticked off. Then it does a couple of tasks in a round-robin fashion so it's not endlessly poking at the same corner of the codebase.

It also keeps a single rolling "Monthly Activity" issue open with a checklist of what needs my attention. So my side of it is pretty simple: the agent opens a draft PR with measured before/after numbers, I read it and run CI, and if I'm happy with it I merge. That's the whole loop. I'm the reviewer, the agent is the workhorse doing the grind I never had time for.

So what did they actually get done?

Over roughly the last two months, here's what these workflows actually got up to on Examine:

124 successful Perf Improver runs and 9 successful Efficiency Improver runs
20 successful Daily Issue Triage runs quietly keeping the issue tracker tidy
24 performance/efficiency PRs reviewed and merged (13 from Perf Improver, 11 from Efficiency Improver) between late May and the end of June
And the bit that actually matters - three releases shipped off this work: v3.8.0, v3.9.0 and the v4.0.0-beta.7 pre-release

That last point is really the whole thing. This isn't a pile of speculative branches rotting in a fork somewhere - it's code that went through my review, passed CI, and is now sitting in NuGet packages that real Umbraco sites are running.

Show me the numbers

Right, this is the part I actually get excited about. One of my favourite things about the Perf Improver is that its prompt tells it to only attempt improvements it can actually measure - establish a baseline first, make the change, then measure again and document both numbers. For the algorithmic hot-path stuff that means benchmarks, so along the way it built out a proper BenchmarkDotNet suite that compares the current source against the published NuGet packages (3.0.1 through 3.3.0), which means I can show you real, reproducible deltas instead of hand waving.

The full-text search hot path

ManagedQuery is the primary full-text search entry point in Examine - it's what runs on basically every search. The agent noticed it had no dedicated benchmark, wrote one, and then stacked up a series of small, individually measured changes: a volatile factory cache in SearchContext.GetFieldValueType, an early return in the extract-terms check, and killing off some redundant ConcurrentDictionary lookups in AddDocument.

Here's the current source vs the most recent 3.3.0 release, on a 1,000-document index:

Method	Version	Mean	Allocated
ManagedQueryAllFields	3.3.0	11.42 ms	1,323 KB
ManagedQueryAllFields	Source	2.17 ms	371 KB

That's about 5.3x faster and roughly 3.6x less memory allocated on the single most travelled code path in the whole library. On the busiest thing Examine does. I'll happily take that.

Building queries

GroupedAnd / GroupedOr / GroupedNot are the workhorses of the query builder. A little string[] fast-path (skipping a defensive .ToArray() copy when the caller already handed it a string[]) plus some allocation trimming got this:

Method	Version	Mean	Allocated
CreateQueryOnly	3.3.0	3,995 ns	8.34 KB
CreateQueryOnly	Source	319 ns	2.20 KB
GroupedAndStringArray	3.3.0	21,377 ns	21.10 KB
GroupedAndStringArray	Source	16,659 ns	14.34 KB

The CreateQuery() baseline dropping from 8.34 KB down to 2.2 KB is a lovely little win, and the grouped clauses are about 25% faster with a third of the allocation shaved off.

Constructing a ValueSet

Every single document you index goes through a ValueSet constructor. The old path allocated an intermediate dictionary and a generator state machine per field, which is exactly the kind of thing you don't notice until you're bulk indexing a big site. The agent got rid of both:

Method	Version	Mean	Allocated
FromDictionary5Fields	3.3.0	1,183 ns	2,200 B
FromDictionary5Fields	Source	226 ns	592 B
FromDictionary20Fields	3.3.0	4,007 ns	6,544 B
FromDictionary20Fields	Source	654 ns	1,520 B

That's roughly 5-6x faster and about 4x less allocated on indexing, and when you're rebuilding the index on a large Umbraco site that adds up fast.

The really nice touch is that these benchmark result tables now live in <remarks> doc-comments right next to the benchmark code, so the numbers are versioned in the repo alongside the thing they measure. The agent did that bit too.

Was it actually worth it?

For me the thing that makes it work is the measurement. Everything comes to me as a small, focused, draft PR with the numbers attached, so I can look at it, sanity check it, run CI, and decide in a couple of minutes. It's not zero effort - I still read every change before it goes anywhere near main - but the ratio is fantastic. I'm getting a steady stream of well-measured, single-purpose performance PRs on a library I care about, on paths I'd genuinely never have found the time to optimise by hand, and I've shipped real releases because of it. For a project I maintain around everything else, that's a pretty great deal.

If you maintain a repo with a backlog you never get to, especially performance work that needs benchmarks to justify it, I'd recommend giving GitHub Next's agentics a go.

You can see all of it out in the open on the Examine repo - the [perf-improver] and [efficiency-improver] PRs, the benchmark suite, and the releases they fed into. And there's a nice bonus here for me too: ExamineX, my managed, cloud-hosted Examine search offering, runs on this exact same internal plumbing. So all this work the agents have been doing to tighten up Examine's query and indexing hot paths feeds straight through into ExamineX - the underlying engine gets faster and leaner, and every ExamineX site gets those wins for free without changing a thing. If you'd rather have your search running as a managed service instead of hosting Lucene indexes on your own servers, that's what ExamineX is there for. 🙂

Articulate 4.3.0 with support for markdown code snippets and syntax highlighting

Thu, 23 Mar 2023 15:10:26 Z

I'm happy to announce that Articulate 4.3.0 is shipped and includes a great new feature that I've been wanting/needing:

The ability to create markdown based posts with support for GitHub style code fences/snippets and syntax highlighting for (almost) any coding language. See #341.

Now I can finally do away with using Live Writer for my blog and having to manually add css classes to the html for syntax highlighting... yes that's what I've been doing ‍♂️

Upgrading

Once you've updated to the 4.3.0 release, you'll probably need to change the Property Editor of the Articulate Markdown Data Type to be Articulate Markdown editor since it's most likely currently configured to the default/basic Umbraco markdown editor.

You'll then need to update your Post.cshtml theme file to include the correct Prism dependencies. For example, here's the update/diff to the VAPOR theme which adds these dependencies. It's just ensuring that the Prism stylesheet is added to the header and the Prism.js dependencies are appended to the Post.cshtml file.

Once that's all done, your set!

Creating markdown posts

If you didn't already know, Articulate has always had a browser based markdown editor to create posts. You can simply go to your Articulate root URL and go to the path: a-new to load the markdown editor. Previously this editor would require you to authenticate at the end of writing your post (if you weren't already authenticated) but now it requires authentication up-front.

Once it's loaded, it's just a text editor and you can use your all the normal markdown syntax. You can even upload or take photos with the editor

Of course you can just use the back office markdown editor to create or update posts too but I find for quickly getting a post written and published it's faster to use the /a-new editor... and it works on mobile.

Using code fences with syntax highlighting

GitHub's documentation shows how this works. The typical code fence is 3x back-ticks above and below your code snippet. If you want to add language specific syntax highlighting you can use 3x back-ticks + the language name/alias. For example, c# would be: ```cs (or c# or csharp) and JavaScript would be ```js. GitHub's implementation is different from what Articulate uses so it may not be a perfect 1:1 result but should be fairly close. Articulate is using a combination of:

Rendered Examples

There are just rendered examples based on the default prism styles. I wrote this blog post with the Articulate markdown editor so you can see the results.

Here's an example of a rendered csharp code fence:

/// <summary>
/// This is an example of the ArticulateComposer
/// </summary>
[RuntimeLevel(MinLevel = RuntimeLevel.Run)]
public class ArticulateComposer : IUserComposer
{
    public void Compose(Composition composition)
    {
        composition.RegisterUnique<ArticulateRoutes>();
        composition.RegisterUnique<ContentUrls>();
        composition.RegisterUnique<ArticulateDataInstaller>();
        composition.RegisterUnique<ArticulateTempFileSystem>(
            x => new ArticulateTempFileSystem("~/App_Data/Temp/Articulate"));

        // TODO: Register remaining services....
    }
}

Here's an example of a rendered js code fence:

(function () {
    'use strict';

    /**
     * An example of the articulateOptionsManagementController
     * @param {any} $scope
     * @param {any} $element
     * @param {any} $timeout
     */
    function articulateOptionsManagementController($scope, $element, $timeout) {

        var vm = this;
        vm.viewState = "list";
        vm.selectedGroup = null;
        
        // TODO: Fill in the rest....
    }

    var articulateOptionsMgmtComponent = {
        templateUrl: '../../App_Plugins/Articulate/BackOffice/PackageOptions/articulatemgmt.html',        
        controllerAs: 'vm',
        controller: articulateOptionsManagementController
    };

    angular.module("umbraco")
        .component('articulateOptionsMgmt', articulateOptionsMgmtComponent);
})();

Here's an example of a rendered ruby code fence:

class Dog  
  def initialize(breed, name)  
    # Instance variables  
    @breed = breed  
    @name = name  
  end  
  
  def bark  
    puts 'Ruff! Ruff!'  
  end  
  
  def display  
    puts "I am of #{@breed} breed and my name is #{@name}"  
  end  
end

Auto upgrade your Nuget packages with Azure Pipelines or GitHub Actions

Thu, 23 Mar 2023 15:10:22 Z

Before we start I just want to preface this with some warnings

This works for me, it might not work for you
To get this working for you, you may need to tweak some of the code referenced
This is not under any support or warranty by anyone
Running Nuget.exe update command outside of Visual Studio will overwrite your files so there is a manual review process (more info below)
This is only for ASP.NET Framework using packages.config – Yes I know that is super old and I should get with the times, but this has been an ongoing behind the scenes project of mine for a long time. When I need this for Package Reference projects, ASP.NET Core/5, I’ll update it but there’s nothing stopping you from tweaking this to work for you
This only works for a specified csproj, not an entire sln – it could work for that but I’ve not tested, there would be a few tweaks to make that work
This does not yet work for GitHub actions but the concepts are all here and could probably very easily be converted UPDATE: This works now!

Now that’s out of the way …

How do I do it?

With a lot of PowerShell :) This also uses a few methods from the PowerShellForGitHub project.

The process is:

Run a pipeline/action on a schedule (i.e. each day)
This checks against your source code for the installed version for a particular package
Then it checks with Nuget (using your Nuget.config file) to see what the latest stable version is
If there’s a newer version:
Create a new branch
Run a Nuget update against your project
Build the project
Commit the changes
Push the changes
Create a PR for review

Azure Pipelines/GitHub Actions YAML

The only part of the YAML that needs editing is the variables, here's what they mean:

ProjectFile = The relative path to your csproj that you want to upgrade
PackageFile = The relative path to your packages.config file for this project
PackageName = The Nuget package name you want upgraded
GitBotUser = The name used for the Git commits
GitBotEmail = The email used for the Git commits

For Azure Pipelines, these are also required:

GitHubOwner = The repository owner (i.e. https://github.com/Shazwazza/Articulate )
GitHubRepository = The repository name (i.e. https://github.com/Shazwazza/Articulate )
There is one Secret setting you need to create in Azure Pipelines: GitHubAccessToken which is a GitHub PAT that you need to create that has access to this repository.

Then there are some variables to assist with testing:

DisableUpgradeStep = If true will just check if there’s an upgrade available and exit
DisableCommit = If true will run the upgrade and will exit after that (no commit, push or PR)
DisablePush = If true will run the upgrade + commit and will exit after that (no push or PR)
DisablePullRequest = If true will run the upgrade + commit + push and will exit after that (no PR)

Each step in the yaml build more or less either calls Git commands or PowerShell functions. The PowerShell functions are loaded as part of a PowerShell Module which is committed to the repository. This module’s functions are auto-loaded by PowerShell because the first step configures the PowerShell environment variable PSModulePath to include the custom path. Once that is in place, all functions exposed by the module are auto-loaded.

In these examples you’ll see that I’m referencing Umbraco Cloud names and that’s because I’m using this on Umbraco Cloud for my own website and the examples are for the UmbracoCms package. But this should in theory work for all packages!

Show me the code

The code for all of this is here in a new GitHub repo and here’s how you use it:

You can copy the folder structure in the repository as-is. Here's an example of what my site's repository folder structure is to make this work (everything except the src folder is in the GitHub repo above):

[root]
- auto-upgrader.devops.yml (If you are using Azure Pipelines)
- .github
  - workflows
    - auto-upgrader.gh.yml (If you are using GitHub Actions)
- build
  - PowershellModules
    - AutoUpgradeFunctions.psd1
    - AutoUpgradeFunctions.psm1
    - AutoUpgradeFunctions
- src
  - Shazwazza.Web
    - Shazwazza.Web.csproj
    - packages.config

All of the steps have descriptive display names and it should be reasonably self documenting.

The end result is a PR, here’s one that was generated by this process:

Nuget overwrites

Nuget.exe works differently than Nuget within Visual Studio’s Package Manager Console. All of those special commands like Install-Package, Update-Package, etc… are all PowerShell module commands built into Visual Studio and they are able to work with the context of Visual Studio. This allows those commands to try to be a little smarter when running Nuget updates and also allows the legacy Nuget features like running PowerShell scripts on install/update to run. This script just uses Nuget.exe and it’s less smart especially for these legacy .NET Framework projects. As such, it will just overwrite all files in most cases (it does detect file changes it seems but isn’t always accurate).

With that warning it is very important to make sure you review the changed files in the PR and revert or adjust any changes you need before applying the PR.

You’ll see a note in the PowerShell script about Nuget overwrites. There are other options that can be used like "Ignore" and "IgnoreAll" but all my tests have showed that for some reason this setting will end up deleting a whole bunch of files so the default overwrite setting is used.

Next steps

Get out there and try it! Would love some feedback on this if/when you get a change to test it.

PackageReference support with .NET Framework projects could also be done (but IMO this is low priority) along with being able to upgrade the entire SLN instead of just the csproj.

Then perhaps some attempts at getting a NET Core/5 version of this running. In theory that will be easier since it will mostly just be dotnet commands.

Powershell script to create an Umbraco package in Umbraco’s native file format

Thu, 23 Mar 2023 15:08:15 Z

Since I like using Powershell for my build scripts for various projects I thought it would be handy to create a Powershell script to create an Umbraco package in it’s native package format. I’ve added this script template to my GitHub Umbraco Scripts repository which you can see here: http://bit.ly/1kM9g9g

I’ve tried to add a bit of documentation to this script and in theory you should only need to set up the paths properly. You’ll also need to ensure that you initially create the package in the back office of Umbraco in order to generate the createdPackages.config as mentioned in the script notes.

I use this script in Articulate’s build script so at least I can also provide an example of using it which you can see here: https://github.com/Shazwazza/Articulate/blob/master/build/build.ps1

Articulate 1.0.4 released

Thu, 23 Mar 2023 15:08:15 Z

I’ve finally got around to releasing Articulate 1.0.4 today. Want to say a big thanks to all those who submitted pull requests, you guys rock! There’s a few nice fixes in this release but most importantly it fixes the issues with multi-tenancy when domains are assigned in Umbraco.

Here’s the release notes on GitHub with links to each issue fixed:

https://github.com/Shandem/Articulate/releases/tag/1.0.4

You can download the Umbraco package from there or from Our here:

http://our.umbraco.org/projects/starter-kits/articulate

Unfortunately I haven’t got around to getting this on Nuget yet – That’s because it has it’s own challenges since I’d like to perform a full Umbraco data install via Nuget… coming soon I hope :)

Enjoy!

This blog now powered by Articulate

Thu, 23 Mar 2023 15:08:14 Z

I’ve recently decided to build a new open source blog engine powered by Umbraco called Articulate. There’s a few reasons why i wanted to do this:

Since my full time job is an Umbraco core developer, I spend most of my time building Umbraco and get very little time for using Umbraco. I wanted to change this dilemma and really use Umbraco and utilize many of the features we’ve been creating and try to push some of it’s boundaries. For me this is a perfect way to find inspiration for new Umbraco features and enhancements.
I’ve wanted to move my blog away from BlogEngine.Net to Umbraco for quite some time (dogfooding) but I really needed to have every feature that I use in BlogEngine.Net available
I wanted to make the blog experience as simple as possible in Umbraco
I wanted to be able to write blog posts directly from my mobile phone easily using the web

I’ve got some documentation written and I’ll keep updating this over time but hopefully there’s enough there to get you started. Since Articulate is open source and hosted on GitHub, any community contributions are hugely welcomed :) Whether that’s core code additions, fixes, documentation, etc… any contribution is a big help.

Articulate templates are based on themes and I think it would be super awesome if people started creating their own themes and releasing them as their own packages or including them in the Articulate core for release. Creating themes is really easy and in fact a few of the themes included with Articulate are open source MIT licensed themes migrated over from the Ghost blogging platform (which is also very easy to do).

The first version of Articulate is out and can be downloaded either from the Umbraco package repository or from GitHub. There are a few minor bugs in this release that have been reported and fixed and I’ll have a newer version out this week.