@Shazwazza

Shannon Deminick's blog all about web development

Building a plugin system in Asp.Net Core is a dream compared to previous Asp.Net versions!

In previous versions it was not really feasible to load Assemblies located outside of the /bin folder for a web application. I battled with this concept quite a long time ago and although it’s sort of possible, the notion of having a plugin system that supported loading DLLs from outside of the /bin folder was riddled with hacks/problems and not really supported OOTB.

A large part of the issues has to do with something called an ‘Assembly Load Context’. In traditional .Net there are 3 of these context types: “Load”, “LoadFrom” and “Neither”, here’s a very old but very relevant post about these contexts from Suzanne Cook. In traditional Asp.Net, the “Load” context is used as the default context and it is managed by something called Fusion (.Net’s normal Assembly Loader/Binder). The problem with this context is that it is difficult to load an assembly into it that isn’t located in Fusion’s probing paths (i.e. /bin folder). If you load in an Assembly with a different Assembly Load Context and then try to mix it’s Types with the Types from the default context  … you’ll quickly see that it’s not going to work.

The “Neither” context

Here is the Neither context definition as defined by Suzanne Cook:

If the user generated or found the assembly instead of Fusion, it's in neither context. This applies to assemblies loaded by Assembly.Load(byte[]) and Reflection Emit assemblies (that haven't been loaded from disk). Assembly.LoadFile() assemblies are also generally loaded into this context, even though a path is given (because it doesn't go through Fusion).

In Asp.Net Core (targeting CoreCLR), the default Assembly Load Context is the “Neither” context. This is a flexible context because it doesn’t use Fusion and  it allows for loading assemblies any way that you want - including loading an assembly from a byte array, from a path or by a name. Since all of Asp.Net Core uses this context it means that all of the types loaded in with this context can talk to each other without having the previous Asp.Net problems.

I would assume that Asp.Net Core targeting Desktop CLR would still operate the same as before and still have the 3 types of Assembly Load Context’s … Maybe someone over at Microsoft can elaborate on that one? (David Fowler… surely you know? :)

Finding referenced plugin assemblies

In many cases if you create a product that supports plugin types, developers will create plugins for your product and ship them via Nuget. This is a pretty standard approach since it allows developers that are using your product to install plugins from the Nuget command line or from within Visual Studio. In this case plugin types will be found in referenced assemblies to your application and will be automatically loaded. Asp.Net Core has an interface called Microsoft.Extensions.PlatformAbstractions.ILibraryManager that can be used to resolve your application’s currently referenced ‘Libraries’ (i.e Nuget packages) and then each ‘Library’ returned exposes the Assemblies that it includes. Asp.Net MVC 6 has an even more helpful interface called Microsoft.AspNet.Mvc.Infrastructure.IAssemblyProvider which returns a list of referenced assemblies that are filtered based on if they are assemblies that reference a subset of MVC assemblies. The default implementation of IAssemblyProvider (DefaultAssemblyProvider) is extensible and we can use it to override it’s property ReferenceAssemblies in order to supply our own product assembly names instead of the MVC ones. This is perfect since this allows us to get a list of candidate assemblies that might contain plugins for your product:

public class ReferencePluginAssemblyProvider : DefaultAssemblyProvider
{
    //NOTE: The DefaultAssemblyProvider uses ILibraryManager to do the library/assembly querying
    public ReferencePluginAssemblyProvider(ILibraryManager libraryManager) : base(libraryManager)
    {
    }

    protected override HashSet<string> ReferenceAssemblies 
        => new HashSet<string>(new[] {"MyProduct.Web", "MyProduct.Core"});
}

now if you want to get a list of candidate assemblies that your application is referencing you could do:

//returns all assemblies that reference your product Assemblies
var candidateReferenceAssemblies = referencedPluginAssemblyProvider.CandidateAssemblies;

Finding and loading non-referenced plugin assemblies

This is where things get fun since this is the type of thing that wasn’t really very feasible with traditional Asp.Net web apps. Lets say you have a plugin framework where a plugin is installed via your web app, not in Visual Studio and therefore not directly referenced in your project. For this example, the plugin is a self contained collection of files and folders which could consist of: Css, JavaScript, Razor Views, and Assemblies. This plugin model is pretty nice since to install the plugin would mean just dropping the plugin folder into the right directory in your app and similarly  to uninstall it you can just remove the folder.  The first step is to be able to load in these plugin Assemblies from custom locations. For an example, let’s assume the web app has the following folder structure:

  • App Root
    • App_Plugins <—This will be the directory that contains plugin folders
      • MyPlugin1
        • bin <—by convention we’ll search for Assemblies in the /bin folder inside of a plugin
        • Views
      • MyPlugin2
        • bin <—by convention we’ll search for Assemblies in the /bin folder inside of a plugin
        • css
    • Views
    • wwwroot

IAssemblyLoader

The first thing we need is an ‘Microsoft.Extensions.PlatformAbstractions.IAssemblyLoader’, this is the thing that will do the assembly loading into the Assembly Load Context based on an AssemblyName and a location of a DLL:

public class DirectoryLoader : IAssemblyLoader
{
    private readonly IAssemblyLoadContext _context;
    private readonly DirectoryInfo _path;

    public DirectoryLoader(DirectoryInfo path, IAssemblyLoadContext context)
    {
        _path = path;
        _context = context;
    }

    public Assembly Load(AssemblyName assemblyName)
    {
        return _context.LoadFile(Path.Combine(_path.FullName, assemblyName.Name + ".dll"));
    }

    public IntPtr LoadUnmanagedLibrary(string name)
    {
        //this isn't going to load any unmanaged libraries, just throw
        throw new NotImplementedException();
    }
}

IAssemblyProvider

Next up we’ll need a custom IAssemblyProvider but instead of using the one MVC ships with, this one will be totally custom in order to load and resolve the assemblies based on the plugin’s /bin folders. The following code should be pretty straight forward, the CandidateAssemblies property iterates over each found /bin folder inside of a plugin’s folder inside of App_Plugins. For each /bin folder found it creates a DirectoryLoader mentioned above and loads in each DLL found by it’s AssemblyName into the current Assembly Load Context.

/// <summary>
/// This will return assemblies found in App_Plugins plugin's /bin folders
/// </summary>
public class CustomDirectoryAssemblyProvider : IAssemblyProvider
{
    private readonly IFileProvider _fileProvider;
    private readonly IAssemblyLoadContextAccessor _loadContextAccessor;
    private readonly IAssemblyLoaderContainer _assemblyLoaderContainer;

    public CustomDirectoryAssemblyProvider(
            IFileProvider fileProvider, 
            IAssemblyLoadContextAccessor loadContextAccessor, 
            IAssemblyLoaderContainer assemblyLoaderContainer)
    {
        _fileProvider = fileProvider;
        _loadContextAccessor = loadContextAccessor;
        _assemblyLoaderContainer = assemblyLoaderContainer;
    }

    public IEnumerable<Assembly> CandidateAssemblies
    {
        get
        {
            var content = _fileProvider.GetDirectoryContents("/App_Plugins");
            if (!content.Exists) yield break;
            foreach (var pluginDir in content.Where(x => x.IsDirectory))
            {
                var binDir = new DirectoryInfo(Path.Combine(pluginDir.PhysicalPath, "bin"));
                if (!binDir.Exists) continue;
                foreach (var assembly in GetAssembliesInFolder(binDir))
                {
                    yield return assembly;
                }
            }
        }
    }

    /// <summary>
    /// Returns assemblies loaded from /bin folders inside of App_Plugins
    /// </summary>
    /// <param name="binPath"></param>
    /// <returns></returns>
    private IEnumerable<Assembly> GetAssembliesInFolder(DirectoryInfo binPath)
    {
        // Use the default load context
        var loadContext = _loadContextAccessor.Default;

        // Add the loader to the container so that any call to Assembly.Load 
        // will call the load context back (if it's not already loaded)
        using (_assemblyLoaderContainer.AddLoader(
            new DirectoryLoader(binPath, loadContext)))
        {
            foreach (var fileSystemInfo in binPath.GetFileSystemInfos("*.dll"))
            {
                //// In theory you should be able to use Assembly.Load() here instead
                //var assembly1 = Assembly.Load(AssemblyName.GetAssemblyName(fileSystemInfo.FullName));
                var assembly2 = loadContext.Load(AssemblyName.GetAssemblyName(fileSystemInfo.FullName));
                yield return assembly2;
            }
        }
    }
}

That’s pretty much it! If you have an instance of CustomDirectoryAssemblyProvider then you can get Assembly references to all of the assemblies found in App_Plugins:

//returns all plugin assemblies found in App_Plugins
var candidatePluginAssemblies = customDirectoryAssemblyProvider.CandidateAssemblies;

Integrating non-referenced plugins/Assemblies with MVC

What if you had custom plugin types as MVC Controllers or other MVC types? By default MVC only knows about assemblies that your project has references to based on the DefaultAssemblyLoader.  If we wanted MVC to know about Controllers that exist in a plugin not referenced by your project (i.e. in App_Plugins) then it’s a case of registering a custom IAssemblyProvider in IoC which will get resolved by MVC. To make this super flexible we can create a custom IAssemblyProvider that wraps multiple other ones and allows you to pass in a custom referenceAssemblies filter if you wanted to use this to resolve your own plugin types:

public class CompositeAssemblyProvider : DefaultAssemblyProvider
{
    private readonly IAssemblyProvider[] _additionalProviders;
    private readonly string[] _referenceAssemblies;

    /// <summary>
    /// Constructor
    /// </summary>
    /// <param name="libraryManager"></param>
    /// <param name="additionalProviders">
    /// If passed in will concat the assemblies returned from these 
    /// providers with the default assemblies referenced
    /// </param>
    /// <param name="referenceAssemblies">
    /// If passed in it will filter the candidate libraries to ones
    /// that reference the assembly names passed in. 
    /// (i.e. "MyProduct.Web", "MyProduct.Core" )
    /// </param>
    public CompositeAssemblyProvider(
        ILibraryManager libraryManager,
        IAssemblyProvider[] additionalProviders = null,
        string[] referenceAssemblies = null) : base(libraryManager)
    {
        _additionalProviders = additionalProviders;
        _referenceAssemblies = referenceAssemblies;
    }

    /// <summary>
    /// Uses the default filter if a custom list of reference
    /// assemblies has not been provided
    /// </summary>
    protected override HashSet<string> ReferenceAssemblies
        => _referenceAssemblies == null
            ? base.ReferenceAssemblies
            : new HashSet<string>(_referenceAssemblies);
    
    /// <summary>
    /// Returns the base Libraries referenced along with any DLLs/Libraries
    /// returned from the custom IAssemblyProvider passed in
    /// </summary>
    /// <returns></returns>
    protected override IEnumerable<Library> GetCandidateLibraries()
    {
        var baseCandidates = base.GetCandidateLibraries();
        if (_additionalProviders == null) return baseCandidates;
        return baseCandidates               
            .Concat(
            _additionalProviders.SelectMany(provider => provider.CandidateAssemblies.Select(
                x => new Library(x.FullName, null, Path.GetDirectoryName(x.Location), null, Enumerable.Empty<string>(),
                    new[] { new AssemblyName(x.FullName) }))));
    }
}

To register this in IoC you just need to make sure it’s registered after you register MVC so that it overrides the last registered IAssemblyProvider:

//Add MVC services
services.AddMvc();
//Replace the default IAssemblyProvider with the composite one
services.AddSingleton<IAssemblyProvider, CompositeAssemblyProvider>(provider =>
{
    //create the custom plugin directory provider
    var hosting = provider.GetRequiredService<IApplicationEnvironment>();
    var fileProvider = new PhysicalFileProvider(hosting.ApplicationBasePath);
    var pluginAssemblyProvider = new CustomDirectoryAssemblyProvider(
        fileProvider,         
        PlatformServices.Default.AssemblyLoadContextAccessor,
        PlatformServices.Default.AssemblyLoaderContainer);
    //return the composite one - this wraps the default MVC one
    return new CompositeAssemblyProvider(
        provider.GetRequiredService<ILibraryManager>(),
        new IAssemblyProvider[] {pluginAssemblyProvider});
});

 

Your all set! Now you have the ability to load in Assemblies from any location you want, you could even load them in as byte array’s from an external data source.  What’s great about all of this is that it just works and you can integrate these external Assemblies into MVC.

Some things worth noting:

  • Parts of the assembly loading APIs are changing a bit in Asp.Net Core RC2: https://github.com/aspnet/Announcements/issues/149
  • The above code doesn’t take into account what happens if you load in the same Assembly from multiple locations. In this case, the last one in wins/is active AFAIK – I haven’t tested this yet but I’m pretty sure that’s how it works.
  • You may have some issues if load in the same Assembly more than once from multiple locations if those Assemblies have different strong names, or major versions applied to them – I also haven’t tested this yet

Last year I created a runtime Js/Css pre-processor for ASP.Net Core (Previously ASP.Net 5) called “Smidge” and have been meaning to blog about how I integrated this with AppVeyor – to run my tests, build the project and output the Nuget files I need, so here it goes…

The build script

I use Powershell for my build scripts for my projects since it’s reasonably easy to read and the same script format has worked quite well for ASP.Net Core projects too. You can see the whole build file here. Here’s the important things to note:

With AppVeyor (and probably other build servers), you need to ensure that it actually has the dnx version you need:

# ensure the correct version
& $DNVM install 1.0.0-rc1-update1

Next you need to make sure that the current process is using the version you need to build:

# use the correct version
& $DNVM use 1.0.0-rc1-update1

Then we need to use DNU to make sure that your project has everything it needs to build:

& $DNU restore "$ProjectJsonPath"

Lastly it’s just building and packaging the project:

& $DNU build "$ProjectJsonPath"
& $DNU pack "$ProjectJsonPath" --configuration Release --out "$ReleaseFolder"

The rest of the build file is normal Powershell bits.

The test script

I’m using xunit for unit tests in this project and similarly to the build script I’m using a simple Powershell script to execute the tests on the build server, the test runner file is here. The important parts are just like the above: Ensure the correct version is installed and being used by the current process and making sure that the project has everything it needs to build and finally to build it. The last missing piece is to actually run the tests:

& $DNX -p "$TestsFolder" test

Where ‘test’ is a command defined in my project.json as part of my unit test project.

AppVeyor configuration

The good news is that there’s really not a lot to setup, it’s super easy. In your AppVeyor settings just go to the ‘Build’ section and tell it to execute the Powershell script with its build version information:

image

Then for the unit tests is basically the same, click on the ‘Tests’ section and tell it to execute the Powershell test script:

image

And that’s pretty much it! The only other part I’ve had to setup is the paths to my Artifacts (Nuget files) based on the current build number.

Now whenever I commit, AppVeyor will execute the build script and test script and we can see the output:

image

And it’s smart enough to know that the test runner executed is for unit tests, so all the unit test output shows up in the ‘Tests’ tab of the build

image

Now that that’s all setup, AppVeyor even gives you a handy Nuget feed that you can use to test your packages based on each build, this can be configured on the ‘NuGet’ settings section, for example here’s the Smidge feed: https://ci.appveyor.com/nuget/smidge

Smidge 1.0.0-RC3

It’s worth noting that I’ve also released a new version of Smidge since I finally had some time to work on it. Couple of bugs fixed in this release and also a handy new feature too! You can see the release notes here: https://github.com/Shazwazza/Smidge/releases/tag/1.0.0-rc3.  I’ve also updated a lot of the documentation, the main readme file was getting quite long so I’ve moved all of the docs over to the project’s Wiki on GitHub and condensed the readme for the most important bits. Have a look here: https://github.com/Shazwazza/Smidge

There’s a chance you might not have heard of FCN (File Change Notification) in ASP.Net and there’s an even bigger chance you didn’t realize how much it might affect you.

What is FCN in ASP.Net?

As you know ASP.Net monitors a few files/folders such as the ~/web.config and ~/App_Code and will restart your app domain when it detects changes. This is part of FCN in ASP.Net but it goes much deeper than that. There are a few other files & folders that ASP.Net monitors which will also cause an app domain restart: bin, App_WebReferences, App_GlobalResources, App_Code, Global.asax, and others. However what you might not realize is that ASP.Net actually monitors every single folder (+ files) in your web app!

Update 29/07/2016! I found a nice MS article about FCN here. I’ve been looking for more resources about this since a few new issues have been cropping up and I’m wondering if another MS update has caused another problem. Recently we’ve seen an increase in the error: “Overwhelming Change Notification in …” which has everything to do with FCN. I’ve also added a few links to the bottom of this post.

Why do I care?

If you have a web app that contains a lot of folders there is a good chance that the performance of your file system is affected in one way or another. To give you an example of how many file system watchers ASP.Net generates I created a Razor view (.cshtml)  (which you should definitely test on your own site!) to display what is actually happening behind the scenes. Here’s the output for the first run page from the Visual Studio 2015 MVC template site:

image

The table above lists all of the “DirectoryMonitor” instances and the folder they are attached to along with all of the “FileMonitor” instances attached to that DirectoryMonitor. It’s worth nothing that these are not simply just .Net’s FileSystemWatcher, but some sort of native windows IO using a delegate called NativeFileChangeNotification. The above table doesn’t seem too scary but these monitors are not static either, they grow with every directory and file that is accessed in your web app.  For example, if I navigate to these pages: /Home/About, /Home/Contact, /Account/Register, /Account/Login and go back to view this table it looks like:

image

Things start to get interesting when you have a web application that has a lot of folders. Some examples of this might be:

  • You are using a dynamic image processor such as Image Resizer or Image Processor since these will create a lot of folders based on hashes to store these dynamic images
  • Maybe you have a lot of members/users on your site and you have one or more folders for each one
  • Maybe you use nodejs or bower and you store these generated folders in your web root and references the assets directly in those folders … there can be tons of folders in bower_components and node_modules
  • You could be using a web framework or CMS like Umbraco or Orchard (I’m sure there are plenty of others) that contain quite a lot of folders by default

Here’s the truncated result of an Orchard site after visiting the admin area, creating a page and displaying it:

image

Whoa!

ASP.Net’s FCNMode

At some stage in ASP.Net’s lifetime somebody must have complained about this to Microsoft and they released a hotfix (seen here: https://support.microsoft.com/en-us/kb/911272). Then I can only assume that other people complained about this and with .Net 4.5 a new configuration setting appeared: FCNMode. This documentation explains a little about what each option does:

  • Default - For each subdirectory, the application creates an object that monitors the subdirectory. This is the default behavior
  • Disabled - File change notification is disabled.
  • NotSet - File change notification is not set, so the application creates an object that monitors each subdirectory. This is the default behavior.
  • Single - The application creates one object to monitor the main directory and uses this object to monitor each subdirectory.

Unfortunately these docs don’t tell us the whole story. It’s obvious that Default/NotSet are the same thing and are the default. Disabled is fairly clear but what it doesn’t mention is that if you set it to Disabled, this will disable all FCN for your web app, so if you change the contents of /bin or /App_Code, the site will not restart. However, Disabled still means that the web.config file is monitored so if you use this setting you can still bump your web.config to restart your site.

What exactly is “Single” though?

The folks over at DNN seem to have quite a bit of experience with FCN and this article seems to be the only place that actually explains “Single” mode correctly:

“FCNMode creates a monitor object with a buffer size of 4KB for each folder. When FCNMode is set to Single, a single monitor object is created with a buffer size of 64KB. When there are file changes, the buffer is filled with file change information. If the buffer gets overwhelmed with too many file change notifications an “Overwhelming File Change Notifications” error will occur and the app domain will recycle. The likelihood of the buffer getting overwhelmed is higher in an environment where you are using separate file server because the folder paths are much larger.”

If I change fcnMode=”Single” in my web.config for the same Orchard site above, the results are:

image

That’s quite a bit less!

I’m sure there are pros to using “Default” instead of “Single” but I’m not actually sure what they are.

Real world problem

The circumstance where “Default” FCN mode becomes a real problem is when you have a website that is running off of a remote file share.  As noted in Shawn Walker’s DNN article mentioned above:

“The likelihood of the buffer getting overwhelmed is higher in an environment where you are using separate file server because the folder paths are much larger.”

I can’t actually see a performance difference between Default or Single when running locally with an SSD HD, but I have seen some big issues running with Default when hosting on a remote file server:

  • Constant app domain restarts – I’ve seen constant app domain restarts even when a site is just serving requests without any IO activity apart from just reading. And by ‘Constant’ … I mean every few seconds all day long.
  • File server performance suffers severely – When multiple sites are active and are being hosted on a remote file server with Default FCNMode, the writing performance of the file server is drastically degraded even though when monitoring IOPS there doesn’t appear to be any issues

To solve this issue we changed fcnMode=”Single” in the machine.config so that all sites would effectively use “Single”… and the result was instant: No more constant app restarts, file server performance was instantly back to normal. And as far as I can tell, there has been no downside to running FCNMode in Single.

So I really wonder what the up-side of Default is when it seems that running in Single mode is perfectly stable running on any hosting environment… ?

As far as I can tell, this won’t be an issue with aspnet5 … let’s just hope!

More info?

Here’s a list of helpful links about FCN in ASP.Net:

Updated 29/07/2016! – More links I’ve discovered. Turns out this has been an issue for IIS + ASP.Net for quite some time with various older hotfixes, some of these new links might shed some light on the particular problem you might be having.

I’ve been working on a side project called Smidge which is a runtime JS & CSS preprocessor for ASP.Net 5. I started this late last year after the 2014 MS MVP Summit as a good starting point to deep dive into ASP.Net 5. I’ve been keeping the codebase up-to-date with the beta releases of ASP.Net 5, I have it cross compiled to both dnx451 and dnxcore50 and recently updated to use Beta 7. This week I decided to give running ASP.Net 5 CoreCLR on Linux… and the result is IT WORKS!

I have next to no experience with Linux and considering that, it wasn’t actually very difficult to get my test site for Smidge up and running. Here’s the info on how I set this up:

Linux setup

I decided to use Ubuntu 14.04.3 LTS. I installed in on Hyper-V on Windows 10 and that was all very easy. I also setup SSH with the server so that I could remote terminal to it which is much nicer than using the terminal through the UI interface of Ubuntu via Hyper-V. Then basically followed the instructions here: https://github.com/aspnet/Home/blob/dev/GettingStartedDeb.md#getting-started-with-aspnet-5-and-linux – except that I didn’t configure any Nuget package sources since that is built into dnvm now. Once that was done I used dnvm to install the default runtime:  dnvm upgrade. This installed mono by default but for my purposes I needed ASP.Net 5 CoreCLR since that’s what Smidge is built against and I wanted to see this CoreCLR cross-platform stuff in action. Issuing this command does the trick: dnvm install 1.0.0-beta7 -r coreclr . Now when I list the runtimes installed (dnvm list) I get:

image

So now dnx is installed! We’re ready to go.

dnu publish & bash

What I really wanted to see was that I could build my solution on my Windows machine in Visual Studio and then export it and see if it would work on the Linux machine. Through the command line on Windows at the root of my project I used dnu publish (https://github.com/aspnet/Home/wiki/DNX-utility#publish-dnu-publish) which outputs a ‘self-contained directory that can be launched’ = great! So I executed that command, it put the folder in my /bin folder for my current project and I copied over that directory to my Linux machine….  and realized I didn’t know what to do next ;)

I had a look through the files that dnu publish exports and the one that is listed in ASP.Net’s docs is the output/kestrel.cmd (since the command in my project is named ‘kestrel’). Inside this file this is listed:

@"dnx.exe" --appbase "%~dp0approot\src\Smidge.Web" Microsoft.Dnx.ApplicationHost --configuration Debug kestrel %*

which if you want to translate to Linux, you could execute this at the Linux terminal at the root of this folder:

dnx --appbase "approot/src/Smidge.Web" Microsoft.Dnx.ApplicationHost --configuration Debug kestrel

… which will actually work, BUT it turns out there’s a way more Linuxy way to do it. dnu publish also creates this file which isn’t in the docs:  output/kestrel Having a look at this file, the first line is: #!/usr/bin/env bash …  so I can only assume this is something for Linux since I’ve heard the term bash before. Turns out on Linux you can just do this in the terminal from the root of this folder!

bash kestrel

Result:

image

WHOOHOOO!

Lets see it in action

Now that it’s running, I’ll jump over to the UI in Ubuntu and fire up the browser… Tada!!

image

Problems along the way

I probably made the above sound a bit easier than it was ;) … I did run into a few setup issues along the way.

Problem #1

The first one was when I first tried to run dnx:

failed to locate libcoreclr with error libunwind.so.8: cannot open shared object file: No such file or directory” when you run dnx or dnu command

I solved this issue from reading about it on this nice post: http://blogs.msdn.com/b/rdcdev/archive/2015/08/28/some-issues-when-hosting-asp-net-5-on-ubuntu-on-azure.aspx which has some other nice tricks if you run into Ubuntu issues with ASP.Net 5.  The solution was that I needed to run this command:

sudo apt-get install libunwind8

Problem #2

Then I got this exception:

The type initializer for 'libcrypto' threw an exception

Which is referenced on this ASP.Net issue: https://github.com/aspnet/dnx/issues/1806 … and turns out that it’s also referenced on the above link. I can’t remember where exactly I found this solution but I had to run:

apt-get install libcurl4-openssl-dev

Problem #3

After fixing those 2 things, the bash kestrel command succeeded but when I went to test this in my browser, I just had a white screen. After Googling, I found this link: http://stackoverflow.com/questions/28845892/blank-white-screen-on-error-with-kestrel-asp-net-5 and as it turns out, I had the same issue. I forgot to add the error handling middleware. Perhaps when running in VS with IIS this is automatically taken care of for you… not sure. But in any case, it’s super important that you add it and you should add it as the first middleware so you can actually see if your other middleware fails, typically your ‘Configure’ method in your Startup class should start with:

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    // Add the following to the request pipeline only in development environment.
    if (env.IsDevelopment())
    {
        app.UseErrorPage();
    }
    else
    {
        // Add Error handling middleware which catches all application specific errors and
        // sends the request to the following path or controller action.
        app.UseErrorHandler("/Home/Error");
    }

Problem #4

We’re not in Windows-land anymore! The errors I was getting were due to invalid file system paths. Turns out that .Net has always had this property: System.IO.Path.DirectorySeparatorChar but there wasn’t much reason to use it since .Net only runs in Windows and that character is always backslash. So I had to change my hard coded backslash use to use this instead. Next file path issue was case sensitivity… DOH. The Smidge configuration file is: ~/smidge.json however in my c# code I was trying to load it in with “Smidge.json” which fails in Linux of course.

Problem #5

Static files… I’m so used to working with IIS I forgot that outside of IIS I’d need to make sure the static file middleware was used:

app.UseStaticFiles();

I fixed that up and everything just worked… very freakin cool!!

Release – beta6

I’ve put up a new release on Nuget with these changes:

PM> Install-Package Smidge -Pre

And the source is on GitHub:

Using AspNet5 OptionsModel

January 12, 2015 04:21

If you’ve used AspNet5 then you’ve probably been using some MVC, in which case you’ve probably seen something like this in your Startup class:

// Add MVC services to the services container
services.AddMvc(configuration)
    .Configure<MvcOptions>(options =>
    {
        //Configure some MVC options like customizing the 
        // view engines, etc...
        options.ViewEngines.Insert(0, typeof(TestViewEngine));
    });

It turns out this syntax for specifying ‘options’ for a given service is a generic pattern that you can use in your own code. In fact the OptionsModel framework is it’s own code repository: https://github.com/aspnet/Options

I’ve implemented custom options in my AspNet5 project called Smidge (a runtime JavaScript/CSS pre-processing engine) and wanted to share the details since as far as I’ve seen there isn’t any documentation about this.

What are options?

Probably the simplest way to describe the options framework is that: Options allow you to configure your application via code during startup.

Options are just a POCO class that can contain configuration options to customize the behavior of your library. These option classes can be injected into any of your services with IoC using an interface called Microsoft.Framework.OptionsModel.IOptions. There’s a caveat to this POCO class however: It must contain an parameter-less/empty constructor which means you cannot have services injected into the your options class via it’s constructor.  This options framework also allows for ‘named’ options. So for example, perhaps you have a single options class that you would like to have configured in 2 different ways, one for ‘staging’ and one for your ‘live’ website.

Creating options

Here’s a really simple example of a POCO options class:

public class CustomMessageOptions
{
    public CustomMessage()
    {
        Message = "";
    }

    public string Message { get; set; }
}

In order to use this options class you need to create an options configuration class. For example:

public class CustomMessageOptionsSetup : ConfigureOptions<CustomMessageOptions>
{
    public CustomMessageOptionsSetup() 
    : base(ConfigureMessageOptions)
    {
    }

    /// <summary>
    /// Set the default options
    /// </summary>
    public static void ConfigureMessageOptions(CustomMessageOptions options)
    {
        options.Message = "Hello world";
    }
}

Then you need to add this class to your IoC container of type Microsoft.Framework.OptionsModel.IConfigureOptions:

services.AddTransient<IConfigureOptions<CustomMessageOptions>, CustomMessageOptionsSetup>();

Using options

To configure your options during startup, you do so in the ConfigureServices method like:

services.Configure<CustomMessageOptions>(options =>
{
    options.Message = "Hello there!";
});

Now you can have these options injected into any of your services using the IOptions interface noted previously:

public class MyCoolService 
{
    public MyCoolService(IOptions<CustomMessageOptions> messageOptions)
    {
        //IOptions exposes an 'Options' property which resolves an instance
        //of CustomMessageOptions
        ConfiguredMessage = messageOptions.Options.Message;
    }

    public string ConfiguredMessage {get; private set;}
}

Named options

As an example, lets say that you want a different message configured for your ‘staging’ and ‘live’ websites. This can be done with named options, here’s an example:

services
    .Configure<CustomMessageOptions>(options =>
    {
        options.Message = "Hi! This is the staging site";
    }, "staging")
    .Configure<CustomMessageOptions>(options =>
    {
        options.Message = "Hi! This is the live site";
    }, "live");

Then in your service you can resolve the option instance by name:

public class MyCoolService 
{
    public MyCoolService(IOptions<CustomMessageOptions> messageOptions)
    {
        //IRL This value would probably be set via some environment variable
        var configEnvironment = "staging";

        //IOptions exposes an 'GetNamedOptions' method which resolves an instance
        //of CustomMessageOptions based on a defined named configuration
        ConfiguredMessage = messageOptions.GetNamedOptions(configEnvironment);
    }

    public string ConfiguredMessage {get; private set;}
}

Configuring options with other services

Since your options class is just a POCO object and must have a parameter-less/empty constructor, you cannot inject services into the options class. However, there is a way to use IoC services in your options classes by customizing the ConfigureOptions class created above.  In many cases this won’t be necessary but this really depends on how you are using options.  As a (bad) example, lets say we wanted to expose a custom helper service called SiteHelper on the CustomMessageOptions class that can be used by a developer to create the message. The end result syntax might look like:

services.Configure<CustomMessageOptions>(options =>
    {
        var siteId = options.SiteHelper.GetSiteId();
        options.Message = "Hi! This is the staging site with id: " + siteId;
    });

In order for that to work the options.SiteHelper property needs to be initialized. This is done with the CustomMessageOptionsSetup class (created above) which has been added to the IoC container, this means it can have other services injected into it. The resulting class would look like:

public class CustomMessageOptionsSetup : ConfigureOptions<CustomMessageOptions>
{
    //SiteHelper gets injected via IoC
    public CustomMessageOptionsSetup(SiteHelper siteHelper) 
    : base(ConfigureMessageOptions)
    {
        SiteHelper = siteHelper;
    }

    public SiteHelper SiteHelper { get; private set; }

    /// <summary>
    /// Set the default options
    /// </summary>
    public static void ConfigureMessageOptions(CustomMessageOptions options)
    {
        options.Message = "Hello world";
    }

    /// <summary>
    /// Allows for configuring the options instance before options are set
    /// </summary>
    public override void Configure(Bundles options, string name = "")
    {
        //Assign the site helper instance
        options.SiteHelper = SiteHelper;

        base.Configure(options, name);
    }
}

IRL to give you an example of why this might be useful, in my Smidge project I allow developers to create named JavaScript/CSS bundles during startup using options. In some cases a developer might want to manipulate the file processing pipeline for a given bundle and in that case they need access to a service called PreProcessPipelineFactory which needs to come from IoC. The usage might look like:

services.AddSmidge()
    .Configure<Bundles>(bundles =>
    {                   
        bundles.Create("test-bundle-3", 
            bundles.PipelineFactory.GetPipeline(
                //add as many processor types as you want
                typeof(DotLess), typeof(JsMin)), 
            WebFileType.Js, 
            "~/Js/Bundle2");
    });

In the above, the bundles.PipelineFactory is a property on the bundles (options) class which gets initialized in my own ConfigureOptions class.

 

Hopefully this helps anyone looking to use custom options in their AspNet5 libraries!

During the past month I decided to dive deep into learning ASP.NET 5, and what better way to learn than to start a new OSS project :)

I chose to make a new new simple and extensible Javascript/CSS runtime pre-processor for ASP.NET 5. It does file minification, combination and compression, has a nice file caching layer and it’s all done in async operations. I ported over a few ideas and code snippets from CDF (client dependency framework) but with a more modern approach. I’ve called it ‘Smidge’ = something really small.

The project is on GitHub, it’s still a work in progress but its functional and there’s even some documentation! In the next few weeks I’ll get more of the code and docs updated and hopefully have a beta release out. In the meantime, you can clone the source, browse the code, build it and of course use it if you like.

Project details

It’s currently only targeting aspnet50 and not the Core CLR… I didn’t start with Core CLR because there was some legacy code I had to port over and I wanted to get something up and working relatively quickly. It shouldn’t be too much work to convert to Core CLR and Mono, hopefully I’ll find time to do that soon. It’s referencing all of the beta-* libraries from the ASP.NET 5 nightly myget feeds since there’s some code I’m using that isn’t available in the current beta1 release (like Microsoft.AspNet.WebUtilities.UriHelper). The target KRE version is currently KRE-CLR-amd64 1.0.0-beta2-10760.

Installation

I’ve put up an Alpha 1 release on Nuget, so you can install it from there:

PM> Install-Package Smidge -Pre

There’s some installation instructions here, you’ll need to add the smidge.json file yourself for now, can’t figure out how to get VS 2015 (kpm pack) to package that up … more learning required!

 

There’s certainly a lot of detective work involved in learning ASP.NET 5 but with the code being open source and browse-able/searchable on GitHub, it makes finding what you need fairly easy.

Here’s a new nifty feature I found in ASP.Net 5 – you can construct your model during model binding with IoC without any additional work. This is available on the dev branch on GitHub and is based on something called ServicesModelBinder. This is actually pretty cool because it means that you can have a model wired up with all of it’s dependencies based on IoC and then bound to your controller action’s parameters.

FromServices attribute

There’s a new attribute called FromServices which is what you use to enable this functionality. For example:

public async Task<ActionResult> GetProduct(
    [FromServices]ProductModel product)
{

}

What this attribute does is tell MVC to bind the model using IServiceActivatorBinderMetadata which is what the FromServices attribute implements. This in turn tells MVC to lookup the model binder that is aware of IServiceActivatorBinderMetadata which happens to be the ServicesModelBinder.

ServicesModelBinder

This model binder is pretty simple, it’s just going to resolve the model type from your IoC container. That could be pretty useful if you need to build up your model properties based on other services. I think some people might argue that this isn’t great practice because this is putting the binding logic in to the model itself instead of using a separate class to perform the binding logic. I suppose it’s up to the individual developer as to what their preference is. You can of course still create your own IModelBinder and use the ModelBinderAttribute to keep the model binding logic in the binder itself.

Incoming route values

Since the model is being created from IoC, how do you get the current route values to build up your model? To do that you’d put a constructor dependency on IContextAccessor<ActionContext>. This will give you all of the current route values, HttpContext, etc… basically everything you’d need to pull the data out of the current request to build your model.

Example

Given the above GetProduct example, the ProductModel class could look like:

public class ProductModel
{
    public ProductModel(IContextAccessor<ActionContext> action, IProductService prodService)
    {
        //TODO: Do some error checking...
        var productId = action.Value.RouteData.Values["product"];
        Value = prodService.Get(productId);
    }

    public IProduct Value { get; private set; }
}

This is pretty simple – the IProduct is looked up from the IProductService based on the incoming ‘product’ route value. Then in the controller you could just do: product.Value to get the value for IProduct.

You then need to ensure that both IProductService and ProductModel are registered as services in your container and it’s really important that the ProductModel is registered as a Transient object

This is part 2 of a series of posts about some fundamental changes in ASP.Net 5 that we’ll need to re-learn (or un-learn!)

Part 1: http://shazwazza.com/post/aspnet-5-re-learning-a-few-things-part-1/

System.Web

This probably isn’t new news to most people since it’s really one of the fundamental shifts for ASP.Net 5 – There won’t be a System.Web DLL. Everything that you’ll install in your website will come as different, smaller, separate libraries. For example, if you want to serve static files, you’d reference the Microsoft.AspNet.StaticFiles package, if you want to use MVC, you’d include the Microsoft.AspNet.Mvc package.

ASP.Net 5: “consists of modular components with minimal overhead, so you retain flexibility while constructing your solutions

Web Forms

Gone! YAY! :-)

Web Forms will still be a part of .Net as part of the main framework in System.Web, just not part of ASP.Net 5.

HttpModules

An HttpModule simply doesn’t exist in Asp.Net 5, but of course there is a new/better way to achieve this functionality, it’s called Middleware. In an HttpModule, you had to execute code based on the various stages of a request, things such as AuthenticateRequest, AuthorizeRequest, PostResolveRequestCache, and other slightly confusingly named events. This is no longer the case with Middleware, things just make sense now … everything is simply a linear execution of your code. You can have multiple middleware’s defined to execute in your application and each one is registered explicitly in your Startup.cs file. As a developer, you are in full control of what get’s executed and in what order instead of not knowing which HttpModules are executing and not really in control of their order of execution. Middleware can simply modify a request and continue calling the next one in the chain, or it can just terminate the pipeline and return a result.

There’s loads of examples in the source for middleware, ranging from the static file middleware to cookie authentication middleware,  etc…

And here’s a good article that explains middeware registration and the flow of control.

HttpHandlers

HttpHandlers are also a thing of the past. All they really were was a request handler that was based on a specific request path. MVC (which now also includes WebApi) has got this covered. If you really wanted, you could create middleware for this type of functionality as well but unless you require something extraordinarily complex that MVC cannot do (and it can do a lot!), I’d recommend just sticking with MVC.

ASP.Net 5 (aka vNext) is now in Beta and the Visual Studio 2015 preview is now out! So, what is this new ASP.Net? The 2 biggest features is that it’s totally open source and it will support a cross platform runtime = great!! But it’s worth knowing that this is more or less a rebuild of ASP.Net and there’s quite a few things that we’ve become accustomed to that will now be totally different.

Configuration files

You know all of that junk in the web.config and perhaps in a bunch of other *.config files? Well that is gone! I think this is wonderful news because creating those configuration sections was a huge pain. Even better news is how easy creating custom configuration inputs will be in  ASP.Net 5 using the new IConfiguration and ConfigurationModel sources. OOTB Microsoft is releasing support for JSON, XML and INI files for configuration but of course you can easily create your own.  The repository for the configuration bits is here and a nice tutorial can be found here.

So what about configuration transforms?? That is also a thing of the past. In ASP.Net 5, “configuration” will mostly be done with code found in your Startup.cs file, anything that you want to enable in your application is done in this class. Any package that is installed in your app, you will need to opt-in to use it in your Startup.cs file. In some cases however, a configuration file might need to be updated/transformed… this could be due to an upgrade of a package. The good news is that the configuration sources in ASP.Net 5 are writable (IConfigurationSource) which means during startup or first access to your config, you could detect what needs to be updated to support your new version, make the updates in code and commit (ICommitableConfigurationSource) the changes.

Wait… isn’t that going to restart the app domain?? 

NOTE: If you are using IIS, there can still be a web.config which can be used to configure IIS settings under the system.webserver section.

AppDomain restarts

This is something that we’ve all become familiar with… you want to restart your app, just bump/touch the web.config and your app domain is restarted. This is something we’ll all have to un-learn. In ASP.Net 5 auto app domain restarts don’t happen. First, there is no web.config (or global.asax for that matter) so there are no files to bump/touch. Next, a big reason why auto app domain restarts can’t occur is because ASP.Net 5 will be able to be run on various different web servers which don’t really know about what it means to restart an app domain. For example if you’ve been playing around with vNext before you had a chance to use VS 2015, you might be familiar with the command line “k web” (see docs for details). This command will start up a simple web server: Microsoft.AspNet.Server.WebListener which will serve web requests. In order for app domain restarts to occur, it would need to know how to restart itself after it’s been shutdown which isn’t exactly possible with a simple command line process. Instead if you made any changes to your code and wanted to restart your app, you’d kill the process (ctrl + c) and just call k web again. Another thing to be aware of is that when you kill a process like this, your app domain does not gracefully shutdown/unwind, it’s simply terminated.

But not to worry! If you have a web app that requires restarting (i.e. maybe it installs plugins, etc…) and needs to gracefully unwind, it’s still possible and actually much more pleasant since you’ll be in full control of how/when it happens. In order for this to work you’ll need to be running a web server that knows how to start itself back up again - like IIS! The way to gracefully shutdown your app domain is by using: IApplicationShutdown when you want to gracefully shutdown your app. You could even use that in combination with an IFileWatcher and your own configuration files … if you really wanted to mimic an app domain restart by bumping/touching a file.

Deployments, bin folder and App_Code

Speaking of app domain restarts, how will ASP.Net 5 work when I put a new assembly in the /bin folder or add a new class to App_Code?? These are a couple more things that need to be un-learned. There really isn’t a /bin folder anymore (well, there is but it only contains one very special assembly if you are running IIS) and there isn’t any App_Code folder.  So where does all of this go?

When you publish a web project in VS 2015 (which uses kpm pack) you end up with a very different looking deployment structure. There’s two folder: approot and wwwroot.

wwwroot – is the content of your website, nothing more. Even things like configuration files don’t exist here, it’s just content that your webserver can serve.

approot – this is the brains of your website. It includes all of the binaries, config files, source code, etc… that is used to run your website.

Here’s a blog post that describes the deployed file structure 

Did you say source code?? Yup! By default kpm pack will deploy your site with all of it’s packages and the source code for all of your projects. The Roslyn compiler will take care of everything for you when your site needs to start serving requests. You can of course opt-out of this and have your site deployed as a compiled package.

Did you say Package?? Yup, as in Nuget package! Instead of a /bin folder full of assemblies, all of your dependencies will actually be Nuget references and stored in approot/packages and if you choose to deploy your website without source, it will be compiled into a Nuget package and deployed in the packages folder as well.

More to come….

So there’s a a few of the things that are pretty different in ASP.Net 5, there’s still more to come and hopefully I’ll find some time to write them all up!