Shazwazza

Automated website deployment with PowerShell and SmartFTP

Thu, 23 Mar 2023 15:08:09 Z

This post was imported from FARMCode.org which has been discontinued. These posts now exist here as an archive. They may contain broken links and images.

SmartFTP is a fantastic FTP application which handles syncing files very effectively. This means that when you upload your entire website, SmartFTP will automatically detect changes and only upload what is required (instead of overwriting all of the files like some FTP applications do). For each project at TheFARM we have build scripts which run and create a time stamped ZIP package for each deployment environment with all of the necessary files formatted appropriately for each. Our deployment process then involves unzipping the contents of this file, opening up SmartFTP, connecting to the deployment destination and transfering all of the deployment files up (which SmartFTP synchronizes for us).

I thought it would be much more efficient if we automated this process. So we did some investigation and it turns out the SmartFTP conveniently has an API! So we decided to see if we could write a PowerShell script to use the SmartFTP api to automagically transfer/sync all of our deployment files in our Zip package to the necessary FTP site and with a bit of trial and error we managed to do it! Now, I’m not PowerShell expert or anything, and in fact this was my very first PowerShell script ever written so I’m sure this could all be done a bit better, but it works! I’m not going to go into detail about the SmartFTP api or how to write PowerShell stuff because this script will work with some basic requirements:

You need both PowerShell and SmartFTP installed
Currently this only supports the standard FTP protocol, but if you need SFTP, etc… you can just change the $fav variable’s ‘Protocol’ property
The parameters, in this order are:
- destination
  - the IP address, or host of your FTP server
- user
  - the username used to login to the FTP server
- password
  - the password used to login to the FTP server
- path
  - The FTP path of where you want your files to go on your FTP server
- port
  - The FTP port to use, default is 21
- source
  - The source folder to copy to the FTP site, if not specified, uses the current directory that the PowerShell script is run from

Example usage:

FTPSync.ps1 123.123.123.123 MyUserName MyPassword 21 “C:\MyWebsiteFolder” “/websites/MyWebsite”

or you can just double click on the ps1 file and it will prompt you for these details.

So without further adieu, here’s the script!

#requires -version 2.0

# Define inputs
param (
    [parameter(Mandatory=$true)]    
    [string]
    $dest,
    [parameter(Mandatory=$true)]
    [string]
    $user,
    [parameter(Mandatory=$true)]
    [string]
    $pass,
    [parameter(Mandatory=$true)]
    [ValidatePattern('\d+')]
    [int]
    $port = 21,
    [parameter(Mandatory=$false)]
    [ValidateScript({ Test-Path -Path $_ -PathType Container })]
    [string]
    $source,
    [parameter(Mandatory=$true)]
    [ValidatePattern('\/+')]
    [string]
    $path
)

# get current folder
$currFolder = (Get-Location -PSProvider FileSystem).ProviderPath;
# set current folder
[Environment]::CurrentDirectory=$currFolder;
# if the source isn't set, then use the current folder
if ($source = "") {
    $source = $currFolder;
}

Write-Host "------------------------------------------------------" -foregroundcolor yellow -backgroundcolor black
Write-Host("{0, -20}{1,20}" -f "Destination", $dest);
Write-Host("{0, -20}{1,20}" -f "User", $user);
Write-Host("{0, -20}{1,20}" -f "Pass", "********");
Write-Host("{0, -20}{1,20}" -f "Port", $port);
Write-Host "";
Write-Host "Source:";
Write-Host $source;
Write-Host "";
Write-Host "Path";
Write-Host $path;
Write-Host "------------------------------------------------------" -foregroundcolor yellow -backgroundcolor black

# Create application
$smartFTP = New-Object -comObject SmartFTP.Application;
$smartFTP.Visible = [bool]0;
$smartFTP.CloseAll();

# create temp favorite item
$fav = $smartFTP.CreateObject("sfFavorites.FavoriteItem");
$fav.Name = $user + " @ " + $dest + " (temp favorite by cmdInterface)";
# 1 = FTP standard protocol
$fav.Protocol = 1;
$fav.Host = $dest;
$fav.Port = $port;
$fav.Path = $path;
$fav.Username = $user;
$fav.Password = $pass;
# forces it not to be saved
$fav.Virtual = "true";

# Add temporary favorite to SmartFTPs FavoriteManager
$favMgr = $smartFTP.FavoritesManager;
$rootFolder = $favMgr.RootFolder;
$rootFolder.AddItem($fav);

# Get the transfer queue
$queue = $smartFTP.TransferQueue;
# stop the queue if it isn't already
if ($queue.State -ne 1) { $queue.Stop(); } # Stopped = 1
# clear the queue
foreach($item in $queue.Items) { $queue.RemoveItem($item); }
# set the thread count for the queue
$queue.MaxWorkers = 20;
#enable logging
$queue.Log = "true";
$queue.LogFolder = $currFolder + "\\LOG";

# create new transfer item
$newItem = $smartFTP.CreateObject("sfTransferQueue.TransferQueueItem");
# set the item as a folder and copy operation, 
$newItem.type = 2;  #FOLDER = 2 
$newItem.Operation = 1; #COPY = 1

# Set the source
$newItem.Source.type = 1;  #LOCAL = 1
$newItem.Source.Path = $source;

# Set the destination
$newItem.Destination.type = 2; #REMOTE = 2
$newItem.Destination.Path = $path;
$newItem.Destination.FavoriteIdAsString = $fav.IdAsString; #links up to our connection favorite

# and finally add it
$queue.AddItemTail($newItem);

Write-Host "STARTING" -foregroundcolor yellow -backgroundcolor black;

$queue.Start();

while ($queue.Items.Count -ne 0) { 
    Write-Host "Processing...bytes transfered: " $queue.TransferredBytes;
    Start-Sleep -s 2; #wait 2 seconds
}

# store the total bytes
$totalBytes = $queue.TransferredBytes;

# cleanup smartftp app
$queue.Quit();
$smartFTP.Exit();

# parse logs

# regex to find "[DATE/TIME] STOR FILENAME
# which indicates a file transfer
$regex = new-object System.Text.RegularExpressions.Regex("\[[\w\-\:]*?\]\sSTOR\s(.+?)\[",,[System.Text.RegularExpressions.RegexOptions]::SingleLine);
$totalFiles = 0;
Write-Host "Files Transfered" -foregroundcolor cyan -backgroundcolor black
Get-ChildItem $queue.LogFolder -include *.log -Recurse | foreach ($_) { 
    $currFile = Get-Content $_.fullname;
    $match = $regex.Matches($currFile);    
    if ($match.Count -gt 0) {        
        foreach($m in $match) { Write-Host $m.Groups[1]; } 
        $totalFiles++;
    }
    remove-item $_.fullname -Force -Recurse ;
}

Write-Host "COMPLETED (total bytes: " $totalBytes ", total files: )" $totalFiles -foregroundcolor cyan -backgroundcolor black;

"------------------------------------------------------"

# cleanup COM
Remove-Variable smartFTP

NDepend review = pretty cool!

Thu, 23 Mar 2023 15:08:08 Z

For a while I’ve been wanting to automate some reporting from our build process to show some fairly simple statistical information such as

Obsoleted code
Internal code planned to be made public
Code flagged to be obsoleted
Code flagged as experimental

NOTE: some of the above are based on our own internal c# attributes

Originally I was just going to write some code to show me this which would have been fairly straight forward but I knew that NDepend had a code query language and natively outputs reports and can be integrated into the build process, so I figured I’d give it a go.

So I went and installed NDepend and ran the analysis against my current project … I wasn’t quite prepared for the amount of information it gave me! :)

Reporting

I quickly found out that the reports that I wanted to create were insanely simple to write. I then lost a large part of my day exploring the reports it generates OOTB and I realized I hadn’t even touched the surface of the information this tool could give me. I was mostly interested in statistical analysis but this will also happily tell you about how bad or good your code is, how much test coverage you’ve got, what backwards compatibility you broke with your latest release, what code is most used in your project and what code isn’t used at all… and the list goes on and on.

To give you an example of the first report I wanted to write, here’s all I wrote to give me what methods in my codebase were obsoleted:

from m in JustMyCode.Methods where m.HasAttribute("System.ObsoleteAttribute") select m

What’s cool about the report output is that you can output to HTML or XML, so if you want to publish it on your site you can just write some XSLT to make it look the way you want.

Project comparison

Another really great feature that I thought would be fun to automate is the code analysis against previous builds so you know straight away if you are breaking compatibility with previous releases. I think it has great potential but what I did discover is that in some cases the OOTB queries for this aren’t perfect. It will tell you what public types, methods, and fields have been removed or changed and the query is pretty straightforward but… In many cases when we’re trying to maintain backwards compatibility while creating new APIs we’ll remove a type’s methods/properties and just make the legacy type inherit from the new API class (or some other trickery similar to that). This might break compatibility if people are using reflection or specific type checking but it wont break the API usage which I’m mostly interested in. I haven’t figured it out yet but I’m sure there’s some zany NDepend query I can write to check that.

Automation

So now that I know I can extract all this info, it’s time to decide what to export. Thankfully NDepend comes fully equipped with it’s own MSBuild integration so automating the report output should be pretty straightforward. I also noticed that if you wanted to be ultra careful you can integrate it with your build server and have your builds fail if some of your queries cause errors, like the above: If you break compatibility the build will fail. Or anything you want really, like failing the build if we detect you’ve written a method that has too many lines of code.

Pretty cool indeed!

I was going to use NDepend just to output reports which it clearly does without issue. What I didn’t realize was how much detail you can get about your code with this tool. There’s some tools in there that were a bit over my head but I suppose if you are working on really huge projects and need anything from just a snapshot or insane details into what dependencies your code has on other code it’ll do that easily as well.

The UI is a bit overwhelming at first and could probably do with some work but TBH I’m not sure how else you’d display that much information nicely. Once I figured out how the whole query and reporting structures work then it was dead simple.

Happy reporting :)

Multi targeting a single .Net project to build for different framework versions

Thu, 23 Mar 2023 15:08:08 Z

Consider this scenario:

I have a project that relies on ASP.Net MVC and currently this project is built against the .Net framework 4.0 and MVC 4
I want to support MVC 5 for this project
To support MVC 5, I’d need to upgrade the project to use .Net framework 4.5
This would mean that developers still running .Net framework 4.0 would no longer be able to use my updated project

I also don’t want to have to create different projects and assemblies that target specific MVC versions (in this case you’d end up with assembly names like MyProject.MVC4.dll and MyProject.MVC5.dll). I have seen this done with other solutions and it works but I feel it is not necessary unless your project is using specific functionality from a particular MVC version.

In my case my project will compile perfectly against MVC 4 and 5 without any codebase changes so I’m not actually targeting against a specific MVC version ( >= 4 ). If your project uses specific functionality from the different MVC versions I think you will have to output different assemblies like MyProject.MVC5.dll. In that case this post will probably help: http://blogs.korzh.com/2013/12/nuget-package-different-mvc-versions.html

The goal

The end result is that I want a single project file that has 4 build configurations:

Debug / Release
- These will build against .Net 4.0
- This will reference MVC 4
Debug-Net45 / Release-Net45
- This will build against .Net 4.5
- This will reference MVC 5

Each of these build configurations exports the same assembly name: MyProject.dll

I then want a single NuGet package which will install the correct DLL based on the .Net version that the user has installed.

How it’s done

Build configuration setup

To start with you’ll have a project that targets .Net framework 4.0 and references MVC 4 and 2 standard build configurations: Debug, Release.

First we need to create 2 new build configurations: Debug-Net45 and Release-Net45 and have these configurations output the DLLs to a custom folder. To do this we launch the Configuration Manager:

Create a new build configuration:

Enter the name of the new configuration and copy the configuration from it’s associated existing one. So Debug-Net45 would copy from Debug and Release-Net45 copies from Release.

Do this for both configurations – Debug-Net45 and Release-Net45.

Project setup

The next step requires manually editing your .csproj file since Visual Studio doesn’t want to normally allow you to do this. This is how you can configure each build type to target a different framework version. You’ll first need to find these 2 entries:

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">

Inside each of these elements you need to explicitly tell these build configurations to use .Net framework 4.0 by adding this element:

<TargetFrameworkVersion>v4.0</TargetFrameworkVersion>

Next you’ll want to copy/paste both of these property groups and:

Change the Condition to check for the new build configurations and to target .Net framework 4.5
Change the OutputPath to have a specific .Net 4.5 folder
Change the TargetFrameworkVersion to be .Net 4.5

You should end up with 2 new elements that look something like:

<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug-Net45|AnyCPU'">
    <DebugSymbols>true</DebugSymbols>
    <OutputPath>bin\Debug-Net45\</OutputPath>
    <DefineConstants>DEBUG;TRACE</DefineConstants>
    <DebugType>full</DebugType>
    <PlatformTarget>AnyCPU</PlatformTarget>
    <ErrorReport>prompt</ErrorReport>
    <TargetFrameworkVersion>v4.5</TargetFrameworkVersion>
    <PlatformTarget>AnyCPU</PlatformTarget>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release-Net45|AnyCPU'">
    <DebugType>pdbonly</DebugType>
    <Optimize>true</Optimize>
    <OutputPath>bin\Release-Net45\</OutputPath>
    <DefineConstants>TRACE</DefineConstants>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
    <TargetFrameworkVersion>v4.5</TargetFrameworkVersion>
    <PlatformTarget>AnyCPU</PlatformTarget>
</PropertyGroup>

Next we need to do this for the MVC references or any framework specific references you need to target. In this example it is just the MVC references and you’ll need to wrap them with an ItemGroup element, you’ll end up with something like this to target the MVC 4 libs:

<ItemGroup Condition=" '$(Configuration)'=='Debug' Or '$(Configuration)'=='Release'">
    <Reference Include="System.Web.Helpers, Version=2.0.0.0, .....">
      <Private>True</Private>
      <HintPath>
        ..\packages\Microsoft.AspNet.WebPages.2.0.30506.0\lib\net40\System.Web.Helpers.dll
      </HintPath>
    </Reference>
    <Reference Include="System.Web.Mvc, Version=4.0.0.0, .....">
      <Private>True</Private>
      <HintPath>
        ..\packages\Microsoft.AspNet.Mvc.4.0.30506.0\lib\net40\System.Web.Mvc.dll
      </HintPath>
    </Reference>
    <Reference Include="System.Web.Razor, Version=2.0.0.0, .....">
      <Private>True</Private>
      <HintPath>
        ..\packages\Microsoft.AspNet.Razor.2.0.30506.0\lib\net40\System.Web.Razor.dll
      </HintPath>
    </Reference>
    <Reference Include="System.Web.WebPages, Version=2.0.0.0, .....">
      <Private>True</Private>
      <HintPath>
        ..\packages\Microsoft.AspNet.WebPages.2.0.30506.0\lib\net40\System.Web.WebPages.dll
      </HintPath>
    </Reference>
    <Reference Include="System.Web.WebPages.Deployment, Version=2.0.0.0, .....">
      <Private>True</Private>
      <HintPath>
        ..\packages\Microsoft.AspNet.WebPages.2.0.30506.0\lib\net40\System.Web.WebPages.Deployment.dll
      </HintPath>
    </Reference>
    <Reference Include="System.Web.WebPages.Razor, Version=2.0.0.0, .....">
      <Private>True</Private>
      <HintPath>
        ..\packages\Microsoft.AspNet.WebPages.2.0.30506.0\lib\net40\System.Web.WebPages.Razor.dll
      </HintPath>
    </Reference>
</ItemGroup>

Then we need to go ahead and copy/paste this block but target the different build configurations and then swap out these MVC 4 version references with the MVC 5 version references.

Nuget package restore

For this tutorial I’m assuming you are using Nuget to reference your MVC libs and an easy way to get Nuget to play reasonably with this setup is to enable Nuget package restore for your solution – right click on your solution and click the button:

Then in your packages.config file you can manually add the MVC 5 references:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Microsoft.AspNet.Mvc" 
           version="4.0.30506.0" targetFramework="net40" />
  <package id="Microsoft.AspNet.Razor" 
           version="2.0.30506.0" targetFramework="net40" />
  <package id="Microsoft.AspNet.WebPages" 
           version="2.0.30506.0" targetFramework="net40" />
  <package id="Microsoft.Web.Infrastructure" 
           version="1.0.0.0" targetFramework="net40" />
  
  <package id="Microsoft.AspNet.Mvc" 
           version="5.0.0" targetFramework="net45" />
  <package id="Microsoft.AspNet.Razor" 
           version="3.0.0" targetFramework="net45" />
  <package id="Microsoft.AspNet.WebPages" 
           version="3.0.0" targetFramework="net45" />
</packages>

Now when you build your solution under the different build configurations Nuget will automatically go and get the correct MVC versions based on the current .Net Framework version.

Thats it!

With all of this in place it should ‘just work’ :) You will notice some odd looking icons in your references list in Visual Studio but it’s nothing to worry about, Visual Studio is just confused because it isn’t familiar with how you’ve tricked it!

So now when you build using your different build configurations, it will output the correct DLLs to the correct build folders:

Both Debug & Release will output to /bin/Debug & bin/Release with MVC 4 libraries and the MyProject.dll will be compiled against .Net Framework 4.0
Both Debug-Net45 & Release-Net45 will be output to /bin/Debug-Net45 & /bin/Release-Net45 with MVC 5 libraries and the MyProject.dll will be compiled against .Net Framework 4.5

And here’s some proof:

If you are using some automated build processes with MSBuild you can then just target the build configuration names you’d like to export.

The Nuget setup is pretty simple since you can target your dependencies by framework:

<group targetFramework="net40">
<!--between 4 and less than version 5-->
<dependency id="Microsoft.AspNet.Mvc" version="(4.0.20710,5)" />
</group>

<group targetFramework="net45">
<!--between 4 and less than version 6 (who knows what'll happen then)-->
<dependency id="Microsoft.AspNet.Mvc" version="(4.0.20710,6)" />
</group>

Then for each of your lib files in Nuget you ensure that you output them to the framework specific /lib folders such as:

target="lib\net40"

target="lib\net45"

This process has been implemented for the Client Dependency Framework version 1.8.0 to support MVC 4 and 5 without having to change the assembly name.