This blog was originally posted on June 30, 2018. Much of the data, information, images, and links are outdated. It is included in its original form as an archive.
Azure Service Fabric is a great platform for container orchestration. It provides a full suite of features to ensure that your container is held up by the five pillars of software quality – ensuring scalability, availability, resiliency, management, and security. Assuming your containerized application may need access to certificates to handle encryption, decryption, signing, or verification, Service Fabric even provides a built-in way to expose certificates installed in the LocalMachine store to the container by using a ContainerHostPolicy. You can also explicitly provide certificate files as part of the Data Package. Both approaches are documented well in the use a certificate in a container topic in the docs. What if you need more control over the certificates? What if they’re not installed on the node and you need to dynamically make them available to your container at the time of service startup? What actually needs to happen in the setupentrypoint.sh script?
This post shows how you can utilize SetupEntryPoint scripts to manage acquiring certificates and making them available to your Service Fabric hosted container. It will depend on the LocalMachine certificate store to provide the initial certificate, but it could also be a certificate file pulled from Azure Key Vault or some other certificate store. This code was originally created using Visual Studio 15.7.4 (taking advantage of the new container tooling) and the Microsoft Azure Service Fabric SDK version 3.1.269.9494.
Create a Web Application for Containerization
With the container tooling available in the Service Fabric tools (Visual Studio 15.7 or greater), creating a Service Fabric hosted container is really simple. I started with an ASP.NET Core Web Application with a controller that handled encryption and decryption actions. I broke out the encryption and decryption into a separate service so that the certificate would only have to be loaded once. When finished, the controller remains very light:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System;
namespace ContainerCrypto.Controllers
{
[Produces("application/json")]
[Route("[controller]")]
public class CryptoController : Controller
{
private IStringCryptoService _stringCryptoService;
public CryptoController(IStringCryptoService stringCryptoService)
{
_stringCryptoService = stringCryptoService;
}
[HttpPost]
[Route("encrypt")]
public IActionResult Encrypt([FromBody] string textToEncrypt)
{
try
{
var encryptedText = _stringCryptoService.Encrypt(textToEncrypt);
return Ok(encryptedText);
}
catch (Exception ex)
{
return StatusCode(StatusCodes.Status500InternalServerError, ex);
}
}
[HttpPost]
[Route("decrypt")]
public IActionResult Decrypt([FromBody] string base64EncodedTextToDecrypt)
{
try
{
var decryptedText = _stringCryptoService.Decrypt(base64EncodedTextToDecrypt);
return Ok(decryptedText);
}
catch (Exception ex)
{
return StatusCode(StatusCodes.Status500InternalServerError, ex);
}
}
}
}
The encryption and decryption service CertStringCryptoService is mainly responsible for loading a certificate and storing its public and private keys for encryption and decryption respectively. Since it will be hosted in a container, the code depends on environment variables to be configured at runtime. In this case, it will need to know the path to the certificate PFX file and the path to a file containing the password to access the PFX. It will then use the retrieved keys for encryption and decryption.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
public string Encrypt(string plainTextToEncrypt)
{
EnsurePublicKey();
var contentToEncrypt = Encoding.Default.GetBytes(plainTextToEncrypt);
var encryptedBytes = publicKey.Encrypt(contentToEncrypt, RSAEncryptionPadding.Pkcs1);
return Convert.ToBase64String(encryptedBytes);
}
public string Decrypt(string base64EncodedTextToDecrypt)
{
EnsurePrivateKey();
var contentToDecrypt = Convert.FromBase64String(base64EncodedTextToDecrypt);
var decryptedBytes = privateKey.Decrypt(contentToDecrypt, RSAEncryptionPadding.Pkcs1);
return Encoding.Default.GetString(decryptedBytes);
}
private void LoadKeysFromCertificateFile()
{
// Get password from file
string password = File.ReadAllLines(
EnvironmentVariables.FullPathToCertificatePfxPasswordFile,
Encoding.Default)[0];
password = password.Replace("\0", string.Empty);
// Load certificate from file
X509Certificate2 certificate = new X509Certificate2(
EnvironmentVariables.FullPathToCertificatePfxFile, password);
publicKey = certificate.GetRSAPublicKey();
privateKey = certificate.GetRSAPrivateKey();
}
With this, we have a solution that can be run and tested as-is if the environment variables have been set. The two used are:
- Custom_PfxFileName
- Custom_PfxPasswordFileName
There is a third environment variable, Fabric_Folder_App_Work, required for the solution to work. This environment variable is automatically set by the Service Fabric runtime; however, you would need to explicitly set it to be the base path in which the PFX and PFX password files would be found for this to work outside of a Service Fabric hosted container.
Make it Service Fabric Ready
How do we containerize our web application and setup a Service Fabric application to host and orchestrate the solution? Easy. Right-click on your ASP.NET Core Application project, and select Add > Container Orchestrator Support. Select Service Fabric from the dialog drop-down and confirm with OK. You now have a Service Fabric application that is configured to host your web application in a container.
Let’s look at what really happened when we added orchestrator support. First, the only change to our existing ASP.NET Core Web Application project is that a PackageRoot directory and a Dockerfile are added. The PackageRoot directory contains Service Fabric service artifacts that build out the service package such as the Config directory representing the Config package and the ServiceManifest.xml that declaratively defines the service type, version, and any other service-specific metadata. The Dockerfile allows Docker to build out the image for your ASP.NET Core Web Application and is used by the build and deploy tasks.
Second, there’s now a Service Fabric Application project in your solution. This declaratively defines the application type, version, and provides information about the services that make up that application. We’ll mainly be interested in the ApplicationManifest.xml under the ApplicationPackageRoot.
Configure the SetupEntryPoint and Scripts
The SetupEntryPoint is a great place to execute any kind of service setup that might require elevated permissions or access on the node. More information can be found in the documentation here.
We’ll need to create the Code directory under our service / ASP.NET Core Web Application project’s PackageRoot directory. This location will hold our setup scripts as they will now be part of the service fabric service code package. The SetupEntryPoint executable host requires either .EXE, .BAT, or .CMD files; however, we want to run a PowerShell script. To do that, simply create a CMD or BAT file with nothing more than a line to invoke the PowerShell executable with our PowerShell script.
1
2
@echo off
powershell.exe -ExecutionPolicy Bypass -Command ".\setup.ps1"
Our PowerShell script will be doing the work. The first thing it does is randomly generate a password that will be used to secure the exported PFX. It also depends on a new environment variable – Custom_CertThumbprint – that will help it find the correct certificate in the LocalMachine store. Once it’s found the certificate, it will create a file containing the password and export a PFX secured with that password to the locations built from both the Service Fabric work directory and provided Custom_PfxFileName and Custom_PfxPasswordFileName environment variables.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# Generate a random password. This is used to secure the exported PFX
# and will be added to the generated password file.
[Reflection.Assembly]::LoadWithPartialName("System.Web")
$password = [System.Web.Security.Membership]::GeneratePassword(20,2)
# Find the cert with the thumbprint matching that provided in the
# Custom_CertThumbprint environment variable. Additional
# checks such as finding the one with the latest expiration date
# may be preferred.
$matchedCerts = Get-ChildItem -Path cert:\LocalMachine\My | ?{ $_.Thumbprint -eq "$env:Custom_CertThumbprint" }
$certToExport = $matchedCerts[0]
# Build the destination full paths for the certificate and password
# file using Service Fabric's work directory for the service and
# provided Custom_PfxFileName and Custom_PfxPasswordFileName
# environment variables
$certDestinationPath = Join-Path -Path $env:Fabric_Folder_App_Work -ChildPath $env:Custom_PfxFileName
$passwordDestinationPath = Join-Path -Path $env:Fabric_Folder_App_Work -ChildPath $env:Custom_PfxPasswordFileName
# Write out the plain text password to file
$password | Out-File $passwordDestinationPath
# Export the PFX with private key
$securePassword = ConvertTo-SecureString -String $password -Force –AsPlainText
Export-PfxCertificate -Cert $certToExport -FilePath $certDestinationPath -Password $securePassword -Verbose
To let Service Fabric know that we have setup scripts for this particular service, we’ll need to update the ServiceManifest.xml by adding a SetupEntryPoint element under the CodePackage element. It will tell Service Fabric to start with the setup.cmd program and to run out of the CodePackage. This is important since our setup script expects our setup.ps1 script to be in the same directory as we’re running. You can also use a ConsoleRedirection element for debugging only.
1
2
3
4
5
6
7
<SetupEntryPoint>
<ExeHost>
<Program>setup.cmd</Program>
<WorkingFolder>CodePackage</WorkingFolder>
<ConsoleRedirection FileRetentionCount="5" FileMaxSizeInKb="2048" />
</ExeHost>
</SetupEntryPoint>
Service Fabric uses the ApplicationManifest.xml to control how services are run – including the SetupEntryPoint. To run our setup scripts with elevated permissions, we’ll need to update the Application Manifest. We’ll do two things – 1) define a SetupAdminUser in the Administrators group, and 2) add a RunAsPolicy element to our service to run the setup as the SetupAdminUser. This Principals element can be added to the end of the manifest, just after the DefaultServices element.
1
2
3
4
5
6
7
8
9
<Principals>
<Users>
<User Name="SetupAdminUser">
<MemberOf>
<SystemGroup Name="Administrators" />
</MemberOf>
</User>
</Users>
</Principals>
This RunAsPolicy element will need to be added to the Policies under the ServiceManifestImport for our encrypting / decrypting service. In this case, there’s only one, but if there is more than one service in your application be sure to add the policy to the correct one. This policy specifies only the Setup entry point execute as this user, so the actual container host will continue to execute as the default user – likely Network Service. We want to avoid increasing the privileges of the container host, so this is exactly what we want, but you could run the container host with administrator privileges as well by changing the EntrypPointType to Main or All for both to run elevated.
1
<RunAsPolicy CodePackageRef="Code" UserRef="SetupAdminUser" EntryPointType="Setup" />
Test it Out
With this in place, the code we wrote in our encryption and decryption service will now be able to pull from a known location at runtime – even when being hosted within the container on Service Fabric. Let’s test it out to be sure. To test, just publish the Service Fabric application to a Service Fabric cluster that supports Windows containers.
Note: Original image with caption ‘Publish Service Fabric Application dialog’ no longer available.
Since the controller was written to expect HTTP POST actions with bodies containing strings to be either encoded or decoded, you can use your favorite HTTP client to test. Just replace the cluster URL with the correct URL hosting your application. Don’t forget that you may need to update your load balancer to allow the port over which your service endpoint has been configured to pass through to your node(s). Also, make sure the certificate for which you’re providing a thumbprint is actually installed in the local machine store on all of the nodes.
Encryption
Note: Original image with caption ‘Postman screen capture showing encryption request and response’ no longer available.
Decryption
Note: Original image with caption ‘Postman screen capture showing decryption request and response’ no longer available.
Summary
If you’re only in need of certificates known to already be installed on the node, you should continue to use the guidance to import a certificate file through the CertificateRef element in the ContainerHostPolicies in your ApplicationManifest.xml. However, this blog covers an approach that allows greater control and flexibility at the time of service setup so that you can have runtime access to certificate files within your container. A complete working example is available on the AwkwardIndustries GitHub site in the servicefabric-containers-crypto repository.
Happy hacking!