Advertisements

Substituting custom web.config sections with environment variables for docker

Substituting custom web.config sections with environment variables for docker

March 29, 2019 Uncategorized 0

Wow. That title is a mouthful.

TLDR: You can substitute web.config sections, even custom ones, using the ConfigurationBuilder in System.Configuration.

The enterprise

If you work with .Net, it’s pretty likely that you work for an enterprise that has an established catalog of applications. When you work in that sort of environment, it can be difficult to move some of those applications to a containerized ecosystem. One of the biggest headaches about moving .Net applications to containers, other than the size of the containers, is the fact that containers are supposed to be immutable. That can be a problem when trying to deploy the same container to multiple environments when you need to change connection strings, app settings, or even custom configuration section values.

ConfigurationBuilder to the rescue

While scouring the internet to find a solution to this problem, I found an interesting blog post discussing a new feature added in .Net 4.7.1; the ConfigurationBuilder. I’m not going to regurgitate the contents of the aforementioned blog post, but the gist of it was that you can replace appSettings and connectionStrings using environment variables. That’s all fine and dandy, but what if you have custom configuration sections? Well, since it gives you access to the XmlNode of the configuration section element before it gets loaded into the ConfigurationManager. That means this works with sections created with ConfigurationSectionDesigner.

The configuration sections

The first thing you need to do is analyze your configuration. In our demo, we have appSettings, connectionStrings, and a custom configuration section called website.

<configuration>
  <configSections>
    <section name="website" type="WebSite.Configuration.WebSite, WebSite.Configuration, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>    
  </configSections>
  
  <website xmlns="urn:WebSite.Configuration" name="My website" tag="development">
    <container name="local"/>
    <externalLinks>
      <link name="Google" url="https://www.google.com"/>
    </externalLinks>
  </website>

  <appSettings>
    <add key="webpages:Version" value="3.0.0.0"/>
    <add key="webpages:Enabled" value="false"/>
    <add key="ClientValidationEnabled" value="true"/>
    <add key="UnobtrusiveJavaScriptEnabled" value="true"/>
    <add key="SomeSetting" value="local setting"/>
  </appSettings>

  <connectionStrings>
    <add name="SomeName" providerName="Some provider" connectionString="local connection string"/>
  </connectionStrings>
  
  <!-- The rest of the configuration was left out for brevity -->
  
</configuration>

 

The environment variables

We wanted to go in a similar direction as configuration environment variables are done in AspNetCore, so we decided to prefix all environment variables for this demo with ASPNET__. This is similar to what is usually done in AspNetCore, with the prefix ASPNETCORE__.

We decided to have three distinct schemas for the environment variables.

appSettings and connectionStrings

The appSettings and connectionStrings sections have a predictable xml schema for the configuration file, so we could have an easy way to translate them to environment variables.

ASPNET__connectionStrings__{name}={value}
ASPNET__appSettings__{key}={value}

Custom configuration sections

If you have a custom configuration section, we also have a way to translate them to environment variables.

ASPNET__{element}__{attribute}={value}
ASPNET__{element}__{childelement}__{attribute}={value}
ASPNET__{element}__{collectionelement}__{collectionitemelement}[index]__{attribute}={value}

One thing, though

In a previous blog post, we ported the IIS service monitor which is the entry point of an IIS Docker container. This was done to fix a bug regarding environment variables keys being converted to upper case. The casing matters for this solution, so you can either use our ported IIS service monitor of wait for Microsoft to deploy their fix.

Our configuration builder

When implementing ConfigurationBuilder, there are two methods that you can override to manipulate the configuration before it gets loaded into memory in the ConfigurationManager.

public virtual ConfigurationSection ProcessConfigurationSection(ConfigurationSection configSection);
public virtual XmlNode ProcessRawXml(XmlNode rawXml);

We chose to process the raw xml, so we create three separate methods to process the appSettings, connectionStrings, and custom sections.

Processing appSettings

With app settings, we decided on the strategy of if there is an environment variable, but no counterpart is found in the web.config, it will be added anyway.

private XmlNode ProcessAppSettingsXml(XmlNode appSettings)
{
    var environment = GetEnvironmentVariables();
    var settings = appSettings.ChildNodes.OfType().ToDictionary(n => n.Attributes["key"].Value, n => n.Attributes["value"]);
    foreach (var variable in environment)
    {
        var prefix = $"{KeyPrefix}__appSettings__";
        if (!variable.Key.StartsWith(prefix)) continue;

        var key = variable.Key.Substring(prefix.Length);
        if (settings.ContainsKey(key))
        {
            settings[key].Value = variable.Value;
            continue;
        }
        // Add the app setting if missing from original configuration
        var element = appSettings.OwnerDocument.CreateElement("add");

        var elementKey = appSettings.OwnerDocument.CreateAttribute("key");
        elementKey.Value = key;
        element.Attributes.Append(elementKey);

        var elementValue = appSettings.OwnerDocument.CreateAttribute("value");
        elementValue.Value = variable.Value;
        element.Attributes.Append(elementValue);

        appSettings.AppendChild(element);
    }

    return appSettings;
}

Processing connectionStrings

To us, connection strings were a bit different than app settings. We felt that if a connection string that came from an environment variable didn’t have a counterpart in the web.config, we wouldn’t need it at all and it would be ignored.

private XmlNode ProcessConnectionStringsXml(XmlNode connectionStrings)
{
    var environment = GetEnvironmentVariables();
    var connections = connectionStrings.ChildNodes.OfType().ToDictionary(n => n.Attributes["name"].Value, n => n.Attributes["connectionString"]);
    foreach (var variable in environment)
    {
        var prefix = $"{KeyPrefix}__connectionStrings__";
        if (!variable.Key.StartsWith(prefix)) continue;

        var name = variable.Key.Substring(prefix.Length);
                
        if (connections.ContainsKey(name))
            connections[name].Value = variable.Value;

        // Skip the connection if missing from original configuration
    }

    return connectionStrings;
}

Processing custom configuration sections

Here come the meat and potatoes, folks.

For custom configuration sections, we need to parse the environment variable keys and navigate through the XmlNode to the specified attribute to change it.

private (ElementDescription[], string) ParsePath(string key)
{
    var split = key.Split(new[] { "__" }, StringSplitOptions.RemoveEmptyEntries).Skip(2); // skip prefix and section element name
    if (split.Count() == 0) return (new ElementDescription[0], null);
    var attribute = split.LastOrDefault();
    var path = split.Take(split.Count() - 1);
    return (path.Select(Convert).ToArray(), attribute);
}

private ElementDescription Convert(string part)
{
    var regex = new Regex(@"^(.*?)\[(\d+)\]$");
    var match = regex.Match(part);
    if (!match.Success) return new ElementDescription { Name = part };

    return new ElementDescription
    {
        Index = int.Parse(match.Groups[2].Value),
        Name = match.Groups[1].Value
    };
}

class ElementDescription
{
    public int? Index { get; set; }
    public string Name { get; set; }
}

Now that we have a way to parse the path, we can process the XmlNode.

private XmlNode ProcessCustomConfigurationSection(XmlNode section)
{
    var prefix = $"{KeyPrefix}__{section.LocalName}__";
    var environment = GetEnvironmentVariables();
    var keys = environment.Keys.Where(key => key.StartsWith(prefix));
    foreach (var key in keys)
    {
        var path = ParsePath(key);
        var xml = section;
        foreach (var part in path.Item1)
        {
            var element = null as XmlElement;
            if (part.Index == null)
                element = xml[part.Name];
            else
                element = xml.ChildNodes.OfType().ElementAtOrDefault(part.Index.Value);

            if (element == null)
            {
                element = xml.OwnerDocument.CreateElement(part.Name, section.NamespaceURI);
                xml.AppendChild(element);
            }

            xml = element;
        }
        var attribute = xml.Attributes[path.Item2];
        if (attribute == null)
        {
            attribute = xml.OwnerDocument.CreateAttribute(path.Item2);
            xml.Attributes.Append(attribute);
        }
        attribute.Value = environment[key];
    }
    return section;
}

Putting it together

Now we override the ProcessRawXml method, check the xml element, and call the corresponding method to

private const string KeyPrefix = "ASPNET";
public override XmlNode ProcessRawXml(XmlNode rawXml)
{
    if (rawXml.LocalName == "appSettings") return ProcessAppSettingsXml(rawXml);
    if (rawXml.LocalName == "connectionStrings") return ProcessConnectionStringsXml(rawXml);
    return ProcessCustomConfigurationSection(rawXml);
}

Using it in the web.config

Using our configuration builder (or your if you implement your own) is pretty easy. You just need to add a couple of things to your web.config to specify that the builder should be run.

<configuration>
  <configSections>
    <section name="website" type="WebSite.Configuration.WebSite, WebSite.Configuration, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
    <section name="configBuilders" type="System.Configuration.ConfigurationBuildersSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" restartOnExternalChanges="false" requirePermission="false"/>    
  </configSections>

  <configBuilders>
    <builders>
      <add name="environment" type="Solid.Containers.Configuration.EnvironmentConfigurationBuilder, Solid.Containers.Configuration" />
    </builders>
  </configBuilders>
  
  <website xmlns="urn:WebSite.Configuration" name="My website" tag="development" configBuilders="environment">
    <container name="local"/>
    <externalLinks>
      <link name="Google" url="https://www.google.com"/>
    </externalLinks>
  </website>

  <appSettings configBuilders="environment">
    <add key="webpages:Version" value="3.0.0.0"/>
    <add key="webpages:Enabled" value="false"/>
    <add key="ClientValidationEnabled" value="true"/>
    <add key="UnobtrusiveJavaScriptEnabled" value="true"/>
    <add key="SomeSetting" value="local setting"/>
  </appSettings>

  <connectionStrings configBuilders="environment">
    <add name="SomeName" providerName="Some provider" connectionString="local connection string"/>
  </connectionStrings>
  
  <!-- The rest of the configuration was left out for brevity -->
  
</configuration>

As you can see, our ConfigurationBuilder implementation has been added to the config and named environment. Any section that is going to be altered needs to reference the environment configBuilder by adding a configBuilder=”environment” attribute to it’s root element.

Running the demo

If you clone the repo that is paired with this blog post, you can try this out yourself.

Local

If you first run the website that is part of the solution locally, you’ll see the default settings being used. The title will just be My Website, only one external link will be in the navigation bar, and the container name in the footer will be local. Also, you will see the app settings, connection strings, and relevant environment variables (prefixed with ASPNET) listed on the page.

local

Docker with environment variables

Building and running the website in a container with environment variables is done with the following command.

PS> docker build --rm -t solid/docker-demo-site -f WebSite/Dockerfile .
PS> docker run -d --rm -p 8080:80 `
      -e ASPNET__website__name='Dockerized' `
      -e ASPNET__website__container__name='solid/docker-demo-site' `
      -e ASPNET__website__externalLinks__link[1]__name='Solid softworks' `
      -e ASPNET__website__externalLinks__link[1]__url='https//solidsoft.works' `
      solid/docker-demo-site

After the loooong process of pulling the huge Windows base docker image and building this container, you can visit this site on http://localhost:8080. You’ll see the website title has changed, another external link has been added, and the container name is in the footer.

docker

You can play around with different environment variables and see how the website changes. Try adding an ASPNET__website__tag environment variable.

Advertisements

 

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.