AWS Lambda Feature Toggles Made Simple

“Feature Toggles are a powerful technique, allowing teams to modify system behavior without changing code…. Toggles introduce complexity. We can keep that complexity in check by using smart toggle implementation practices and appropriate tools to manage our toggle configuration, but we should also aim to constrain the number of toggles in our system” — Martin Fowler

Feature toggles are a powerful tool but as Martin Fowler mentioned in this article, they can also be quite hard to implement.

In this blog I’ll demonstrate how feature toggles in AWS can become a simple task and a great tool that you must include in your AWS Lambdas.

I’ll implement both static and dynamic feature toggles. I’ll demonstrate that by using Python, AWS AppConfig service, AWS Lambda Powertools and Pydantic, you can start using feature toggles in your lambdas with ease right away.

Static hardcoded feature toggles

You can find below a cool one liner trick to parse & validate Lambda environment variables with Pydantic.

In this example, we have 2 environment variables: env_var_1 and env_var_2. They are modelled into a Pydantic schema class in line 4 as two strings parameters.

In line 18, the lambda_handler function calls the check_env_vars which parses the os.enviorn variable (line 15) and extracts from it only the defined environment variables, env_var_1 and env_var_2. In case of a missing parameter, a detailed ValidationError exception is raised.

load & validate environment variables into a global singleton with Pydantic

CONF is now initialised with the environment variables and can be used across the lambda code in the same manner it’s used in line 19, as a dataclass.

You can read more about Pydantic and it’s parsing prowess in my first blog.

Ok great, this works BUT…

There’s a better way, use dynamic feature toggles. Use AWS App Config.

Dynamic Feature Toggle Configuration

“Using a general purpose DB which is already part of the system architecture to store toggle configuration is very common; it’s an obvious place to go once Feature Flags are introduced and start to gain traction. However nowadays there are a breed of special-purpose hierarchical key-value stores which are a better fit for managing application configuration…..

These services form a distributed cluster which provides a shared source of environmental configuration…. Configuration can be modified dynamically whenever required..” — Martin Fowler

What is AWS AppConfig?

How does it work?

‘test-service’ application at the top level

An application has a list of environments (dev, stage, production etc.)

Then you create an environment.

‘dev’ environment of test-service application

Each environment can have multiple configurations profiles.

Each profile defines the current version of a configuration, its’ values (in JSON/YAML/plain text format) and the deployment strategy to use when deploying it. You can then finally deploy the configuration profile which is listed in the deployments tab. Our configuration is consisted of 3 parameters: 2 booleans and one string.

JSON configuration that will be deployed under the name ‘test-applicaton-profile’
‘test-applicaton-profile’ configuration is deployed

You can read more about deployment strategies here.

I’d like to stop here and not dive too deep into AppConfig specifics and move on. AppConfig is awesome and you should read more about it here.

Let’s consume the configuration

The lambda will fetch the configuration from AWS AppConfig. If a new configuration is deployed into AppConfig, the lambda will fetch the new configuration and change its’ behavior (according to the fetched values) without the need to redeploy the Lambda itself, hence being dynamic.

This sounds simple, but as Martin Fowler stated, this is no easy feat. So how do we do that?

AWS Lambda Powertools to the rescue!

This repo, which started mostly Python oriented (but now supports other languages such as Java and more to follow) provides an easy to use solutions for lambda logging, tracing (with CloudWatch metrics), SSM utilities, and even validation and advanced parsing for incoming AWS Lambda events. Check out my previous blog on the latter.

The new AppConfig utility provides an easy way to consume configuration from AWS AppConfig and save it in a dedicated local cache. The cache also has a configurable TTL.

Once the configuration is received, we will parse it with Pydantic and create a singleton global instance of the feature toggles schema in a similar way we did with the environment variables CONF instance.

Building upon our first environment variable example with Pydantic, we will now add 3 new environment variables: application name, environment name, and configuration name which define the configuration we wish to extract from AWS AppConfig.

These variables will determined during lambda deployment. If you deploy to a dev environment — environment name should be ‘dev’, for test ‘test’ and so forth. This will allow you to deploy to multiple environment while using a different configuration per environment.

In our case the configuration name is ‘test-application-profile’, the application name is ‘test-service’ and the environment name is ‘dev’ which are represented by the APP_CONF_APPLICATION, APP_CONF_ENVIORMENT and APP_CONF_CONFNAME environment variables at lines 8–10 below.

Now let’s define our feature toggles schema. We define 2 boolean toggles: feature1, feature2 and 1 string log_level parameter. I’ll explain later on why these are defined as Optional with a default value.

Now, let’s import AppConfigProvider class from Lambda Powertools and use it to load a configuration from AWS AppConfig according to the environment variables: APP_CONF_APPLICATION, APP_CONF_ENVIORMENT and APP_CONF_CONFNAME.

In line 8, AppConfigProvider is configured with the AWS AppConfig parameters: application and environment.

Line 11 is the actual call to AWS AppConfig. The utility AppConfigProvider get function is used to retrieve a specific configuration, to store it in its’ internal cache for 600 seconds and to also return the value as a JSON Python Dict object which will be saved to conf_json variable.

Each configuration ‘get’ call to AWS AppConfig costs money, which makes the cache option very useful. As long as the 600 seconds TTL haven’t passed from the first ‘get’, you wont connect again to AppConfig and won’t be charged. Very nice indeed.

Line 14 causes the configuration Dict, conf_json to be parsed, validated and loaded into the FEATURE_TOGGLES global class instance of type AppConfToggles. In case of an error, a Pydantic ValidationError exception is raised with a detailed error message.

Combining it all together will look like this:

Full implementation

Finally, now that we have consumed the feature toggles configuration, parsed and validated them, we can use them in line 54 and 56 accordingly.

How do I change my AppConfig configurations?

Following this logic, the AWS AppConfig creation and configuration must be managed though a different code repository and CI/CD pipeline. This separation will allow to deploy changes to AppConfig without triggering the entire Lambda pipeline and make this solution truly dynamic.

different repos hold different responsibilities

Feature Toggles Use Cases

However, since the toggles are defined in the AppConfToggles class as Optional with a default value, toggles can be removed or added without fear of breaking the code during runtime. This definition minimizes the coupling.

Adding a new feature toggle

  1. AppConfToggles doesn’t include feature3 yet so it won’t crash when parsing the new configuration from AWS AppConfig (it ignores feature3 since it’s not defined in the AppConfToggles schema). Add the new toggle to the AppConfToggles class with type Optional and set a default value. Run CI/CD pipeline, deploy the lambda.

Modifying a value of an existing toggle, i.e feature1

  1. Lambda’s repo doesn't require any change. It will load the new value once the AppConf provider’s cache expires (600 seconds in our case).

Removing a toggle, i.e feature2

  1. Since all toggles are defined as Optional with a default value in the Pydantic schema, removing feature1 from AppConf won’t break the Lambda in runtime. Remove feature1 from the Lambda repo, run CI/CD pipeline.

As you can see, all possible changes are safe and won’t crash your Lambda during runtime.

Summary

For the dynamic version, we used AWS AppConfig in combination of the excellent PowerTools utility to easily consume and validate the feature toggles.

Pros:

  1. Feature toggle changes are dynamic and take short time to deploy via a dedicated & tracked CI/CD pipeline.
  2. Once the new toggle’s configuration is deployed and AppConfigProvider cache expires, the lambda will use the new configuration.

4. No change can break your lambda during runtime in any scenario.

Cons:

  1. Each feature toggle change requires a PR and a code review (can also be listed under pros).

I hope you found this blog useful, feel free to leave any comment below.

Acknowledgements

Alon Sadovski

Heitor Lessa

Software Architect at CyberArk. Designs and develops SaaS applications and K8s microservices applications.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store