Config Source

View Licence Agreement

Config manager component. Has methods for getting configuration for Lambdas.

The default is DynamoConfig. It is automatically called from the app.Processor.__init__ and if your Lambda has permissions to access the config table, the Processor will look for the record: YOUR_FUNCTION_NAME_config, and recursively update the DEFAULT_CONFIG with it.

You can also import and use the following functions. They can be directly imported from this module and will be automatically switched to DDB / SSM / Secrets manager.

Warning

Using these methods requires the Role to have relevant permissions to access DynamoDB / SSM Parameter Store / AWS Secrets.

Usage example (pseudo code):

from SOME_DB_DRIVER import connect
from sosw.components.config import get_secrets_credentials, get_config, update_config
from sosw.app import Processor as SoswProcessor

class Processor(SoswProcessor):

    def work_with_db(self):

        db_settings = get_credentials_by_prefix('db_')
        db_password = get_secrets_credentials(type='name', value='db_password')['db_password']

        connection = connect(**db_settings, password=db_password)

        last_processed_row = get_config('last_row')

        db_result = connection.query(f"SOME QUERY LIMIT last_processed_row, 10;")
        self.do_something(db_result)

        update_config('last_row', last_processed_row + 10)
class sosw.components.config.ConfigSource(test=False, sources=None, config=None, **kwargs)[source]

A strategy adapter for config. Returns config from the selected config source. You can implement your own functions using these clients, and they can even call different configurations.

Parameters:

  • sources (str) – Config clients to initialize. Supported: SSM and Dynamo (default). Could be both, comma-separated. The first one then becomes default.

  • config (dict) – Custom configurations for clients. Should be in ssm_config, dynamo_config, etc. Don’t be confused, but sometimes configs also need their own configs. :)

class sosw.components.config.DynamoConfig(**kwargs)[source]

This is a manager to operate with custom configurations for Lambdas stored in the config DynamoDB table. It tries to find the table config in the same region where the Lambda runs with the following structure:

'env':          'S',
'config_name':  'S',
'config_value': 'S',

If the table exists and the Lambda has permissions to access it, the class will look for the record: YOUR_FUNCTION_config, and recursively update the DEFAULT_CONFIG with it.

get_config(name, env='production')[source]

Retrieve the Config from DynamoDB config table and return as a JSON parsed dictionary. If not in JSON format, returns a string.

Note

In case the environment is test the table name will be automatically changed to autotest_config. This might be relevant only for complex integration tests.

Parameters:

  • name (str) – Name of config to extract

  • env (str) – Environment name, usually: ‘production’ or ‘dev’

Return type:

dict|string

Returns:

Configuration

get_credentials_by_prefix(prefix: str, env: str = 'production') → dict[source]

Fetches multiple records from the config table. Filters rows that start with the prefix and returns them as a dict with this prefix trimmed.

Example in config DDB:

{'env': 'production', 'config_name': 'project_a_db_username', 'config_value': 'john'}
{'env': 'production', 'config_name': 'project_a_db_port', 'config_value': '27019'}
{'env': 'production', 'config_name': 'another_db_username', 'config_value': 'silver'}


credentials = get_credentials_by_prefix('project_a_db')

# {'username': 'john', 'port': '27019'}
update_config(name, val, **kwargs)[source]

Update a field in the DynamoDB config table with a new value. May be used to store not sensitive tokens.

Parameters:

  • name (str) – Field name to address.

  • val (object) – Field value to update.

class sosw.components.config.SecretsManager(test=False, **kwargs)[source]
call_boto_secrets_with_pagination(f: str, **kwargs) → list[source]

Invoke SecretsManager functions with the ability to paginate results.

Parameters:

f – SecretsManager function to invoke.

Returns:

If the function can be the paginate the response will return as paginate iterator. Else it will return as a list

get_secrets_credentials(**kwargs) → dict[source]

Retrieve the credentials with given name or tag from AWS SecretsManager and return as a dictionary.

Must provide type and value to search. Type is either 'tag' or 'name'. value is a string.

Usage example:

my_secret =  get_secrets_credentials(type='name', value='my_secret_name')

my_secrets_by_tag = get_secrets_credentials(type='tag', value='project_a_credentials')
class sosw.components.config.SSMConfig(test=False, **kwargs)[source]

Methods to access some configurations and/or credentials stored in AWS SSM ParameterStore. Please note that SSM has a pretty low limit of concurrent calls and it THROTTLES. For high load Lambdas it is recommended to use DynamoConfig instead.

call_boto_with_pagination(f, **kwargs) → list[source]

Invoke SSM functions with the ability to paginate results.

Parameters:

  • f (str) – SSM function to invoke.

  • kwargs (object) – Keyword arguments for the function to invoke.

get_config(name)[source]

Retrieve the Config from AWS SSM ParameterStore and return as a JSON parsed dictionary.

Parameters:

name (str) – Name of config to extract

Return type:

dict

Returns:

Config of some Controller

get_credentials_by_prefix(prefix)[source]

Retrieve the credentials with given prefix from AWS SSM ParameterStore and return as a dictionary.

In ParameterStore the values Name must begin with prefix_ and they must have Tag:Environment (production|dev). The type of elements is expected to be SecureString. Regular strings could work, but not guaranteed.

Parameters:

prefix (str) – prefix of records to extract

Return type:

dict

Returns:

Some credentials

update_config(name, val, **kwargs)[source]

Update a parameter in SSM ParameterStore with a new value.

Parameters:

  • name (str) – Parameter name to address.

  • val (object) – Parameter value to update.