Processor¶

sosw.app.Processor is the core class of the framework. Your Lambda implements a subclass of it (conventionally called Processor in your app.py), declares its dependencies and defaults in DEFAULT_CONFIG, and puts the business logic into __call__.

from sosw.app import LambdaGlobals, get_lambda_handler, Processor as SoswProcessor


class Processor(SoswProcessor):

    DEFAULT_CONFIG = {
        'init_clients': ['DynamoDb'],
        'dynamo_db_config': {
            'table_name': 'things',
            'row_mapper': {'thing_id': 'S'},
        },
    }


    def __call__(self, event, **kwargs):
        super().__call__(event)
        ...


global_vars = LambdaGlobals()
lambda_handler = get_lambda_handler(Processor, global_vars)

Initialization pipeline¶

Processor.__init__ runs the following steps:

Resolve the effective test flag: an explicitly passed test keyword always wins, otherwise it derives from the STAGE environment variable (test / autotest stages run in test mode).
Capture the AWS account id from the Lambda context (when available in global_vars).
Assemble self.config (see Configuration layering below).
Initialize self.stats and self.result counters (collections.defaultdict(int)).
Register the clients listed in the init_clients config parameter (see Client registration).

Configuration layering¶

The configuration of a Processor is assembled by init_config() from three layers, each recursively updating the previous one:

DEFAULT_CONFIG — the class attribute of your Processor. Code-level defaults.
The per-function config from DynamoDB / SSM: the record named {AWS_LAMBDA_FUNCTION_NAME}_config. This is how you change the behavior of a deployed Lambda without redeploying it. Read more: Configuration.
custom_config — the argument passed to the constructor (usually via get_lambda_handler(Processor, global_vars, custom_config)). The final word.

The external lookup of layer 2 can be skipped — for functions that have no config table, no permissions to read it, or simply no use for runtime overrides — in either of two ways:

class Processor(SoswProcessor):
    DISABLE_DDB_CONFIG = True                       # class attribute, or:
    DEFAULT_CONFIG = {'disable_ddb_config': True}   # config flag (works in custom_config too)

DEFAULT_CONFIG and custom_config are still applied in this case.

Note

disable_ddb_config was absorbed from community PR #376.

Anywhere in your code, read configuration values through the self._c() shortcut with dot notation and an optional default:

chunk_size = self._c('processing.chunk_size', 100)

Client registration¶

register_clients() initializes the clients listed in the init_clients config parameter and assigns them to the Processor with the _client suffix. For every name it tries, in order:

a module in the components or managers package of your Lambda;
a module in sosw.components — names are the CamelCase class stem, e.g. 'DynamoDb' → DynamoDbClient, 'Siblings' → SiblingsManager;
a plain boto3.client() — e.g. 'sts' → self.sts_client = boto3.client('sts').

Class-based clients receive their config from the {module_name}_config key of the Processor config (as custom_config for *Manager classes and config for *Client classes).

For DynamoDB specifically, prefer the lazy factory get_ddbc(prefix): declare any number of {prefix}_dynamo_db_config blocks in the config and get fully configured, cached DynamoDbClient instances on demand:

DEFAULT_CONFIG = {
    'things_dynamo_db_config':  {'table_name': 'things', 'row_mapper': {'thing_id': 'S'}},
    'orders_dynamo_db_config':  {'table_name': 'orders', 'row_mapper': {'order_id': 'S'}},
}

def __call__(self, event, **kwargs):
    super().__call__(event)
    things = self.get_ddbc('things').get_by_scan()

The `stats` / `result` contract¶

Two counters with two different lifetimes:

self.result: Per-invocation state. Reset to an empty defaultdict(int) at the beginning of every __call__ (unless you call super().__call__(event, reset_result=False)). Put anything here that describes the outcome of the current invocation.
self.stats: Per-container counters. __call__ increments processor_calls; the @benchmark decorator adds execution times; your code adds anything else. After every invocation the generated handler logs get_stats() and calls reset_stats(recursive=True) exactly once: ordinary counters are folded into total_* accumulators which survive for the container lifetime, and the parameters listed in the lifetime_stats_params config are preserved as-is.

Note

In the 0.7.x line the handler called reset_stats() twice per invocation; it is now called once (recursively). See the migration guide.

Custom clients participate in both get_stats() and reset_stats() recursively when they implement these methods themselves.

Failing loudly¶

sosw code is expected to fail fast and loud. When a Processor cannot continue, call self.die(message): it logs the message with the current stats, attempts a best-effort SNS notification to the topic from the dead_sns_topic config parameter (default SoswWorkerErrors), and terminates the invocation.

The single deliberate exception to the rule is LambdaApi, which converts exceptions into well-formed HTTP error responses — an API must always answer its clients.

API reference¶

View Licence Agreement

class sosw.app.Processor(custom_config=None, **kwargs)[source]¶

Core Processor class template. This is the base class of the framework: every Lambda built on sosw subclasses it (directly, or through specializations like LambdaApi). It provides layered configuration, automatic client registration, statistics counters and a uniform entry point.

get_ddbc(prefix: str) -> DynamoDbClient:

Lazily initializes and retrieves a DynamoDB client configured for a specific table and schema validation.

This method initializes custom DynamoDB clients based on the provided prefix. If a client with the specified prefix has already been initialized, it returns the existing client. If not, it looks in the Processor config for a prefixed dynamodb config (e.g. for prefix project_a -> project_a_dynamo_db_config). The dynamo_db client will be initialized as self.project_a_dynamo_db_client.

This is particularly useful for scenarios requiring schema validation, transformation of DynamoDB syntax to dictionary format, and other operations beyond the capabilities of the raw boto3 client.

Parameters:: prefix (str) – The prefix for the DynamoDB client configuration and naming.
Raises:: ValueError – If the provided prefix is not supported by the available configuration.

init_config(custom_config: Dict = None)[source]¶

By default, tries to initialize config from DEFAULT_CONFIG or as an empty dictionary. After that, a specific custom config of the Lambda will recursively update the existing one. The last step is update config recursively with a passed custom_config.

The lookup of the specific custom config of the Lambda ({AWS_LAMBDA_FUNCTION_NAME}_config from DynamoDB / SSM) can be skipped either by setting the class attribute DISABLE_DDB_CONFIG = True in your Processor, or by passing a truthy disable_ddb_config key in the DEFAULT_CONFIG or custom_config. DEFAULT_CONFIG and custom_config are still applied in this case.

Overwrite this method if custom logic of recursive updates in configs is required.

Note

Processor¶

Initialization pipeline¶

Configuration layering¶

Client registration¶

The stats / result contract¶

Failing loudly¶

API reference¶

The `stats` / `result` contract¶