Resources#

Resources are used in Dagster to make external resources (like a database connection) available to solids during pipeline execution. When used alongside modes, resources make it easier to write pipelines that can be developed locally and tested.

Relevant APIs#

Name	Description
@resource	The decorator used to define resources. The decorated function is called a resource_fn. The decorator returns a ResourceDefinition
ResourceDefinition	Base class for solids. You almost never want to use initialize this class directly. Instead, you should use the @resource which returns a ResourceDefinition

Overview#

We recommend learning about Modes before learning about resources.

Resources ...

Defining a Resource#

To define a resource, we use the @resource decorator:

class DatabaseConnection():
    def execute_query():
        pass

@resource
def db_resource(init_context):
    return DatabaseConnection()

The @resource decorator decorates a function that returns a resource. The object returned from the resource function itself can be any standard python object - it doesn't need to inherit any particular interface or implement any particular functionality.

Resource Configuration#

ResourceDefinitions can have a config schema, which allows you to customize behavior at runtime through pipeline configuration.

For example, let's say we wanted to pass a connection string to our DatabaseConnection resource.

class DatabaseConnection():
    def __init__(connection: str):
        self.connection = connection

@resource(config_schema={"connection": str})
def db_resource(init_context):
    connection = init_context.resource_config["connection"]
    return DatabaseConnection(connection)

Resource to Resource Dependencies#

Resources can depend upon other resources. If a resource key is included in the required_resource_keys set provided to the resource definition, then the resource initializer can access a required resource via the "resources" attribute of its context object.

Default Resources#