Skip to main content

Workers

Workers are standalone processes that host workflow components for Stepflow. They allow you to implement custom logic, integrate with external systems, and extend the capabilities of Stepflow beyond the built-in components.

What is a Worker?​

A worker is a process that:

  • Hosts components: Provides one or more executable components for workflows
  • Pulls tasks: Pulls tasks from named queues hosted by the orchestrator
  • Returns results: Executes components and reports results back to the orchestrator via gRPC
  • Calls back: Can make gRPC requests back to the orchestrator (e.g., submit sub-workflows, store blobs)

Because workers communicate over an open protocol (task queues for dispatch, gRPC for results and callbacks), they can be written in any language, deployed independently, and scaled horizontally.

Getting Started​

There are two main approaches to creating workers:

The Python SDK provides a high-level API that handles all protocol details:

from stepflow_py import StepflowServer

server = StepflowServer()

@server.component
def my_component(input: MyInput) -> MyOutput:
    return MyOutput(result=process(input.data))

if __name__ == "__main__":
    server.run()

2. Implementing the Protocol Directly​

For other languages, you can implement the Stepflow Protocol directly. The protocol is based on gRPC with Protocol Buffers.

Next Steps​