argo-logo

How to use Argo Workflows¶

Prerequisites¶

Request access to Argo Workflows by contacting support@eodc.eu. You will receive a token required to authenticate. Mention in your request if you intend to use the eodc SDK.

Install¶

pip install eodc hera-workflows

Configure the connection¶

Before submitting any workflow, set the three required settings:

import eodc
from eodc.settings import settings

settings.FAAS_URL = "https://services.eodc.eu/workflows/"
settings.NAMESPACE = "default"
settings.ARGO_WORKFLOWS_TOKEN = ""  # your token here

Then create a service client:

service = eodc.faas.CustomWorkflow(
    processor_details=eodc.faas.FaasProcessor.custom
)

Submit a container-based workflow¶

The most common use case is running an existing Docker image across your data. Use a Container template to specify the image and command:

from hera.workflows import Container, Parameter, Step, Steps, Workflow

def my_workflow(workflows_service):
    with Workflow(
        workflows_service=workflows_service,
        namespace=workflows_service.namespace,
        generate_name="eo-processing-",
        entrypoint="steps",
    ) as w:
        process = Container(
            name="process",
            image="registry.eodc.eu/my-org/my-processor:latest",
            command=["python", "run.py"],
        )

        with Steps(name="steps"):
            Step(
                name="run-processor",
                template=process,
                arguments=[Parameter(name="input", value="s3://my-bucket/input/")],
            )
    return w

name = service.submit_workflow(
    workflow=my_workflow(service.workflows_service)
)
print(f"Submitted: {name}")

Container(image=..., command=...) — defines the Docker image and entrypoint. The image must be accessible from the eodc container registry or a public registry.
generate_name — used as a prefix for the workflow instance name. The full name is returned by submit_workflow.
Step(arguments=[Parameter(...)]) — passes runtime arguments into the container.
service.submit_workflow(...) — submits the workflow to the eodc Argo cluster and returns the workflow name.

Submit a multi-step workflow with parallel steps¶

For workflows with sequential and parallel stages, use s.parallel() to group steps that run concurrently:

from hera.workflows import Container, Step, Steps, Workflow

def pipeline(workflows_service):
    with Workflow(
        workflows_service=workflows_service,
        namespace=workflows_service.namespace,
        generate_name="pipeline-",
        entrypoint="steps",
    ) as w:
        task = Container(
            name="task",
            image="busybox",
            command=["echo"],
        )

        with Steps(name="steps") as s:
            Step(name="prepare", template=task,
                 arguments=[Parameter(name="message", value="preparing")])
            with s.parallel():
                Step(name="process-a", template=task,
                     arguments=[Parameter(name="message", value="tile-A")])
                Step(name="process-b", template=task,
                     arguments=[Parameter(name="message", value="tile-B")])
            Step(name="merge", template=task,
                 arguments=[Parameter(name="message", value="merging results")])
    return w

name = service.submit_workflow(
    workflow=pipeline(service.workflows_service)
)

Monitor and get logs¶

After submitting, retrieve logs programmatically:

service.get_logs(name)

To monitor visually, open the Argo dashboard — see Using the dashboard for a walkthrough with screenshots.

Tips¶

Workflows can read from and write to the eodc Object Store — useful for passing data between steps and storing results.
For scheduled processing (daily runs, etc.), use Argo’s CronWorkflow resource. Contact support@eodc.eu for help setting one up.
For a full tutorial series on Argo Workflows with Hera, see the Argo Workflows SDK notebook.