{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Demo Argo Workflow - Submit Custom\n", "\n", "First we need to provide some settings configuration in order to be able to interact with the service.\n", "\n", "1. FAAS_URL. This is the URL of the Argo Workflow service.\n", "2. NAMESPACE. This is the intended namespace in the kubernetes cluster for creating the Argo Workflows.\n", "3. ARGO_WORKFLOWS_TOKEN. This is the token that will be used to authenticate with the Argo Workflows server." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import eodc" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from eodc.settings import settings\n", "\n", "settings.FAAS_URL =\"https://services.eodc.eu/workflows/\"\n", "settings.NAMESPACE = \"default\"\n", "settings.ARGO_WORKFLOWS_TOKEN = \"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Within eodc.faas ther CustomWorkflow class makes use of the FaasProcessorBase that has a number of functions defined for interacting with Argo Workflow resources. We will use the eodc.faas.FaasProcessor.custom in order to instantiate a service connection that will allow us to submit an arbitrary Argo Workflow.\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "service = eodc.faas.CustomWorkflow(\n", " processor_details=eodc.faas.FaasProcessor.custom\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using [Hera](https://hera.readthedocs.io/en/stable/), it is possible to define Argo Workflow resources in code. The following is an example resource taken from their documentation." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from hera.workflows import Container, Parameter, Step, Steps, Workflow, WorkflowsService\n", "\n", "def hello(service: WorkflowsService):\n", " with Workflow(\n", " workflows_service=service,\n", " namespace=service.namespace,\n", " generate_name=\"steps-\",\n", " entrypoint=\"hello-hello-hello\",\n", " ) as w:\n", " print_message = Container(\n", " name=\"print-message\",\n", " image=\"busybox\",\n", " command=[\"echo\"],\n", " )\n", "\n", " with Steps(name=\"hello-hello-hello\") as s:\n", " Step(\n", " name=\"hello1\",\n", " template=print_message,\n", " arguments=[Parameter(name=\"message\", value=\"hello1\")],\n", " )\n", "\n", " with s.parallel():\n", " Step(\n", " name=\"hello2a\",\n", " template=print_message,\n", " arguments=[Parameter(name=\"message\", value=\"hello2a\")],\n", " )\n", " Step(\n", " name=\"hello2b\",\n", " template=print_message,\n", " arguments=[Parameter(name=\"message\", value=\"hello2b\")],\n", " )\n", " return w" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once you have an Argo Workflow resource that is has been wrapped in a function that will passes the workflow service on instantiation. You will be able to submit it to the Argo Workflow server in the following way." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "name = service.submit_workflow(\n", " workflow=hello(service.workflows_service)\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A number of functions are available via the eodc-sdk to monitor and interact with Argo Workflows. To get the workflow logs, we can use the .get_logs() function." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "service.get_logs(name)" ] } ], "metadata": { "kernelspec": { "display_name": "Python (env_zarr)", "language": "python", "name": "env_zarr" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.5" } }, "nbformat": 4, "nbformat_minor": 2 }