hi all, I have a task to call a client which used to run the pipeline in certain platform, and I wan...

millions-summer-85484

08/30/2023, 4:42 AM

hi all, I have a task to call a client which used to run the pipeline in certain platform, and I want to cancel the pipeline running when I click the terminate button in flyte UI, I have tried to use

atexist

to register

clinet.cancel_running

in the task, but termination of flyte workflow does not call

atexist

somehow, anyone knows why and how to fix? thanks!

millions-summer-85484

08/30/2023, 4:47 AM

Copy code

@task()
def client_task():
    import Client
    client = Client()
    client.execute_flow("a", "b")
    execid = client.execid
    import atexit
    atexit.register(client.cancel_execution, execid)

@workflow
def azkaban_client_workflow() -> None:
    azkaban_client_task()

tall-lock-23197

08/30/2023, 6:57 AM

@millions-summer-85484, since you want to terminate an execution that's running elsewhere, please make sure you're terminating it correctly. Is it working when you aren't including the same in a Flyte task?

freezing-airport-6809

08/30/2023, 2:41 PM

@millions-summer-85484 today we do not handle docker termination. https://docs.python.org/3/library/atexit.html Hence it does not. Get invoked. You can contribute a signal handler to pyflyte-execute

millions-summer-85484

08/30/2023, 4:05 PM

Is it working when you aren't including the same in a Flyte task?

it's working when I explicitly call the method of canceling. @freezing-airport-6809 I see, is there any other method to achieve the same effect?

freezing-airport-6809

08/30/2023, 5:47 PM

if you are simply calling external services, recommend Agents

tall-lock-23197

08/31/2023, 5:11 AM

https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/extend_flyte/agent_service.html

👀 1

Open in Slack

Previous Next

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.