clean-glass-36808
05/23/2024, 11:03 PMclean-glass-36808
05/23/2024, 11:06 PMArmadaTasks
are run locally they are actually just run against Armada, because that is what the agent code does. Realistically we're going to be wrapping most general compute tasks as an ArmadaTask
so we need some way for ArmadaTasks
to behave like PythonTasks
when run locally and exercise the plugin logic only when running remotely. I am digging into how pyflyte works now and I'm not seeing a clean way to do this yet.clean-glass-36808
05/23/2024, 11:25 PMclean-glass-36808
05/23/2024, 11:37 PMthankful-minister-83577
thankful-minister-83577
glamorous-carpet-83516
05/24/2024, 5:26 PMexecute
method
https://github.com/flyteorg/flytekit/blob/91cfb973eecca2ef35f0f5959f9754a1f51af451/plugins/flytekit-spark/flytekitplugins/spark/task.py#L186glamorous-carpet-83516
05/24/2024, 5:26 PMdef execute(self, **kwargs) -> Any:
return PythonFunctionTask.execute(self, **kwargs)
clean-glass-36808
05/24/2024, 5:57 PMclean-glass-36808
05/24/2024, 6:21 PM