Spoiler: I'm pretty much a Python noob. I've start...
# ask-the-community
b
Spoiler: I'm pretty much a Python noob. I've started writing a plugin, deriving my plugin Task from
PythonCustomizedContainerTask[T]
and
PythonTask[PluginConfig]
. I then started using my plugin in the workflow, which works out fine. Now I would like to apply caching and other usual decorators like requests and limit. How can I achieve this, as I don't have the classic
Task(...)
decorator for my plugin?
Copy code
class GlueTask(PythonCustomizedContainerTask[T], PythonTask[GlueConfig]):
    # following the usual init

class GlueTaskExecutor(ShimTaskExecutor[GlueTask]):
    def execute_from_model(self, tt: task_models.TaskTemplate, **kwargs) -> typing.Any:
        # my plugin execution code

# usage in workflow
GlueTask(name="testing", task_config=GlueConfig(crawler_name="test-crawler"), container_image="bla",cache=True, cache_version="1.0")()
That
cache
parameters don't throw errors, but also don't do caching.
Yes that worked 😄
GlueTask(name="testing", task_config=GlueConfig(crawler_name="test-crawler-delete-me"), container_image="bla", metadata=TaskMetadata(cache=True, cache_version="1.0"))()
y
nice. welcome @Broder Peters
so the background on this "PythonCustomizedContainerTask" is that we wanted to offer users a way to run tasks where the task definition (ie everything you need to run the task, is all encapsulated in the task template). in these cases, you don't need to build different containers for each task, you can just have one.
there's a new style forthcoming as well that will do something similar - the interface for which should we think is a bit simpler. be on the lookout for information on the flyte 'agent service' and agent tasks if interested.
b
Cool, thanks for the input!
118 Views