I'm currently trying to enhance the functionality ...
# ask-the-community
b
I'm currently trying to enhance the functionality of my tasks by decorating tasks. I am trying to use FlyteFile within the decorators themselves. The key thing is that the decorators will not be returning the
FlyteFile
, but the output of the function which it's decorating. For example:
Copy code
def my_decorator(fn):
    @wraps(fn)
    def _wrapper(*args, **kwargs):
        out = fn(*args, **kwargs)
        ... # process and save 'out' to some local file 
        file_handler = FlyteFile(path=some_path, remote_path=remote_path)
        return out
    return _wrapper

@task
@my_decorator
def some_func(value: int) -> int:
    return value * 2
However, it seems that when I use it in this way, the file does not get uploaded to the
remote_path
specified. I'm not very clear about how the internals of
FlyteFile
works, but I was wondering if the file will only be uploaded if the
FlyteFile
were created and returned directly within the decorated function itself? Is there any other way that I could still get this to work within my decorator function instead? Thanks!
j
but your task doesnt return FlyteFile so it will not be uploaded right?
y
the upload is triggered by the type engine. if you call
<http://TypeEngine.to|TypeEngine.to>_literal
on the instance that will suffice.
though if you discard that result, that’s also a bit weird
b
Thanks, will look into this
I managed to get this to work, but I'm curious to know why you say that discarding the output is weird, are there any unintended side effects that you foresee @Yee?
y
no
just like, you upload.
but where you uploaded to is returned by the type engine, the s3 or gs path right?
and if you ignore that, how do you keep track of it?
149 Views