Hi! I'm porting some stuff to Flyte to see how it compares to our current solution, but I get stuck at a very simple thing. I need to do datetime arithmetic in numpy, so how to I pass datetype parameters to a workflow? np.datetime64 is not supported and trying to parse a string errors. Thanks!

import numpy as np
from flytekit import workflow

@workflow
def wf(date: str = "2022-01-01"):
    date = np.datetime64(date) # this errors

You should be able to use a python datetime as input type: https://docs.flyte.org/projects/cookbook/en/latest/auto/core/type_system/flyte_python_types.html Also the actual processing needs to be done inside a task. Workflows are basically a DSL to compose tasks. I.e. something like this (not tested):

from datetime import datetime
@task
def t(date: datetime):
    date = np.datetime64(date)
    ...

@workflows
def wf(date: datetime = datetime(2022,1,1)):
    t(date=date)

hi @Sebastian so one distinction between

@task

and

@workflow

is that you can think of

@task

as a regular python function, whereas the code within a

@workflow

is actually execution graph constructor language. So

@task

functions implements the code that does some unit of work, whereas

@workflow

functions define the logic of how to put multiple tasks (units of work) together in some sequence

Thank you for your response @Niels Bantilan. I somehow didn't find the cookbook before, but after reading it this is much clearer. I moved everything to a task and started splitting it up from there and it all makes sense now

hi @Sebastian just following up here: are you unblocked now on porting your current solution to Flyte? Don’t hesitate to reach out here. Just so we can improve the experience in the future, where did you start in the Flyte docs, and how can we make the cookbook more discoverable?

Hi Niels, thank you very much for your assistance. I have managed to set it up. I think the docs have changed since I browsed them last? I don't recognise them. One thing I have not really seen though: Can you please point me to any documentation on how I could do backfilling in a time range using Flyte? If I change the way some derived tables are computed, and those computations are run on Flyte, I would like to be able to re-run them. Thanks!