https://flyte.org logo
#ask-the-community
Title
# ask-the-community
s

Sebastian

04/20/2022, 9:17 AM
Hi! I'm porting some stuff to Flyte to see how it compares to our current solution, but I get stuck at a very simple thing. I need to do datetime arithmetic in numpy, so how to I pass datetype parameters to a workflow? np.datetime64 is not supported and trying to parse a string errors. Thanks!
Copy code
import numpy as np
from flytekit import workflow

@workflow
def wf(date: str = "2022-01-01"):
    date = np.datetime64(date) # this errors
s

Sören Brunk

04/20/2022, 12:25 PM
You should be able to use a python datetime as input type: https://docs.flyte.org/projects/cookbook/en/latest/auto/core/type_system/flyte_python_types.html Also the actual processing needs to be done inside a task. Workflows are basically a DSL to compose tasks. I.e. something like this (not tested):
Copy code
from datetime import datetime
@task
def t(date: datetime):
    date = np.datetime64(date)
    ...

@workflows
def wf(date: datetime = datetime(2022,1,1)):
    t(date=date)
n

Niels Bantilan

04/20/2022, 1:52 PM
hi @Sebastian so one distinction between
@task
and
@workflow
is that you can think of
@task
as a regular python function, whereas the code within a
@workflow
is actually execution graph constructor language. So
@task
functions implements the code that does some unit of work, whereas
@workflow
functions define the logic of how to put multiple tasks (units of work) together in some sequence
When porting code over to Flyte, I’d recommend: • starting out by factoring the main components of your code with
@task
functions (maybe even a single task) • then string those tasks together into a
@workflow
, much like how you might define a
main
function to tie all of the functions in your module together into a single command.
s

Sebastian

04/21/2022, 8:16 AM
Thank you for your response @Niels Bantilan. I somehow didn't find the cookbook before, but after reading it this is much clearer. I moved everything to a task and started splitting it up from there and it all makes sense now
n

Niels Bantilan

04/25/2022, 1:27 PM
hi @Sebastian just following up here: are you unblocked now on porting your current solution to Flyte? Don’t hesitate to reach out here. Just so we can improve the experience in the future, where did you start in the Flyte docs, and how can we make the cookbook more discoverable?
s

Sebastian

05/18/2022, 7:36 AM
Hi Niels, thank you very much for your assistance. I have managed to set it up. I think the docs have changed since I browsed them last? I don't recognise them. One thing I have not really seen though: Can you please point me to any documentation on how I could do backfilling in a time range using Flyte? If I change the way some derived tables are computed, and those computations are run on Flyte, I would like to be able to re-run them. Thanks!
28 Views