Niels Bantilan
04/11/2023, 5:59 PM@channel
, the core Flyte team has heard reports of long import times when simply running a script locally that does import flytekit
(the number I’ve heard of is ~8 seconds).
I’ve been unable to reproduce this locally, but can anyone with this experience provide instructions on how to reproduce this behavior? We’d like to look into reducing this import time.Bernhard Stadlbauer
04/11/2023, 6:56 PMPYTHONPROFILEIMPORTTIME=1
produced the attached logs.
You should be able to produce an interactive flame-graph using tuna
(docs).
tuna flyte_profile.log
Seems like most of it is related to serialization and FlyteDecks - so potentially if you import FlyteDecks locally it could be slow? 🤔Niels Bantilan
04/11/2023, 6:58 PMBernhard Stadlbauer
04/11/2023, 7:00 PMflytekit 1.4.x
(not 100% certain though)Ketan (kumare3)
Bernhard Stadlbauer
04/11/2023, 8:41 PM@task(
environment={"PYTHONPROFILEIMPORTTIME": "1"}
)
def demo_task():
print("foobar")
@workflow
def demo_workflow():
demo_task()
And then I’ve copied the import time logs from k8sKetan (kumare3)