broad-monitor-993
04/11/2023, 5:59 PM@channel
, the core Flyte team has heard reports of long import times when simply running a script locally that does import flytekit
(the number I’ve heard of is ~8 seconds).
I’ve been unable to reproduce this locally, but can anyone with this experience provide instructions on how to reproduce this behavior? We’d like to look into reducing this import time.agreeable-kitchen-44189
04/11/2023, 6:56 PMPYTHONPROFILEIMPORTTIME=1
produced the attached logs.
You should be able to produce an interactive flame-graph using tuna
(docs).
tuna flyte_profile.log
Seems like most of it is related to serialization and FlyteDecks - so potentially if you import FlyteDecks locally it could be slow? 🤔broad-monitor-993
04/11/2023, 6:58 PMagreeable-kitchen-44189
04/11/2023, 7:00 PMflytekit 1.4.x
(not 100% certain though)freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
agreeable-kitchen-44189
04/11/2023, 8:41 PM@task(
environment={"PYTHONPROFILEIMPORTTIME": "1"}
)
def demo_task():
print("foobar")
@workflow
def demo_workflow():
demo_task()
And then I’ve copied the import time logs from k8sagreeable-kitchen-44189
04/11/2023, 8:41 PMfreezing-airport-6809
freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
freezing-airport-6809