Channels
announcements
ask-the-community
auth
conference-talks
contribute
databricks-integration
datahub-flyte
deployment
ecosystem-unionml
engineeringlabs
events
feature-discussions
flyte-bazel
flyte-build
flyte-console
flyte-deployment
flyte-documentation
flyte-github
flyte-on-gcp
flyte-ui-ux
flytekit
flytekit-java
flytelab
great-content
hacktoberfest-2022
helsing-flyte
in-flyte-conversations
integrations
introductions
jobs
konan-integration
linkedin-flyte
random
ray-integration
ray-on-flyte
release
scipy-2022-sprint
show-and-tell
sig-large-models
torch-elastic
workflow-building-ui-proj
writing-w-sfloris
Title
k
Ketan (kumare3)
04/04/2023, 5:06 AMso it should be possible to pickle and run tasks directly from notebook onto a flyte-server
g
Greg Gydush
04/04/2023, 5:07 AMthat's epic
I'm imagining map tasks from within notebook would be incredible
k
Ketan (kumare3)
04/04/2023, 5:07 AMyup
exactly
also distributed training
and others
so i think we should just rewrite this resolverrr https://github.com/flyteorg/flytekit/blob/master/flytekit/extras/cloud_pickle_resolver.py
or use this resolver to use in fast-execute
and update pyflyte-fast-execute to use cloudpickle as an option
and then it should just work IMO?
Also @Greg Gydush have you seen the new map-tasks
support for multiple-lists etc and partials?
g
Greg Gydush
04/04/2023, 5:09 AMI haven't yet
k
Ketan (kumare3)
04/04/2023, 5:09 AMthis is in beta, some small bugs being fixed, but keep an eye