Hi This is such a great project i m having problems when i a

Question

Hi This is such a great project i m having problems when i add from flytekitplugins spark import Spark to my py after running remote way im not having problems using pyflyte run py my wf the process is complete but the wokflow appears failed with the following message from web UI 1 1 currentAttempt done Last Error USER trap gt line 688 in load unlocked File line 883 in exec module File line 241 in call with frames removed File root flyte py line 5 in from flytekitplugins spark import Spark ModuleNotFoundError No module named flytekitplugins spark Traceback most recent call last File usr local bin pyflyte fast execute line 8 in sys exit fast execute task cmd

Ketan (kumare3) · Answer

You need to install flytekit plugins spark in the container

Ketan (kumare3) · Answer

But for this join todays community sync great work by < Kevin Su> and < Evan Sadler> makes this effortless

Gabriel Molina · Answer

great i have it installed in my local python env how can i install it in the container

Gabriel Molina · Answer

< Kevin Su> < Evan Sadler> can u please help me

Kevin Su · Answer

< Gabriel Molina> you have to write a dockerfile to build an image btw you could try to use image spec Here is an example to build image for spark task by using image spec

Gabriel Molina · Answer

Thanks < Kevin Su> i tried image spec and it works with run but not with run remote

Kevin Su · Answer

you have to install envd plugin as well pip install flytekitplugins envd==1 6 1

Ketan (kumare3) · Answer

I think we should include envd in default And then allow it to be overridden

Samhita Alla · Answer

+1 to including it in the flytekit library

Gabriel Molina · Answer

< Kevin Su> i installed it sucesfully and got this failed to build the imageSpec autorization failed

Samhita Alla · Answer

Have you specified the registry I think you should authenticate to dockerhub from the CLI <https docs docker com docker hub access tokens >

Samhita Alla · Answer

< Kevin Su> is dockerhub the only supported registry

Samhita Alla · Answer

< Gabriel Molina> changing the registry name to your username should work

Samhita Alla · Answer

Kevin is spark working on the demo cluster

Gabriel Molina · Answer

< Samhita Alla> im able to run the spark example shared by kevin now with remote flag but got the following error in the web UI 1 1 currentAttempt done Last Error USER core py 760 in invoke 760 return callback args kwargs usr local lib python3 10 site packages flytekit bin entrypoint py 508 in fast execute task cmd 508 subprocess run cmd check=True usr local lib python3 10 subprocess py 526 in run 526 raise CalledProcessError retcode process args CalledProcessError Command pyflyte execute inputs c8 n0 data inputs pb output prefix c8 n0 data 0 raw output data prefix checkpoint path prev checkpoint dynamic addl distro pt mode tar gz dynamic dest dir root resolver flytekit core python auto container default task resolver task module flyte3 task name hello spark returned non zero exit status 1

Gabriel Molina · Answer

Additionally if i try to pyflyte run remote more than once it doesn t works I attached the error at the end of this comment but it works again if i rename the workflow in the py file Were you able to reproduce the sample code shared by Kevin RPC Failed with Status StatusCode INVALID ARGUMENT details launch plan with different structure already exists with id resource type LAUNCH PLAN project flytesnacks domain development name wf version jMuCe4OGZ9LuvDqx089w Q== Debug string UNKNOWN Error received from peer grpc message launch plan with different structure already exists with id resource type LAUNCH PLAN project flytesnacks domain development name wf version jMuCe4OGZ9LuvDqx089w Q== grpc status 3 created time 2023 05 18T09 14 13 318343238 04 00

Ketan (kumare3) · Answer

This is weird It is not understanding that something has changed cc < Kevin Su> < Eduardo Apolinario eapolinario > this is unexpected < Gabriel Molina> is this after you use the new Imagespec

Gabriel Molina · Answer

< Ketan kumare3 > yes im using spark image = ImageSpec registry= lt mydockerhubuser gt and running it with flytekit==1 6 1 and flytekitplugins envd==1 6 1

Samhita Alla · Answer

gt im able to run the spark example shared by kevin now with remote flag but got the following error in the web UI Would you be able to fetch the k8s log <https docs flyte org en latest community troubleshoot html troubleshoot> ```kubectl get pods n flytesnacks development kubectl logs lt pod name gt n flytesnacks development```

Gabriel Molina · Answer

< Samhita Alla> Sure this is the output ```tar Removing leading ` from member names Traceback most recent call last usr local bin pyflyte execute 8 in lt module gt 8 sys exit execute task cmd usr local lib python3 10 site packages click core py 1130 in call 1130 return self main args kwargs usr local lib python3 10 site packages click core py 1055 in main 1055 rv = self invoke ctx usr local lib python3 10 site packages click core py 1404 in invoke 1404 return ctx invoke self callback ctx params usr local lib python3 10 site packages click core py 760 in invoke 760 return callback args kwargs usr local lib python3 10 site packages flytekit bin entrypoint py 471 in execute task cmd 471 execute task usr local lib python3 10 site packages flytekit exceptions scopes py 160 in system entry point 160 return wrapped args kwargs usr local lib python3 10 site packages flytekit bin entrypoint py 347 in execute task 347 task def = resolver obj load task loader args=resolver args usr local lib python3 10 site packages flytekit core utils py 295 in wrapper 295 return func args kwargs usr local lib python3 10 site packages flytekit core python auto container py 235 in load task 235 task module = importlib import module name=task module typ usr local lib python3 10 importlib init py 126 in import module 126 return bootstrap gcd import name level package level in gcd import 1050 in find and load 1027 in find and load unlocked 1006 in load unlocked 688 in exec module 883 in call with frames removed 241 root flyte3 py 7 in lt module gt 7 from flytekitplugins spark import Spark ModuleNotFoundError No module named flytekitplugins spark Traceback most recent call last usr local bin pyflyte fast execute 8 in lt module gt 8 sys exit fast execute task cmd usr local lib python3 10 site packages click core py 1130 in call 1130 return self main args kwargs usr local lib python3 10 site packages click core py 1055 in main 1055 rv = self invoke ctx usr local lib python3 10 site packages click core py 1404 in invoke 1404 return ctx invoke self callback ctx params usr local lib python3 10 site packages click core py 760 in invoke 760 return callback args kwargs usr local lib python3 10 site packages flytekit bin entrypoint py 508 in fast execute task cmd 508 subprocess run cmd check=True usr local lib python3 10 subprocess py 526 in run 526 raise CalledProcessError retcode process args CalledProcessError Command pyflyte execute inputs 5a n0 data inputs pb output prefix 5a n0 data 0 raw output data prefix checkpoint path prev checkpoint dynamic addl distro pt mode tar gz dynamic dest dir root resolver flytekit core python auto container default task resolver task module flyte3 task name hello spark returned non zero exit status 1 ```

Kevin Su · Answer

sorry actually you can t use fast register if spark task is in the workflow because spark worker won t download the code try to use `pyflyte register ` instead

Ketan (kumare3) · Answer

< Kevin Su> why is that

Ketan (kumare3) · Answer

should it not download the code

Kevin Su · Answer

when I use fast register only driver downloaded code