Yuan Wang (Mike)
07/27/2023, 2:15 PMpyflyte run
on the relevant workflow. I have encountered this issue two times today, same PC, but different python virtual environments. I think this is also the reason why previously I failed to execute pyflyte run
on the first day and succeeded on the second day.
(mike) ➜ mike pip list | grep nlpaug
nlpaug 1.1.11
# Just add one custom python library nlpaug in the example.py file
(mike) ➜ mike cat workflows/example.py
import typing
from flytekit import task, workflow
import nlpaug
@task
def say_hello(name: str) -> str:
...
(mike) ➜ mike pyflyte run workflows/example.py wf
Failed with Unknown Exception <class 'ModuleNotFoundError'> Reason: No module named 'nlpaug'
# However, after about 30 minutes, this command works fine
(mike) ➜ mike pyflyte run workflows/example.py wf
DefaultNamedTupleOutput(o0='hello union!', o1=12)
# I set these two alias in my ~/.zshrc file. However, it seems that it would not have an impact.
alias python='python3'
alias pip='pip3'
--no-cache
option to install the requirements, and it did not solve the issue. Another interesting finding is that if I use python
to run the example workflow program, it will work immediately after executing pip install -r requirements.txt
. But pyflyte run
requires a restart of the shell. I am using iterm2
with oh~my~zsh
. I tested this on two MacOS computers. Both have this problem. One of them needs about 30 minutes if I do not restart the shell. The other one is faster, but still requires about 2 to 3 minutes.
Here is how to reproduce the issue and the workaround.
If I call pyflyte run
directly after pip install -r requirements.txt
, it will fail. However, if I call python workflows/example.py
, it will work.
(yuan) ➜ yuan pyflyte run workflows/example.py wf
Failed with Unknown Exception <class 'ModuleNotFoundError'> Reason: No module named 'nlpaug'
No module named 'nlpaug'
(yuan) ➜ yuan python workflows/example.py
Running wf() DefaultNamedTupleOutput(o0='hello passengers!', o1=17)
After restart the iterm2 session, both the python
and pyflyte run
works.
➜ ~ cd PycharmProjects/flyte/yuan
➜ yuan source ~/venvs/yuan/bin/activate
(yuan) ➜ yuan python workflows/example.py
Running wf() DefaultNamedTupleOutput(o0='hello passengers!', o1=17)
(yuan) ➜ yuan pyflyte run workflows/example.py wf
DefaultNamedTupleOutput(o0='hello union!', o1=12)
(yuan) ➜ yuan python --version
Python 3.11.4
Samhita Alla
Yuan Wang (Mike)
07/28/2023, 8:42 AM