wave ~Back with another basic question I ve been using `pyf Flyte #ask-the-community

:wave: ~Back with another basic question. I've be...

Terence Kent

09/22/2023, 8:00 PM

👋 Back with another basic question. I've been using
pyflyte run
to mess around with some basic Flyte workflows, just getting familiar with it. For one test, I wanted to use the
map_task
feature, running the workflow against a dev cluster. Unfortunately, I'm doing something wrong. In every attempt, using either
pyflyte build
,
pyflyte run
, etc, I get the following as soon as I include a
map_task
call in my
@workflow
function.

Copy code

Failed with Unknown Exception <class 'ModuleNotFoundError'> Reason: Error encountered while executing 'maptask_sample':
  Module from file ../flyte-local-playground/simple-py-workflow/venv/bin/pyflyte cannot be loaded
Error encountered while executing 'maptask_sample':
  Module from file ../flyte-local-playground/simple-py-workflow/venv/bin/pyflyte cannot be loaded

The code is setup so a regular python execution (via
main
) works just fine and if I remove any
map_task
calls it works in the dev cluster. I'm clearly missing something here, can somebody point me in the right direction?

Terence Kent

09/22/2023, 10:59 PM

Never mind, found the issue. It looks like I had an environment setup problem that didn't show up for me until using

map_task

. I had placed my python

venv

folder inside of the flyte project folder and also had

pyflyte

installed there. Apparently, in that condition, the error above happens. Just setting up

pyflyte

in a different parent directory's

venv

fixed the problem. That took a minute.

Joe Hartshorn

09/28/2023, 10:45 AM

I’m running into the same issue I think..

placed my python
venv
folder inside of the flyte project folder

This is standard for all our python projects, why does it cause issues here?

Terence Kent

09/28/2023, 4:20 PM

Yea, it does seem like a bug, doesn’t it? I saw you posted later about a different situation with

map_task

usage. Were you actually running into the same situation related to `pytflyte`'s

venv

and the other issue, or just the other issue?

Joe Hartshorn

10/03/2023, 8:01 AM

Good point, I’ll check

9 Views

Open in Slack

Previous Next