https://flyte.org logo
k

KS Tarun

09/01/2022, 5:33 PM
Hi, I've been exploring flyte and want to use it to build ML Model pipelines. After following the steps to setup flytectl in an EC2 instance,I've tried to run the diabetes classification model. I'm getting various types of errors, some of which are listed below: 1. No module named joblib or sometimes sklearn 2. Not able to read the .csv file from the github link & also from the s3 bucket 3. Sometimes the code is not running citing that the workflow version is already available in the console Pls provide some clarifications on how these errors can be avoided.
y

Yuvraj

09/01/2022, 5:36 PM
Hey can you describe the steps that you used ?
k

KS Tarun

09/01/2022, 5:48 PM
Followed the steps mentioned here: https://docs.flyte.org/en/latest/getting_started/index.html 1. Installed docker engine using the steps mentioned here: https://docs.docker.com/engine/install/ubuntu/ 2. Next pip installed flytekit 3. Then setup flytectl CLI
s

Samhita Alla

09/02/2022, 5:23 AM
@KS Tarun, can you try running the following command in the
pima_diabetes
directory?
Copy code
pyflyte run --remote --image <http://ghcr.io/flyteorg/flytecookbook:pima_diabetes-latest|ghcr.io/flyteorg/flytecookbook:pima_diabetes-latest> diabetes.py diabetes_xgboost_model
You haven’t used the
--image
flag; can you run
pyflyte run --remote --image <http://ghcr.io/flyteorg/flytecookbook:pima_diabetes-latest|ghcr.io/flyteorg/flytecookbook:pima_diabetes-latest> diabetes.py diabetes_xgboost_model
?
k

KS Tarun

09/02/2022, 5:58 AM
Yes, when --image was used; I'm getting this error: "task with different structure already exists with id resource_type:TASK project:"flytesnacks" domain:"development" name:"diabetes.split_traintest_dataset" version:"dk9ZbNYRXQT4R-F7Eq6g1g==" ","grpc_status":3}"
s

Samhita Alla

09/02/2022, 6:02 AM
@Kevin Su, we shouldn’t be getting this error, right?
k

Kevin Su

09/02/2022, 6:05 AM
Yes, it seems like joblib isn’t in that image. Let me check it
s

Samhita Alla

09/02/2022, 6:06 AM
No joblib is present in the image. The different structure already exists shouldn’t crop up, right?
k

KS Tarun

09/02/2022, 6:09 AM
I changed the domain to staging and executed : pyflyte run --remote --domain staging --image ghcr.io/flyteorg/flytecookbook:pima_diabetes-latest diabetes.py:diabetes_xgboost_model Now I'm getting the below mentioned error related to the input data file: [1/1] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes. [fdc141bb7439e4eab912-n0-0] terminated with exit code (1). Reason [Error]. Message: n_val, python_type, expected) File "/opt/venv/lib/python3.8/site-packages/flytekit/types/file/file.py", line 311, in to_literal raise TypeTransformerFailedError(f"Error converting {python_val} because it's not a file.") flytekit.core.type_engine.TypeTransformerFailedError: Error converting pima-indians-diabetes_data.csv because it's not a file. Traceback (most recent call last): File "/opt/venv/bin/pyflyte-fast-execute", line 8, in <module> sys.exit(fast_execute_task_cmd()) File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1130, in call return self.main(*args, **kwargs) File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/opt/venv/lib/python3.8/site-packages/flytekit/bin/entrypoint.py", line 502, in fast_execute_task_cmd subprocess.run(cmd, check=True) File "/usr/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['pyflyte-execute', '--inputs', 's3://my-s3-bucket/metadata/propeller/flytesnacks-staging-fdc141bb7439e4eab912/n0/data/inputs.pb', '--output-prefix', 's3://my-s3-bucket/metadata/propeller/flytesnacks-staging-fdc141bb7439e4eab912/n0/data/0', '--raw-output-data-prefix', 's3://my-s3-bucket/test/yg/fdc141bb7439e4eab912-n0-0', '--checkpoint-path', 's3://my-s3-bucket/test/yg/fdc141bb7439e4eab912-n0-0/_flytecheckpoints', '--prev-checkpoint', '""', '--dynamic-addl-distro', 's3://my-s3-bucket/os/flytesnacks/staging/OZHVS3GWCFOQJ6CH4F5RFLVA2Y======/scriptmode.tar.gz', '--dynamic-dest-dir', '.', '--resolver', 'flytekit.core.python_auto_container.default_task_resolver', '--', 'task-module', 'diabetes', 'task-name', 'split_traintest_dataset']' returned non-zero exit status 1. .
k

Kevin Su

09/02/2022, 6:09 AM
It shouldn’t. We should register a new version of wf
s

Samhita Alla

09/02/2022, 6:18 AM
Let me try to repro, @KS Tarun.
k

KS Tarun

09/02/2022, 6:26 AM
Ok !
s

Samhita Alla

09/02/2022, 6:28 AM
I assume you’re seeing the
TypeTransformerFailedError
in the
split_traintest_dataset
task. It works for me, though. Can you install a newer flytekit beta version (
pip install flytekit==1.2.0b0
) and trigger the pyflyte run command I’ve sent?
BTW, the code’s working for me.
Also, please modify the resource limits:
limits=Resources(mem="500Mi")
k

KS Tarun

09/02/2022, 6:36 AM
Ok, I'll try this
The code worked.
Thanks @Samhita Alla
5 Views