Hi, I've been exploring flyte and want to use it to build ML Model pipelines. After following the steps to setup flytectl in an EC2 instance,I've tried to run the diabetes classification model. I'm getting various types of errors, some of which are listed below: 1. No module named joblib or sometimes sklearn 2. Not able to read the .csv file from the github link & also from the s3 bucket 3. Sometimes the code is not running citing that the workflow version is already available in the console Pls provide some clarifications on how these errors can be avoided.

Hey can you describe the steps that you used ?

Followed the steps mentioned here: https://docs.flyte.org/en/latest/getting_started/index.html 1. Installed docker engine using the steps mentioned here: https://docs.docker.com/engine/install/ubuntu/ 2. Next pip installed flytekit 3. Then setup flytectl CLI

@KS Tarun, can you try running the following command in the

pima_diabetes

directory?

pyflyte run --remote --image <http://ghcr.io/flyteorg/flytecookbook:pima_diabetes-latest|ghcr.io/flyteorg/flytecookbook:pima_diabetes-latest> diabetes.py diabetes_xgboost_model

Yes, when --image was used; I'm getting this error: "task with different structure already exists with id resource_type:TASK project:"flytesnacks" domain:"development" name:"diabetes.split_traintest_dataset" version:"dk9ZbNYRXQT4R-F7Eq6g1g==" ","grpc_status":3}"

@Kevin Su, we shouldn’t be getting this error, right?

Yes, it seems like joblib isn’t in that image. Let me check it

No joblib is present in the image. The different structure already exists shouldn’t crop up, right?

I changed the domain to staging and executed : pyflyte run --remote --domain staging --image ghcr.io/flyteorg/flytecookbook:pima_diabetes-latest diabetes.py:diabetes_xgboost_model Now I'm getting the below mentioned error related to the input data file: [1/1] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes. [fdc141bb7439e4eab912-n0-0] terminated with exit code (1). Reason [Error]. Message: n_val, python_type, expected) File "/opt/venv/lib/python3.8/site-packages/flytekit/types/file/file.py", line 311, in to_literal raise TypeTransformerFailedError(f"Error converting {python_val} because it's not a file.") flytekit.core.type_engine.TypeTransformerFailedError: Error converting pima-indians-diabetes_data.csv because it's not a file. Traceback (most recent call last): File "/opt/venv/bin/pyflyte-fast-execute", line 8, in <module> sys.exit(fast_execute_task_cmd()) File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1130, in call return self.main(*args, **kwargs) File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/opt/venv/lib/python3.8/site-packages/flytekit/bin/entrypoint.py", line 502, in fast_execute_task_cmd subprocess.run(cmd, check=True) File "/usr/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['pyflyte-execute', '--inputs', 's3://my-s3-bucket/metadata/propeller/flytesnacks-staging-fdc141bb7439e4eab912/n0/data/inputs.pb', '--output-prefix', 's3://my-s3-bucket/metadata/propeller/flytesnacks-staging-fdc141bb7439e4eab912/n0/data/0', '--raw-output-data-prefix', 's3://my-s3-bucket/test/yg/fdc141bb7439e4eab912-n0-0', '--checkpoint-path', 's3://my-s3-bucket/test/yg/fdc141bb7439e4eab912-n0-0/_flytecheckpoints', '--prev-checkpoint', '""', '--dynamic-addl-distro', 's3://my-s3-bucket/os/flytesnacks/staging/OZHVS3GWCFOQJ6CH4F5RFLVA2Y======/scriptmode.tar.gz', '--dynamic-dest-dir', '.', '--resolver', 'flytekit.core.python_auto_container.default_task_resolver', '--', 'task-module', 'diabetes', 'task-name', 'split_traintest_dataset']' returned non-zero exit status 1. .

It shouldn’t. We should register a new version of wf

Let me try to repro, @KS Tarun.

Ok !

I assume you’re seeing the

TypeTransformerFailedError

in the

split_traintest_dataset

task. It works for me, though. Can you install a newer flytekit beta version (

pip install flytekit==1.2.0b0

) and trigger the pyflyte run command I’ve sent?

Ok, I'll try this