Or you can also write a Dockerfile, build an image and send it to the relevant pyflyte command.

or can you tell me in the imagespec.yaml what exactly needs to be added # imageSpec.yaml python_version: 3.10 registry: pingsutw packages: - sklearn - opencv-python env: Debug: "True" what do i put in place of registry should I leave it the same?

I tried creating a new conda env and installed flytekitplugins-envd and then when i run pyflyte run --remote --image imagespec.yaml new.py standard_scale_workflow --values '[1.0, 2.0, 3.0, 4.0, 5.0]' It is building a docker image but I get this error Failed with Unknown Exception <class 'Exception'> Reason: failed to run command envd build --path /tmp/flyte-s5m5ghk_/sandbox/local_flytekit/e3c3143d14bce47e61050188b2791fa7 --platform linux/amd64 --output type=image,name=pingsutw/flytekit:t93nYMZ9tvO68GDt0g1xRg..,push=true with error b'time="2023-09-05T092544Z" level=fatal msg="failed to create the builder: failed to create buildkit client: failed to bootstrap the buildkitd: failed to create container: Error response from daemon: invalid mount config for type \\"bind\\": bind source path does not exist: /home/ngupta/.config/envd"\n' failed to run command envd build --path /tmp/flyte-s5m5ghk_/sandbox/local_flytekit/e3c3143d14bce47e61050188b2791fa7 --platform linux/amd64 --output type=image,name=pingsutw/flytekit:t93nYMZ9tvO68GDt0g1xRg..,push=true with error b'time="2023-09-05T092544Z" level=fatal msg="failed to create the builder: failed to create buildkit client: failed to bootstrap the buildkitd: failed to create container: Error response from daemon: invalid mount config for type \\"bind\\": bind source path does not exist: /home/ngupta/.config/envd"\n' I guess if you help me out with this then my work will be done

[1/1] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes. [f6d65c22f831647dca1e-n0-0] terminated with exit code (247). Reason [OOMKilled]. Message: tar: Removing leading `/' from member names do you have any idea why I get this message while running a task , the task is to download mnist data? the task is failing by the way.

I know it is because of resource limit but I have 900GB of memory but how do I set the memory for a project ?

if you could let me know how pvc can be attached to the flyte tasks , it would really be helpful

It'll have to do with the roles you assigned.

Thanks

I fixed it, thanks, but how do I convert Promise(node:n1.o0) to str I mean a task is returning a string and I can see the same in console , but when I call it in workflow it is returning Promise(node:n1.o0). Below is the code can you help me fix it @workflow def optimize_model() -> Tuple[float, str, List[int]]: best_accuracy, best_params = optimize_hyp() # Train the best model with best hyperparameters run_id = train_best_model(best_params=best_params) print(run_id) # Specify the MLflow model URI model_uri = f"runs:/a4519e1f7e00433886646a2bfb51600f/best_random_forest_model" # Sample data for inference (for the Iris dataset) data = [[5.1, 3.5, 1.4, 0.2]] # Perform inference using the ray_inference task inference_results = ray_inference(model_uri=model_uri, data=data) This is the workflow and run_id is what giving me the promise instead of str below is the task code for run_id return best_accuracy, run_id, inference_results @task(requests=Resources(cpu="2", mem="1Gi")) def train_best_model(best_params: Dict[str, Any]) -> str: import mlflow import mlflow.sklearn from sklearn.datasets import load_iris from sklearn.ensemble import RandomForestClassifier import numpy as np iris = load_iris() X, y = iris.data, iris.target # Initialize and train a Random Forest Classifier with the best hyperparameters clf = RandomForestClassifier(n_estimators=best_params["n_estimators"], max_depth=best_params["max_depth"], min_samples_split=best_params["min_samples_split"], random_state=42) accuracy = model_accuracy(n_estimators=best_params["n_estimators"], max_depth=best_params["max_depth"], min_samples_split=best_params["min_samples_split"]) # Fit the model clf.fit(X, y) # Log the model to MLflow with mlflow.start_run(run_name="BestRandomForestModel") as run: mlflow.sklearn.log_model(clf, "best_random_forest_model") mlflow.log_metric("accuracy", accuracy) run_id = mlflow.active_run().info.run_id return str(run_id)

I fixed it thanks

Hey @tall-lock-23197 is there anyway we can add resource values in flyte-binary for task section, I see in flyte-core that we can configure task resources like storage memory as well as cpu and gpu.

Hey @tall-lock-23197 for the below code in flyte I get a certain error import flytekit from flytekit import task, workflow, Resources from typing import List, Tuple @task(requests=Resources(gpu="1", cpu="2", mem="1Gi"),container_image="822795565729.dkr.ecr.us-west-2.amazonaws.com/prime-analysis:prime-analysis-reksi-flyte_pipeline-0.0.10") def sleep(): import time time.sleep(3600) @workflow def toy_workflow(): sleep() I run it using pyflyte run --remote --project cpa-test toy_pipeline.py toy_workflow the error is /opt/nvidia/nvidia_entrypoint.sh: line 67: exec: pyflyte-fast-execute: not found I installed flyte and related packages. pyflyte-fast-execute is in PATH, but it still fails. I am not sure how to debug it, pod fails to start. can you please help me?

is it something because of the gpu's?