containers with unready status: [f7592492fe4da4f14b2f-n0-0]|Failed to apply default image tag "<http://cr.flyte.org/flyteorg/flytekit:py3.10-1.3.2:image|cr.flyte.org/flyteorg/flytekit:py3.10-1.3.2:image>": couldn't parse image reference "<http://cr.flyte.org/flyteorg/flytekit:py3.10-1.3.2:image|cr.flyte.org/flyteorg/flytekit:py3.10-1.3.2:image>": invalid reference format

storage:
  # -- Sets the storage type. Supported values are sandbox, s3, gcs and custom.
  type: custom
  custom:
    type: minio
    container: flyte
    stow:
      kind: s3
      config:
        access_key_id: XXXX
        auth_type: accesskey
        secret_key: XXXX
        disable_ssl: true
        endpoint: <http://minio-hl.minio.svc.cluster.local:9000>
        region: us-east-1

Matplotlib created a temporary config/cache directory at /tmp/matplotlib-dqrfqnc3 because the default path (/home/flytekit/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.

@task()
def get_data() -> pd.DataFrame:
    data_dict = load_breast_cancer(as_frame=True)
    print(data_dict['data'].head())
    return data_dict['data']

@task()
def clean_data_pre_split(data:pd.DataFrame) -> pd.DataFrame:

    return data

@workflow()
def training_workflow():
    data  = get_data()
    data = clean_data_pre_split(data=data)
    return

serviceAccount

create: false

serviceAccount:
  create: false
  annotations:
  <http://eks.amazonaws.com/role-arn|eks.amazonaws.com/role-arn>: "arn:aws:iam::<id-redacted>:role/eksctl-flyte-cluster-cluster-ServiceRole"

from flytekit import workflow, LaunchPlan, task
from dataclasses import dataclass
from dataclasses_json import dataclass_json

@dataclass_json
@dataclass
class MyDataClass(object):
    def __init__(self, a: int, b: str):
        self.a = a
        self.b = b

@task
def example_node(input: MyDataClass):
    print(f'{input.a}')
    

@workflow
def my_dataclass_workflow(input: MyDataClass):
    return example_node(input=input)

Events:
  Type     Reason            Age    From               Message
  ----     ------            ----   ----               -------
  Warning  FailedScheduling  11m    default-scheduler  0/1 nodes are available: 1 Insufficient <http://nvidia.com/gpu|nvidia.com/gpu>, 1 node(s) didn't match Pod's node affinity/selector. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.

kubectl describe node 31e2e2ba6f9f

31e2e2ba6f9f

>kubectl describe node 31e2e2ba6f9f
...
Capacity:
  cpu:                8                                                                                                                                                       ephemeral-storage:  944801904Ki
  hugepages-1Gi:      0                                                                                                                                                   hugepages-2Mi:      0
  memory:             65774996Ki
  pods:               110
Allocatable:
  cpu:                8
  ephemeral-storage:  919103291491                                                                                                                                        hugepages-1Gi:      0
  hugepages-2Mi:      0                                                                                                                                                   memory:             65774996Ki
  pods:               110
...

nvidia-smi

❯ docker exec -it 31e2e2ba6f9f /bin/sh
/ # nvidia-smi
/bin/sh: nvidia-smi: not found
/ #

--gpus all

docker run --gpus all <task_image_with_gpu_req>

--gpus all

flytectl demo start