StructuredDataset

FlyteFile[Format]

RuntimeError

serviceaccount

trainer-service

default

@task(
    requests=Resources(cpu="250m", mem="512Mi"),
    pod_template=PodTemplate(
        pod_spec=V1PodSpec(
            service_account_name="trainer-service",
            containers=[],
            tolerations=...
)

@task
def test(a, b, c):
    return a+b+c

@workflow
def workflow_test():
    kwargs = {"a": 4, "b": 3, "c":0}
    test(**kwargs)

No Flyte entities were detected. Aborting!

pyflyte register

cluster_resource_manager

- key: ac_project_efs_shared
      value: |
        apiVersion: v1
        kind: PersistentVolumeClaim
        metadata:
          name: efs-shared
          namespace: {{ namespace }}
        spec:
          accessModes:
            - ReadWriteMany
          storageClassName: efs-sc
          resources:
            requests:
              storage: 5Gi
          volumeName: efs-shared

validate download

validate download

node_name

with_overrides(node_name=)

@flytekit.task
def task1(x: int) -> str:
    return str(x)

@flytekit.task
def task2(x: str) -> str:
    return x


@flytekit.workflow
def baby_training_wf() -> str:
    t1 = task1(x=3).with_overrides(node_name='t_1')
    return task2(x=t1).with_overrides(node_name='t_2')

Failed with Exception: Reason: SYSTEM:Unknown
RPC Failed, with Status: StatusCode.INTERNAL
        details: failed to compile workflow for [resource_type:WORKFLOW project:"flytesnacks" domain:"development" name:"tmp.workflow.baby_training_wf" version:"yXlTbWXJo4nsbhCWJe5PGw==" ] with err failed to compile workflow with err Collected Errors: 4
        Error 0: Code: NodeReferenceNotFound, Node Id: t_1, Description: Referenced node [t_1] not found.
        Error 1: Code: NodeReferenceNotFound, Node Id: t_2, Description: Referenced node [t_2] not found.
        Error 2: Code: ParameterNotBound, Node Id: end-node, Description: Parameter not bound [o0].
        Error 3: Code: ParameterNotBound, Node Id: t-2, Description: Parameter not bound [x].

flytectl create project

Value error!  Received: 403. Request to send data

003-storage.yaml

010-inline-config.yaml

003-storage.yaml: |
    propeller:
      rawoutput-prefix: s3://<bucket-name>/flyte/data
    storage:
      type: stow
      stow:
        kind: s3
        config:
          region: us-east-1
          disable_ssl: true
          v2_signing: true
          endpoint: <vast-endpoint>
          auth_type: accesskey
          access_key_id: <key_id>
          secret_key: <secret>
      container: <bucket-name>
  010-inline-config.yaml: |
    plugins:
      k8s:
        default-env-vars:
        - AWS_METADATA_SERVICE_TIMEOUT: 5
        - AWS_METADATA_SERVICE_NUM_ATTEMPTS: 20
        - FLYTE_AWS_ENDPOINT: <vast-endpoint>
        - FLYTE_AWS_ACCESS_KEY_ID: <key_id>
        - FLYTE_AWS_SECRET_ACCESS_KEY: <secret>

nodeName

{{.hostname}}

<gs://mybucket/metadata/propeller/myproject-myns-myexecid/n1/data/>

data
|  0/
|  | start-node/
|  |    |  0/
|  |    |  | outputs.pb
|  
|  inputs.pb

n0/

data/0/

data/0/start-node/0/outputs.pb

flytekit 1.5.0

flyte-core:1.5.0

1.4.0

@task
def fail_task() -> List[Any]:
    return []

@workflow
def fail_workflow():
    fail_task()

Traceback (most recent call last):

      File "/opt/venv/lib/python3.8/site-packages/flytekit/exceptions/scopes.py", line 165, in system_entry_point
        return wrapped(*args, **kwargs)
      File "/opt/venv/lib/python3.8/site-packages/flytekit/core/base_task.py", line 587, in dispatch_execute
        raise TypeError(msg) from e

Message:

    Failed to convert outputs of task 'pipelines.parallel.fail_task' at position 0:
  range() arg 3 must not be zero

SYSTEM ERROR! Contact platform administrators.

wait_for_input

admin:
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: dns:///127.0.0.1:8080
  authType: Pkce
  insecure: true
  clientId: <My Client ID here>
logger:
  show-source: true
  level: 10

kubectl -n flyte port-forward service/flyteadmin 8080:81

flytectl create project --file project.yaml

Error: authentication error! Original Error: rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken, Auth Error: context was canceled during auth flow
{"json":{"src":"main.go:13"},"level":"error","msg":"authentication error! Original Error: rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken, Auth Error: context was canceled during auth flow","ts":"2023-04-20T16:10:46Z"}