#3612 [BUG] Explicitly error on default Optional parameters in workflows and tasks Issue created by hamersaw Describe the bug Workflows that include a task with

cache_serialize

enabled can fail to correctly abort if the task with

cache_serialize

is processed during abort but has not yet started. This is because the

ReleaseCatalogReservation

function attempts to read the input values (to compute the cache key). If the node has not yet started, then Flyte has not yet written the input values. Expected behavior Aborts over

NotYetStarted

tasks with

cache_serialize

should not error. Additional context to reproduce This error is reproducible with the follow workflow:

import time
import flytekit

from flytekit import workflow, task
from flytekit.types.file import FlyteFile
from pathlib import Path
from typing import Optional


@task(cache=True, cache_serialize=True, cache_version='0.0.0')
def t1() -> FlyteFile:
    out = Path(flytekit.current_context().working_directory) / 'out.txt'
    out.write_text('Hi from t1')
    return FlyteFile(path=str(out))


@task(cache=True, cache_serialize=True, cache_version='0.0.0')
def t2(t1_: FlyteFile, optional: Optional[int] = None) -> FlyteFile:
    out = Path(flytekit.current_context().working_directory) / 'out.txt'
    with open(t1_, 'r') as f:
        out.write_text(f.read() + f'\nHi from t2, optional={optional}')
    return FlyteFile(path=str(out))


@workflow
def inner_wf(t1_: FlyteFile, optional: Optional[int] = None) -> FlyteFile:
    return t2(t1_=t2(t1_=t1_, optional=optional), optional=optional)


@workflow
def wf() -> FlyteFile:
    return inner_wf(t1_=t1())

This is certainly not a minimal reproduction, but the workflow fails because node

n0

(subworkflow

inner_wf

) is unable to resolve the

optional

value. In attempting to abort the workflow, Flyte attempts to abort task

t1

. Since

t1

is

cache_serialize

during the abort process there is a failure during

ReleaseCatalogReservation

. The failure is reported as:

{
  "json": {
    "exec_id": "myexecid",
    "ns": "myns",
    "res_ver": "445663335",
    "routine": "worker-7",
    "wf": "myproject:myns:mywf.mytask"
  },
  "level": "error",
  "msg": "Error when trying to reconcile workflow. Error [0: failed at Node[n0]. CatalogCallFailed: failed to release reservation, caused by: failed to read inputs when trying to query catalog: [READ_FAILED] failed to read data from dataDir [<gs://mybucket/metadata/propeller/myproject-myns-myexecid/n1/data/0/n0/inputs.pb>]., caused by: path:<gs://mybucket/metadata/propeller/myproject-myns-myexecid/n1/data/0/n0/inputs.pb>: not found]. Error Type[errors.ErrorCollection]",
  "ts": "2023-04-20T12:13:39Z"
}

Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte