@ancient-wolf-19325 pyflyte run --remote without checking if code is ok

It seems likely that pyflyte run --remote will attempt to execute your code on the Flyte cluster without first checking for syntax errors or other issues in your local code. If your code contains errors, the remote execution may fail, and you will see the error in the Flyte UI or logs, rather than catching it locally first. This is consistent with the documentation, which recommends running workflows locally to catch immediate syntax errors before deploying remotely, as local execution allows for faster iteration and debugging of such issues before incurring the overhead of remote runs (Union.ai Docs: Running). If you want to ensure your code is valid before running it remotely, it's best to run pyflyte run locally first, or simply execute your script with python to catch syntax and import errors. The remote run does not perform a pre-check for code correctness—it just uploads and executes as-is (Union.ai Docs: Running). Would you like more detail or a step-by-step example? Sources: - Union.ai Docs: Running 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 can you fix my map_task implementation

@fl.workflow
def train_by_comet_key_dynamic_wf(
    keys: list[str],
    ddh_uuid: str,
    rcddh_uuid: str,
    project: str = "rama",
):
    fl.map_task(train_by_comet_key_task)(keys, ddh_uuid, rcddh_uuid, project)

@fl.task(
    container_image=hv_image_spec,
    resources=fl.Resources(cpu="16", mem="32Gi", gpu="1", ephemeral_storage="15Gi"),
    shared_memory=True,
    environment=dotenv_values(),
)
def train_by_comet_key_task(
    key: str,
    ddh_uuid: str,
    rcddh_uuid: str,
    project: str,
    module_name: Optional[str] = None,
):
    run_train_by_comet_key(
        key=key,
        ddh_uuid=ddh_uuid,
        rcddh_uuid=rcddh_uuid,
        project=project,
        module_name=module_name,
    )

Your map_task implementation is incorrect because map_task expects keyword arguments, and the mapped argument (the one to be iterated over) must be passed as a keyword with a list value. All other arguments should be passed as fixed values (not lists). The correct usage is:

@fl.workflow
def train_by_comet_key_dynamic_wf(
    keys: list[str],
    ddh_uuid: str,
    rcddh_uuid: str,
    project: str = "rama",
):
    return fl.map_task(train_by_comet_key_task)(
        key=keys, ddh_uuid=ddh_uuid, rcddh_uuid=rcddh_uuid, project=project
    )

This ensures that key is mapped over, while the other arguments are fixed for each invocation (Flyte map_task docs, Flytesnacks example). Would you like a step-by-step explanation or more advanced usage examples? Sources: - Flyte map_task docs - Flytesnacks example 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325

@fl.workflow
def train_by_comet_key_map_tasks_wf(
    model_ids: list[str],
    ddh_uuid: str,
    rcddh_uuid: str,
    project: str = "rama",
):
    partial_task = functools.partial(
        train_by_comet_key_task,
        ddh_uuid=ddh_uuid,
        rcddh_uuid=rcddh_uuid,
        project=project,
        module_name=None,
    )
    fl.map_task(partial_task)(model_ids)

@fl.task(
    container_image=hv_image_spec,
    resources=fl.Resources(cpu="16", mem="32Gi", gpu="1", ephemeral_storage="15Gi"),
    shared_memory=True,
    environment=dotenv_values(),
)
def train_by_comet_key_task(
    key: str,
    ddh_uuid: str,
    rcddh_uuid: str,
    project: str,
    module_name: Optional[str] = None,
):
    run_train_by_comet_key(
        key=key,
        ddh_uuid=ddh_uuid,
        rcddh_uuid=rcddh_uuid,
        project=project,
        module_name=module_name,
    )

I'm running into an error saying that the image is not accessible.

failed to authorize: failed to fetch anonymous token: unexpected status from GET request to <https://gitlab.com/jwt/auth>.... 403 Forbidden

The error failed to authorize: failed to fetch anonymous token: ... 403 Forbidden indicates that the container image specified by hv_image_spec is in a private registry (e.g., GitLab) and Flyte cannot access it due to missing authentication credentials. Flyte requires proper imagePullSecrets or service account configuration to pull private images (Flyte private images guide). To fix this, create a Kubernetes secret with your registry credentials and configure Flyte to use it via imagePullSecrets, either on the service account or via a custom pod template. This will allow Flyte to authenticate and pull the image. Would you like detailed steps on how to set up imagePullSecrets for your Flyte deployment? Sources: - Flyte private images guide 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.