https://flyte.org
Join Slack
hey Flyte folks, so i have been seen this weird/breaking behavior in relaunching a workflow that has...
# flyte-support
m
hey Flyte folks, so i have been seen this weird/breaking behavior in relaunching a workflow that has optional input(FlyteFile)
Copy code
Workflow[myworkflow.fastq2bam_workflow] failed. RuntimeExecutionError: max number of system retry attempts [11/10] exhausted. Last known status message: Failed to check Catalog for previous results: unexpected artifactData: [trimmed_fastqs] type: [map_value_type:<union_type:<variants:<blob:<> > variants:<union_type:<variants:<blob:<> structure:<tag:"FlyteFilePath" > > > > > > ] does not match any task output type: [map_value_type:<union_type:<variants:<blob:<> structure:<tag:"FlyteFilePath" > > variants:<simple:NONE structure:<tag:"none" > > > > ]
unfortunately i am unable to replicate it in flyte sandbox yet. Essentially when we relaunch a workflow due to failure, in that workflow the successful task that has optional input is stuck in 
Queued
state and flytepropeller is unable to resolve it. did anyone encountered this issue?
maybe it is an issue on 
datacatalog
side? I also noticed that for individual relaunch optional types input is also getting ignored(input becomes None) and thus do not utilize caching
g
how did you run the workflow at the beginning. pyflyte run?
m
from flyteconsole
g
so launch works, but failed to relaunch
m
yeah brand new launch will work but relaunch fails
g
are you able to share the workflow code to reproduce? we’ll investigate it tomorrow
m
so far no luck yet 😅 yeah once i have something definitely will post it here
h
This sounds very similar to this issue.
4 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by