#3720 [Core feature] Show FlyteDeck even when task fails
Issue created by
peridotml
Motivation: Why do you think this is important?
Users want to surface failing validation checks on data and models in Decks. For example,
• Show which checks failed in pandera
• A model did not pass quality checks, i.e. the model returned random results
This also allows users to use Decks early in tasks that could be used for debugging. I can see someone saving summary statistics on a dataset to the Deck before training. Then when training fails, they could inspect the deck and be like "Oh lol I had 10000 columns instead of 5"
Goal: What should the final outcome look like, ideally?
A user will go through this flow
1. See an error was raised that says their validation check failed
2. Click on FlyteDecks
3. Look at the reasons it failed
For plugins, like Pandera, the FlyteDeck cloud be built into the plugin!
Describe alternatives you've considered
Have the validation task succeed and then return a boolean to see if it failed. Then raise in a separate task.
@task(container_image=python_image, disable_deck=False)
def validate_dataset(df: pd.DataFrame) -> bool:
try:
InputSchema.validate(df, lazy=True)
except SchemaErrors as err:
Deck(
"Schema Validation Failures",
build_table(err.failure_cases, "orange_light"),
)
return True
return False
@workflow
def data_validaition_wf(valid: bool):
df = load_dataset()
failure = validate_dataset(df=df)
raise_error(failure=failure) >> etl_task(df=df)
I don't like the extra steps!
Propose: Link/Inline OR Additional context
slack convo:
https://flyte-org.slack.com/archives/CP2HDHKE1/p1684947483699369?thread_ts=1684945136.431699&cid=CP2HDHKE1
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyte