acoustic-carpenter-78188
03/10/2024, 11:31 AMpyflyte backfill
command.
Ideally when running a backfill, it would be possible to ignore cache for all executions in that backfill. Backfills are sometimes necessary when upstream data had a data quality issue which was later fixed, requiring downstream jobs to re-run on the restated data.
Goal: What should the final outcome look like, ideally?
The pyflyte backfill
command should have a flag named something like --ignore-cache
or --overwrite-cached-outputs
that behaves similar to the web UI when selecting "Overwrite cached outputs".
Describe alternatives you've considered
I don't know of simple alternatives to running a backfill from the command line that has an option to overwrite the cache.
Propose: Link/Inline OR Additional context
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteacoustic-carpenter-78188
03/10/2024, 11:31 AM