#3362 [Core feature] Task time estimation / progress bar capacity Issue created by zeryx Motivation: Why do you think this is important? Workflow visibility is fantastic for seeing the status of a workflow, and when a task launched. However for managed solutions it can be hard to view the direct kubernetes logs. Specifically the problem is most pronounced with long running tasks that have multiple steps that are doing an inner map or parallel map operation. Goal: What should the final outcome look like, ideally? The user should be able to go to the flyte dashboard, and for their long running mapping task - they could see an estimate for how long it will take for that task to complete, that the task is moving forward and progressing; and how many steps have been completed already. Describe alternatives you've considered printing task level logging to the user can provide this same level of observability, if for whatever reason there were security issues with providing this to a practitioner within a larger managed environment we might not be able to use a simple stdout / stderr log tail solution. Propose: Link/Inline OR Additional context TQDM: https://github.com/tqdm/tqdm This allows for wrapping for loops and iterators and print to some logging service / solution. For tasks with a large scale mapping flow, or parallel mapping flow this is normal and extremely practical and easy to use. Compatible with every ML framework and is an easy interaction on the users side. My proposal is that we wrap tqdm within the flytekit service, something like

import Task, Workflow, Tqdm from flytekit
import torch as tf
@Task
def some_func_with_mapping(...) -> ...:
  model = tf.load(...)
  output = []
  tqdm(for row in ...):
     output.append(model.forward(row))
  return output

Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte