Question, does flyte support continues training and hyper parameter optimisation?

These features aren’t supported natively as such in Flyte. However, you could use, say, AWS Lambda with flyte to perform continual learning. Hyperparameter optimization can also be implemented as a flyte workflow. I recommend you to check out https://github.com/unionai-oss/unionml, a wrapper built on top of flyte to facilitate building and deploying ML models, where hyperparam opt and streaming data are on the roadmap. cc: @Niels Bantilan

At its core Flyte has a workflow engine. Grid search is easy to implement. You can use map tasks and / or use dynamic workflows. But anything fancier which needs pruning etc is not great