Machine learning orchestration is not a mature discipline yet.
Building ML pipelines per se is a complex process, which is why we developed new ML add-ons to take the load off your shoulders:
• Flyte Decks 📊 is Flyte’s way of visualizing data, such as outputs and intermediary data.
• The Flyte PyTorch types enables automatic conversion of tensors from GPU ➡️ CPU.
• Bridge the gap between developing and deploying ML models with Flyte ONNX types for ScikitLearn, PyTorch, and TensorFlow.
• Spark ✨ Pipelines can now be passed seamlessly among the Flyte tasks.
• Data and model monitoring are important phases of ML pipelines. They help gauge how efficient your data and ML models are. Use WhyLab’s whylogs 🪵 from within Flyte to monitor your data and models.
More details in @tall-lock-23197's latest blog How We Are Simplifying the Orchestration of Machine Learning Pipelines.