I have a more general question pertaining to Feature Stores vs Orchestrators with Caches: For a web application (exploratory analysis dashboard / BI and decision workflow tracking), that shall make use of DataFrame-like tables with • raw data, • as well as refined features from those … what makes more sense to use? An orchestrator like Flyte or a Feature Store, if on-demand (lazy) access of these data by the app is the main goal, so processing resources are used conservatively but well defined and up-to-date refined data is available to the app?

depending on what you’re doing, you may end up needing both. flyte helps with things like running things in a reproducible way, it helps you manage the infrastructure for the compute. if you want to run tasks with different containers, passing data between them, on a schedule, where one task needs gpus and the other doesn’t, flyte makes this very easy. but it’s not a feature store. data is copy-on-write and immutable. a feature store is more akin to a database. you will probably end up using an orchestrator to hit the feature store in a consistent way.