#707 [Feature] [Plugins] EMR Hive/Spark
Issue created by
EngHabu
Motivation: Why do you think this is important?
AWS EMR offers a Hadoop managed service to run hive and containerized spark workloads that promises scalability and less management overhead.
Goal: What should the final outcome look like, ideally?
backend webAPI plugins to run hive and spark workloads on AWS EMR.
Ideally these plugins should not require flytekit work and should be able to leverage existing spark/hive protos to target EMR.
Example for AWS Athena plugin
AWS EMR Spark Example
Current k8s-based Spark Plugin
Current Hive plugin
Flyte component
☐ Overall
☐ Flyte Setup and Installation scripts
☐ Flyte Documentation
☐ Flyte communication (slack/email etc)
☐ FlytePropeller
☐ FlyteIDL (Flyte specification language)
☑︎ Flytekit (Python SDK)
☐ FlyteAdmin (Control Plane service)
☑︎ FlytePlugins
☐ DataCatalog
☐ FlyteStdlib (common libraries)
☐ FlyteConsole (UI)
☐ Other
flyteorg/flyte