Here are some potential details Iam thinking about on decentralization without compromising system stability…
Discovery of Custom Agent ConfigMaps:
• Implement a mechanism that systematically discovers and collects custom agent ConfigMaps across namespaces or predefined locations. This ensures that any new custom agent configurations are properly detected and processed.
Validation of ConfigMaps:
• Before applying or loading any custom agent configuration, each ConfigMap should go through an automated validation process (via webhooks or other validation tools). If a ConfigMap fails validation, it would be excluded from being applied or loaded, without affecting other valid configurations.
• This ensures that invalid custom agent configurations do not propagate errors or make FlytePropeller unable to start.
Aggregation and Conflict Resolution:
• After validation, the valid ConfigMaps should be aggregated. This aggregation should handle potential *conflicts*—for example, if two agents have the same name or endpoint, a mechanism should resolve or flag the conflict.
• This ensures that only conflict-free and valid configurations are applied, maintaining isolation between configurations and preventing a bad agent config from affecting the entire Flyte setup.