#3328 Add Flytekit-Lepton Plugin for Lepton AI Inference Endpoints Pull request opened by ansjindal This commit introduces a new flytekit plugin for seamless integration with Lepton AI for deploying and managing AI inference endpoints within Flyte workflows. Key Features: • Support for multiple deployment types (vLLM, NIM, SGLang, custom containers) • Dynamic endpoint creation and lifecycle management • Storage mount configuration for model caching • Environment variable and secret management • Auto-scaling configuration support • Comprehensive example usage patterns Main Components: • LeptonConfig: Task configuration class for Lepton deployments • LeptonEndpointTask: Flyte task for creating inference endpoints • LeptonConnector: Backend connector for managing Lepton resources • Examples: Basic, advanced, and vLLM inference patterns ## How was this patch tested?

pyflyte run --remote examples/basic_inference.py basic_inference_workflow
pyflyte run --remote examples/vllm_example.py vllm_inference_workflow
pyflyte run --remote examples/advanced_usage.py multi_model_comparison_workflow

### Setup process 1. Run Flyte sandbox 2. Agent build

cd flytekit/plugins/flytekit-dgxc-lepton
docker build -f Dockerfile.agent -t localhost:30000/lepton-agent:latest .
docker push localhost:30000/lepton-agent:latest

1. Deployment update image tag in agent-deployment.yaml file along with secrets for workspace and token.

kubectl apply -f deployment/agent-deployment.yaml
kubectl get pods -n flyte -l app=lepton-agent
kubectl logs -n flyte deployment/lepton-agent --follow

1. Update flyte config:

# deployment/lepton-agent-config.yaml
tasks:
  task-plugins:
    enabled-plugins:
      - container
      - sidecar
      - k8s-array
      - lepton_endpoint_task
    default-for-task-types:
      - container: container
      - container_array: k8s-array
      - lepton_endpoint_task: lepton-agent-service.flyte:8000

### Screenshots Basic: [image](https://private-user-images.githubusercontent.com/170848644/491252269-ad9dfd1a-fbbd-408b-8853-28f5b9f8748b.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTk1MjQwNjIsIm5iZiI6MTc1OTUyMzc2MiwicGF0aCI6Ii8xNzA4NDg2NDQvNDkxMjUyMjY5LWFkOWRmZDFhLWZiYmQtNDA4Yi04ODUzLTI4ZjViOWY4NzQ4Yi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUxMDAzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MTAwM1QyMDM2MDJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1hOGE2MTBkNmRlZTJlZDkzZjIyMTQxYzhiZTI2YThiMTE5OTE3ZDBjNjJkZTlmNjI3YTEwYWM2OTQxYzFkMTYwJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.aRpvx_7HDY84R5e7oo4HS6VCX6LU8z7WocvfAfiN_qM) Advanced: [image](https://private-user-images.githubusercontent.com/170848644/491252379-cf9a0dfa-789a-49c9-a673-7e5b1e1bfb80.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTk1MjQwNjIsIm5iZiI6MTc1OTUyMzc2MiwicGF0aCI6Ii8xNzA4NDg2NDQvNDkxMjUyMzc5LWNmOWEwZGZhLTc4OWEtNDljOS1hNjczLTdlNWIxZTFiZmI4MC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUxMDAzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MTAwM1QyMDM2MDJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1iYmRiNjM5ZmQwMGYyMDM4OGMzODgxMjY5Njc0Yjk4OTk4ZjA2ZWQ4OGI5ZDQ4N2RjNTY5ZmUyNzhkMjYzYThkJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.s1GKDlBarCpAxE-5o8yBPADA-nQqe84s3yFPBHAolTE) corresponding lepton endpoints were created successfuly. ## Check all the applicable boxes • I updated the documentation accordingly. • All new and existing tests passed. • All commits are signed-off. https://bito.ai#summarystart| ### Summary by Bito This pull request introduces the Flytekit-Lepton plugin, enhancing Flyte workflows by integrating Lepton AI for managing AI inference endpoints. Key features include support for multiple deployment types, dynamic endpoint creation, and comprehensive user guidance examples, along with new deployment configurations and a configuration map for dynamic endpoint management. The updates significantly improve resource management and configuration for AI models, ensuring robust functionality through an updated test suite. flyteorg/flytekit