<#6583 Flyte 2: Cut over to Flyte 2 new backend im...
# flyte-github
a
#6583 Flyte 2: Cut over to Flyte 2 new backend implementation Pull request opened by EngHabu Summary This PR merges the entire Flyte 2 backend from the
v2
branch into
main
. It represents a ground-up rewrite of the Flyte control plane: Kubernetes-native, gRPC/buf-connect-based, PostgreSQL-backed, and structured as a set of independently runnable services that can also be composed into a single binary. This is a cutover, not an incremental change. After merge,
main
becomes the home of Flyte 2; Flyte 1 continues to live on
master
. What's in v2 New protobuf surface —
flyteidl2/
A clean v2 schema replacing
flyteidl
. Domains include
core
,
workflow
,
task
,
actions
,
event
,
auth
,
cluster
,
org
,
project
,
notification
,
secret
,
imagebuilder
,
trigger
,
cacheservice
,
dataproxy
,
connector
,
plugins
,
logs
,
settings
, and
common
. Code generation supports Go, TypeScript, Python, and Rust via
buf
. New gRPC services (each with its own
cmd/
,
config/
,
service/
,
repository/
,
migrations/
)
•
runs/
— RunService: workflow run lifecycle, with a built-in scheduler. •
queue/
— QueueService: action enqueue/abort. •
events/
— EventService: real-time event streaming via PostgreSQL
LISTEN/NOTIFY
. •
cache_service/
— Cache lookup/upload for task outputs. •
dataproxy/
— Signed-URL data proxy + presigned log streaming. •
secret/
— SecretService. •
app/
— AppLogsService (tail/stream pod logs). •
manager/
— Unified host process composing the services for single-binary deployments. All services share a common pattern: `pgx`/`gorm` for Postgres,
flytestdlib
for config/logging/database/health, buf-connect handlers over
net/http
, graceful shutdown, and standardized health endpoints. See
IMPLEMENTATION_SPEC.md
for the architectural contract. Executor —
executor/
Kubernetes-native task execution. Schedules tasks as pods, supports multi-cluster routing, service-account-based identity, and per-pod log tracking. Includes its own
Dockerfile
,
Makefile
, controller scaffolding, and dev workflow. Triggers Triggers API + validation annotations end-to-end (proto, server, scheduler). Generated clients —
gen/
Build artifacts for Go, TypeScript, Python, and Rust generated from
flyteidl2
. The Python SDK lives in its own repo (
flyteorg/flyte-sdk
). Deployment • New Helm charts:
charts/flyte-binary
,
charts/flyte-devbox
,
charts/flyte-sandbox
. • New top-level
Dockerfile
for the unified binary;
DOCKER_QUICK_START.md
for the local Docker workflow. • Reworked
docker/
and CI image-build workflows. Foundational packages •
flytestdlib/
— config, database, logging, promutils, etc., updated for v2 patterns. •
flyteplugins/
,
flytecopilot/
— carried forward and adapted. Repository & developer experience • Module path:
github.com/flyteorg/flyte/v2
. • Top-level
Makefile
,
buf.gen.{go,ts,python,rust}.yaml
,
.mockery.yaml
. • Contributor docs (
BACKEND_README.md
,
CONTRIBUTING.md
, per-service
README.md
/
DEVELOPMENT.md
). • New CODEOWNERS for v2 modules. Why now
main
exists specifically as the v2 line — it forked from
master
at the "Clear flyte to prep for v2" commit and is not being kept in sync with Flyte 1 work. Flyte 1 maintenance continues on
master
. This PR makes
main
reflect the actual state of v2 development that has been happening on the
v2
branch. DCO The
v2 → main
history contains 133 commits across ~20 contributors that fail the strict default DCO check (mostly GitHub
noreply
-vs-personal-email mismatches on commits that were originally merged through DCO-clean PRs to
master
). Resolved via: • #7297 — adds
.github/dco.yml
to
master
enabling third-party remediation commits. • #7298 — adds the remediation commit to
v2
covering all 133 SHAs. Merge strategy Squash • Flyte 2: Cut over to Flyte 2 new backend implementation 👈 • #6646 • #6690 • #6715 • #6748 • #6766 • #6806 • #6879 • #6906 • #6909 • #6933 • #6938 • #6944 • #6963 • #6983 • #6998 • #7023 • #7037 • #7052 • #7066 • #7121 • #7138 • #7154 • #7162 • #7171 • #7222 • #7238 • #7249 • #7263 • #7267 • #7269 • #7285 • #7316 • #7397 flyteorg/flyte
#6583 Flyte 2 Pull request ready for review by EngHabu flyteorg/flyte