<#7317 feat(connector): record connector endpoint ...
# flyte-github
a
#7317 feat(connector): record connector endpoint on LogContext Pull request opened by pingsutw Summary Plumbs the connector endpoint through to the log-streaming path so the dataplane dataproxy can proxy log requests directly to the connector instead of trying to tail pod logs that don't exist for connector-served actions. Proto changes •
flyteidl2/core/execution.proto
— adds
ConnectorLogContext { string endpoint }
and a new
LogContext.connector
field. The connector plugin records the resolved deployment's gRPC endpoint here, and the dataproxy reads it at log-stream time. •
flyteidl2/connector/connector.proto
— adds structured
repeated flyteidl2.logs.dataplane.LogLine lines = 2
to
GetTaskLogsResponseBody
(carries timestamp + originator). Marks the existing
repeated string results = 1
deprecated. Old connectors that emit
results
keep working; new connectors emit
lines
. •
flyteidl2/dataproxy/dataproxy_service.proto
— adds
string connector_endpoint = 4
on
TailLogsRequest
so the UI can pass the endpoint sourced from
ActionAttempt.LogContext.connector.endpoint
(avoids the dataproxy needing to re-resolve it from a possibly-stale event). Connector plugin changes (
flyteplugins/go/tasks/plugins/webapi/connector/plugin.go
)
•
ResourceWrapper
carries the resolved connector endpoint. •
Get
stamps the endpoint from the resolved
ConnectorDeployment
onto each returned
ResourceWrapper
. •
Status
writes
taskInfo.LogContext = { Connector: { Endpoint: ... } }
whenever the endpoint is set, so the action event carries it forward. Misc •
runs/service/run_service.go
— drops a stray
connect.NotFound
check that returned early when only
LogContext
was nil but the cluster name was still available; callers (dataproxy) now decide whether the missing LogContext is fatal. • Regenerated Go / Python / Rust / TypeScript bindings + swagger. Why Connector tasks have no Kubernetes pod, so the dataplane dataproxy can't tail logs the usual way. Recording the connector's gRPC endpoint on
LogContext
and passing it through
TailLogsRequest
lets the dataproxy proxy log requests straight to the connector's
AsyncConnectorService.GetTaskLogs
without a separate registry, ConfigMap watcher, or stale-event dance. The structured
lines
field lets connectors emit timestamps and originator metadata that survive end-to-end (UI included), instead of being stamped with observe-time on the dataproxy. Out of scope • Resource metadata for
GetTaskLogs
(
resource_meta
,
task_category
) still lives in plugin state and is fetched at log-stream time. • The dataproxy / operator-proxy wiring that consumes these proto changes ships in the unionai/cloud companion PR. flyteorg/flyte