<#7271 fix(spark): populate timestamp fields in lo...
# flyte-github
a
#7271 fix(spark): populate timestamp fields in log template input Pull request opened by cmditch Summary The Spark plugin's
getEventInfoForSpark
constructs
tasklog.Input
without populating any timestamp fields (
PodUnixStartTime
,
PodUnixFinishTime
,
PodRFC3339StartTime
,
PodRFC3339FinishTime
). This causes log template URIs referencing
{{ .podUnixStartTime }}
to render as
0
, making it impossible to construct time-bounded log links for external logging services like Grafana Loki or Kibana. The standard K8s plugin (
logging_utils.go
) and kfoperators (
common_operator.go
) already populate these fields from the resource's
CreationTimestamp
. This change applies the same pattern to all four
tasklog.Input
structs in
getEventInfoForSpark
. Changes • Compute
startTime
from
sj.CreationTimestamp.Unix()
(always available) • Compute
finishTime
from
sj.Status.TerminationTime
only when the app is in a terminal state (
Completed
or
Failed
) and
TerminationTime
is non-zero — otherwise
finishTime
remains
0
• Populate
PodUnixStartTime
,
PodUnixFinishTime
,
PodRFC3339StartTime
, and
PodRFC3339FinishTime
in all four
tasklog.Input
structs (Driver, User, System, and All-User logs) • Added
TestGetEventInfoTimestamps
test with two subtests: • Running app: verifies start time is populated and finish time is
0
• Completed app: verifies both start and finish times are populated from
CreationTimestamp
and
TerminationTime
respectively Testing • All existing Spark plugin tests pass • New test verifies that a log template URI containing
{{ .podUnixStartTime }}
renders the actual creation timestamp rather than
0
• New test verifies that finish time is only populated for terminal states using the CRD's
TerminationTime
• Manually tested against a Flyte Binary deployment with Grafana Loki log links using
{{ .podUnixStartTime }}000
for epoch-ms time ranges Fixes #6328 flyteorg/flyte