Hi all, I encountered the problem that the Spark D...
# ask-the-communityc
chung-che lai
02/02/2024, 4:37 AMHi all, I encountered the problem that the Spark Driver pod log cannot be displayed on the console.
General
Python TaskSpark Taskchung-che lai
02/02/2024, 4:39 AMFrom the web url, it seems to be the
default settingchung-che lai
02/02/2024, 4:43 AMThis is the Helm values.yaml file that has been deployed.
I setting
logs Copy code
logs:
mixed:
kubernetes-enabled: true
kubernetes-url: |-
<http://10.233.112.73/#/log/{{.namespace> }}/{{ .podName }}/pod?namespace={{ .namespace }}chung-che lai
02/02/2024, 5:15 AMviewing spark logs setting in the flyte-flyte-sandbox-config configmap
k
Kevin Su
02/05/2024, 6:54 AMformat is wrong. should be
Copy code
<http://10.233.112.73:30082/#!/log/{{> .namespace }}/{{ .podName }}/pod?namespace={{ .namespace }}c
chung-che lai
02/05/2024, 6:55 AMgot it, I try it.
chung-che lai
02/05/2024, 7:32 AMIs this setting correct in the Helm values file?
Copy code
plugins:
k8s:
default-env-vars:
- FLYTE_AWS_ENDPOINT: http://{{ printf "%s-minio" .Release.Name | trunc 63 | trimSuffix "-" }}.{{ .Release.Namespace }}:9000
- FLYTE_AWS_ACCESS_KEY_ID: minio
- FLYTE_AWS_SECRET_ACCESS_KEY: miniostorage
spark:
spark-config-default:
- spark.driver.cores: "1"
- spark.hadoop.fs.s3a.aws.credentials.provider: "org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider"
- spark.hadoop.fs.s3a.endpoint: http://{{ printf "%s-minio" .Release.Name | trunc 63 | trimSuffix "-" }}.{{ .Release.Namespace }}:9000
- spark.hadoop.fs.s3a.access.key: "minio"
- spark.hadoop.fs.s3a.secret.key: "miniostorage"
- spark.hadoop.fs.s3a.path.style.access: "true"
- spark.kubernetes.allocation.batch.size: "50"
- spark.hadoop.fs.s3a.acl.default: "BucketOwnerFullControl"
- spark.hadoop.fs.s3n.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem"
- spark.hadoop.fs.AbstractFileSystem.s3n.impl: "org.apache.hadoop.fs.s3a.S3A"
- spark.hadoop.fs.s3.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem"
- spark.hadoop.fs.AbstractFileSystem.s3.impl: "org.apache.hadoop.fs.s3a.S3A"
- spark.hadoop.fs.s3a.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem"
- spark.hadoop.fs.AbstractFileSystem.s3a.impl: "org.apache.hadoop.fs.s3a.S3A"
logs:
mixed:
kubernetes-enabled: true
kubernetes-url: |-
<http://10.233.112.73:30082/#!/log/{{> .namespace }}/{{ .podName }}/pod?namespace={{ .namespace }}chung-che lai
02/05/2024, 7:55 AMsame result
chung-che lai
02/05/2024, 7:56 AMScreenshot 2024-02-05 at 3.55.58 PM.png
chung-che lai
02/05/2024, 8:00 AMI manually adjusted the URL and found the correct result
k
Kevin Su
02/05/2024, 8:20 AMseems like helm can’t parse
{{}}Kevin Su
02/05/2024, 8:21 AMcould you edit the config directly, and see if that works?
Copy code
kubectl edit cm flyte-flyte-sandbox-configc
chung-che lai
02/05/2024, 8:21 AMok , I try it
chung-che lai
02/05/2024, 8:24 AMupdated and
k -n flyte rollout restart deployment flyte-flyte-sandboxchung-che lai
02/05/2024, 8:27 AMsame url result
<http://localhost:30082/#!/log/flytesnacks-development/a9wqtmj775jsbgvwwfdc-n0-0-driver/pod?namespace=flytesnacks-development>chung-che lai
02/05/2024, 8:28 AMI re-launch a workflow (new execution id*)*
k
Kevin Su
02/05/2024, 8:33 AMmind creating an issue here? [flyte-bug]
s
Slackbot
02/05/2024, 8:33 AMk
Kevin Su
02/05/2024, 8:33 AMI’ll take a look tomorrow morning
c
chung-che lai
02/05/2024, 8:33 AMok, thx
chung-che lai
02/05/2024, 8:52 AM7 Views