Jan Costandius
11/07/2023, 2:14 PM@task(
task_config=Spark(...),
container_image=...,
pod_template=PodTemplate(
...,
pod_spec=V1PodSpec(
...
tolerations=[V1Toleration(key="app", operator="Equal", value="flyte", effect="NoSchedule")],
),
),
)
Dunno whether this will display correctly as I had to write this on my phone.
I've tried pretty much everything I can think of, from setting the spark configs, to creating the pod template on k8s and referencing it with the pod_template_name kwarg.L godlike
11/07/2023, 2:56 PMJan Costandius
11/07/2023, 3:09 PMspark.kubernetes
configurations, but I have had no luck with doing the same with the podTemplateFile
spark k8s configs. Either way this route shouldn't be necessary, considering the completed PR I included above.Kevin Su
11/07/2023, 5:59 PMSlackbot
11/07/2023, 6:00 PMJan Costandius
11/07/2023, 7:04 PMKevin Su
11/07/2023, 7:29 PMJan Costandius
11/07/2023, 7:46 PMKevin Su
11/07/2023, 11:14 PMJan Costandius
11/08/2023, 5:29 AM