Jan Costandius11/07/2023, 2:14 PM
Dunno whether this will display correctly as I had to write this on my phone. I've tried pretty much everything I can think of, from setting the spark configs, to creating the pod template on k8s and referencing it with the pod_template_name kwarg.
@task( task_config=Spark(...), container_image=..., pod_template=PodTemplate( ..., pod_spec=V1PodSpec( ... tolerations=[V1Toleration(key="app", operator="Equal", value="flyte", effect="NoSchedule")], ), ), )
L godlike11/07/2023, 2:56 PM
Jan Costandius11/07/2023, 3:09 PM
configurations, but I have had no luck with doing the same with the
spark k8s configs. Either way this route shouldn't be necessary, considering the completed PR I included above.
Kevin Su11/07/2023, 5:59 PM
Slackbot11/07/2023, 6:00 PM
Jan Costandius11/07/2023, 7:04 PM
Kevin Su11/07/2023, 7:29 PM
Jan Costandius11/07/2023, 7:46 PM
Kevin Su11/07/2023, 11:14 PM
Jan Costandius11/08/2023, 5:29 AM