<#3282 [cherry-pick] KeyError: 'sparkConf' occurs...
# flyte-github
c
#3282 [cherry-pick] KeyError: 'sparkConf' occurs when running a Databricks task without spark_conf Pull request opened by pingsutw ## Tracking issue Closes flyteorg/flyte#6471 ## Why are the changes needed? Using Databricks connector without setting optional
spark_conf
field will cause
KeyError: 'sparkConf'
## What changes were proposed in this pull request? If
sparkConf
is empty, use
{}
## How was this patch tested? Added unit test for DatabricksV2 with and without
spark_conf
### Setup process ### Screenshots ## Check all the applicable boxes • I updated the documentation accordingly. • All new and existing tests passed. • All commits are signed-off. ## Related PRs ## Docs link https://bito.ai#summarystart| ### Summary by Bito This pull request resolves a KeyError in Databricks tasks by using an empty dictionary when the optional spark_conf field is absent. It also includes unit tests for the new DatabricksV2 configuration to ensure functionality in both scenarios. flyteorg/flytekit