#933 Removing spark-user-id Pull request opened by pmahindrakar-oss Signed-off-by: pmahindrakar-oss prafulla.mahindrakar@gmail.com Without this change the spark driver pod fails with permission denied issue when writing to /.aws and anything to /tmp directory

+ exec /usr/bin/tini -s -- /opt/spark/bin/spark-submit --conf spark.driver.bindAddress=10.0.130.1 --deploy-mode client --properties-file /opt/spark/conf/spark.properties --class org.apache.spark.deploy.PythonRunner local:///opt/venv/bin/entrypoint.py pyflyte-execute --inputs <s3://union-j1-us-east-2-mockjackorg/metadata/propeller/flytesnacks-development-a7wbdflrdzs7sr7lhrff/n0/data/inputs.pb> --output-prefix <s3://union-j1-us-east-2-mockjackorg/metadata/propeller/flytesnacks-development-a7wbdflrdzs7sr7lhrff/n0/data/0> --raw-output-data-prefix <s3://union-j1-us-east-2-mockjackorg/5a/a7wbdflrdzs7sr7lhrff-n0-0> --checkpoint-path <s3://union-j1-us-east-2-mockjackorg/5a/a7wbdflrdzs7sr7lhrff-n0-0/_flytecheckpoints> --prev-checkpoint '""' --resolver flytekit.core.python_auto_container.default_task_resolver -- task-module k8s_spark.pyspark_pi task-name hello_spark
22/12/22 22:55:23 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Matplotlib created a temporary config/cache directory at /tmp/matplotlib-t41x9d5w because the default path (/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.

{"asctime": "2022-12-22 22:55:26,788", "name": "flytekit", "levelname": "ERROR", "message": "Error from command '['aws', 's3', 'cp', '<s3://union-j1-us-east-2-mockjackorg/metadata/propeller/flytesnacks-development-a7wbdflrdzs7sr7lhrff/n0/data/inputs.pb>', '/tmp/flyte-bmtatnmx/sandbox/local_flytekit/inputs.pb']':\nb\"fatal error: [Errno 13] Permission denied: '/.aws'\\n\"\n"}
{"asctime": "2022-12-22 22:55:27,312", "name": "flytekit", "levelname": "ERROR", "message": "Error from command '['aws', '--no-sign-request', 's3', 'cp', '<s3://union-j1-us-east-2-mockjackorg/metadata/propeller/flytesnacks-development-a7wbdflrdzs7sr7lhrff/n0/data/inputs.pb>', '/tmp/flyte-bmtatnmx/sandbox/local_flytekit/inputs.pb']':\nb'fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden\\n'\n"}

Removing the usage of spark-user-id flyteorg/flytesnacks GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/kfmpi) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/spark_horovod) ✅ 28 other checks have passed 28/30 successful checks