Hi! I tried joining the office hours tonight but had some connection issues unfortunately 😞. I had reached out previously here . I've tried running the sandbox with the k8s operator for spark instructions again with the most recent release, and I'm still running into the same issues. I've followed the instructions here and tried running the most recent release registered via:

flytectl register files --config ~/.flyte/config.yaml <https://github.com/flyteorg/flytesnacks/releases/download/v0.3.70/snacks-cookbook-integrations-kubernetes-k8s_spark.tar.gz> --archive -p flytesnacks -d development --version latest

(I can run the examples locally)

++ id -u
+ myuid=0
++ id -g
+ mygid=0
+ set +e
++ getent passwd 0
+ uidentry=root:x:0:0:root:/root:/bin/bash
+ set -e
+ '[' -z root:x:0:0:root:/root:/bin/bash ']'
+ SPARK_CLASSPATH=':/opt/spark/jars/*'
+ env
+ grep SPARK_JAVA_OPT_
+ sed 's/[^=]*=\(.*\)/\1/g'
+ sort -t_ -k4 -n
+ readarray -t SPARK_EXECUTOR_JAVA_OPTS
Non-spark-on-k8s command provided, proceeding in pass-through mode...
+ '[' -n '' ']'
+ '[' '' == 2 ']'
+ '[' '' == 3 ']'
+ '[' -n '' ']'
+ '[' -z ']'
+ case "$1" in
+ echo 'Non-spark-on-k8s command provided, proceeding in pass-through mode...'
+ CMD=("$@")
+ exec /usr/bin/tini -s -- pyflyte-execute --inputs <s3://my-s3-bucket/metadata/propeller/flytesnacks-development-kdu1tzd712/n0/data/inputs.pb> --output-prefix <s3://my-s3-bucket/metadata/propeller/flytesnacks-development-kdu1tzd712/n0/data/0> --raw-output-data-prefix <s3://my-s3-bucket/72/kdu1tzd712-n0-0> --checkpoint-path <s3://my-s3-bucket/72/kdu1tzd712-n0-0/_flytecheckpoints> --prev-checkpoint '""' --resolver flytekit.core.python_auto_container.default_task_resolver -- task-module k8s_spark.dataframe_passing task-name create_spark_df
22/04/21 04:27:54 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
{"asctime": "2022-04-21 04:28:13,716", "name": "flytekit.entrypoint", "levelname": "ERROR", "message": "!! Begin System Error Captured by Flyte !!"}
{"asctime": "2022-04-21 04:28:13,717", "name": "flytekit.entrypoint", "levelname": "ERROR", "message": "Traceback (most recent call last):\n\n      File \"/opt/venv/lib/python3.8/site-packages/flytekit/exceptions/scopes.py\", line 165, in system_entry_point\n        return wrapped(*args, **kwargs)\n      File \"/opt/venv/lib/python3.8/site-packages/flytekit/core/base_task.py\", line 463, in dispatch_execute\n        new_user_params = self.pre_execute(ctx.user_space_params)\n      File \"/opt/venv/lib/python3.8/site-packages/flytekitplugins/spark/task.py\", line 124, in pre_execute\n        self.sess = sess_builder.getOrCreate()\n      File \"/opt/venv/lib/python3.8/site-packages/pyspark/sql/session.py\", line 228, in getOrCreate\n        sc = SparkContext.getOrCreate(sparkConf)\n      File \"/opt/venv/lib/python3.8/site-packages/pyspark/context.py\", line 392, in getOrCreate\n        SparkContext(conf=conf or SparkConf())\n      File \"/opt/venv/lib/python3.8/site-packages/pyspark/context.py\", line 144, in __init__\n        SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)\n      File \"/opt/venv/lib/python3.8/site-packages/pyspark/context.py\", line 339, in _ensure_initialized\n        SparkContext._gateway = gateway or launch_gateway(conf)\n      File \"/opt/venv/lib/python3.8/site-packages/pyspark/java_gateway.py\", line 108, in launch_gateway\n        raise RuntimeError(\"Java gateway process exited before sending its port number\")\n\nMessage:\n\n    Java gateway process exited before sending its port number\n\nSYSTEM ERROR! Contact platform administrators."}
{"asctime": "2022-04-21 04:28:13,717", "name": "flytekit.entrypoint", "levelname": "ERROR", "message": "!! End Error Captured by Flyte !!"}

hey Katrina I am on the office hours right now

Hey @limited-dog-47035. So we fixed the issue! 😅 You’d have to bump the hadoop-aws and aws-java-sdk versions if you’re using spark 3.2.1. We created a PR for the same. And, please set SPARK_VERSION to 3.2.1 in the Dockerfile and also pyspark to 3.2.1 in the requirements. Here’s the relevant PR. You would have to wait for a new flytekit version to pull the latest changes, but you can try this out. Hope this works for you.

@limited-dog-47035 again sorry for the trouble and thank you for the patience. Thank you for all the work @tall-lock-23197