Hi everyone! 👋 I'm new here, and this is my first message in the Flyte community. 😊 I'm working on deploying Flyte on AWS to test its integration with the Spark operator. I followed the documentation for the Spark plugin and tried running the provided example with:

pyflyte run --remote pyspark_pi.py my_spark

(from the Flytesnacks examples). While Spark seems to execute, I keep getting the following error:

Traceback (most recent call last):
...
ValueError: ZIP does not support timestamps before 1980

I’m a bit stuck, as I can't seem to find what's causing the issue. Has anyone encountered this error before or have any insights into what might be happening? Thanks so much for any help you can provide! 🙏

ohh we clear out the timestamps so that the checksum remains constant. where do you see this problem? When i run spark i dont see this problem cc @glamorous-carpet-83516?

It's on the hello_spark task from the flytesnacks example https://github.com/flyteorg/flytesnacks/tree/master/examples/k8s_spark_plugin/k8s_spark_plugin The exact error is

Traceback (most recent call last):

      File "/opt/micromamba/envs/runtime/lib/python3.9/site-packages/flytekit/bin/entrypoint.py", line 126, in _dispatch_execute
    outputs = task_def.dispatch_execute(ctx, idl_input_literals)
      File "/opt/micromamba/envs/runtime/lib/python3.9/site-packages/flytekit/core/base_task.py", line 719, in dispatch_execute
    new_user_params = self.pre_execute(ctx.user_space_params)
      File "/opt/micromamba/envs/runtime/lib/python3.9/site-packages/flytekitplugins/spark/task.py", line 207, in pre_execute
    shutil.make_archive(file_name, file_format, os.getcwd())
      File "/opt/micromamba/envs/runtime/lib/python3.9/shutil.py", line 1093, in make_archive
    filename = func(base_name, base_dir, **kwargs)
      File "/opt/micromamba/envs/runtime/lib/python3.9/shutil.py", line 990, in _make_zipfile
    zf.write(path, path)
      File "/opt/micromamba/envs/runtime/lib/python3.9/zipfile.py", line 1748, in write
    zinfo = ZipInfo.from_file(filename, arcname,
      File "/opt/micromamba/envs/runtime/lib/python3.9/zipfile.py", line 519, in from_file
    zinfo = cls(arcname, date_time)
      File "/opt/micromamba/envs/runtime/lib/python3.9/zipfile.py", line 362, in __init__
    raise ValueError('ZIP does not support timestamps before 1980')

Message:

    ValueError: ZIP does not support timestamps before 1980