Gabriel Molina
05/16/2023, 3:13 PMKetan (kumare3)
Gabriel Molina
05/16/2023, 4:17 PMKevin Su
05/16/2023, 5:42 PMGabriel Molina
05/16/2023, 11:17 PMKevin Su
05/16/2023, 11:26 PMKetan (kumare3)
Samhita Alla
Gabriel Molina
05/17/2023, 2:55 PMSamhita Alla
Gabriel Molina
05/18/2023, 1:18 PMKetan (kumare3)
Gabriel Molina
05/18/2023, 2:11 PMSamhita Alla
im able to run the spark example shared by kevin now with --remote flag! but got the following error in the web UI:Would you be able to fetch the k8s log? https://docs.flyte.org/en/latest/community/troubleshoot.html#troubleshoot
kubectl get pods -n flytesnacks-development
kubectl logs <pod-name> -n flytesnacks-development
Gabriel Molina
05/18/2023, 3:54 PMtar: Removing leading `/' from member names
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /usr/local/bin/pyflyte-execute:8 in <module> │
│ │
│ ❱ 8 │ sys.exit(execute_task_cmd()) │
│ │
│ /usr/local/lib/python3.10/site-packages/click/core.py:1130 in __call__ │
│ │
│ ❱ 1130 │ │ return self.main(*args, **kwargs) │
│ │
│ /usr/local/lib/python3.10/site-packages/click/core.py:1055 in main │
│ │
│ ❱ 1055 │ │ │ │ │ rv = self.invoke(ctx) │
│ │
│ /usr/local/lib/python3.10/site-packages/click/core.py:1404 in invoke │
│ │
│ ❱ 1404 │ │ │ return ctx.invoke(self.callback, **ctx.params) │
│ │
│ /usr/local/lib/python3.10/site-packages/click/core.py:760 in invoke │
│ │
│ ❱ 760 │ │ │ │ return __callback(*args, **kwargs) │
│ │
│ /usr/local/lib/python3.10/site-packages/flytekit/bin/entrypoint.py:471 in │
│ execute_task_cmd │
│ │
│ ❱ 471 │ _execute_task( │
│ │
│ /usr/local/lib/python3.10/site-packages/flytekit/exceptions/scopes.py:160 in │
│ system_entry_point │
│ │
│ ❱ 160 │ │ │ │ return wrapped(*args, **kwargs) │
│ │
│ /usr/local/lib/python3.10/site-packages/flytekit/bin/entrypoint.py:347 in │
│ _execute_task │
│ │
│ ❱ 347 │ │ _task_def = resolver_obj.load_task(loader_args=resolver_args) │
│ │
│ /usr/local/lib/python3.10/site-packages/flytekit/core/utils.py:295 in │
│ wrapper │
│ │
│ ❱ 295 │ │ │ │ return func(*args, **kwargs) │
│ │
│ /usr/local/lib/python3.10/site-packages/flytekit/core/python_auto_container. │
│ py:235 in load_task │
│ │
│ ❱ 235 │ │ task_module = importlib.import_module(name=task_module) # typ │
│ │
│ /usr/local/lib/python3.10/importlib/__init__.py:126 in import_module │
│ │
│ ❱ 126 │ return _bootstrap._gcd_import(name[level:], package, level) │
│ in _gcd_import:1050 │
│ in _find_and_load:1027 │
│ in _find_and_load_unlocked:1006 │
│ in _load_unlocked:688 │
│ in exec_module:883 │
│ in _call_with_frames_removed:241 │
│ │
│ /root/flyte3.py:7 in <module> │
│ │
│ ❱ 7 from flytekitplugins.spark import Spark │
╰──────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: No module named 'flytekitplugins.spark'
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /usr/local/bin/pyflyte-fast-execute:8 in <module> │
│ │
│ ❱ 8 │ sys.exit(fast_execute_task_cmd()) │
│ │
│ /usr/local/lib/python3.10/site-packages/click/core.py:1130 in __call__ │
│ │
│ ❱ 1130 │ │ return self.main(*args, **kwargs) │
│ │
│ /usr/local/lib/python3.10/site-packages/click/core.py:1055 in main │
│ │
│ ❱ 1055 │ │ │ │ │ rv = self.invoke(ctx) │
│ │
│ /usr/local/lib/python3.10/site-packages/click/core.py:1404 in invoke │
│ │
│ ❱ 1404 │ │ │ return ctx.invoke(self.callback, **ctx.params) │
│ │
│ /usr/local/lib/python3.10/site-packages/click/core.py:760 in invoke │
│ │
│ ❱ 760 │ │ │ │ return __callback(*args, **kwargs) │
│ │
│ /usr/local/lib/python3.10/site-packages/flytekit/bin/entrypoint.py:508 in │
│ fast_execute_task_cmd │
│ │
│ ❱ 508 │ subprocess.run(cmd, check=True) │
│ │
│ /usr/local/lib/python3.10/subprocess.py:526 in run │
│ │
│ ❱ 526 │ │ │ raise CalledProcessError(retcode, process.args, │
╰──────────────────────────────────────────────────────────────────────────────╯
CalledProcessError: Command '['pyflyte-execute', '--inputs',
'<s3://my-s3-bucket/metadata/propeller/flytesnacks-development-f52f3d40c3cf849a89>
5a/n0/data/inputs.pb', '--output-prefix',
'<s3://my-s3-bucket/metadata/propeller/flytesnacks-development-f52f3d40c3cf849a89>
5a/n0/data/0', '--raw-output-data-prefix',
'<s3://my-s3-bucket/data/rr/f52f3d40c3cf849a895a-n0-0>', '--checkpoint-path',
'<s3://my-s3-bucket/data/rr/f52f3d40c3cf849a895a-n0-0/_flytecheckpoints>',
'--prev-checkpoint', '""', '--dynamic-addl-distro',
'<s3://my-s3-bucket/flytesnacks/development/U67AW4X3WQLPZXY2HBRFM7GDWY======/scri>
pt_mode.tar.gz', '--dynamic-dest-dir', '/root', '--resolver',
'flytekit.core.python_auto_container.default_task_resolver', '--',
'task-module', 'flyte3', 'task-name', 'hello_spark']' returned non-zero exit
status 1.
Kevin Su
05/18/2023, 5:48 PMpyflyte register …
instead.Ketan (kumare3)
Kevin Su
05/19/2023, 4:58 AM