Erik Dao
05/29/2023, 6:52 AMfrom flytekit.remote import FlyteRemote
from flytekit.configuration import Config, PlatformConfig, ImageConfig, SerializationSettings
remote = FlyteRemote(
config=Config(
platform=PlatformConfig(
endpoint="dns:///flyte-backend-flyte-binary-grpc.namespace.svc.cluster.local:8089",
insecure=True,
insecure_skip_verify=True,
)
),
default_project="flytesnacks",
default_domain="development",
data_upload_location="<s3a://flyte-storage/>"
)
HYPERPARAMS = {
"hyperparameters": {"C": 0.1}
}
flyte_entity = training_workflow
flyte_workflow = remote.register_workflow(
entity=flyte_entity,
serialization_settings=SerializationSettings(image_config=ImageConfig.auto_default_image()),
version="v1"
)
launch_plan = LaunchPlan.get_or_create(
workflow=flyte_workflow,
name="wine_classification_lp",
default_inputs=HYPERPARAMS,
)
remote.register_launch_plan(launch_plan, "v1", project="flytesnacks", domain="development")
And the workflow failed right at the first task get_data
, here is the error I’ve got
[1/1] currentAttempt done. Last Error: USER::rapped(*args, **kwargs) │
│ │
│ /usr/local/lib/python3.8/site-packages/flytekit/bin/entrypoint.py:347 in │
│ _execute_task │
│ │
│ ❱ 347 │ │ _task_def = resolver_obj.load_task(loader_args=resolver_args) │
│ │
│ /usr/local/lib/python3.8/site-packages/flytekit/core/utils.py:295 in wrapper │
│ │
│ ❱ 295 │ │ │ │ return func(*args, **kwargs) │
│ │
│ /usr/local/lib/python3.8/site-packages/flytekit/core/python_auto_container.p │
│ y:235 in load_task │
│ │
│ ❱ 235 │ │ task_module = importlib.import_module(name=task_module) # typ │
│ │
│ /usr/local/lib/python3.8/importlib/__init__.py:127 in import_module │
│ │
│ ❱ 127 │ return _bootstrap._gcd_import(name[level:], package, level) │
│ in _gcd_import:1011 │
│ in _sanity_check:950 │
╰──────────────────────────────────────────────────────────────────────────────╯
ValueError: Empty module name
Any suggestion on which might have caused this error and how to fix it? I’m suspecting that I haven’t registered the workflow or the launch plan properly.Samhita Alla
Erik Dao
05/29/2023, 10:53 AMValueError
issue.
However, I’m dealing with an S3 issue. Basically I have MinIO installed on the same k8s cluster as Flyte. I have supply data_config
in FlyteRemote.
remote = FlyteRemote(
config=Config(
platform=PlatformConfig(
endpoint="dns:///flyte-backend-flyte-binary-grpc.namespace.svc.cluster.local:8089",
insecure=True,
insecure_skip_verify=True,
),
data_config=DataConfig(s3=S3Config(
endpoint="<https://my-minio-endpoint.company.com>",
access_key_id="access-key",
secret_access_key="secret-key"
))
),
default_project="flytesnacks",
default_domain="development",
data_upload_location="<s3a://flyte-storage/>",
)
However, I can see that the get_data
is failing with error
FlyteAssertion: Failed to get data from
<s3://flyte-storage/flytesnacks/development/Z6G7IM5HUD77U7SVV2UQRHP5KQ===>
===/script_mode.tar.gz to ./ (recursive=False).
Original exception: Unable to locate credentials
I guess I’m still not configuring the S3/data config properly.
Any idea how to fix this issue?Samhita Alla
data_upload_location
have s3a?Erik Dao
05/29/2023, 3:04 PMSamhita Alla
Erik Dao
05/30/2023, 4:45 AMSamhita Alla
kubectl -n flyte edit cm flyte-sandbox-config
): https://discuss.flyte.org/t/9671922/running-the-greeting-example-i-am-getting-the-following-erro#ae1029d6-077a-4081-bee7-4fe85f57f3ae.Erik Dao
05/30/2023, 3:06 PM