Hi everyone, I was trying to test integration betw...
# ask-the-community
f
Hi everyone, I was trying to test integration between feast and minio s3 store using flytekit pipeline I have faced this error could anybody help
Copy code
"name": "flytekit", "levelname": "WARNING", "message": "Unsupported Type <class 'feast.repo_config.RepoConfig'> found, Flyte will default to use PickleFile as the transport. Pickle can only be used to send objects between the exact same version of Python, and we strongly recommend to use python type that flyte support."}
this is the code
Copy code
import os
import boto3
import logging
from flytekit import task
from feast.infra.offline_stores.file import FileOfflineStoreConfig
from feast.infra.online_stores.sqlite import SqliteOnlineStoreConfig
from feast.repo_config import RepoConfig
logger = logging.getLogger(__file__)

# execution on demo cluster
os.environ["FEAST_S3_ENDPOINT_URL"] = ENDPOINT = "<http://192.168.1.117:30002>"
os.environ["AWS_ACCESS_KEY_ID"] = "minio"
os.environ["AWS_SECRET_ACCESS_KEY"] = "miniostorage"

# flytectl config set s3.endpoint_url http://<minio_host>:<minio_port>
# flytectl config set s3.access_key <minio_access_key>
# flytectl config set s3.secret_key <minio_secret_key>

bucket_name = "my-s3-bucket"
registry_path = "data/registry.db"
online_store_path = "data/online_store.db"

@task
def create_bucket(
    bucket_name: str, registry_path: str, online_store_path: str
) -> RepoConfig:
    client = boto3.client(
        "s3",
        aws_access_key_id="minio",
        aws_secret_access_key="miniostorage",
        use_ssl=False,
        endpoint_url="<http://192.168.1.117:30002>",
    )
    return RepoConfig(
        registry=f"<s3://my-s3-bucket/data/online_store.db>",
        project="my_project",
        provider="local",
        offline_store=FileOfflineStoreConfig(),
        online_store=SqliteOnlineStoreConfig(path=online_store_path),
        entity_key_serialization_version=2,
    )
s
@Fi fi, you can ignore that warning. It isn't an error. It basically means that the type isn't registered with Flyte, and Flyte defaults to pickle for serialization of the task output.
256 Views