• Matheus Moreno

    Matheus Moreno

    1 month ago
    Hi, everyone. I'm having some trouble using the sandbox to pass DataFrames between tasks. No IAM roles configured, no Service Accounts, just a task that returns a DataFrame. This is the error I'm having:
    File "/home/flyte/.venv/lib/python3.9/site-packages/flytekit/core/data_persistence.py", line 447, in put_data
        raise FlyteAssertion(
    flytekit.exceptions.user.FlyteAssertion: Failed to put data from /tmp/flyte-9ibktn0s/sandbox/local_flytekit/engine_dir to <s3://my-s3-bucket/metadata/propeller/search-filter-improvement-by-semantic-search-development-alqv5l4nv4kms759s7k2/n0/data/0> (recursive=True).
    
    Original exception: Called process exited with error code: 1.  Stderr dump:
    
    b'upload failed: ../../tmp/flyte-9ibktn0s/sandbox/local_flytekit/engine_dir/error.pb to <s3://my-s3-bucket/metadata/propeller/search-filter-improvement-by-semantic-search-development-alqv5l4nv4kms759s7k2/n0/data/0/error.pb> An error occurred (AccessDenied) when calling the PutObject operation: Access Denied\n'
    Why is this happening? Sandbox image I'm using: cr.flyte.org/flyteorg/flyte-sandbox:dind-106a8147446a6f0221162f47a9260ea0a764426e Signature of the task:
    def check_new_amnt(sql_path: str, config_path: str) -> pd.DataFrame:
  • Kevin Su

    Kevin Su

    1 month ago
    message has been deleted
  • would you mind sharing the whole code snippet? I just ran the example above with the same sandbox image, it works for me
  • Matheus Moreno

    Matheus Moreno

    1 month ago
    Nevermid, I discovered the problem... I was overwriting the values-sandbox.yaml file because I didn't know it had configuration data in it. I am now appending the extra configurations and it works. Thank you!