• Han

    Han

    3 months ago
    Quick question. we have people working on a shared server. I have written a workflow that runs locally under my env. But when my colleague tries to run the same workflow on the server, he got the following error
    $ python tasks/run_workflow.py
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/pathlib.py", line 1284, in mkdir
        self._accessor.mkdir(self, mode)
    FileNotFoundError: [Errno 2] No such file or directory: '/tmp/flyte/20220422_220444/sandbox/local_flytekit'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/pathlib.py", line 1284, in mkdir
        self._accessor.mkdir(self, mode)
    FileNotFoundError: [Errno 2] No such file or directory: '/tmp/flyte/20220422_220444/sandbox'
    
    During handling of the above exception, another exception occurred:
    ...
    ...
    PermissionError: [Errno 13] Permission denied: '/tmp/flyte/20220422_220444'
    How do I ensure reproducibility for different local environments?
  • Eduardo Apolinario (eapolinario)

    Eduardo Apolinario (eapolinario)

    3 months ago
    @Han, can you share more details about your workflow? How are you running it? And how is your colleague running it?
  • Han

    Han

    3 months ago
    We both work on the same shared server. We are both just doing
    python run_workflow.py
    from our user accounts on that server.
  • Ketan (kumare3)

    Ketan (kumare3)

    3 months ago
    aah you mean shared-server for the local development?
  • it is like a jupyter shared server?
  • it seems you do not have permissions to create
    /tmp/flyte
    folder?
  • Eduardo Apolinario (eapolinario)

    Eduardo Apolinario (eapolinario)

    3 months ago
    Can you double check if your colleague has access to /tmp/flyte?
  • Ketan (kumare3)

    Ketan (kumare3)

    3 months ago
    @Eduardo Apolinario (eapolinario) i think what happened is one of them first created
    /tmp/flyte
    and the second person cannot access it, as we restrict it to the uid only
  • we should probably create a random sandbox per vm?
  • Han

    Han

    3 months ago
    Yes! It's a shared server for development (bare metal). I am guessing that I ran a workflow “locally”, and flyte setup local temp, but others don't have permissions. So when they try to run flyte workflow on their accounts they don't have access and get blocked.
  • I can try to ask admin to open up access of that folder for everyone, not sure how would flyte sandbox change the permissions after a workflow run tho.
  • Ketan (kumare3)

    Ketan (kumare3)

    3 months ago
    No we will change this, if you can get by for a bit
  • Han

    Han

    3 months ago
    Ok. Please keep me posted.
  • Ketan (kumare3)

    Ketan (kumare3)

    3 months ago
    @Eduardo Apolinario (eapolinario) made a PR
  • @Han this was merged, slated for the next beta
  • will keep you posted
  • Eduardo Apolinario (eapolinario)

    Eduardo Apolinario (eapolinario)

    3 months ago
    @Han, can you give flytekit 1.0.0 a try?
  • Han

    Han

    3 months ago
    Yes, lemme grab it
  • guess it will take a while to propagate to our internal repo