https://flyte.org
Join Slack
Hi All, I have a script with a few tasks and the workflow defined. Certain information is stored in...
# flyte-support
n
Hi All, I have a script with a few tasks and the workflow defined. Certain information is stored in a config file which I am importing into this script. But when I run the workflow, it is giving a module not found error. What should I be doing to import other scripts ?
g
were you using pyflyte run?
If so, pyflyte run doesn’t work for this case because it only upload workflow script to s3. We’re trying to improve it. Try to use pyflyte register instead
n
Yes, I was using pyflyte run
Should I now run it as "pyflyte register --remote ..... " ?
g
yes, and you have to launch the workflow by yourself on flyteconsole
sorry, no remote flag. something like
Copy code
pyflyte --config ~/.flyte/config-remote.yaml register example_union1.py (or a folder example/)
n
In this directory : ~/.flyte/ ; I currently have only config-sandbox.yaml.
Should I create the config-remote.yaml file ?
g
that’s fine, you can use sandbox.yaml
n
Ok, I'll try it
Thanks Kevin !
I tried it : Getting this error
Any thoughts on this ? Since the same workflow is running well when I use pyflyte run --remote.
g
No need “main”
n
It is showing successfully serialized flyte objects. But where would the console be to view the DAG ?
g
pyflyte register only registers your workflow, and won’t execute it. therefore, you have to go to http://localhost:30081/console/projects/flytesnacks/workflows, and launch the wf
👍 1
n
Understood. Then what modifications I need to do to store the model outputs at the required location (say our own s3 buckets in this case) ?
g
you have to overwrite the aws credentials in the pods. By default, the task output will be written to minio
Copy code
$ kubectl edit cm flyte-propeller-config
  k8s.yaml: |
    plugins:
      k8s:
        default-cpus: 600m
        default-env-vars:
        - FLYTE_AWS_ENDPOINT: <http://minio.flyte.svc.cluster.local:9000>
        - FLYTE_AWS_ACCESS_KEY_ID: minio
        - FLYTE_AWS_SECRET_ACCESS_KEY: miniostorage
        default-memory: 600Mi
👍 1
n
Got it
g
hmm, do you see any error in command line after you register the workflow?
n
No, I don't see any error. But I see the registered workflows in the console now
There's no issue regarding it, now.
g
great
n
Thanks
If I modify a script after registering the workflow, how should I ensure that when I re-launch the wf, the code changes are in effect. Currently I don't see that happening unless I register everytime I make a change.
g
yes, you need to re-register a new workflow if you modify the script. This make every version of the workflow reproducible
n
Ok 👍
The changes are not getting reflected even when I am re-registering as a different version in the same workflow. It's only working when I register it as a new workflow everytime. Any reason why it might be happening ?
t
Can you share with us the command you’re using to register the workflow?
n
pyflyte --config ~/.flyte/config-sandbox.yaml register --image fib:1.0 Test1.py
t
Could you share the error, please?
n
There is no error. The problem I'm facing is: After I register a workflow & launch it (version 1.0); And when I make some minor changes to the script, I re register the workflow and make this version 1.1. So now when I launch the workflow with the 1.1 version, the changes made in the script are not getting reflected in the output. I'm seeing the same output which was available from the previous run (basically everything is fetched from the cache).
👀 1
I want to know the reason behind this .
t
Could you send us the code you’re trying to change? A redacted version of your code should suffice.
n
The code is structured to query certain data from a mysql db. A config.py is imported which would have the info about the table to query from and the start and end dates. The first task will query certain records between 2 mentioned dates ( taken from the config file ). Another task will do certain operations and return the output as a list. Now after registering the version 1.0, I go ahead and make some changes in the config file and change the start and end date and re-register it as version 1.1. So theoretically this new version should give a different output compared to the first one.
g
got you. good catch, thanks. It seems like a bug. I can reproduce this issue 1, register the workflow 2. run the workflow 3. change output to 3 4. re-register the workflow 5. The output of t1 is still 2 (fetch from cache)
Copy code
from flytekit import task, workflow


@task(cache=True, cache_version="1.0")
def t1() -> int:
    return 2


@workflow
def wf() -> int:
    return t1()


if __name__ == "__main__":
    wf()
👍 1
@nice-judge-43061 ^^^ is it the issue you run into?
n
Yes, operation wise it's similar. Just that I'm making the changes in a different script (config.py in my case) and importing that into this.
Copy code
from flytekit import Resources,task
from flytekit import workflow


@task(cache_version="1.0", cache=True, limits=Resources(mem="500Mi"))
def add(a:int,b:int) -> int:
    return a+b

@task(cache_version="1.0", cache=True, limits=Resources(mem="500Mi"))
def multiply(x:int,y:int) -> int:
    return x*y

@workflow
def output(l:int,m:int,n:int) -> int:
    return multiply(x=n,y=add(a=l,b=m))
@tall-lock-23197 I'm registering this workflow. In this code, for version 1.0, I'm multiplying x & y. And later in the next version I'm adding x & y. The output when I run the 1.1 version is same as 1.0.
👍 1
t
@glamorous-carpet-83516, do we have an issue for this?
g
No, we don’t
t
I feel this is a high priority issue. WDYT, Kevin?
Also, this isn’t working when run locally, right?
g
yes, will work on it today.
Also, this isn’t working when run locally, right?
yes, and running remotely doesn’t work as well
🙏 1
t
@high-accountant-32689
oh nm
with dan on this one, i think this is the intended behavior
h
we talked about deriving the version of tasks from the code itself in the past, but that's not currently implemented.
Something like https://github.com/flyteorg/flyte/issues/770, but for all tasks
🆗 1
169 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by