Hi guys I m using Flyte on a project that predicts glass typ

Question

Hi guys I m using Flyte on a project that predicts glass type using `KNeighborsClassifier` It seems Flyte doesn t accept `KNeighborsClassifier` as the type of my function What would be the right type hint for this I get the following error ``` 4 4 currentAttempt done Last Error USER Pod failed No message received from kubernetes sd4h34hqd15wyc3wy32t n0 3 terminated with exit code 137 Reason OOMKilled Message asctime 2023 03 05 20 24 43 219 name flytekit levelname WARNING message Unsupported Type lt class sklearn neighbors classification KNeighborsClassifier gt found Flyte will default to use PickleFile as the transport Pickle can only be used to send objects between the exact same version of Python and we strongly recommend to use python type that flyte support 2023 03 05 20 24 43 WARNING mlflow utils git utils Failed to import Git the Git executable is probably not on your PATH so Git SHA is not available Error Failed to initialize Bad git executable The git executable must be specified in one of the following ways be included in your $PATH be set via $GIT PYTHON GIT EXECUTABLE explicitly set via git refresh All git commands will error until this is rectified This initial warning can be silenced or aggravated in the future by setting the $GIT PYTHON REFRESH environment variable Use one of the following values quiet|q|silence|s|none|n|0 for no warning or exception warn|w|warning|1 for a printed warning error|e|raise|r|2 for a raised exception Example export GIT PYTHON REFRESH=quiet asctime 2023 03 05 20 24 43 424 name flytekit levelname WARNING message Unsupported Type lt class sklearn neighbors classification KNeighborsClassifier gt found Flyte will default to use PickleFile as the transport Pickle can only be used to send objects between the exact same version of Python and we strongly recommend to use python type that flyte support ``` Can you please point me to the right direction

Kevin Su · Answer

Flyte By default will serialize the output int str list pd dataframe to `Flyte Literal` if the type is unrecognized by Flyte flytekit will serialize it to pickle you could register a custom type transformer yourself here is an

Taeef Najib · Answer

Thanks for your response Can I create a custom class in my script and use it as a custom type As shown here <https docs flyte org projects cookbook en latest auto core extend flyte custom types html advanced custom types> Was also wondering if I could simply use `ClassifierMixin` `BaseEstimator` or `Any` or maybe `None`

Samhita Alla · Answer

gt Can I create a custom class in my script and use it as a custom type Yes you can You ll need to have the methods specified in the guide gt Was also wondering if I could simply use `ClassifierMixin` `BaseEstimator` or `Any` or maybe `None` You can The error you re seeing is OOMKilled right In that case please increase the memory you re allocating to your task The unsupported one is a warning You can either register a custom type which is preferred or leave it as is

Niels Bantilan · Answer

interesting shouldn t `KNeighborsClassifier` be picked up by the `SklearnTypeTransformer` <https github com flyteorg flytekit blob master flytekit extras sklearn native py L72>

Taeef Najib · Answer

Thanks guys

Niels Bantilan · Answer

which version of flytekit are you using < Taeef Najib>

Taeef Najib · Answer

< Niels Bantilan> it s 1 2 7

Niels Bantilan · Answer

kk are you able to update or do you need to stick with 1 2 7

Niels Bantilan · Answer

1 2 7 won t have the support for sklearn estimators

Taeef Najib · Answer

We can upgrade to a later version Will that support KNeighborsClassifier as a type hint

Niels Bantilan · Answer

yep any version gt = 1 3 2 should support `KNeighborsClassifier`

Taeef Najib · Answer

Thanks I ll try upgrading then

Taeef Najib · Answer

< Niels Bantilan> After upgrading it to 1 4 0 I m getting this error ``` Traceback most recent call last File opt venv bin pyflyte line 5 in lt module gt from flytekit clis sdk in container pyflyte import main File opt venv lib python3 10 site packages flytekit init py line 202 in lt module gt from flytekit core base sql task import SQLTask File opt venv lib python3 10 site packages flytekit core base sql task py line 4 in lt module gt from flytekit core base task import PythonTask TaskMetadata File opt venv lib python3 10 site packages flytekit core base task py line 27 in lt module gt from flytekit core context manager import File opt venv lib python3 10 site packages flytekit core context manager py line 30 in lt module gt from flytekit clients import friendly as friendly client noqa File opt venv lib python3 10 site packages flytekit clients friendly py line 4 in lt module gt from flyteidl admin import common pb2 as common pb2 File opt venv lib python3 10 site packages flyteidl admin common pb2 py line 5 in lt module gt from google protobuf internal import builder as builder ImportError cannot import name builder from google protobuf internal opt venv lib python3 10 site packages google protobuf internal init py ```

Niels Bantilan · Answer

we had to yank 1 4 0 can you try installing 1 4 1

Taeef Najib · Answer

let me try

Taeef Najib · Answer

So we upgraded to 1 4 1 and got this error ``` 4 4 currentAttempt done Last Error USER Pod failed No message received from kubernetes snbh2jiispjmvrt5faat n0 3 terminated with exit code 1 Reason Error Message asctime 2023 03 08 16 51 11 316 name flytekit levelname WARNING message FlyteSchema is deprecated use Structured Dataset instead Traceback most recent call last File opt venv bin pyflyte execute line 8 in lt module gt sys exit execute task cmd File opt venv lib python3 10 site packages click core py line 1130 in call return self main args kwargs File opt venv lib python3 10 site packages click core py line 1055 in main rv = self invoke ctx File opt venv lib python3 10 site packages click core py line 1404 in invoke return ctx invoke self callback ctx params File opt venv lib python3 10 site packages click core py line 760 in invoke return callback args kwargs File opt venv lib python3 10 site packages flytekit bin entrypoint py line 476 in execute task cmd execute task File opt venv lib python3 10 site packages flytekit exceptions scopes py line 160 in system entry point return wrapped args kwargs File opt venv lib python3 10 site packages flytekit bin entrypoint py line 348 in execute task task def = resolver obj load task loader args=resolver args File opt venv lib python3 10 site packages flytekit core class based resolver py line 34 in load task return self mapping idx IndexError list index out of range ``` < Niels Bantilan>

Niels Bantilan · Answer

can you share the code and commands you re running

Taeef Najib · Answer

Here s the workflow code ```import sklearn import os import sys import typing from flytekit import Resources task workflow from main import Hyperparameters from main import generate dataset from main import train model wf outputs=typing NamedTuple WfOutputs train model 0=sklearn ensemble forest RandomForestClassifier workflow def test classification 2 wf args Hyperparameters gt wf outputs generate dataset=task requests=Resources cpu= 1 limits=Resources cpu= 1 retries=3 generate dataset train model=task requests=Resources cpu= 1 limits=Resources cpu= 1 retries=3 train model generate dataset o0 generate dataset o1 generate dataset o2 generate dataset o3 =generate dataset hp= wf args train model o0 =train model hp= wf args X train=generate dataset o0 y train=generate dataset o2 return wf outputs train model o0 ```

Taeef Najib · Answer

did it give any clue about the possible problem

Taeef Najib · Answer

< Niels Bantilan> do you need to see the code from the imported modules

Niels Bantilan · Answer

you need to define tasks in the top level scope of the module not inside a workflow function body You also need to make sure the output of the `train model task` needs to be ` wf outputs` directly you can t do ` wf outputs train model o0 ` because `train model o0 ` is a promise you can t operate on it like a regular python value ```import sklearn import os import sys import typing from flytekit import Resources task workflow from main import Hyperparameters from main import generate dataset from main import train model generate dataset task = task requests=Resources cpu= 1 limits=Resources cpu= 1 retries=3 generate dataset train model task = task requests=Resources cpu= 1 limits=Resources cpu= 1 retries=3 train model wf outputs=typing NamedTuple WfOutputs train model 0=sklearn ensemble forest RandomForestClassifier workflow def test classification 2 wf args Hyperparameters gt wf outputs generate dataset o0 generate dataset o1 generate dataset o2 generate dataset o3 = generate dataset task hp= wf args train model o0 =train model task hp= wf args X train=generate dataset o0 y train=generate dataset o2 return wf outputs train model o0 ``` Note that the output of `generate dataset` needs to be a `tuple` or `NamedTuple` for you to be able to unpack the outputs like `generate dataset o0 generate dataset o1 generate dataset o2 generate dataset o3 = generate dataset task hp= wf args `

Niels Bantilan · Answer

actually not confident whether ```generate dataset task = task requests=Resources cpu= 1 limits=Resources cpu= 1 retries=3 generate dataset ``` will work though it should in theory

Niels Bantilan · Answer

the main thing to remember about Flyte workflows is that the function body within ` workflow` decorated functions only supports certain operations You can read more about it here <https docs flyte org projects cookbook en latest getting started tasks and workflows html workflows>

Taeef Najib · Answer

< Niels Bantilan> The problem is deploying with v1 4 1 breaks with `index out of range` error whereas it works fine in v1 2 7 Is something different in v1 4 1 causing the error Workflow code is exactly same To confirm I tried what you suggested moving the task definitions outside the workflow but it throws this erros in both v1 2 7 and v1 4 1 ```Traceback most recent call last File opt venv bin pyflyte execute line 8 in lt module gt sys exit execute task cmd File opt venv lib python3 10 site packages click core py line 1130 in call return self main args kwargs File opt venv lib python3 10 site packages click core py line 1055 in main rv = self invoke ctx File opt venv lib python3 10 site packages click core py line 1404 in invoke return ctx invoke self callback ctx params File opt venv lib python3 10 site packages click core py line 760 in invoke return callback args kwargs File opt venv lib python3 10 site packages flytekit bin entrypoint py line 476 in execute task cmd execute task File opt venv lib python3 10 site packages flytekit exceptions scopes py line 160 in system entry point return wrapped args kwargs File opt venv lib python3 10 site packages flytekit bin entrypoint py line 354 in execute task handle annotated task ctx task def inputs output prefix File opt venv lib python3 10 site packages flytekit bin entrypoint py line 297 in handle annotated task dispatch execute ctx task def inputs output prefix File opt venv lib python3 10 site packages flytekit bin entrypoint py line 80 in dispatch execute logger debug f Starting dispatch execute for task def name AttributeError function object has no attribute name ``` Another thing I noticed when I made this change in the flyte console task details I see this ``` pyflyte execute inputs input output prefix outputPrefix raw output data prefix rawOutputDataPrefix checkpoint path checkpointOutputPrefix prev checkpoint prevCheckpointPrefix resolver flytekit core python auto container default task resolver task module main task name generate dataset ``` Instead of ``` pyflyte execute inputs input output prefix outputPrefix raw output data prefix rawOutputDataPrefix checkpoint path checkpointOutputPrefix prev checkpoint prevCheckpointPrefix resolver wf wf 1 my workflow 1 0 ```

Niels Bantilan · Answer

did you refactor your code can you share it

Niels Bantilan · Answer

The first task detail looks correct the second one looks off are you using a custom task resolver or custom container interface

Taeef Najib · Answer

```import sklearn import os import sys import typing from flytekit import Resources task workflow from main import Hyperparameters from main import generate dataset from main import train model generate dataset=task requests=Resources cpu= 1 limits=Resources cpu= 1 retries=3 generate dataset train model=task requests=Resources cpu= 1 limits=Resources cpu= 1 retries=3 train model wf outputs=typing NamedTuple WfOutputs train model 0=sklearn ensemble forest RandomForestClassifier workflow def my workflow 3 wf args Hyperparameters gt wf outputs generate dataset o0 generate dataset o1 generate dataset o2 generate dataset o3 =generate dataset hp= wf args train model o0 =train model hp= wf args X train=generate dataset o0 y train=generate dataset o2 return wf outputs train model o0 ```

Taeef Najib · Answer

No not using a custom task resolver or custom container interface The first one is before I took the tasks outside the workflow But when I take them out I get the `AttributeError function object has no attribute name ` and can t figure out why

Niels Bantilan · Answer

can you try renaming the tasks Like ```generate dataset task=task generate dataset train model task=task train model ```

Niels Bantilan · Answer

and then using `generate dataset task` and `train model task` in your workflow

Taeef Najib · Answer

```import sklearn import os import sys import typing from flytekit import Resources task workflow from main import Hyperparameters from main import generate dataset from main import train model generate dataset task=task requests=Resources cpu= 1 limits=Resources cpu= 1 retries=3 generate dataset train model task=task requests=Resources cpu= 1 limits=Resources cpu= 1 retries=3 train model wf outputs=typing NamedTuple WfOutputs train model task 0=sklearn ensemble forest RandomForestClassifier workflow def my workflow 3 wf args Hyperparameters gt wf outputs generate dataset task o0 generate dataset task o1 generate dataset task o2 generate dataset task o3 =generate dataset task hp= wf args train model task o0 =train model task hp= wf args X train=generate dataset task o0 y train=generate dataset task o2 return wf outputs train model task o0 ``` Same result

Niels Bantilan · Answer

the result being `AttributeError function object has no attribute name ` or ` ```packages flytekit core class based resolver py line 34 in load task return self mapping idx IndexError list index out of range``` or both

Taeef Najib · Answer

I think your suggestion may have fixed the `list index out of range` error But still fails with `AttributeError function object has no attribute name ` error during execution

Niels Bantilan · Answer

is this a local execution using `pyflyte run` it would also be help if you can provide the contents of the `main` module of you can

Niels Bantilan · Answer

can you go to this line of your flytekit installation ``` File opt venv lib python3 10 site packages flytekit bin entrypoint py line 80 in dispatch execute logger debug f Starting dispatch execute for task def name ``` And print out `task def` and it s `type` before line 80 I don t think it s an actual `PythonTask` object

Taeef Najib · Answer

regarding your previous message ``` Import necessary libraries import typing from sklearn datasets import make classification from sklearn model selection import train test split from sklearn ensemble import RandomForestClassifier from sklearn metrics import accuracy score import numpy as np from dataclasses import dataclass from dataclasses json import dataclass json dataclass json dataclass class Hyperparameters object n samples int = 1000 n features int = 20 n informative int = 15 n classes int = 2 test size float = 0 2 n estimator int = 100 Generate the dataset def generate dataset hp Hyperparameters gt typing Tuple np ndarray np ndarray np ndarray np ndarray X y = make classification n samples=hp n samples n features=hp n features n informative=hp n informative n classes=hp n classes return train test split X y test size=hp test size Train a random forest classifier on the train data def train model hp Hyperparameters X train np ndarray y train np ndarray gt RandomForestClassifier return RandomForestClassifier hp n estimator fit X train y train ``` No this is a remote execution via flyte REST API

Taeef Najib · Answer

Regarding your last message give me some time I ll give you an update about the result

Niels Bantilan · Answer

so I just got this working on a sandbox cluster basically flytekit currently doesn t support defining tasks like this ```generate dataset task=task requests=Resources cpu= 1 limits=Resources cpu= 1 retries=3 generate dataset ``` It basically needs to be used as a decorator Here s a one scripter that works ``` Import necessary libraries import typing import sklearn from flytekit import Resources task workflow from sklearn datasets import make classification from sklearn model selection import train test split from sklearn ensemble import RandomForestClassifier from sklearn metrics import accuracy score import numpy as np from dataclasses import dataclass from dataclasses json import dataclass json dataclass json dataclass class Hyperparameters object n samples int = 1000 n features int = 20 n informative int = 15 n classes int = 2 test size float = 0 2 n estimator int = 100 Generate the dataset task requests=Resources cpu= 1 limits=Resources cpu= 1 retries=3 def generate dataset hp Hyperparameters gt typing Tuple np ndarray np ndarray np ndarray np ndarray X y = make classification n samples=hp n samples n features=hp n features n informative=hp n informative n classes=hp n classes return train test split X y test size=hp test size Train a random forest classifier on the train data task requests=Resources cpu= 1 limits=Resources cpu= 1 retries=3 def train model hp Hyperparameters X train np ndarray y train np ndarray gt RandomForestClassifier return RandomForestClassifier hp n estimator fit X train y train wf outputs=typing NamedTuple WfOutputs train model task 0=sklearn ensemble forest RandomForestClassifier workflow def my workflow 3 wf args Hyperparameters gt wf outputs generate dataset task o0 generate dataset task o1 generate dataset task o2 generate dataset task o3 =generate dataset hp= wf args train model task o0 =train model hp= wf args X train=generate dataset task o0 y train=generate dataset task o2 return wf outputs train model task o0 if name == main print my workflow 3 wf args=Hyperparameters ```

Niels Bantilan · Answer

you can do the ` task` decoration directly where the functions are define in `main`

Taeef Najib · Answer

That s strange because it was working for 1 2 7 Decorators are just functions and isn t flyte just plain python outside of ` workflow` The reason we need to do this is because we re using flyte in an automated setup and we need to be able to apply the decorator to the imported functions How would we do that

Taeef Najib · Answer

Also using `pyflyte run` works just fine with the code we currently have doesn t that mean flyte should work in a remote execution as well

Niels Bantilan · Answer

< Yee> < Kevin Su> is this behavior supported ^^ this might be a bug

Niels Bantilan · Answer

I was able to reproduce this error here <https development uniondemo run console projects flytesnacks domains development executions arz5gpmmwlr2lbj49nwn duration=all amp fromExecutionNav=true|https development uniondemo run console projects flytesnacks domains development execut arz5gpmmwlr2lbj49nwn duration=all amp fromExecutionNav=true>

Niels Bantilan · Answer

ah I think I misled you a little you ll need to overwrite the function name after all as you originally had it ```generate dataset=task requests=Resources cpu= 1 limits=Resources cpu= 1 retries=3 generate dataset train model=task requests=Resources cpu= 1 limits=Resources cpu= 1 retries=3 train model ``` I got this to work on my end can you try it

Niels Bantilan · Answer

basically flytekit got confused it was looking for `generate dataset` as the task name but it was a function in the module s namespace so re defining it as a task makes it work I m not sure why this wasn t working earlier perhaps we ll get another error on your side

Taeef Najib · Answer

OK so it s the same error when we change the task function name to overwrite the imported function name

Niels Bantilan · Answer

can you say how exactly you re 1 packaging the workflow 2 registering it 3 running it UI or `FlyteRemote` pyflyte run

Niels Bantilan · Answer

< Yee> < Eduardo Apolinario eapolinario > above use case seems to be supported in earlier flytekit versions but doesn t work in 1 4 1 basically we import some functions that we then use as tasks in a separate module ```from other module import fn from flytekit import task workflow fn = task fn workflow def wf fn ``` This will raise an error ``` 4 4 currentAttempt done Last Error USER Pod failed No message received from kubernetes apdmwk4bc9m9nq27gfth n0 3 terminated with exit code 1 Reason Error Message bin entrypoint py line 80 in dispatch execute logger debug f Starting dispatch execute for task def name AttributeError function object has no attribute name Traceback most recent call last File usr local bin pyflyte fast execute line 8 in lt module gt sys exit fast execute task cmd File usr local lib python3 8 site packages click core py line 1130 in call return self main args kwargs File usr local lib python3 8 site packages click core py line 1055 in main rv = self invoke ctx File usr local lib python3 8 site packages click core py line 1404 in invoke return ctx invoke self callback ctx params File usr local lib python3 8 site packages click core py line 760 in invoke return callback args kwargs File usr local lib python3 8 site packages flytekit bin entrypoint py line 513 in fast execute task cmd subprocess run cmd check=True File usr local lib python3 8 subprocess py line 516 in run raise CalledProcessError retcode process args subprocess CalledProcessError Command pyflyte execute inputs output prefix raw output data prefix checkpoint path prev checkpoint dynamic addl distro dynamic dest dir root resolver flytekit core python auto container default task resolver task module other module task name fn returned non zero exit status 1 ```

Niels Bantilan · Answer

it s failing because the flyte task resolver thinks the task is defined in `other module fn` so it tries to grab the task from there but it s not a task it s a function

Niels Bantilan · Answer

< Taeef Najib> so I tried running this on flytekit 1 2 7 and it doesn t work it gives me the same error message ```logger debug f Starting dispatch execute for task def name ```

Yee · Answer

is it really called `fn` in multiple places can we rename

Yee · Answer

shouldn t matter

Taeef Najib · Answer

Inside a CI CD pipeline ````pyflyte pkgs wf package output flyte workflow package tgz image <http gcr io blah|gcr io blah> force` `flytectl register files project valor unicorn domain development archive flyte workflow package tgz config root flyte config yaml version 0 0 16```` Yes I can confirm that this code fails with the same error on 1 2 7 as well 1 2 7 succeeds if I put the task defs inside the workflow edited But the same code runs fine locally via `pyflyte run` on 1 4 1 so I d say either the way we re registering running the workflow is somehow deprecated or this seems like a bug I think it s fair to assume that the code that runs fine via `pyflyte run` should also run without issues remotely What would you suggest is the right way to write the flyte workflow if I had to import functions from other modules like this How can I get this to work

Taeef Najib · Answer

< Niels Bantilan> < Yee> It must be weekend in your time zone Could you please take a look at it when you return to work I m extremely excited about making it work Thank you for what you all are doing for me pray enjoy your weekends

Eduardo Matus · Answer

have the same error ``` 1 1 currentAttempt done Last Error USER Pod failed No message received from kubernetes feaac56274c144d7f8a6 n0 0 terminated with exit code 1 Reason Error Message ted task dispatch execute ctx task def inputs output prefix File opt venv lib python3 9 site packages flytekit bin entrypoint py line 80 in dispatch execute logger debug f Starting dispatch execute for task def name AttributeError function object has no attribute name ```

Niels Bantilan · Answer

hi < Taeef Najib> would you mind filing a bug report for this issue We ve been discussing on this thread for some time and I think it s work creating an issue for this esp with < Eduardo Matus> s error as well Basically it should contain 1 a minimally reproducible code snippet 2 instructions on how to reproduce the error 3 description of expected behavior flyte bug point down

Slackbot · Answer

ladybug Create a new Flyte Bug issue <https github com flyteorg flyte issues new assignees= amp labels=bug 2Cuntriaged amp template=bug report yaml amp title= 5BBUG 5D+>

Samhita Alla · Answer

< Niels Bantilan> < Eduardo Matus> was able to resolve this issue by downgrading Flyte to 1 3 0

Samhita Alla · Answer

< Taeef Najib> can you test and confirm

Taeef Najib · Answer

< Niels Bantilan> I will Thanks < Samhita Alla> Of course I can test

Taeef Najib · Answer

< Samhita Alla> OK on v1 3 0 packaging step fails with this error ``` 12 44 AM ERROR pip s dependency resolver does not currently take into account all the packages that are installed This behaviour is the source of the following dependency conflicts grpcio status 1 51 3 requires protobuf gt =4 21 6 but you have protobuf 3 20 3 which is incompatible flytekit 1 3 0 requires pyarrow lt 11 0 0 gt =4 0 0 but you have pyarrow 11 0 0 which is incompatible flyteidl 1 3 10 requires protobuf lt 5 0 0 gt =4 21 1 but you have protobuf 3 20 3 which is incompatible awscli 1 27 85 requires botocore==1 29 85 but you have botocore 1 29 89 which is incompatible awscli 1 27 85 requires PyYAML lt 5 5 gt =3 10 but you have pyyaml 6 0 which is incompatible Successfully installed aiohttp 3 8 4 aiosignal 1 3 1 alembic 1 10 2 anyio 3 6 2 appdirs 1 4 4 asgiref 3 6 0 async timeout 4 0 2 attrs 22 2 0 backoff 2 2 1 bentoml 1 0 15 boto3 1 26 89 botocore 1 29 89 cattrs 22 2 0 circus 0 18 0 click option group 0 5 5 contextlib2 21 6 0 contourpy 1 0 7 cycler 0 11 0 databricks cli 0 17 5 deepmerge 1 1 0 entrypoints 0 4 exceptiongroup 1 1 1 flask 2 2 3 fonttools 4 39 0 frozenlist 1 3 3 fs 2 4 16 greenlet 2 0 2 gunicorn 20 1 0 h11 0 14 0 itsdangerous 2 1 2 kiwisolver 1 4 4 llvmlite 0 39 1 mako 1 2 4 markdown 3 4 1 markdown it py 2 2 0 matplotlib 3 7 1 mdurl 0 1 2 mlflow 2 2 1 multidict 6 0 4 numba 0 56 4 oauthlib 3 2 2 opentelemetry api 1 14 0 opentelemetry exporter otlp proto http 1 14 0 opentelemetry instrumentation 0 35b0 opentelemetry instrumentation aiohttp client 0 35b0 opentelemetry instrumentation asgi 0 35b0 opentelemetry proto 1 14 0 opentelemetry sdk 1 14 0 opentelemetry semantic conventions 0 35b0 opentelemetry util http 0 35b0 pathspec 0 11 0 pillow 9 4 0 pip requirements parser 32 0 1 prometheus client 0 16 0 protobuf 3 20 3 psutil 5 9 4 pyarrow 11 0 0 pygments 2 14 0 pyjwt 2 6 0 pynvml 11 5 0 pyparsing 3 0 9 python multipart 0 0 6 pyzmq 25 0 1 querystring parser 1 2 4 rich 13 3 2 schema 0 7 5 scikit learn 1 2 0 scipy 1 10 1 setuptools 67 6 0 shap 0 41 0 sidetrekutils 0 0 14 simple di 0 1 5 slicer 0 0 7 sniffio 1 3 0 sqlalchemy 2 0 6 sqlparse 0 4 3 starlette 0 26 1 tabulate 0 9 0 threadpoolctl 3 1 0 tornado 6 2 tqdm 4 65 0 urllib3 1 26 15 uvicorn 0 21 0 watchfiles 0 18 1 werkzeug 2 2 3 yarl 1 8 2 Traceback most recent call last File opt venv lib python3 10 site packages git init py line 89 in lt module gt refresh File opt venv lib python3 10 site packages git init py line 76 in refresh if not Git refresh path=path File opt venv lib python3 10 site packages git cmd py line 392 in refresh raise ImportError err ImportError Bad git executable The git executable must be specified in one of the following ways be included in your $PATH be set via $GIT PYTHON GIT EXECUTABLE explicitly set via git refresh All git commands will error until this is rectified This initial warning can be silenced or aggravated in the future by setting the $GIT PYTHON REFRESH environment variable Use one of the following values quiet|q|silence|s|none|n|0 for no warning or exception warn|w|warning|1 for a printed warning error|e|raise|r|2 for a raised exception Example export GIT PYTHON REFRESH=quiet The above exception was the direct cause of the following exception Traceback most recent call last File opt venv bin pyflyte line 5 in lt module gt from flytekit clis sdk in container pyflyte import main File opt venv lib python3 10 site packages flytekit clis sdk in container pyflyte py line 7 in lt module gt from flytekit clis sdk in container package import package File opt venv lib python3 10 site packages flytekit clis sdk in container package py line 13 in lt module gt from flytekit tools repo import NoSerializableEntitiesError serialize and package File opt venv lib python3 10 site packages flytekit tools repo py line 14 in lt module gt from flytekit remote import FlyteRemote File opt venv lib python3 10 site packages flytekit remote init py line 98 in lt module gt from flytekit remote remote import FlyteRemote File opt venv lib python3 10 site packages flytekit remote remote py line 22 in lt module gt from git import Repo File opt venv lib python3 10 site packages git init py line 91 in lt module gt raise ImportError Failed to initialize 0 format exc from exc ImportError Failed to initialize Bad git executable The git executable must be specified in one of the following ways be included in your $PATH be set via $GIT PYTHON GIT EXECUTABLE explicitly set via git refresh All git commands will error until this is rectified This initial warning can be silenced or aggravated in the future by setting the $GIT PYTHON REFRESH environment variable Use one of the following values quiet|q|silence|s|none|n|0 for no warning or exception warn|w|warning|1 for a printed warning error|e|raise|r|2 for a raised exception Example export GIT PYTHON REFRESH=quiet``` So it won t even deploy can t get to the execute step

Yee · Answer

try `1 3 4`

Eduardo Matus · Answer

< Taeef Najib> Create a new environment using python 3 8 3 9 with flyte force the installation with the version that Yee mentioned above

Taeef Najib · Answer

< Eduardo Matus> < Yee> Able to register package but same error on python 3 9 16 with flyte 1 3 4

Yee · Answer

sorry what error

Yee · Answer

git

Yee · Answer

or the other one

Taeef Najib · Answer

This one ```AttributeError function object has no attribute name ```

Yee · Answer

can i ask what the goal is

Yee · Answer

i get that the code is how it is

Yee · Answer

but why the desire to separate the task decorator call from the function

Yee · Answer

there s a fundamental thing happening here <https github com flyteorg flytekit blob 152080fa27c6d07edda9fcc2a602ca3ca9f4739a flytekit core python auto container py L287>

Taeef Najib · Answer

That way we can import functions from other files otherwise you have to put all your task and workflow code in the same file

Yee · Answer

i would have to check tomorrow but I believe it s done that way to support the other use case where the decoration happens together but then that task is imported as a whole unit into other python modules

Yee · Answer

but why not decorate first and then import

Yee · Answer

if you truly need access to the underlying function it is still there in the ` task function` property

Taeef Najib · Answer

OK so the problem is using the decorator this way then right

Yee · Answer

i think so

Yee · Answer

at least for now

Yee · Answer

python module loading is complicated and almost always a rabbit hole when i venture down it

Yee · Answer

and we do some possibly <https github com flyteorg flytekit blob 152080fa27c6d07edda9fcc2a602ca3ca9f4739a flytekit core tracker py L26|esoteric> things already that i think might be challenging to increase the flexibility of

Taeef Najib · Answer

Lol so true OK thank you so much for looking into this we ll try one of the other ways you mentioned I really appreciate your patience with all the back and forth

Yee · Answer

but leave the issue open we will look at it when we have a bit more time there s some other module loading clean up things we need to investigate as well

Yee · Answer

thank you

Taeef Najib · Answer

Will do thanks again