Taeef Najib
03/05/2023, 9:30 PMKNeighborsClassifier
It seems Flyte doesn't accept KNeighborsClassifier
as the type of my function. What would be the right type hint for this? I get the following error:
[4/4] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes.
[sd4h34hqd15wyc3wy32t-n0-3] terminated with exit code (137). Reason [OOMKilled]. Message:
{"asctime": "2023-03-05 20:24:43,219", "name": "flytekit", "levelname": "WARNING", "message": "Unsupported Type <class 'sklearn.neighbors._classification.KNeighborsClassifier'> found, Flyte will default to use PickleFile as the transport. Pickle can only be used to send objects between the exact same version of Python, and we strongly recommend to use python type that flyte support."}
2023/03/05 20:24:43 WARNING mlflow.utils.git_utils: Failed to import Git (the Git executable is probably not on your PATH), so Git SHA is not available. Error: Failed to initialize: Bad git executable.
The git executable must be specified in one of the following ways:
- be included in your $PATH
- be set via $GIT_PYTHON_GIT_EXECUTABLE
- explicitly set via git.refresh()
All git commands will error until this is rectified.
This initial warning can be silenced or aggravated in the future by setting the
$GIT_PYTHON_REFRESH environment variable. Use one of the following values:
- quiet|q|silence|s|none|n|0: for no warning or exception
- warn|w|warning|1: for a printed warning
- error|e|raise|r|2: for a raised exception
Example:
export GIT_PYTHON_REFRESH=quiet
{"asctime": "2023-03-05 20:24:43,424", "name": "flytekit", "levelname": "WARNING", "message": "Unsupported Type <class 'sklearn.neighbors._classification.KNeighborsClassifier'> found, Flyte will default to use PickleFile as the transport. Pickle can only be used to send objects between the exact same version of Python, and we strongly recommend to use python type that flyte support."}
.
Can you please point me to the right direction?Kevin Su
03/05/2023, 10:44 PMFlyte Literal
(protobuf message)
if the type is unrecognized by Flyte, flytekit will serialize it to pickle. you could register a custom type transformer yourself.
here is an exampleTaeef Najib
03/06/2023, 1:19 AMClassifierMixin
, BaseEstimator
or Any
or maybe None
?Samhita Alla
03/06/2023, 4:22 AMCan I create a custom class in my script and use it as a custom type?Yes, you can! You'll need to have the methods specified in the guide.
Was also wondering if I could simply useYou can. The error you're seeing is OOMKilled, right? In that case, please increase the memory you're allocating to your task. The unsupported one is a warning. You can either register a custom type (which is preferred) or leave it as is.,ClassifierMixin
orBaseEstimator
or maybeAny
?None
Niels Bantilan
03/06/2023, 6:31 PMKNeighborsClassifier
be picked up by the SklearnTypeTransformer
? https://github.com/flyteorg/flytekit/blob/master/flytekit/extras/sklearn/native.py#L72Taeef Najib
03/06/2023, 9:47 PMNiels Bantilan
03/06/2023, 11:53 PMTaeef Najib
03/07/2023, 5:31 PMNiels Bantilan
03/07/2023, 5:39 PMTaeef Najib
03/07/2023, 5:41 PMNiels Bantilan
03/07/2023, 5:45 PMKNeighborsClassifier
Taeef Najib
03/07/2023, 5:46 PMTraceback (most recent call last):
File "/opt/venv/bin/pyflyte", line 5, in <module>
from flytekit.clis.sdk_in_container.pyflyte import main
File "/opt/venv/lib/python3.10/site-packages/flytekit/__init__.py", line 202, in <module>
from flytekit.core.base_sql_task import SQLTask
File "/opt/venv/lib/python3.10/site-packages/flytekit/core/base_sql_task.py", line 4, in <module>
from flytekit.core.base_task import PythonTask, TaskMetadata
File "/opt/venv/lib/python3.10/site-packages/flytekit/core/base_task.py", line 27, in <module>
from flytekit.core.context_manager import (
File "/opt/venv/lib/python3.10/site-packages/flytekit/core/context_manager.py", line 30, in <module>
from flytekit.clients import friendly as friendly_client # noqa
File "/opt/venv/lib/python3.10/site-packages/flytekit/clients/friendly.py", line 4, in <module>
from flyteidl.admin import common_pb2 as _common_pb2
File "/opt/venv/lib/python3.10/site-packages/flyteidl/admin/common_pb2.py", line 5, in <module>
from google.protobuf.internal import builder as _builder
ImportError: cannot import name 'builder' from 'google.protobuf.internal' (/opt/venv/lib/python3.10/site-packages/google/protobuf/internal/__init__.py)
Niels Bantilan
03/08/2023, 1:53 PMTaeef Najib
03/08/2023, 2:51 PM[4/4] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes.
[snbh2jiispjmvrt5faat-n0-3] terminated with exit code (1). Reason [Error]. Message:
{"asctime": "2023-03-08 16:51:11,316", "name": "flytekit", "levelname": "WARNING", "message": "FlyteSchema is deprecated, use Structured Dataset instead."}
Traceback (most recent call last):
File "/opt/venv/bin/pyflyte-execute", line 8, in <module>
sys.exit(execute_task_cmd())
File "/opt/venv/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/opt/venv/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/opt/venv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/opt/venv/lib/python3.10/site-packages/flytekit/bin/entrypoint.py", line 476, in execute_task_cmd
_execute_task(
File "/opt/venv/lib/python3.10/site-packages/flytekit/exceptions/scopes.py", line 160, in system_entry_point
return wrapped(*args, **kwargs)
File "/opt/venv/lib/python3.10/site-packages/flytekit/bin/entrypoint.py", line 348, in _execute_task
_task_def = resolver_obj.load_task(loader_args=resolver_args)
File "/opt/venv/lib/python3.10/site-packages/flytekit/core/class_based_resolver.py", line 34, in load_task
return self.mapping[idx]
IndexError: list index out of range
.
@Niels BantilanNiels Bantilan
03/08/2023, 4:59 PMTaeef Najib
03/08/2023, 5:07 PMimport sklearn
import os
import sys
import typing
from flytekit import Resources, task, workflow
from main import Hyperparameters
from main import generate_dataset
from main import train_model
_wf_outputs=typing.NamedTuple("WfOutputs",train_model_0=sklearn.ensemble._forest.RandomForestClassifier)
@workflow
def test_classification_2(_wf_args:Hyperparameters)->_wf_outputs:
generate_dataset=task(requests=Resources(cpu="1"),limits=Resources(cpu="1"),retries=3)(generate_dataset)
train_model=task(requests=Resources(cpu="1"),limits=Resources(cpu="1"),retries=3)(train_model)
generate_dataset_o0_,generate_dataset_o1_,generate_dataset_o2_,generate_dataset_o3_=generate_dataset(hp=_wf_args)
train_model_o0_=train_model(hp=_wf_args,X_train=generate_dataset_o0_,y_train=generate_dataset_o2_)
return _wf_outputs(train_model_o0_)
Niels Bantilan
03/09/2023, 8:19 PMtrain_model_task
needs to be _wf_outputs
directly: you can’t do _wf_outputs(train_model_o0_)
because train_model_o0_
is a promise: you can’t operate on it like a regular python value
import sklearn
import os
import sys
import typing
from flytekit import Resources, task, workflow
from main import Hyperparameters
from main import generate_dataset
from main import train_model
generate_dataset_task = task(requests=Resources(cpu="1"),limits=Resources(cpu="1"),retries=3)(generate_dataset)
train_model_task = task(requests=Resources(cpu="1"),limits=Resources(cpu="1"),retries=3)(train_model)
_wf_outputs=typing.NamedTuple("WfOutputs",train_model_0=sklearn.ensemble._forest.RandomForestClassifier)
@workflow
def test_classification_2(_wf_args:Hyperparameters)->_wf_outputs:
generate_dataset_o0_,generate_dataset_o1_,generate_dataset_o2_,generate_dataset_o3_ = generate_dataset_task(hp=_wf_args)
train_model_o0_=train_model_task(hp=_wf_args,X_train=generate_dataset_o0_,y_train=generate_dataset_o2_)
return _wf_outputs(train_model_o0_)
Note that the output of generate_dataset
needs to be a tuple
or NamedTuple
for you to be able to unpack the outputs like generate_dataset_o0_,generate_dataset_o1_,generate_dataset_o2_,generate_dataset_o3_= generate_dataset_task(hp=_wf_args)
generate_dataset_task = task(requests=Resources(cpu="1"),limits=Resources(cpu="1"),retries=3)(generate_dataset)
will work, though it should in theory.@workflow
-decorated functions only supports certain operations. You can read more about it here: https://docs.flyte.org/projects/cookbook/en/latest/getting_started/tasks_and_workflows.html#workflowsTaeef Najib
03/10/2023, 1:58 PMindex out of range
error whereas it works fine in v1.2.7. Is something different in v1.4.1 causing the error? Workflow code is exactly same.
To confirm, I tried what you suggested (moving the task definitions outside the @workflow) but it throws this erros (in both v1.2.7 and v1.4.1)
Traceback (most recent call last):
File "/opt/venv/bin/pyflyte-execute", line 8, in <module>
sys.exit(execute_task_cmd())
File "/opt/venv/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/opt/venv/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/opt/venv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/opt/venv/lib/python3.10/site-packages/flytekit/bin/entrypoint.py", line 476, in execute_task_cmd
_execute_task(
File "/opt/venv/lib/python3.10/site-packages/flytekit/exceptions/scopes.py", line 160, in system_entry_point
return wrapped(*args, **kwargs)
File "/opt/venv/lib/python3.10/site-packages/flytekit/bin/entrypoint.py", line 354, in _execute_task
_handle_annotated_task(ctx, _task_def, inputs, output_prefix)
File "/opt/venv/lib/python3.10/site-packages/flytekit/bin/entrypoint.py", line 297, in _handle_annotated_task
_dispatch_execute(ctx, task_def, inputs, output_prefix)
File "/opt/venv/lib/python3.10/site-packages/flytekit/bin/entrypoint.py", line 80, in _dispatch_execute
logger.debug(f"Starting _dispatch_execute for {task_def.name}")
AttributeError: 'function' object has no attribute 'name'
Another thing I noticed when I made this change - in the flyte console task details I see this:
"pyflyte-execute"
"--inputs"
"{{.input}}"
"--output-prefix"
"{{.outputPrefix}}"
"--raw-output-data-prefix"
"{{.rawOutputDataPrefix}}"
"--checkpoint-path"
"{{.checkpointOutputPrefix}}"
"--prev-checkpoint"
"{{.prevCheckpointPrefix}}"
"--resolver"
"flytekit.core.python_auto_container.default_task_resolver"
"--"
"task-module"
"main"
"task-name"
"generate_dataset"
Instead of:
"pyflyte-execute"
"--inputs"
"{{.input}}"
"--output-prefix"
"{{.outputPrefix}}"
"--raw-output-data-prefix"
"{{.rawOutputDataPrefix}}"
"--checkpoint-path"
"{{.checkpointOutputPrefix}}"
"--prev-checkpoint"
"{{.prevCheckpointPrefix}}"
"--resolver"
"wf.wf_1.my_workflow_1"
"--"
"0"
Niels Bantilan
03/10/2023, 2:38 PMTaeef Najib
03/10/2023, 4:16 PMimport sklearn
import os
import sys
import typing
from flytekit import Resources, task, workflow
from main import Hyperparameters
from main import generate_dataset
from main import train_model
generate_dataset=task(requests=Resources(cpu="1"),limits=Resources(cpu="1"),retries=3)(generate_dataset)
train_model=task(requests=Resources(cpu="1"),limits=Resources(cpu="1"),retries=3)(train_model)
_wf_outputs=typing.NamedTuple("WfOutputs",train_model_0=sklearn.ensemble._forest.RandomForestClassifier)
@workflow
def my_workflow_3(_wf_args:Hyperparameters)->_wf_outputs:
generate_dataset_o0_,generate_dataset_o1_,generate_dataset_o2_,generate_dataset_o3_=generate_dataset(hp=_wf_args)
train_model_o0_=train_model(hp=_wf_args,X_train=generate_dataset_o0_,y_train=generate_dataset_o2_)
return _wf_outputs(train_model_o0_)
AttributeError: 'function' object has no attribute 'name'
and can’t figure out why.Niels Bantilan
03/10/2023, 4:35 PMgenerate_dataset_task=task(...)(generate_dataset)
train_model_task=task(...)(train_model)
generate_dataset_task
and train_model_task
in your workflow?Taeef Najib
03/10/2023, 5:03 PMimport sklearn
import os
import sys
import typing
from flytekit import Resources, task, workflow
from main import Hyperparameters
from main import generate_dataset
from main import train_model
generate_dataset_task=task(requests=Resources(cpu="1"),limits=Resources(cpu="1"),retries=3)(generate_dataset)
train_model_task=task(requests=Resources(cpu="1"),limits=Resources(cpu="1"),retries=3)(train_model)
_wf_outputs=typing.NamedTuple("WfOutputs",train_model_task_0=sklearn.ensemble._forest.RandomForestClassifier)
@workflow
def my_workflow_3(_wf_args:Hyperparameters)->_wf_outputs:
generate_dataset_task_o0_,generate_dataset_task_o1_,generate_dataset_task_o2_,generate_dataset_task_o3_=generate_dataset_task(hp=_wf_args)
train_model_task_o0_=train_model_task(hp=_wf_args,X_train=generate_dataset_task_o0_,y_train=generate_dataset_task_o2_)
return _wf_outputs(train_model_task_o0_)
Same resultNiels Bantilan
03/10/2023, 5:06 PMAttributeError: 'function' object has no attribute 'name'
or `
packages/flytekit/core/class_based_resolver.py", line 34, in load_task
return self.mapping[idx]
IndexError: list index out of range
or both?Taeef Najib
03/10/2023, 5:14 PMlist index out of range
error
But still fails with AttributeError: 'function' object has no attribute 'name'
error during executionNiels Bantilan
03/10/2023, 5:15 PMpyflyte run
? it would also be help if you can provide the contents of the main
module of you canFile "/opt/venv/lib/python3.10/site-packages/flytekit/bin/entrypoint.py", line 80, in _dispatch_execute
logger.debug(f"Starting _dispatch_execute for {task_def.name}")
And print out task_def
and it’s type
before line 80? I don’t think it’s an actual PythonTask
objectTaeef Najib
03/10/2023, 5:24 PM# Import necessary libraries
import typing
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
import numpy as np
from dataclasses import dataclass
from dataclasses_json import dataclass_json
@dataclass_json
@dataclass
class Hyperparameters(object):
n_samples: int = 1000
n_features: int = 20
n_informative: int = 15
n_classes: int = 2
test_size: float = 0.2
n_estimator: int = 100
# Generate the dataset
def generate_dataset(hp: Hyperparameters) -> typing.Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
X, y = make_classification(n_samples=hp.n_samples, n_features=hp.n_features, n_informative=hp.n_informative, n_classes=hp.n_classes)
return train_test_split(X, y, test_size=hp.test_size)
# Train a random forest classifier on the train data
def train_model(hp: Hyperparameters, X_train: np.ndarray, y_train: np.ndarray) -> RandomForestClassifier:
return RandomForestClassifier(hp.n_estimator).fit(X_train, y_train)
No this is a remote execution via flyte REST APINiels Bantilan
03/10/2023, 6:15 PMgenerate_dataset_task=task(requests=Resources(cpu="1"),limits=Resources(cpu="1"),retries=3)(generate_dataset)
It basically needs to be used as a decorator. Here’s a one-scripter that works:
# Import necessary libraries
import typing
import sklearn
from flytekit import Resources, task, workflow
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
import numpy as np
from dataclasses import dataclass
from dataclasses_json import dataclass_json
@dataclass_json
@dataclass
class Hyperparameters(object):
n_samples: int = 1000
n_features: int = 20
n_informative: int = 15
n_classes: int = 2
test_size: float = 0.2
n_estimator: int = 100
# Generate the dataset
@task(requests=Resources(cpu="1"),limits=Resources(cpu="1"),retries=3)
def generate_dataset(hp: Hyperparameters) -> typing.Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
X, y = make_classification(n_samples=hp.n_samples, n_features=hp.n_features, n_informative=hp.n_informative, n_classes=hp.n_classes)
return train_test_split(X, y, test_size=hp.test_size)
# Train a random forest classifier on the train data
@task(requests=Resources(cpu="1"),limits=Resources(cpu="1"),retries=3)
def train_model(hp: Hyperparameters, X_train: np.ndarray, y_train: np.ndarray) -> RandomForestClassifier:
return RandomForestClassifier(hp.n_estimator).fit(X_train, y_train)
_wf_outputs=typing.NamedTuple("WfOutputs",train_model_task_0=sklearn.ensemble._forest.RandomForestClassifier)
@workflow
def my_workflow_3(_wf_args:Hyperparameters)->_wf_outputs:
generate_dataset_task_o0_,generate_dataset_task_o1_,generate_dataset_task_o2_,generate_dataset_task_o3_=generate_dataset(hp=_wf_args)
train_model_task_o0_=train_model(hp=_wf_args,X_train=generate_dataset_task_o0_,y_train=generate_dataset_task_o2_)
return _wf_outputs(train_model_task_o0_)
if __name__ == "__main__":
print(my_workflow_3(_wf_args=Hyperparameters()))
@task
decoration directly where the functions are define in main
Taeef Najib
03/10/2023, 6:19 PM@workflow
?
The reason we need to do this is because we’re using flyte in an automated setup and we need to be able to apply the decorator to the imported functions
How would we do that?pyflyte run
works just fine with the code we currently have - doesn’t that mean flyte should work in a remote execution as well?Niels Bantilan
03/10/2023, 6:21 PMgenerate_dataset=task(requests=Resources(cpu="1"),limits=Resources(cpu="1"),retries=3)(generate_dataset)
train_model=task(requests=Resources(cpu="1"),limits=Resources(cpu="1"),retries=3)(train_model)
I got this to work on my end, can you try it?generate_dataset
as the task name, but it was a function in the module’s namespace, so re-defining it as a task makes it work.
I’m not sure why this wasn’t working earlier, perhaps we’ll get another error on your sideTaeef Najib
03/10/2023, 7:55 PMNiels Bantilan
03/10/2023, 8:14 PMFlyteRemote
? pyflyte run?)from other_module import fn
from flytekit import task, workflow
fn = task(...)(fn)
@workflow
def wf():
fn()
This will raise an error:
[4/4] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes.
[apdmwk4bc9m9nq27gfth-n0-3] terminated with exit code (1). Reason [Error]. Message:
bin/entrypoint.py", line 80, in _dispatch_execute
logger.debug(f"Starting _dispatch_execute for {task_def.name}")
AttributeError: 'function' object has no attribute 'name'
Traceback (most recent call last):
File "/usr/local/bin/pyflyte-fast-execute", line 8, in <module>
sys.exit(fast_execute_task_cmd())
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flytekit/bin/entrypoint.py", line 513, in fast_execute_task_cmd
subprocess.run(cmd, check=True)
File "/usr/local/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['pyflyte-execute', '--inputs', '<s3://flyte-development-data/metadata/propeller/flytesnacks-development-apdmwk4bc9m9nq27gfth/n0/data/inputs.pb>', '--output-prefix', '<s3://flyte-development-data/metadata/propeller/flytesnacks-development-apdmwk4bc9m9nq27gfth/n0/data/3>', '--raw-output-data-prefix', '<s3://flyte-development-data/data/bh/apdmwk4bc9m9nq27gfth-n0-3>', '--checkpoint-path', '<s3://flyte-development-data/data/bh/apdmwk4bc9m9nq27gfth-n0-3/_flytecheckpoints>', '--prev-checkpoint', '<s3://flyte-development-data/data/mi/apdmwk4bc9m9nq27gfth-n0-2/_flytecheckpoints>', '--dynamic-addl-distro', '<s3://flyte-development-data/flytesnacks/development/VB6EEUDVKPK3GOU5HSHIETCXVU======/fasta42e5e6df1c146b6ec0cf0dd67ed937b.tar.gz>', '--dynamic-dest-dir', '/root', '--resolver', 'flytekit.core.python_auto_container.default_task_resolver', '--', 'task-module', 'other_module', 'task-name', 'fn']' returned non-zero exit status 1.
.
other_module.fn
so it tries to grab the task from there, but it’s not a task… it’s a functionlogger.debug(f"Starting _dispatch_execute for {task_def.name}")
Yee
03/10/2023, 8:23 PMfn
in multiple places? can we rename?Taeef Najib
03/11/2023, 1:25 AM`pyflyte --pkgs wf package --output flyte-workflow-package.tgz --image <http://gcr.io/blah|gcr.io/blah> --force`
`flytectl register files --project valor-unicorn --domain development --archive flyte-workflow-package.tgz --config /root/flyte_config.yaml --version 0.0.16`
Yes, I can confirm that this code fails with the same error on 1.2.7 as well - 1.2.7 succeeds if I put the task defs inside the @workflow (edited)
But the same code runs fine locally via pyflyte run
on 1.4.1 so I’d say either the way we’re registering/running the workflow is somehow deprecated or this seems like a bug. I think it’s fair to assume that the code that runs fine via pyflyte run
should also run without issues remotely?
What would you suggest is the right way to write the flyte workflow if I had to import functions from other modules like this? How can I get this to work?Eduardo Matus
03/12/2023, 9:46 PM[1/1] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes.
[feaac56274c144d7f8a6-n0-0] terminated with exit code (1). Reason [Error]. Message:
ted_task
_dispatch_execute(ctx, task_def, inputs, output_prefix)
File "/opt/.venv/lib/python3.9/site-packages/flytekit/bin/entrypoint.py", line 80, in _dispatch_execute
logger.debug(f"Starting _dispatch_execute for {task_def.name}")
AttributeError: 'function' object has no attribute 'name'
Niels Bantilan
03/13/2023, 2:43 PMSlackbot
03/13/2023, 2:43 PMSamhita Alla
03/13/2023, 3:58 PMTaeef Najib
03/13/2023, 5:32 PM[12:44 AM] ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
grpcio-status 1.51.3 requires protobuf>=4.21.6, but you have protobuf 3.20.3 which is incompatible.
flytekit 1.3.0 requires pyarrow<11.0.0,>=4.0.0, but you have pyarrow 11.0.0 which is incompatible.
flyteidl 1.3.10 requires protobuf<5.0.0,>=4.21.1, but you have protobuf 3.20.3 which is incompatible.
awscli 1.27.85 requires botocore==1.29.85, but you have botocore 1.29.89 which is incompatible.
awscli 1.27.85 requires PyYAML<5.5,>=3.10, but you have pyyaml 6.0 which is incompatible.
Successfully installed aiohttp-3.8.4 aiosignal-1.3.1 alembic-1.10.2 anyio-3.6.2 appdirs-1.4.4 asgiref-3.6.0 async-timeout-4.0.2 attrs-22.2.0 backoff-2.2.1 bentoml-1.0.15 boto3-1.26.89 botocore-1.29.89 cattrs-22.2.0 circus-0.18.0 click-option-group-0.5.5 contextlib2-21.6.0 contourpy-1.0.7 cycler-0.11.0 databricks-cli-0.17.5 deepmerge-1.1.0 entrypoints-0.4 exceptiongroup-1.1.1 flask-2.2.3 fonttools-4.39.0 frozenlist-1.3.3 fs-2.4.16 greenlet-2.0.2 gunicorn-20.1.0 h11-0.14.0 itsdangerous-2.1.2 kiwisolver-1.4.4 llvmlite-0.39.1 mako-1.2.4 markdown-3.4.1 markdown-it-py-2.2.0 matplotlib-3.7.1 mdurl-0.1.2 mlflow-2.2.1 multidict-6.0.4 numba-0.56.4 oauthlib-3.2.2 opentelemetry-api-1.14.0 opentelemetry-exporter-otlp-proto-http-1.14.0 opentelemetry-instrumentation-0.35b0 opentelemetry-instrumentation-aiohttp-client-0.35b0 opentelemetry-instrumentation-asgi-0.35b0 opentelemetry-proto-1.14.0 opentelemetry-sdk-1.14.0 opentelemetry-semantic-conventions-0.35b0 opentelemetry-util-http-0.35b0 pathspec-0.11.0 pillow-9.4.0 pip-requirements-parser-32.0.1 prometheus-client-0.16.0 protobuf-3.20.3 psutil-5.9.4 pyarrow-11.0.0 pygments-2.14.0 pyjwt-2.6.0 pynvml-11.5.0 pyparsing-3.0.9 python-multipart-0.0.6 pyzmq-25.0.1 querystring-parser-1.2.4 rich-13.3.2 schema-0.7.5 scikit-learn-1.2.0 scipy-1.10.1 setuptools-67.6.0 shap-0.41.0 sidetrekutils-0.0.14 simple-di-0.1.5 slicer-0.0.7 sniffio-1.3.0 sqlalchemy-2.0.6 sqlparse-0.4.3 starlette-0.26.1 tabulate-0.9.0 threadpoolctl-3.1.0 tornado-6.2 tqdm-4.65.0 urllib3-1.26.15 uvicorn-0.21.0 watchfiles-0.18.1 werkzeug-2.2.3 yarl-1.8.2
Traceback (most recent call last):
File "/opt/venv/lib/python3.10/site-packages/git/__init__.py", line 89, in <module>
refresh()
File "/opt/venv/lib/python3.10/site-packages/git/__init__.py", line 76, in refresh
if not Git.refresh(path=path):
File "/opt/venv/lib/python3.10/site-packages/git/cmd.py", line 392, in refresh
raise ImportError(err)
ImportError: Bad git executable.
The git executable must be specified in one of the following ways:
- be included in your $PATH
- be set via $GIT_PYTHON_GIT_EXECUTABLE
- explicitly set via git.refresh()
All git commands will error until this is rectified.
This initial warning can be silenced or aggravated in the future by setting the
$GIT_PYTHON_REFRESH environment variable. Use one of the following values:
- quiet|q|silence|s|none|n|0: for no warning or exception
- warn|w|warning|1: for a printed warning
- error|e|raise|r|2: for a raised exception
Example:
export GIT_PYTHON_REFRESH=quiet
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/venv/bin/pyflyte", line 5, in <module>
from flytekit.clis.sdk_in_container.pyflyte import main
File "/opt/venv/lib/python3.10/site-packages/flytekit/clis/sdk_in_container/pyflyte.py", line 7, in <module>
from flytekit.clis.sdk_in_container.package import package
File "/opt/venv/lib/python3.10/site-packages/flytekit/clis/sdk_in_container/package.py", line 13, in <module>
from flytekit.tools.repo import NoSerializableEntitiesError, serialize_and_package
File "/opt/venv/lib/python3.10/site-packages/flytekit/tools/repo.py", line 14, in <module>
from flytekit.remote import FlyteRemote
File "/opt/venv/lib/python3.10/site-packages/flytekit/remote/__init__.py", line 98, in <module>
from flytekit.remote.remote import FlyteRemote
File "/opt/venv/lib/python3.10/site-packages/flytekit/remote/remote.py", line 22, in <module>
from git import Repo
File "/opt/venv/lib/python3.10/site-packages/git/__init__.py", line 91, in <module>
raise ImportError("Failed to initialize: {0}".format(exc)) from exc
ImportError: Failed to initialize: Bad git executable.
The git executable must be specified in one of the following ways:
- be included in your $PATH
- be set via $GIT_PYTHON_GIT_EXECUTABLE
- explicitly set via git.refresh()
All git commands will error until this is rectified.
This initial warning can be silenced or aggravated in the future by setting the
$GIT_PYTHON_REFRESH environment variable. Use one of the following values:
- quiet|q|silence|s|none|n|0: for no warning or exception
- warn|w|warning|1: for a printed warning
- error|e|raise|r|2: for a raised exception
Example:
export GIT_PYTHON_REFRESH=quiet
So it won’t even deploy - can’t get to the execute stepYee
03/13/2023, 8:27 PM1.3.4
Eduardo Matus
03/13/2023, 11:54 PMTaeef Najib
03/14/2023, 3:44 AMYee
03/14/2023, 4:00 AMTaeef Najib
03/14/2023, 4:07 AMAttributeError: 'function' object has no attribute 'name'
Yee
03/14/2023, 4:14 AMTaeef Najib
03/14/2023, 4:16 AMYee
03/14/2023, 4:16 AM.task_function
propertyTaeef Najib
03/14/2023, 4:21 AMYee
03/14/2023, 4:25 AMTaeef Najib
03/14/2023, 4:28 AMYee
03/14/2023, 4:28 AMTaeef Najib
03/14/2023, 4:29 AM