Yash Panchwatkar
10/16/2022, 2:37 PMkarthikraj
10/16/2022, 8:37 PMFile "/home/kkanagar/kkr/kuberay_test.py", line 15, in <module>
def ray_task() -> list[int]:
TypeError: 'type' object is not subscriptable
Changing this list[int] to any is working fine in local but when trying to register to remote flyte.
Need your help, pleaseTaeef Najib
10/17/2022, 12:56 AMfrom flytekit.types.file import PythonPickledFile
and in the example:
https://docs.flyte.org/projects/cookbook/en/latest/auto/case_studies/ml_training/house_price_prediction/house_price_predictor.html I see
from flytekit.types.file import JoblibSerializedFile
These are used to output the model state. Couldn’t we use regular Python type hints for these? May I know why PythonPickedFile
and JoblibSerializedFile
were used in the examples?Nada Saiyed
10/17/2022, 3:23 PMexecute()
of a custom plugin, if i pass a templated value to container_image
.
E.g. is my task definition
@task(task_config=my_task_config, container_image="{{.image.img1.fqn }}:{{.image.img1.version}}"
I can see that the container_image
is set as a property and if I retrieve it, I see it’s value is still a templated string.
How can I get the actual rendered image uri for that task? (i.e. basically I want what this function returns)Laura Lin
10/17/2022, 6:04 PMfrom flytekit.sdk.types import Types
the Types.Generic
? This used to be the type for arbitrary dict/json objects but it doesn't seem to be here anymore?Laura Lin
10/17/2022, 8:55 PMpyflyte run
without the remote option. e.g. if I define a @task(container_image="XXXX"
, can I use this image locally when I run?karthikraj
10/17/2022, 10:30 PMpanic: rpc error: code = Unauthenticated desc = transport: per-RPC creds failed due to error: oauth2: cannot fetch token: 400 Bad Request
Response: {"errorCode":"invalid_client","errorSummary":"Invalid value for 'client_id' parameter.","errorLink":"invalid_client","errorId":"oae4BTFncguRFCNZUfFEpIFhA","errorCauses":[]}
goroutine 1 [running]:
main.main()
/go/src/github.com/flyteorg/flyteadmin/cmd/scheduler/main.go:12 +0x85
This is my sample configmap for this.
configmap:
auth:
appAuth:
authServerType: External
externalAuthServer:
baseUrl: <https://www.oktadev.com/oauth2/default>
thirdPartyConfig:
flyteClient:
clientId: "a1b2c3xxxxyz9N3"
redirectUri: <https://flytedev.myapp.com/callback>
scopes:
- offline
- all
userAuth:
openId:
baseUrl: <https://www.oktadev.com/oauth2/default>
scopes:
- profile
- openid
clientId: "a1b2c3xxxxyz9N3"
authorizedUris:
- <https://flytedev.myapp.com>
- <http://flyteadmin:80>
- <http://flyteadmin.flyte.svc.cluster.local:80>
cryptic
10/18/2022, 3:46 AMtf.tensor
. The tf.io.serialize_tensor
returns a string, should we store it in tf.train.Feature
?
and further put this into tf.train.Example
and use SerializeToString()
method to serialize the proto? Is there a need to store it on to the disk? i'm not sure if storing on disk is possible or notKetan (kumare3)
10/18/2022, 4:04 PMChris Hairfield
10/18/2022, 7:21 PMfast register
to develop in the sandbox
environment, and I’d like to configure it to use a storage backend other than awscli
. Is there any documentation for doing this?
We run exclusively in GCP and install gsutil
in each of our Flyte-ready Docker images. I’m hoping to avoid installing awscli
in our images just to support running in sandbox
.
Thanks!Martin Hwasser
10/19/2022, 7:57 AMFabio Grätz
10/19/2022, 8:57 AMflytectl get projects
, am redirected to the browser, and then retrieve the projects. pyflyte register ...
works as well. So far so good.
When I manually launch a registered workflow from the console, it doesn't start however.
k -n development describe <http://flyteworkflows.flyte.lyft.com|flyteworkflows.flyte.lyft.com> ah9ghn272q7v22dbsvvd
gives:
Status:
Failed Attempts: 9
Message: Workflow[] failed. ErrorRecordingError: failed to publish event, caused by: EventSinkError: Error sending event, caused by [rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"]
Phase: 0
Tasks:
resource_type:TASK project:"3dod" domain:"development" name:"project.workflows.workflow.deploy_model" version:"2"
Details about config in 🧵
A pointer in the right direction would be much appreciated.Tom Melendez
10/19/2022, 1:42 PMFelix Ruess
10/19/2022, 7:21 PMruntimeClassName
to nvidia
so that it can actually use the GPU?
I added requests=Resources(gpu="1")
but can I add the runtimeClassName?varsha Parthasarathy
10/19/2022, 9:46 PMFlyteFile
Laura Lin
10/19/2022, 10:10 PMLeiqing
10/20/2022, 2:25 PMpyflyte-fast-execute
gets invoked? Or is there any other recommended way of doing this?Luísa Braga Simões
10/20/2022, 2:47 PMAndrew Korzhuev
10/20/2022, 3:11 PMError: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp ...:81: i/o timeout"
panic: authentication error! Original Error: rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken, Auth Error: failed to initialized token source provider. Err: failed to fetch auth metadata. Error: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp ...:81: i/o timeout"
Could this be related to auth config? And if so where do I look to fix it?Frank Shen
10/20/2022, 10:16 PMkarthikraj
10/21/2022, 2:59 AMraise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INTERNAL
details = "failed to create workflow in propeller json: error calling MarshalJSON for type *v1alpha1.Inputs: Marshal called with nil"
debug_error_string = "{"created":"@1666318439.715331527","description":"Error received from peer ipv4:35.172.9.181:443","file":"src/core/lib/surface/call.cc","file_line":1074,"grpc_message":"failed to create workflow in propeller json: error calling MarshalJSON for type *v1alpha1.Inputs: Marshal called with nil","grpc_status":13}"
Sanjay Chouhan
10/21/2022, 9:51 AMRequested MEMORY default [6Gi] is greater than the limit [150Mi]. Please fix your configuration
where and how to configure the limit? I am using the AWS EKS setup.Martin Hwasser
10/21/2022, 12:29 PMflytekit.LaunchPlan.get_or_create
. However, when I try to register this archive with the launchplans, it fails on a particular one with the name: a-cst-old
, giving the error: Error: rpc error: code = InvalidArgument desc = missing name
It’s just this name and the fixed input (string) that’s changed, nothing else, if I prefix this name with something else like, it works.
And on a related note, is there a way to delete, not just archive, all launchplans?Louis DiNatale
10/21/2022, 3:29 PMLaura Lin
10/21/2022, 3:29 PMOnly my executions
filtering. Right now when I click on that, I get Request failed with status code 400 invalid value for filters
What needs to be done for this to work? Do I have to set up OAuth login for the UI?Hitarth Srivastava
10/21/2022, 4:38 PMFrank Shen
10/21/2022, 5:24 PMLast Error: USER::Spark Job Failed with Error: driver container failed with ExitCode: 127, Reason: ContainerCannotRun
I have the following in the requirements.txt file for spark related dependencies.
flytekit>=0.20.0
flytekitplugins-snowflake
flytekitplugins-spark
scikit-learn
xgboost
unionml
SHAP
pandas
apache-beam
joblib
pyyaml==6.0
boto3==1.9.207
aws-secretsmanager-caching==1.1.1.5
aws-requests-auth==0.4.3
Frank Shen
10/21/2022, 5:25 PMFrank Shen
10/21/2022, 5:57 PMRUN python3 -m venv ${VENV}
ENV PATH="${VENV}/bin:$PATH"
RUN pip3 install wheel
# Install Python dependencies
COPY k8s_spark/requirements.txt /root
RUN pip install -r /root/requirements.txt
RUN flytekit_install_spark3.sh
Frank Shen
10/21/2022, 5:57 PMFrank Shen
10/21/2022, 5:57 PM