quaint-diamond-37493
11/07/2022, 1:30 PMhallowed-mouse-14616
11/07/2022, 3:07 PMhallowed-mouse-14616
11/07/2022, 3:10 PMhallowed-mouse-14616
11/07/2022, 3:32 PMquaint-diamond-37493
11/07/2022, 3:56 PMquaint-diamond-37493
11/07/2022, 4:02 PM/etc
hallowed-mouse-14616
11/07/2022, 4:03 PM@task
decorator and ContainerTask
use the same backend plugin - adding the functionality you request should be VERY similar for both, really just a flytekit side update.hallowed-mouse-14616
11/07/2022, 4:04 PMquaint-diamond-37493
11/07/2022, 4:10 PMquaint-diamond-37493
11/07/2022, 4:12 PMquaint-diamond-37493
11/07/2022, 4:13 PMquaint-diamond-37493
11/07/2022, 4:14 PMhallowed-mouse-14616
11/07/2022, 4:17 PMquaint-diamond-37493
11/07/2022, 4:20 PMquaint-diamond-37493
11/07/2022, 4:21 PMhallowed-mouse-14616
11/07/2022, 4:23 PMhallowed-mouse-14616
11/07/2022, 4:24 PMquaint-diamond-37493
11/07/2022, 4:28 PMquaint-diamond-37493
11/07/2022, 4:30 PMenv_from
and using a projected volume should also work, I can also try to create minimal examples for thosehallowed-mouse-14616
11/07/2022, 4:34 PMquaint-diamond-37493
11/07/2022, 4:35 PMquaint-diamond-37493
11/07/2022, 4:37 PMquaint-diamond-37493
11/09/2022, 8:12 PMhigh-accountant-32689
11/09/2022, 9:10 PMquaint-diamond-37493
11/09/2022, 9:12 PMhigh-accountant-32689
11/09/2022, 9:12 PMquaint-diamond-37493
11/10/2022, 6:33 PMhigh-accountant-32689
11/10/2022, 8:53 PMquaint-diamond-37493
11/10/2022, 9:32 PMelegant-australia-91422
11/13/2022, 11:02 PMnum_threads
in LightGBM (o/w LGBM will incorrectly use the number of cpus on the node to set its internal thread count, as opposed to the cpu allocation for that particular pod). In this particular case, we can likely work around this by setting a Kyverno policy to add those env vars to each pod created in one of the *-development, *-staging, *-production
namespaces, but in general big +1 to addressing this.elegant-australia-91422
11/14/2022, 11:34 PMPodFunctionTask
task & adding the appropriate fix was simpler to unblock short-term.
This was the simplest fix for our usage, but heavy caveat that I'm not sure if this breaks other cases/fully captures all the intended behavior:
container.env = [V1EnvVar(name=key, value=val) for key, val in sdk_default_container.env.items()] + container.env
quaint-diamond-37493
11/15/2022, 5:32 PMhigh-accountant-32689
11/16/2022, 6:42 PMquaint-diamond-37493
11/16/2022, 8:07 PMtask_config
in container_task.py, but that is not enough apparently....high-accountant-32689
11/16/2022, 8:52 PMquaint-diamond-37493
11/16/2022, 9:07 PMquaint-diamond-37493
11/16/2022, 9:08 PMquaint-diamond-37493
11/16/2022, 9:10 PMgrpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INTERNAL
details = "json: cannot unmarshal object into Go struct field Container.containers.envFrom of type []v1.EnvFromSource"
debug_error_string = "{"created":"@1668631993.197113078","description":"Error received from peer ipv4:10.0.0.101:443","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"json: cannot unmarshal object into Go struct field Container.containers.envFrom of type []v1.EnvFromSource","grpc_status":13}"
high-accountant-32689
11/16/2022, 10:52 PMquaint-diamond-37493
11/16/2022, 10:55 PMelegant-australia-91422
11/16/2022, 10:57 PMquaint-diamond-37493
11/16/2022, 11:03 PM@task
decorator and passing pod spec via task_config it seems only some things can be passed/serialized... in my case I would need to mount the secret as file as projected volume to /etc/my_file_name...high-accountant-32689
11/16/2022, 11:10 PMenv_from
should take a list of V1EnvFromSource
quaint-diamond-37493
11/16/2022, 11:16 PMquaint-diamond-37493
11/19/2022, 6:52 PMget_k8s_pod
doesn't seem the way to go here...