Alex Bain
02/08/2022, 11:00 PMflyteconsole:v0.41.0
that in the console, there are no log links to subtasks or any UI to show the status of subtasks. Can I ask how soon you plan to land these features in flyteconsole and if there are GitHub issues or PRs that I can follow / contribute to?Ketan (kumare3)
Alex Bain
02/08/2022, 11:05 PMKetan (kumare3)
Dan Rammer (hamersaw)
02/08/2022, 11:40 PMJason Porter
02/09/2022, 12:25 AMAlex Bain
02/09/2022, 1:41 AMflyteconsole:v0.41.0
. Let me double check if I get the log links for subtasks...Ketan (kumare3)
Alex Bain
02/11/2022, 7:11 PMk8s-array
plugin tasks to use Stackdriver links, but I still get the k8s log links (like in the screenshot a couple of comments above) when I execute my map tasks.Dan Rammer (hamersaw)
02/11/2022, 7:26 PMAlex Bain
02/11/2022, 10:31 PMdeployment.yaml
has two ConfigMaps, the first is:
apiVersion: v1
kind: ConfigMap
metadata:
name: flyte-propeller-config
namespace: "prod"
data:
propeller: |-
propeller:
... (stuff here)
tasks:
max-plugin-phase-versions: 1000000
task-plugins:
enabled-plugins:
- container
- sidecar
- spark
- k8s-array
default-for-task-types:
container: container
sidecar: sidecar
spark: spark
container_array: k8s-array
k8s-array:
logs:
stackdriver-enabled: true
gcp-project: lyft-level5
stackdriver-logresourcename: aws_ec2_instance
^^^^ I added this k8s-array
block above, I sort of just took my best guess on declaring this here.logs
block there is the same block that we have in a second ConfigMap for the existing k8s task plugins (so this ConfigMap should be ok). It looks like:
apiVersion: v1
kind: ConfigMap
metadata:
name: flyte-plugin-config
namespace: "prod"
data:
plugins: |-
plugins:
enabled-plugins:
- container
- sidecar
- k8s-array
- spark
logs:
stackdriver-enabled: true
gcp-project: lyft-level5
stackdriver-logresourcename: aws_ec2_instance
Dan Rammer (hamersaw)
02/11/2022, 11:14 PMk8s-array
block should be defined in the plugin configuration instead of the propeller configuration. Also, I don't believe the enabled-plugins
block in the plugin configuration is necessary.apiVersion: v1
kind: ConfigMap
metadata:
name: flyte-propeller-config
namespace: "prod"
data:
propeller: |-
propeller:
... (stuff here)
tasks:
max-plugin-phase-versions: 1000000
task-plugins:
enabled-plugins:
- container
- sidecar
- spark
- k8s-array
default-for-task-types:
container: container
sidecar: sidecar
spark: spark
container_array: k8s-array
apiVersion: v1
kind: ConfigMap
metadata:
name: flyte-plugin-config
namespace: "prod"
data:
plugins: |-
plugins:
k8s-array:
logs:
stackdriver-enabled: true
gcp-project: lyft-level5
stackdriver-logresourcename: aws_ec2_instance
logs:
stackdriver-enabled: true
gcp-project: lyft-level5
stackdriver-logresourcename: aws_ec2_instance