@Prafulla Mahindrakar, could you help?

@Sathish kumar Venkatesan are you able to see other non spark task logs in cloudwatch. Also for spark can you check this which requires to segregate user, system logs https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/k8s_spark/index.html#step-3-optionally-setup-visibility Would be good to see first if normal tasks logs are flowing to cloud watch correctly using your initial configuration . And also when you click the log link you need to be in the same aws account where logs are being published to.

@Prafulla Mahindrakar currently i am testing only spark. and aws account in same between flyte config and published account. is there any command like kubectle logs to check the clouwatch agent services logs at flyte side.

Can you check this thread https://flyte-org.slack.com/archives/C01P3B761A6/p1656366983976689 You might be missing configuring the mechanism for pushing the logs and we seem to missing docs on it. cc @Samhita Alla

is fluent-bit mandatory for pushing logs from flyte to cloudwatch?

Amazon recommends using that https://aws.amazon.com/premiumsupport/knowledge-center/cloudwatch-stream-container-logs-eks/ for streaming logs from eks containers https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-EKS-logs.html

thanks for that. since fluent-bit nowhere mentioned in flyte document, my assumption was by default flyte should able to steam logs to cloudwatch.

fair assumption with missing docs. Would you help with contributing this bit of docs once you have this setup in your environment. would really benefit the community.

@Prafulla Mahindrakar sure. once i make it work from my side

@Sathish kumar Venkatesan Flyte cannot talk about how you push logs. There are multiple options. I do not think it is fair for Flyte to document how to push the logs- this is completely independent