Thread
#announcements
    Samhita Alla

    Samhita Alla

    2 weeks ago
    @Prafulla Mahindrakar, could you help?
    p

    Prafulla Mahindrakar

    2 weeks ago
    @Sathish kumar Venkatesan are you able to see other non spark task logs in cloudwatch. Also for spark can you check this which requires to segregate user, system logs https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/k8s_spark/index.html#step-3-optionally-setup-visibility Would be good to see first if normal tasks logs are flowing to cloud watch correctly using your initial configuration . And also when you click the log link you need to be in the same aws account where logs are being published to.
    Sathish kumar Venkatesan

    Sathish kumar Venkatesan

    2 weeks ago
    @Prafulla Mahindrakar currently i am testing only spark. and aws account in same between flyte config and published account. is there any command like kubectle logs to check the clouwatch agent services logs at flyte side.
    p

    Prafulla Mahindrakar

    2 weeks ago
    Can you check this thread https://flyte-org.slack.com/archives/C01P3B761A6/p1656366983976689 You might be missing configuring the mechanism for pushing the logs and we seem to missing docs on it. cc @Samhita Alla
    Sathish kumar Venkatesan

    Sathish kumar Venkatesan

    2 weeks ago
    is fluent-bit mandatory for pushing logs from flyte to cloudwatch?
    Sathish kumar Venkatesan

    Sathish kumar Venkatesan

    2 weeks ago
    thanks for that. since fluent-bit nowhere mentioned in flyte document, my assumption was by default flyte should able to steam logs to cloudwatch.
    p

    Prafulla Mahindrakar

    2 weeks ago
    fair assumption with missing docs. Would you help with contributing this bit of docs once you have this setup in your environment. would really benefit the community.
    Sathish kumar Venkatesan

    Sathish kumar Venkatesan

    2 weeks ago
    @Prafulla Mahindrakar sure. once i make it work from my side
    Ketan (kumare3)

    Ketan (kumare3)

    2 weeks ago
    @Sathish kumar Venkatesan Flyte cannot talk about how you push logs. There are multiple options. I do not think it is fair for Flyte to document how to push the logs- this is completely independent
    The document should say, if you have logs in cloudwatch then