https://flyte.org
Join Slack
Hi! Question on Logging. I'm running on GCP (configured using Opta + some customizing). I noticed I ...
# flyte-support
a
Hi! Question on Logging. I'm running on GCP (configured using Opta + some customizing). I noticed I am only getting logs from 
print
statements in my tasks, but not 
<http://logging.info|logging.info>
. I was wondering what's needed to set up logging for GCP (GKE). I found this document, but wasn't sure what settings would be appropriate for GCP. FIgured I'd check here first before diving too deep. https://docs.flyte.org/projects/cookbook/en/latest/auto/deployment/configure_logging_links.html#sphx-glr-auto-deployment-configure-logging-links-py Thank you!
đź‘€ 1
f
logging.info logs probably needs some log level to be set - cc @thankful-minister-83577
t
@able-ice-99180 do you have a sec to talk about this?
a
@thankful-minister-83577 were you thinking voice or slack message? I can slack but can't do voice right now.
@freezing-airport-6809 you were correct, thank you! It appears I forgot to set up the logger. Specifically, I needed to set up the logger from somewhere inside the workflow execution, i.e. inside the 
@workflow
function is good for me.
Copy code
@workflow
def my_wf(a: int, b: str) -> Tuple[int, str]:
    logging.basicConfig(
        format="%(asctime)s %(levelname)-8s %(module)s - %(funcName)s: %(message)s",
        level=<http://logging.INFO|logging.INFO>,
        datefmt="%Y-%m-%d %H:%M:%S"
    )
Untitled.py
The screenshot shows it showing up in Google Cloud Logging (see "Hello from t1: B")
f
Workflow is not executed at runtime
a
@freezing-airport-6809 Would that imply that the code addition of 
logging.basicConfig(...)
should not have solved the issue then?
f
No so it did, just that it was almost like luckily the same module was loaded and workflows are module loading time constructs
You can add it to some init module?
a
Good call. I'll do that. đź‘Ť
t
we need to clean up our logging docs
a
Just added to init.py and that worked.
t
yeah it will
but you shouldn’t need to do that
there’s a fair number of loggers already configured in flytekit.
and here should be one attached to flytekit.current_context().logger
by default yeah, the python logging module will not be at the info level.
and to set the level for all the loggers you can use the 
FLYTE_SDK_LOGGING_LEVEL
env var. just set that to 10 and you should get everything including debug
i thought your question was more on how to get the link to the gcp logging page hooked up properly so that when you click on the log link, the url has the right filters to show your pod’s logs
that will take a backend change to config
a
which takes a backend change to config? the gcp logging page" Or default value for 
FLYTE_SDK_LOGGING_LEVEL
?
o
I still don’t understand. How can we use the flytekit provided logging tools? I’m trying to do: 
from flytekit.loggers import logger as flyte_logger, user_space_logger
and using it like a standard python logger. It doesn’t work, and I have to resort to print statements everywhere in my code.
f
Why not simply import logging?
o
So I solved this when I realized that my original logging wasn’t setting the 
BasicConfig(force=True)
166 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by