https://flyte.org logo
Powered by
j

Jay Ganbat

03/25/2022, 8:45 PM
hi flytekit šŸ˜„, is there anyway i can suppress flytekit logger INFO part, im using FlyteRemote to launch workflow/task but everytime im getting bunch of this INFO message
Copy code
2022-03-25 20:39:40,203 [INFO] Found existing task template for resource_type: TASK
i want to show other INFO logs from my script but want to ignore FlyteRemote INFO messages, is there a way to do this? Also it looks like this is coming from this code
Copy code
remote_logger.debug(f"Found existing task template for {task.id}, will not retrieve from Admin")
it says debug but somehow tagged as INFO type logging
y

Yee

03/25/2022, 8:58 PM
FLYTE_SDK_LOGGING_LEVEL_REMOTE=40
let me know if that works.
i added this a while backā€¦ didnā€™t really expect anyone to actually use it
j

Jay Ganbat

03/25/2022, 8:59 PM
so the tricky part is i want flytekit INFO during task execution but not during launching,
so im actually running from a venv, i set this like
Copy code
export FLYTE_SDK_LOGGING_LEVEL=40
but im still seeing the logging INFO
y

Yee

03/25/2022, 9:00 PM
i think the remote logger is separate from the other loggers though
_REMOTE
j

Jay Ganbat

03/25/2022, 9:00 PM
ohhhhhh
y

Yee

03/25/2022, 9:00 PM
but that still might not work, let me know
j

Jay Ganbat

03/25/2022, 9:00 PM
yeah not working either šŸ˜ž
y

Yee

03/25/2022, 9:00 PM
logging is complicatedā€¦
one sec
j

Jay Ganbat

03/25/2022, 9:01 PM
yeah šŸ˜… tell me about it haha
y

Yee

03/25/2022, 9:06 PM
so the root logger is set to debug, the channel is set to that 
FLYTE_SDK_LOGGING_LEVEL
env var. that should workā€¦ youā€™re saying that if you 
export FLYTE_SDK_LOGGING_LEVEL=40
you get info lines still? can you copy paste a few more (redacted) log lines? or is that the only one?
j

Jay Ganbat

03/25/2022, 9:07 PM
yeah same still seeing
yeah here is the full line
Copy code
2022-03-25 21:07:02,739 [INFO] Found existing task template for resource_type: TASK
project: "balrog"
domain: "development"
name: "balrog_pipeline.workflows.utility_workflows.log_process_flowcell_exec_id_task"
version: "fix-test-logging-6f52a81c-7248eb2451c6"
, will not retrieve from Admin
2022-03-25 21:07:02,740 [INFO] Found existing task template for resource_type: TASK
project: "balrog"
domain: "development"
name: "balrog_pipeline.workflows.process_flowcell_workflow.process_samples_task"
version: "fix-test-logging-6f52a81c-7248eb2451c6"
, will not retrieve from Admin
2022-03-25 21:07:02,747 [INFO] Found existing task template for resource_type: TASK
project: "balrog"
domain: "development"
name: "balrog_pipeline.workflows.utility_workflows.generate_metadata_csv_task"
version: "fix-test-logging-6f52a81c-7248eb2451c6"
, will not retrieve from Admin
2022-03-25 21:07:02,748 [INFO] Found existing task template for resource_type: TASK
project: "balrog"
domain: "development"
name: "balrog_pipeline.workflows.process_flowcell_workflow.compute_batch_qc_task"
version: "fix-test-logging-6f52a81c-7248eb2451c6"
, will not retrieve from Admin
2022-03-25 21:07:02,751 [INFO] Found existing task template for resource_type: TASK
project: "balrog"
domain: "development"
name: "balrog_pipeline.workflows.bcl2bam.extract_contigs_tasks.aggregate_extract_contigs_task"
version: "fix-test-logging-6f52a81c-7248eb2451c6"
, will not retrieve from Admin
2022-03-25 21:07:02,758 [INFO] EXECUTION_NAME=run-kzyf
y

Yee

03/25/2022, 9:15 PM
just sanity check can you pip show flytekit?
j

Jay Ganbat

03/25/2022, 9:18 PM
Copy code
% pip show flytekit
WARNING: Ignoring invalid distribution -rpcio (/home/jganbat/.virtualenvs/balrog/lib/python3.9/site-packages)
Name: flytekit
Version: 0.30.3
Summary: Flyte SDK for Python
Home-page: <https://github.com/flyteorg/flytekit>
Author:
Author-email:
License: apache2
Location: /home/jganbat/.virtualenvs/balrog/lib/python3.9/site-packages
Requires: checksumdir, click, cloudpickle, cookiecutter, croniter, dataclasses-json, deprecated, diskcache, docker-image-py, docstring-parser, flyteidl, grpcio, keyring, marshmallow-jsonschema, natsort, pandas, protobuf, pyarrow, python-dateutil, python-json-logger, pytimeparse, pytz, requests, responses, retry, sortedcontainers, statsd, typing-extensions, urllib3, wheel, wrapt
Required-by: flytekitplugins-pod
y

Yee

03/25/2022, 9:19 PM
k at least one thing solved
j

Jay Ganbat

03/25/2022, 9:20 PM
ohh right, my IDE flytekit is using old version haha osrry about that
y

Yee

03/25/2022, 9:20 PM
oh thatā€™s okay
oh
also that just uses logging.
yeah
can you bump
sorry i just saw the ā€œinfoā€ bit, missed the ā€œ_loggingā€ bit
j

Jay Ganbat

03/25/2022, 9:25 PM
do you think if there is any breaking change between 0.30.3 => 0.32
y

Yee

03/25/2022, 9:25 PM
oh donā€™t do 0.32
just 0.31
0.32 is not ready yet
definitely bugs weā€™re ironing out.
j

Jay Ganbat

03/25/2022, 9:25 PM
gothca, and for 0.31 it should be fine right šŸ˜…
ohh just read the release log, ok gonna need to check flyte backend repos, thank you for the help šŸ˜„
2 Views
Powered by