<#6440 [BUG] Flyte Deck doesn't appear for Spark (...
# flyte-github
c
#6440 [BUG] Flyte Deck doesn't appear for Spark (Databricks) tasks Issue created by nsarang ### Describe the bug Hi, I'm testing a simple code to visualize HTML output using Flyte Deck. Running the task using a K8s backend successfully produces a tab in Flyte Deck. However, running the same task code with Databricks (Spark plugin) doesn't make the Flyte Deck tab appear, even though the task completes with no errors. ### Expected behavior Flyte Deck should appear for Databricks (Spark) tasks, similar to how it does for K8s tasks. ### Additional context to reproduce
Copy code
flytekit==1.11.0
flytekitplugins-deck-standard==1.14.4
flytekitplugins-spark==1.12.0
# Databricks task import pandas as pd import flytekit from flytekit import task from flytekitplugins.spark import Databricks @task( container_image="{{.image.databricks}}", task_config=Databricks( databricks_conf=env_conf["databricks"], spark_conf=env_conf["spark"]["setup"], ), enable_deck=True, environment=RUNTIME_ENV_VARS, ) def test_db(): df = pd.DataFrame({"x": [1, 2, 3]}) flytekit.Deck("preview", df.to_html()) return df # k8s task @task( container_image="{{.image.kube}}", task_config=MlPlatformK8sConfig(job_type=MlPlatformJobType.CUSTOM), enable_deck=True, environment=RUNTIME_ENV_VARS, ) def test_kube(): df = pd.DataFrame({"x": [1, 2, 3]}) flytekit.Deck("preview", df.to_html()) return df ### Screenshots Databricks output - No deck [Image](https://private-user-images.githubusercontent.com/25885841/440703939-fb1c5095-365c-489d-af1b-282d68a7654c.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTUyMTQ1NzAsIm5iZiI6MTc1NTIxNDI3MCwicGF0aCI6Ii8yNTg4NTg0MS80NDA3MDM5MzktZmIxYzUwOTUtMzY1Yy00ODlkLWFmMWItMjgyZDY4YTc2NTRjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA4MTQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwODE0VDIzMzExMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWY3NGFlNjQ4YjBlYmI1NzE0NzFlOGZjYzE5ZDc5ZGIxMDRkOWU0NTUxNjhkMGJlZjJkYTdkYzE2MTY0NmZiZGEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.l4DjSiVFlNah35BxI7SEmttyT3zBVZVTiF_tX2DU3ys) K8s output [Image](https://private-user-images.githubusercontent.com/25885841/440703940-81c87c50-5d47-4ebe-a1b0-ea5ee9a03544.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTUyMTQ1NzAsIm5iZiI6MTc1NTIxNDI3MCwicGF0aCI6Ii8yNTg4NTg0MS80NDA3MDM5NDAtODFjODdjNTAtNWQ0Ny00ZWJlLWExYjAtZWE1ZWU5YTAzNTQ0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA4MTQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwODE0VDIzMzExMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTRkM2JkNzA3NjIyOTBmODQ0MzFmMjFlYWQyOGEzNzg4ZjZlNDQ1NDQ5Y2FkZmM2NTE1YmUxOWJhMGM2MmNmYTMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.v8Focgfi0VNFUijPuWJQ8IF4gL1HYGUGXOcKFABIfXw) ### Are you sure this issue hasn't been raised already? • Yes ### Have you read the Code of Conduct? • Yes flyteorg/flyte