aloof-painting-18735
06/09/2023, 9:43 AMaloof-painting-18735
06/09/2023, 12:14 PMaloof-painting-18735
06/09/2023, 12:14 PM$ kubectl create secret generic -n flyte flyte-secret-auth
aloof-painting-18735
06/09/2023, 12:14 PMfreezing-boots-56761
freezing-boots-56761
aloof-painting-18735
06/09/2023, 2:09 PMaloof-painting-18735
06/09/2023, 2:11 PMaloof-painting-18735
06/09/2023, 2:11 PMFLYTE_DATABRICKS_API_TOKEN
env variablefreezing-boots-56761
aloof-painting-18735
06/09/2023, 2:12 PMaloof-painting-18735
06/09/2023, 2:16 PMsandbox
cluster (started by flytectl demo start), I don't have helm chart release that could be upgraded.
So this last does not seem to be compatible with the Sandbox
cluster. Is that correct?freezing-boots-56761
freezing-boots-56761
aloof-painting-18735
06/09/2023, 2:21 PMfreezing-boots-56761
billowy-winter-86593
06/09/2023, 9:27 PMDatabricks
config: flytekitplugins/spark/task.py#LL44C7-L44C17. It might work for testing purposes before @glamorous-carpet-83516 gets a chance to weigh in 🙂billowy-winter-86593
06/09/2023, 9:28 PMaloof-painting-18735
06/12/2023, 8:39 AMaloof-painting-18735
06/12/2023, 1:02 PMdatabricks_instance
alsobillowy-winter-86593
06/12/2023, 1:03 PMbillowy-winter-86593
06/12/2023, 1:03 PMaloof-painting-18735
06/12/2023, 1:03 PMbillowy-winter-86593
06/12/2023, 1:06 PMaloof-painting-18735
06/12/2023, 1:07 PMaloof-painting-18735
06/13/2023, 6:09 PMDatabricks
config, but the run failed with:aloof-painting-18735
06/13/2023, 6:09 PM{"json":{"exec_id":"f5a4876209ef64132940","ns":"flyte-user-flytesnacks-development","res_ver":"5168558","routine":"worker-38","wf":"flytesnacks:development:dbx_example.my_databricks_job"},"level":"error","msg":"Error when trying to reconcile workflow. Error [failed at Node[n0]. RuntimeExecutionError: failed during plugin execution, caused by: failed to execute handle for plugin [databricks]: secrets not found - file [/etc/secrets/FLYTE_DATABRICKS_API_TOKEN], Env [FLYTE_SECRET_FLYTE_DATABRICKS_API_TOKEN]].
aloof-painting-18735
06/13/2023, 6:09 PMbillowy-winter-86593
06/13/2023, 6:12 PMglamorous-carpet-83516
06/13/2023, 6:14 PMFLYTE_SECRET_FLYTE_DATABRICKS_API_TOKEN
aloof-painting-18735
06/13/2023, 6:14 PMglamorous-carpet-83516
06/13/2023, 6:14 PMaloof-painting-18735
06/13/2023, 6:15 PMaloof-painting-18735
06/21/2023, 3:18 PMglamorous-carpet-83516
06/21/2023, 8:03 PMaloof-painting-18735
06/22/2023, 7:21 AMaloof-painting-18735
06/22/2023, 7:21 AMaloof-painting-18735
06/22/2023, 7:39 AMaloof-painting-18735
06/29/2023, 11:53 AMFlyte
to Databricks
, probably that would shed some light on the issue.
Is there an option for debug logging? Please note that I have already added the --verbose
flag to pyflyte
command, like this: pyflyte --verbose run -i <docker_registry>.<http://dkr.ecr.us-east-1.amazonaws.com/|dkr.ecr.us-east-1.amazonaws.com/><prefix>/<image>:<version> --remote --destination-dir . dbx_example_job_cluster.py my_databricks_job
👉 did not helpaloof-painting-18735
06/29/2023, 1:37 PMaloof-painting-18735
06/29/2023, 1:37 PMaloof-painting-18735
06/29/2023, 1:37 PMaloof-painting-18735
06/29/2023, 1:38 PM