https://flyte.org logo
#ask-the-community
Title
# ask-the-community
f

Ferdinand von den Eichen

07/10/2023, 9:39 AM
Our inputs and outputs are always empty, why is this? The tasks themselves definitely have both:
k

Kevin Su

07/10/2023, 1:50 PM
Seems like a UI bug. are you able to share the code? which version of flyteconsole are you using?
j

Jan Fiedler

07/10/2023, 1:57 PM
What code do you mean. We are using flyteconsole version: v1.4.1
k

Kevin Su

07/10/2023, 1:58 PM
workflow code
j

Jan Fiedler

07/10/2023, 2:04 PM
It happens to all our workflow code. Even with the simplest one:
Copy code
from flytekit import workflow, task
import pandas as pd


@task
def load_sample_data(environment: str) -> pd.DataFrame:
    print(f"Loading Sample Data from {environment}")
    sample_data_path = f"<s3://flyte-demo>-{environment}-sample-data-bucket"

    sample_data = pd.DataFrame(
        {
            "Name": ["Broder", "Ferdi", "Jan", "Korbi", "Robert"],
            "Job": ["MLOPS", "MLOPS", "MLOPS", "ML", "SALES"],
        }
    )
    return sample_data


@task
def filter_for_job(sample_data: pd.DataFrame, job_filter: str) -> pd.DataFrame:
    sample_data_preprocessed = sample_data[sample_data["Job"] == job_filter]
    return sample_data_preprocessed


@task
def output_result(sample_data: pd.DataFrame):
    print(sample_data.head())


@workflow
def wf(environment: str = "dev"):
    job_filter = "MLOPS"
    data = load_sample_data(environment=environment)
    filtered_data = filter_for_job(sample_data=data, job_filter=job_filter)
    output_result(sample_data=filtered_data)
We are using the multi cluster setup btw. dont know if this causes problems
k

Kevin Su

07/10/2023, 2:07 PM
thanks, looking
it works for me.
We are using the multi cluster setup btw. dont know if this causes problems
did you see any error in the flyteconsole pod?
feel like flyteconsole doesn’t have access to download input / output data
f

Ferdinand von den Eichen

07/10/2023, 2:27 PM
These are the logs for the console service
Copy code
::ffff:122.175.45.58 - - [10/Jul/2023:14:25:19 +0000] "GET / HTTP/1.1" 200 1240 "-" "ELB-HealthChecker/2.0"
::ffff:122.175.23.11 - - [10/Jul/2023:14:25:19 +0000] "GET / HTTP/1.1" 200 1240 "-" "ELB-HealthChecker/2.0"
::ffff:122.175.88.7 - - [10/Jul/2023:14:25:34 +0000] "GET / HTTP/1.1" 200 1240 "-" "ELB-HealthChecker/2.0"
::ffff:122.175.45.58 - - [10/Jul/2023:14:25:34 +0000] "GET / HTTP/1.1" 200 1240 "-" "ELB-HealthChecker/2.0"
::ffff:122.175.23.11 - - [10/Jul/2023:14:25:34 +0000] "GET / HTTP/1.1" 200 1240 "-" "ELB-HealthChecker/2.0"
::ffff:122.175.88.7 - - [10/Jul/2023:14:25:49 +0000] "GET / HTTP/1.1" 200 1240 "-" "ELB-HealthChecker/2.0"
::ffff:122.175.45.58 - - [10/Jul/2023:14:25:49 +0000] "GET / HTTP/1.1" 200 1240 "-" "ELB-HealthChecker/2.0"
::ffff:122.175.23.11 - - [10/Jul/2023:14:25:49 +0000] "GET / HTTP/1.1" 200 1240 "-" "ELB-HealthChecker/2.0"
::ffff:122.175.88.7 - - [10/Jul/2023:14:26:04 +0000] "GET / HTTP/1.1" 200 1240 "-" "ELB-HealthChecker/2.0"
::ffff:122.175.45.58 - - [10/Jul/2023:14:26:04 +0000] "GET / HTTP/1.1" 200 1240 "-" "ELB-HealthChecker/2.0"
::ffff:122.175.23.11 - - [10/Jul/2023:14:26:04 +0000] "GET / HTTP/1.1" 200 1240 "-" "ELB-HealthChecker/2.0"
::ffff:122.175.88.7 - - [10/Jul/2023:14:26:19 +0000] "GET / HTTP/1.1" 200 1240 "-" "ELB-HealthChecker/2.0"
::ffff:122.175.45.58 - - [10/Jul/2023:14:26:19 +0000] "GET / HTTP/1.1" 200 1240 "-" "ELB-HealthChecker/2.0"
::ffff:122.175.23.11 - - [10/Jul/2023:14:26:19 +0000] "GET / HTTP/1.1" 200 1240 "-" "ELB-HealthChecker/2.0"
During this time frame we frequently clicked on the respective UI section…What role is the flyteconsole service using?
k

Kevin Su

07/10/2023, 2:32 PM
depends on your cluster setup, but it should have access to s3 bucket
j

Jan Fiedler

07/10/2023, 2:48 PM
We found the correct error logs for our problem:
Copy code
{
  "json": {
    "exec_id": "al4ncm8fx74ss8pjk4bf",
    "node": "n6"
  },
  "level": "error",
  "msg": "Failed to read from the raw store [<s3://supply-fc-prod-data-lake-cluster-bucket/metadata/propeller/energy-supply-forecast-production-al4ncm8fx74ss8pjk4bf/n6/data/inputs.pb>] Error: Conf container:customer-flyte-central-cluster-bucket != Passed Container:supply-fc-prod-data-lake-cluster-bucket. Dynamic loading is disabled: not found",
  "ts": "2023-07-10T14:37:06Z"
}
Looks like flyteadmin is expecting some files in his own cluster bucket not in the data plane bucket
k

Kevin Su

07/10/2023, 3:00 PM
j

Jan Fiedler

07/10/2023, 9:54 PM
Is there a way to control this variable via helm or how would I enable this?
k

Kevin Su

07/11/2023, 1:07 AM
update flyteadmin configmap
j

Jan Fiedler

07/11/2023, 7:00 AM
We got this working now 🙂 Thank you so much Kevin!
3 Views