https://flyte.org logo
#flyte-on-gcp
Title
# flyte-on-gcp
j

Jake Dodd

01/25/2024, 6:22 PM
Hi! I’m wondering what the default service account for a task will be, or how to find this out? I’m trying to run some `BigQueryTask`’s using the BigQuery plugin, but keep getting a permission error… I’ve given BigQuery permission to the flytepropeller and flyteworker service accounts, but to no avail. Anyone able to help?
d

David Espejo (he/him)

01/25/2024, 6:43 PM
hey Jake Tasks will use the
default
KSA on their corresponding namespace unless you append
---service-account
to your pyflyte command. In any case, that SA has to include an annotation that "connects" it with a GSA. The TF modules do this for the initial projects but if this is a new project (and new/different namespace) you should check that the KSA has the annotation
kubectl describe sa <default> -n <project-domain>
j

Jake Dodd

01/25/2024, 7:07 PM
hi David, I’m using the flytesnacks > development project, from the flyte-core tf deployment
d

David Espejo (he/him)

01/25/2024, 9:07 PM
ok, the corresponding GSA uses a custom role defined here and that one doesn't include permissions for BigQuery. I haven't used that integration so not sure in general what BQ needs here. I can see there are multiple predefined roles for BQ but not sure what would be the minimum
j

Jake Dodd

01/26/2024, 9:34 AM
I’ve update the flyteworkers role as such:
Copy code
flyteworkers = [
      "storage.buckets.get",
      "storage.objects.create",
      "storage.objects.delete",
      "storage.objects.get",
      "storage.objects.list",
      "storage.objects.update",
      "bigquery.tables.create",
      "bigquery.tables.delete",
      "bigquery.tables.export",
      "bigquery.tables.list",
      "bigquery.tables.get",
      "bigquery.tables.updateData",
      "bigquery.routines.create",
      "bigquery.models.create",
      "bigquery.models.create",
      "bigquery.jobs.update",
      "bigquery.jobs.create",
      "bigquery.datasets.delete",
      "bigquery.datasets.create",
    ],
However, my
BigQueryTask
fails with the following error:
Copy code
Access Denied: Project my_flyte_project: User does not have bigquery.jobs.create permission in project my_flyte_project.
I can see from the GCP UI that the
flyteworker
role has the correct updated permissions
Update: I was able to progress past this error by giving
flytepropeller
the
bigquery.jobs.create
permission. Now I get the following error:
Copy code
Access Denied: Table my-flyte-project:my_dataset.my_table: User does not have permission to query table my-flyte-project.my_dataset.my_table, or perhaps it does not exist in location EU.
• I have confirmed that the table exists and the name is correct • I have confirmed the table exists in the
EU
region • I have checked the dataset and table and can see that the
flyteworkers
service account has the inherited permissions on both from the
flyteworker
role
Even if I manually assign the role
BigQuery Admin
to the
flyteworker
service account, it still has the permission error
After further digging, it seems the request to BIgQuery are coming from the
flytepropeller
service account