https://flyte.org
Join Slack
Menu
Home
Archive
Hi! I’m wondering what the default service account for a task will be, or how to find this out? I’m ...
# flyte-on-gcp
s
Hi! I’m wondering what the default service account for a task will be, or how to find this out? I’m trying to run some `BigQueryTask`’s using the BigQuery plugin, but keep getting a permission error… I’ve given BigQuery permission to the flytepropeller and flyteworker service accounts, but to no avail. Anyone able to help?
a
hey Jake Tasks will use the 
default
KSA on their corresponding namespace unless you append 
---service-account
to your pyflyte command. In any case, that SA has to include an annotation that "connects" it with a GSA. The TF modules do this for the initial projects but if this is a new project (and new/different namespace) you should check that the KSA has the annotation 
kubectl describe sa <default> -n <project-domain>
s
hi David, I’m using the flytesnacks > development project, from the flyte-core tf deployment
a
ok, the corresponding GSA uses a custom role defined here and that one doesn't include permissions for BigQuery. I haven't used that integration so not sure in general what BQ needs here. I can see there are multiple predefined roles for BQ but not sure what would be the minimum
s
I’ve update the flyteworkers role as such:
Copy code
flyteworkers = [
      "storage.buckets.get",
      "storage.objects.create",
      "storage.objects.delete",
      "storage.objects.get",
      "storage.objects.list",
      "storage.objects.update",
      "bigquery.tables.create",
      "bigquery.tables.delete",
      "bigquery.tables.export",
      "bigquery.tables.list",
      "bigquery.tables.get",
      "bigquery.tables.updateData",
      "bigquery.routines.create",
      "bigquery.models.create",
      "bigquery.models.create",
      "bigquery.jobs.update",
      "bigquery.jobs.create",
      "bigquery.datasets.delete",
      "bigquery.datasets.create",
    ],
However, my 
BigQueryTask
fails with the following error:
Copy code
Access Denied: Project my_flyte_project: User does not have bigquery.jobs.create permission in project my_flyte_project.
I can see from the GCP UI that the 
flyteworker
role has the correct updated permissions
Update: I was able to progress past this error by giving 
flytepropeller
the 
bigquery.jobs.create
permission. Now I get the following error:
Copy code
Access Denied: Table my-flyte-project:my_dataset.my_table: User does not have permission to query table my-flyte-project.my_dataset.my_table, or perhaps it does not exist in location EU.
• I have confirmed that the table exists and the name is correct • I have confirmed the table exists in the 
EU
region • I have checked the dataset and table and can see that the 
flyteworkers
service account has the inherited permissions on both from the 
flyteworker
role
Even if I manually assign the role 
BigQuery Admin
to the 
flyteworker
service account, it still has the permission error
After further digging, it seems the request to BIgQuery are coming from the 
flytepropeller
service account
96 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by