Hi, I am getting this below error when registering...
# flyte-support
f
Hi, I am getting this below error when registering the workflows in one of our flyte application. It was working fine few days back but now it fails to register one of them with 403 error. Also tried to add a dummy task instead of the failing one, even that fails to register. Its hard to understand why it gives a 403.
Copy code
------------------------------------------------------------------------------------------------- --------- ----------------------------------------------------- 
| NAME                                                                                            | STATUS  | ADDITIONAL INFO                                     |
 ------------------------------------------------------------------------------------------------- --------- ----------------------------------------------------- 
| /tmp/register1703962791/00_pipeline.tasks.config_task.load_task_1_exec_config_1.pb    | Success | Successfully registered file                        |
 ------------------------------------------------------------------------------------------------- --------- ----------------------------------------------------- 
| /tmp/register1703962791/01_pipeline.tasks.config_task.load_task2_exec_config_1.pb | Failed  | Error registering file due to rpc error: code =     |
|                                                                                                 |         | PermissionDenied desc = unexpected HTTP status code |
|                                                                                                 |         | received from server: 403 (Forbidden)               |
 ------------------------------------------------------------------------------------------------- --------- -----------------------------------------------------
I get this problem when we deploy from github actions but I am able to deploy from my local machine without any issue.
f
Hmm this is interesting
We would love to reproduce any example would be great
f
Its a big application, but I can give you a sample. Not sure if this helps
Copy code
@workflow
def p_t_workflow() -> None:
    config: ExecConfigTakeouts = load_takeouts_exec_config()
    privacy_takeouts_task(config=config)

@task(retries=3, timeout=timedelta(hours=1))
def load_takeouts_exec_config() -> ExecConfigTakeouts:
    <http://logger.info|logger.info>("Loading configuration for takeouts")
    return ExecConfigLoader().takeouts()

@dataclass_json
@dataclass
class ExecConfigTakeouts:
    snowflake_config: SnowflakeConfig
    snowflake_tables: SnowflakeTables
    p_b_config: PBConfig
    sqs_config: SqsConfig

@staticmethod
    def takeouts() -> ExecConfigTakeouts:

        return ExecConfigTakeouts(
            snowflake_config=SnowflakeConfig(**CONFIG["snowflake"]["config"]),
            snowflake_tables=SnowflakeTables(
                user_demographics=TableConfig(
                    **CONFIG["snowflake"]["tables"]["demographics"]
                ),
                user_interest_predictions=TableConfig(
                    **CONFIG["snowflake"]["tables"]["interest_predictions"]
                ),
            ),
            pb_config=PBConfig(
                **CONFIG["pb_config"]
            ),
            sqs_config=SqsConfig(**CONFIG["sqs"]["takeout"]),
        )
The problem is that all these tasks were previously registered successfully and suddenly it started failing on this
/tmp/register1703962791/01_pipeline.tasks.config_task.load_task2_exec_config_1.pb
and I even tried to add a copy of the same task, to see if it would register it but that too failed. But then you can also see that the first one got successful
Copy code
| /tmp/register1703962791/00_pipeline.tasks.config_task.load_task_1_exec_config_1.pb    | Success | Successfully registered
also there were no code changes made after the earlier successful runs, it was just updating some config values
Could it be that something is wrong with the failing task ? Only the .pb related to that tasks are failing
This is something related to our cluster setup and not specific to flyte workflows. Our devFo team is looking into this.