@ancient-wolf-19325 In flyte-core helmchart https://github.com/flyteorg/flyte/blob/master/charts/flyte-core/values.yaml I found a way to set

projectQuotaCpu

per domain. Helmchart itself does not show how to set it up per project. Is it even possible?

Setting

projectQuotaCpu

per domain is supported directly in the Flyte-core Helm chart via the

cluster_resource_manager.config.cluster_resources.customData

field, but setting it per project (within a domain) is not natively supported in the Helm values file. The Helm chart only allows you to specify quotas at the domain level, not at the project+domain granularity. This is confirmed by the structure in values.yaml and related config examples, which only nest quotas under domains, not projects. For per-project quotas, you must use Flyte's dynamic resource management features after deployment, such as updating settings via

flytectl

or the FlyteAdmin API, as discussed in this issue. The Helm chart does not provide a direct way to set project-level quotas at install time. Sources: - Flyte Issue #2070 - values.yaml 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 how about

task_resource_defaults

- any way of setting different task resources for specific projects?

No, the Flyte-core Helm chart's

task_resource_defaults

only sets global (platform-wide) defaults and limits for all tasks; it does not support per-project overrides via the values.yaml file. To set different task resources for specific projects, you must use Flyte's matchable resource system after deployment, typically via

flytectl update task-resource-attribute

or the FlyteAdmin API, as described in Flyte Issue #2070 and the overridable defaults RFC. The Helm chart and values.yaml do not provide a mechanism to specify project-level task resource defaults at install time; this must be managed post-deployment. Sources: - Flyte Issue #2070 - RFC: Sane overridable defaults 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 I have setup

initial-project

in helmchart and removed one of the projects that was already used. Will it be removed and will all runs from it be removed or do I have to make manual steps?

No, removing a project from

initialProjects

in the Helm chart does not delete the project or its runs from Flyte. The

initialProjects

field only seeds projects at install time; it does not manage or remove existing projects or their data. Manual steps are required to archive or clean up projects and their executions, as Flyte does not support project deletion via Helm or the API—only archiving is supported, and data remains unless explicitly cleaned up, as discussed in Flyte Issue #1619 and Flyte Issue #3234. Would you like more detail on how to archive or clean up projects and their data? Sources: - Flyte Issue #1619 - Flyte Issue #3234 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 how to cleanup the project and remove it entirely

Flyte does not support fully deleting a project and all its resources via the API or CLI; only archiving (tombstoning) is supported, which hides the project but does not remove its data or executions. Complete removal would require manual deletion from the database and object storage, which is not recommended and can lead to inconsistencies, as discussed in Flyte Issue #1619 and Flyte Issue #3234. If you need to fully remove a project, you must manually delete all related database entries and storage objects, but this is risky and not officially supported. Would you like more detail on the manual cleanup process and its risks? Sources: - Flyte Issue #1619 - Flyte Issue #3234 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 ok so how to archive the project

To archive a project in Flyte, use the command-line tool

flytectl

with the

update project --archive

flag. For example:

flytectl update project -p <project-id> --archive

This will mark the project as archived, making it hidden in the UI and unavailable for new executions, but it will not delete any data. For more details, see Flyte Issue #1619 and Flytectl project update docs. Would you like more detail on archiving or unarchiving projects? Sources: - Flyte Issue #1619 - flytectl update project command 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 how do I get project_id If I get the name of it

You can retrieve the project ID by listing all projects with

flytectl get project

, which displays both the project name and ID. For example:

flytectl get project

This will show a table with columns for ID, NAME, and DESCRIPTION, allowing you to match the name to its corresponding ID. For more, see Flytectl get project. Would you like more detail or examples? Sources: - flytectl get project command 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 can a new run be submitted to archived project?

No, you cannot submit a new run to an archived project. Once a project is archived, it is treated as inactive and new executions, updates, or registrations are blocked, as described in Flyte Issue #1619 and Flyte Issue #569. Would you like more detail on the behavior of archived projects? Sources: - Flyte Issue #569 - Flyte Issue #1619 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 can I control project description via helmchart? or only via flytectl

You cannot control or update a project description via the Helm chart after initial creation; the Helm chart's

initialProjects

only seeds project names/IDs at install time and does not manage descriptions. To set or update a project description, you must use

flytectl

or the FlyteAdmin API, as shown in flytectl create project and flytectl update project. Would you like more detail on using

flytectl

to manage project descriptions? Sources: - flytectl create project command - flytectl update project command 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 give me example of exact command that updates description of single project using flytectl

To update the description of a single project using flytectl, run:

flytectl update project -p <project-id> --description "New project description"

Replace

<project-id>

with your project's ID. This command updates only the description field. For reference, see flytectl update project command. Sources: - flytectl update project command 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 how can i setup RBAC so only few people can update projects but more people can submit jobs? is there a way to do that

Currently, Flyte open source does not support fine-grained RBAC (Role-Based Access Control) for restricting who can update projects versus who can submit jobs. All authenticated users can access all projects and perform actions like submitting workflows or updating projects, as discussed in Flyte Issue #5189, Flyte Issue #3909, and Flyte Issue #555. There are ongoing discussions and proposals for RBAC, but no official support is available in open source Flyte as of now. Would you like more detail on current workarounds or the status of RBAC proposals? Sources: - Flyte Issue #5189 - Flyte Issue #3909 - Flyte Issue #555 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 How to modify logging level using flyte-core helmchart? In flyte-binary there is logging section which is not present in flyte-binary

It seems likely that in the flyte-core Helm chart, logging level is not set via a top-level

logging

section in

values.yaml

(as in flyte-binary), but instead is configured through the FlyteAdmin and FlytePropeller config files. You can override these by providing custom config files or using the

inlineConfigMap

or

externalConfigMap

fields to inject your own logging configuration, as described in flyte-core values.yaml. There is no direct

logging.level

field in flyte-core's values file. Would you like more detail on how to inject a custom logging config for FlyteAdmin or Propeller? Sources: - flyte-core values.yaml 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 there is loggin plugin also enabled on the flyte binary. can I achieve something similar in flyte-core?

Yes, you can achieve similar logging plugin functionality in flyte-core by configuring the

task_logs

section in the Helm chart's values.yaml or by providing a custom config file. The

task_logs

configuration allows you to set up log links for various providers (e.g., Kubernetes, CloudWatch, Stackdriver) using templated URIs, just like in flyte-binary. See the flyte-core values.yaml and log configuration structure for details. Would you like an example of how to configure a specific logging provider in flyte-core? Sources: - flyte-core values.yaml 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 am I missing something? log config: https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/logs#TemplateLogPluginConfig does not allow to set the logging level if im not incorrect

You are correct: the log config at TemplateLogPluginConfig only defines how log links are generated and displayed in the UI, not the logging level. Logging level is controlled separately in the Flyte platform's logger configuration (e.g.,

logger.level

), not in the log plugin config. Would you like more detail on how to set the logging level in Flyte? Sources: - TemplateLogPluginConfig 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 If I manually want to edit configmap of flyte admin to enable new loggin plugin, whats the configmap entry I should modify? read the flyte sourcecode to find the answer preferably

To enable a new logging plugin in FlyteAdmin, you should modify the

task_logs

section of the FlyteAdmin configmap (commonly named

flyte-admin-config

or similar). Specifically, update or add the

plugins.logs

section under

task_logs

to include your new log plugin configuration, following the structure defined in LogConfig and TemplateLogPluginConfig. For example, to add a new CloudWatch log plugin, you would add:

task_logs:
  plugins:
    logs:
      templates:
        - displayName: CloudWatch
          templateUris:
            - "https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logEventViewer:group=/flyte-production/kubernetes;stream=var.log.containers.{{.podName}}_{{.namespace}}_{{.containerName}}-{{.containerId}}.log"

This entry should be placed in the FlyteAdmin configmap under the appropriate YAML hierarchy. After editing, restart the FlyteAdmin deployment for changes to take effect. Sources: - TemplateLogPluginConfig 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 which yaml file should I edit for the above? is that server.yaml?

No, you should not edit

server.yaml

for logging plugins. The correct file to edit is the config file that contains the

task_logs

section, typically named

task_logs.yaml

or included as part of the FlyteAdmin configmap (often named

flyte-admin-config

). In the Helm chart, this is usually managed via a ConfigMap template that merges multiple config files, and

task_logs.yaml

is the standard location for log plugin configuration, as shown in flyte-core values.yaml and log config docs. Would you like more detail on how to locate or inject this file in your deployment? Sources: - https://github.com/flyteorg/flyte/blob/master/deployment/eks/flyte_helm_controlplane_generated.yaml - https://github.com/flyteorg/flyte/blob/master/deployment/sandbox/flyte_helm_generated.yaml 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 I applied the configuration above, but still am getting

Logs
No logs found

in the task in console. I rebooted admin and console pods after applying the configuration. how to debug the problem

To debug missing logs in the Flyte Console after updating log configuration, first verify that your

task_logs

config is correct and applied to the right ConfigMap. Then, check if the pods for your task are running in the expected namespace and inspect their status using:

kubectl get pods -n <project>-<domain>

If pods exist, describe them for errors (e.g., image pull, crashloop). Also, ensure your log link template matches your logging provider and that the provider (e.g., Kubernetes dashboard, CloudWatch) is accessible. For more, see debugging executions and configuring logging links. Would you like step-by-step troubleshooting for a specific logging provider? Sources: - Debugging executions - configuring logging links 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 should the configmap be modified in propeller or admin cluster?

The

task_logs

configmap should be modified in the FlytePropeller (data plane) cluster, not the FlyteAdmin (control plane) cluster. FlytePropeller is responsible for task execution and log link generation, so log plugin configuration must be present in its environment, as shown in flyte-core values.yaml and deployment configs. Would you like more detail on how to update the configmap for FlytePropeller? Sources: - deployment/eks/flyte_helm_generated.yaml 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 I am deploying multi-cluster flyte-core scenario in multiple AWS accounts. I am trying to minimize the S3 bucket access for workers. What S3 access does the role that is assumed by worker pod need? To the s3 bucket in the target propeller, that is clear to me - but any access in admin bucket as well?

The IAM role assumed by the worker pod only needs access to the S3 bucket(s) configured for the FlytePropeller (data plane) cluster where the workflow is running. It does not require access to the S3 bucket used by the FlyteAdmin (control plane) cluster. Workers only interact with the data and metadata buckets specified for their own data plane, as confirmed in Flyte multi-cluster discussions and Flyte S3 access patterns. Would you like more detail on S3 policy examples or cross-account scenarios? Sources: - Flyte S3 access patterns - Flyte multi-cluster discussions 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 I configured IAM role so it only had access to propeller bucket, and now my tasks are failign with 403 - forbiden sayign they cannot acesss S3 bucket (i see project name is the path inside the back) that is used by flyte-admin (control plane)

This happens because Flyte stores metadata and references to raw data in the S3 bucket configured for the control plane (FlyteAdmin), and worker pods may need to access both the data plane (Propeller) and control plane (Admin) buckets to fetch or store inputs/outputs, especially for metadata or when accessing historical executions. As described in Flyte S3 access patterns and Flyte Issue #6496, restricting worker IAM roles to only the Propeller bucket can cause 403 errors if any references or metadata point to the Admin bucket. To resolve this, ensure the IAM role for worker pods has access to both the Propeller (data plane) and Admin (control plane) S3 buckets. This is necessary for correct operation in multi-cluster or cross-account setups. Sources: - Flyte S3 access patterns - Flyte Issue #6496 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.