https://flyte.org
Join Slack
I want to change the log url displayed in flyte console to where our logs are stored, so I added the...
r

Robin Kahlow

over 3 years ago
I want to change the log url displayed in flyte console to where our logs are stored, so I added the entry below to the values. The url does change but it seems like podName is always empty so the url is https://my.papertrailapp.com/systems//events, is there something else I need to do to make the podName variable work? I just looked at the defaults which uses podName too https://docs.flyte.org/en/stable/deployment/cluster_config/flytepropeller_config.html#logs-logs-logconfig 
task_logs:
    plugins:
      logs:
        kubernetes-enabled: true
        kubernetes-template-uri: <https://my.papertrailapp.com/systems/{{> .podName }}/events
r
d
  • 2
  • 25
  • 176
Hi! We are currently trying to fast-register workflows using `flytectl==0.5.21`. The workflows were ...
b

Bernhard Stadlbauer

over 3 years ago
Hi! We are currently trying to fast-register workflows using 
flytectl==0.5.21
. The workflows were serialized using 
pyflyte
(from `flytekit=1.0.0`; using 
fast
). The storage configuration looks like this: 
... 
storage:
  type: stow
  stow:
    kind: google
    config:
      project_id: "<project>"
      scopes: <https://www.googleapis.com/auth/devstorage.read_write>
  container: "<bucket>"
Now, when running 
flytectl register files tmp/workflows/* -p acorn -d development
, we see: 
Error: failed to upload source code from [tmp/workflows/fasta438eafa1aa98e76043e131e481a7a23.tar.gz]. Error: failed to create an upload location. Error: rpc error: code = Unknown desc = failed to create a signed url. Error: storage: missing required GoogleAccessID
{"json":{},"level":"error","msg":"failed to upload source code from [tmp/workflows/fasta438eafa1aa98e76043e131e481a7a23.tar.gz]. Error: failed to create an upload location. Error: rpc error: code = Unknown desc = failed to create a signed url. Error: storage: missing required GoogleAccessID","ts":"2022-04-27T13:37:26+02:00"}
GOOGLE_APPLICATION_CREDENTIALS
env is set to a service account which has access to the bucket. Registering against flyteadmin in version 1.0.0
b
p
h
  • 3
  • 19
  • 176
In Flyte UI, is it possible to populate the description field of a workflow? Google style docstrings...
g

Greg Gydush

over 3 years ago
In Flyte UI, is it possible to populate the description field of a workflow? Google style docstrings work for documenting parameters, but the description on the UI is always “This workflow has no description”
g
y
+3
  • 5
  • 7
  • 176
Hi team, I am a DS from gojek, currently working on using Flyte for model pipeline. One question I c...
c

Chen Yuanyuan

over 3 years ago
Hi team, I am a DS from gojek, currently working on using Flyte for model pipeline. One question I came across is that the resource requested (cpu, memory) is hard coded in the decorator for each task for the current implementation, so it’s the same for different environments. What will be a better way to do overrides for each task in the workflow for different environments? Will it be possible to define it in a config file instead? cc @Pradithya Aria Pura @Michael Cheng Jan Kao
c
k
+2
  • 4
  • 31
  • 176
Another issue I found: I get this error for `flytectl config validate` command, although I have `co...
c

Chandrashekhar Kotekar

over 2 years ago
Another issue I found: I get this error for 
flytectl config validate
command, although I have 
config-sandbox-yaml
and 
config.yaml
files in 
~/.flyte
directory 
Couldn't find a config file.
Validated config file successfully.
c
j
s
  • 3
  • 4
  • 175
TLDR; Configuring cluster resources in Helm chart (on GCP) - specific for permissions with IAM servi...
a

Ariel Kaspit

over 2 years ago
TLDR; Configuring cluster resources in Helm chart (on GCP) - specific for permissions with IAM service accounts Hi! I have some issues with configuring cluster resources for IAM permissions (in Console and CLI) I deployed Flyte helm chart on GCP and configured 
cluster_resource_manager
as documented, but still getting permissions errors. I followed the documentation, specifically in here: https://docs.flyte.org/en/latest/deployment/configuration/general.html#cluster-resources This is my configuration in `values.yaml`: 
configmap:
  domain:
    domains:
      - id: development
        name: development
      - id: staging
        name: staging
  namespace_config:
    namespace_mapping:
      template: "{{ domain }}"
cluster_resource_manager:
  config:
    cluster_resources:
      customData:
        - development:
          - projectQuotaCpu:
              value: "5"
          - projectQuotaMemory:
              value: 4000Mi
          - gsa:
              value: <mailto:flyte@projectid.iam.gserviceaccount.com|flyte@projectid.iam.gserviceaccount.com>
        - staging:
          - projectQuotaCpu:
              value: "5"
          - projectQuotaMemory:
              value: 4000Mi
          - gsa:
              value: <mailto:flyte@projectid.iam.gserviceaccount.com|flyte@projectid.iam.gserviceaccount.com>
  templates:
    - key: aa_namespace
      value: |
        apiVersion: v1
        kind: Namespace
        metadata:
          name: {{ namespace }}
        spec:
          finalizers:
          - kubernetes
    - key: aab_default_service_account
      value: |
        apiVersion: v1
        kind: ServiceAccount
        metadata:
          name: default
          namespace: {{ namespace }}
          annotations:
            <http://iam.gke.io/gcp-service-account|iam.gke.io/gcp-service-account>: <mailto:flyte@projectid.iam.gserviceaccount.com|flyte@projectid.iam.gserviceaccount.com>
    - key: ab_project_resource_quota
      value: |
        apiVersion: v1
        kind: ResourceQuota
        metadata:
          name: project-quota
          namespace: {{ namespace }}
        spec:
          hard:
            limits.cpu: {{ projectQuotaCpu }}
            limits.memory: {{ projectQuotaMemory }}
In console, I don’t see any IAM and serviceaccounts assigned to the project (attached a screenshot). Using 
pyflyte
, I’m trying to run the 
hello-world
workflow (I use the basic workflow just for testing, it’s from 
flytesnacks/cookbook/core/flyte_basics/hello_world.py
) - and I get 403 Permissions denied. Is there something I need to configure in the workflow itself / in 
./flyte/config.yaml
?
a
d
  • 2
  • 6
  • 175
<#CP2HDHKE1|> Hello All, I am trying to install custom packages into the flyte container. In my buil...
s

Sujith Samuel

over 2 years ago
#CP2HDHKE1 Hello All, I am trying to install custom packages into the flyte container. In my build job prior to flyte package I am installing numpy which is 1.24.3. But after packaging when i trigger the workflow the version which is shown to me in workflow output is 1.20.0 Another example is I am installing transformers python package but in the workflow the same says module not found. I saw that in kubeflow I can install custom packages using 
add_op = create_component_from_func(
with package_to_install option. Is there a Flyte way of doing this
s
s
a
  • 3
  • 17
  • 175
Hi, when using flyte-binary how can I view the flyte-propeller logs to see any errors with workflow ...
b

Blair Anson

over 2 years ago
Hi, when using flyte-binary how can I view the flyte-propeller logs to see any errors with workflow execution? And what about project creation, are there logs on the server showing changes done as the result of a 
flytectl create project
command?
b
k
d
  • 3
  • 10
  • 175
Does anyone have experience with using AWS Secrets Manager with flyte instead of the default k8 secr...
l

Laura Lin

over 2 years ago
Does anyone have experience with using AWS Secrets Manager with flyte instead of the default k8 secrets? a little confused on how to set up the secret names.
l
r
y
  • 3
  • 42
  • 175
```/opt/venv/lib/python3.10/site-packages/flytekit/types/schema/types.py:323: FutureWarning: In the ...
s

seunggs

almost 3 years ago
/opt/venv/lib/python3.10/site-packages/flytekit/types/schema/types.py:323: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.  (This may have returned Python scalars in past versions.
  _np.bool: SchemaType.SchemaColumn.SchemaColumnType.BOOLEAN,  # type: ignore
Traceback (most recent call last):
  File "/opt/venv/bin/pyflyte", line 5, in <module>
    from flytekit.clis.sdk_in_container.pyflyte import main
  File "/opt/venv/lib/python3.10/site-packages/flytekit/__init__.py", line 195, in <module>
    from flytekit.types import directory, file, numpy, schema
  File "/opt/venv/lib/python3.10/site-packages/flytekit/types/schema/__init__.py", line 1, in <module>
    from .types import (
  File "/opt/venv/lib/python3.10/site-packages/flytekit/types/schema/types.py", line 313, in <module>
    class FlyteSchemaTransformer(TypeTransformer[FlyteSchema]):
  File "/opt/venv/lib/python3.10/site-packages/flytekit/types/schema/types.py", line 323, in FlyteSchemaTransformer
    _np.bool: SchemaType.SchemaColumn.SchemaColumnType.BOOLEAN,  # type: ignore
  File "/opt/venv/lib/python3.10/site-packages/numpy/__init__.py", line 284, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'bool'. Did you mean: 'bool_'?
s
k
+5
  • 7
  • 49
  • 175
Previous585960Next

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by