https://flyte.org logo
Join the conversationJoin Slack
Channels
announcements
ask-the-community
auth
conference-talks
contribute
databricks-integration
datahub-flyte
deployment
ecosystem-unionml
engineeringlabs
events
feature-discussions
flyte-bazel
flyte-build
flyte-console
flyte-deployment
flyte-documentation
flyte-github
flyte-ui-ux
flytekit
flytekit-java
flytelab
great-content
hacktoberfest-2022
helsing-flyte
in-flyte-conversations
introductions
jobs
konan-integration
linkedin-flyte
random
ray-integration
ray-on-flyte
release
scipy-2022-sprint
sig-large-models
workflow-building-ui-proj
writing-w-sfloris
Powered by Linen
announcements
  • j

    Jing Li

    03/23/2022, 7:38 AM
    hello, where can I download the images for flyte-sandbox
  • j

    Jing Li

    03/23/2022, 7:39 AM
    like 'cr.flyte.org/flyteorg/flytescheduler-release:v0.19.3'
    y
    • 2
    • 1
  • j

    Jing Li

    03/23/2022, 7:41 AM
    sorry I have a problem with installing flyte-sandbox from a command 'flytectl sandbox start', I must download all docker images locally, and install them.
    y
    k
    • 3
    • 6
  • e

    Eduardo Apolinario (eapolinario)

    03/23/2022, 2:00 PM
    :flyte: It’s time for our weekly Office 🏢 Hours ⌛! This week I’ll be on the zoom link for the next 30 mins. Zoom link
    ❤️ 4
  • s

    Sandra Youssef

    03/23/2022, 11:21 PM
    Hi Flyers, Watch the scoop on Flyte's spot instances and interruptible tasks, checkpointing, and the all-new Flyte Decks by Union.ai's @Ketan (kumare3). Also catch this week's

    community updates▾

    . Meeting minutes: https://bit.ly/3NdCL2h

    https://youtu.be/9dkd0rxy4bl ▾

    See you on April 5th!
    🚀 3
    ❤️ 4
  • k

    Ketan (kumare3)

    03/24/2022, 3:51 AM
    Its time for our weekly office hours 🏢 - :flyte: . I will be on the Zoom link in 10 minutes and stay there for 30 minutes until 9:30 PST
    • 1
    • 1
  • s

    Samhita Alla

    03/24/2022, 3:29 PM
    📣 Contributors of the Month 📣 Hello everyone! With the inflow of contributions, Flyte is being constantly improved. On that note, we want to give shout-outs and send swag to all our contributors who are sweating it out to make Flyte even better! The shout-outs are going to happen every month. The following is the list of contributors we’d like to thank this month. • @Sören Brunk has been extensively working on simplifying the deployment of Flyte on GCP, among other backend improvements. He’s one of the core contributors! • @Max Smolin worked on a significant PR that added support for Union type, one of the most requested features. • @Kenny Workman worked on adding type annotation support to the Flytekit codebase. We hope to have many more of your contributions, Soren, Max, and Kenny! All contributors of the month should be available on flyte.org.
    🎉 9
    :flyte: 6
    🙌 9
    k
    s
    • 3
    • 2
  • k

    Ketan (kumare3)

    03/24/2022, 6:40 PM
    Please clap on this post by Lyft - pretty awesome!!!! = https://eng.lyft.com/orchestrating-data-pipelines-at-lyft-comparing-flyte-and-airflow-72c40d143aad
    👏 18
    🙌 9
    c
    • 2
    • 4
  • j

    Jing Li

    03/25/2022, 9:35 AM
    If I want to deploy flyte-sandbox to AWS, what is the minimum EC2 that I need?
    y
    • 2
    • 1
  • h

    Hampus Rosvall

    03/25/2022, 10:53 AM
    Hi, I want to deploy Flyte Helm charts to our EKS cluster. We currently have an older version of AWS Ingress Controller than required for the Ingress. Is it possible to bypass this requirement by e.g., port-forwarding and still being able to deploy workflows to
    FlyteAdmin
    ? It is in our road map to bump the ALB Ingress Controller version, but until that is done I would still like to experiment with Flyte on EKS.
    y
    p
    • 3
    • 5
  • j

    jeev

    03/25/2022, 6:56 PM
    has anyone successfully benchmarked propeller for >5k concurrent workflows? what does the propeller config look like for something like that?
  • h

    Hafsa Junaid

    03/25/2022, 7:04 PM
    Hi! I was to access flyteconsole service externalize endpoint. I have followed the complete document: https://docs.flyte.org/en/latest/deployment/gcp/manual.html#deployment-gcp-manual The workload and service are up (the screenshot is shared): • The docment says --> Use the https://<FLYTE-ENDPOINT>/console to get access to flyteconsole UI I have done the same and used the domain as flyte-endpoint but nothing is showing up. Are there any further steps to access flyteconsole UI ?? like create external IP instead of cluster IP or mapping service to domain
    y
    y
    • 3
    • 22
  • h

    Hafsa Junaid

    03/26/2022, 12:38 AM
    Hi I am doing
    helm install incubator/sparkoperator --namespace spark-operator --kubeconfig=~/.flyte/k3s/k3s.yaml
    Error: Error: INSTALLATION FAILED: must either provide a name or specify --generate-name What is generate name here?
    k
    • 2
    • 4
  • a

    Alex Bain

    03/26/2022, 1:08 AM
    Hey Flyte Team, I just updated to Flyte
    v0.19.3
    . No matter what I add to my Propeller
    deployment.yaml
    , I cannot seem to get Stackdriver log links to render in Flyte Console for map tasks (i.e.
    k8s_array
    tasks). Map task subtasks always render with Kubernetes log links (even when declaring
    kubernetes-enabled: false
    ). I'm pretty sure that Stackdriver log links just don't work yet for map tasks. Could you confirm that?
    h
    • 2
    • 8
  • h

    Hafsa Junaid

    03/26/2022, 2:55 PM
    @Yuvraj Propeller workload pod is still crashing. Annotation with a namespace is done. Also, the actual problem statement is still there. As I need to access external IP for UI services. @Yee
    y
    s
    • 3
    • 25
  • a

    Alex Pozimenko

    03/28/2022, 7:32 PM
    hi folks, i'm looking at available Flyte metrics to help us monitor the platform as we run large workloads (and stress test the cluster). Are there good docs/samples available? I found this but it's very high level. One specific question: Is this metric cumulative -
    flyte-propeller-all-workflow-acceptance.latency.ms.sum
    ? Sample chat below. Or perhaps I should be looking at percentiles only (.ms.0.9 and .ms.0.99)?
    k
    • 2
    • 2
  • f

    Fernando Diaz

    03/29/2022, 10:17 AM
    Hi! We added a timeout for one node in our workflow. This node is handled by a backend plugin (
    bigqueryjob
    ). We see in the logs that the node timeouts as expected:
    "Current execution for the node timed out; timeout configured: 3h0m0s"
    However, the
    Delete
    method of the plugin (https://github.com/flyteorg/flyteplugins/blob/master/go/tasks/plugins/webapi/bigquery/plugin.go#L226) is never called after the timeout, so the job is not cancelled. Is this expected? How can I tell propeller to abort the node after a timeout?
    d
    • 2
    • 4
  • h

    Hampus Rosvall

    03/29/2022, 12:33 PM
    Hi, I have deployed Flyte on EKS and I am experimenting with the fast register approach as I am only iterating my source code. I saw that I should configure flytectl with S3 configuration. It works with “non-fast” registration but I am getting this error with fast registration. I have serialized using the fast approach initially. Any ideas what might cause this error?
    (.venv) ☁  ~ ⚡ make register PROJECT=determined DOMAIN=development
    flytectl register files \
    		 --project determined \
    		 --domain development \
    		 --archive flyte-package.tgz \
    		 --version v1-fast1
    Error: please check your Storage Config. It failed while uploading the source code. Failed to write data [69b] to path [fast/v1-fast1-fast9bb248f6c3ae89386ec7105633eaaee8.tar.gz].: PutObject, putting object: NoCredentialProviders: no valid providers in chain. Deprecated.
    	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
    gmake: *** [Makefile:22: register] Error 1
    (.venv) ☁  ~  cat ~/.flyte/config.yaml
    admin:
      endpoint: dns:///localhost:8089
      authType: Pkce
      insecure: true
    logger:
      show-source: true
      level: 0
    storage:
      type: stow	
      stow:
        kind: s3
        config:
          auth_type: iam
          region: eu-west-1
      container: <name-of-my-bucket>
    y
    k
    • 3
    • 5
  • m

    Mohamed Nabil H

    03/29/2022, 5:02 PM
    hello there 🙂
    🖖 1
  • m

    Mohamed Nabil H

    03/29/2022, 5:06 PM
    i’m really struggling with getting flyte work on k3d , also the docker-desktop kubernetes , are the docs up to date ? anyone else tried to have it deployed on k3d. ?
    y
    • 2
    • 5
  • m

    Matheus Moreno

    03/29/2022, 5:20 PM
    Hey, everyone! I'm trying to integrate our MLflow server to our Flyte projects. My current approach consists of passing the MLflow username and password as Secrets to the required tasks, but I was wondering: does anyone else here uses this stack, and if so, are you willing to share a little bit about how it's configured? 🙂
    l
    k
    s
    • 4
    • 13
  • m

    Miggy

    03/29/2022, 10:00 PM
    There’s a new use case that came up and we’re trying to figure how best to do this. I was wondering if I could some help with resources/ideas. So we have a bunch of tasks that are already registered and at runtime we want to be able to construct the graph. One thought was to basically create the workflow graph spec and submit a request to flyteadmin similar to how flytekit does. Then the newly created workflow can be launched. In short we want to be able to generate a workflow without needing to write python code for it. Just specify the proto spec. Thoughts? @katrina @Ketan (kumare3) @Yee cc: @Rupsha Chaudhuri @Danielle Curammeng
    y
    d
    k
    • 4
    • 29
  • m

    Mike Ossareh

    03/30/2022, 5:13 PM
    Are there any examples of flyte scheduling workloads on CoiledCloud ?
    k
    • 2
    • 12
  • a

    Alex Pozimenko

    03/30/2022, 5:49 PM
    Hi Flyers.
    flyte-admin-execution.manager-active.executions.gauge
    shows negative values. Is this by design?
    k
    • 2
    • 2
  • s

    Sandra Youssef

    03/30/2022, 7:38 PM
    Hi Flyers, Save the date! Help us welcome @Jake Neyer from Striveworks, who will be presenting at our upcoming Flyte Community Sync next week. Details coming... Tuesday April 5, 9am PT
    🔥 6
    🚀 1
    👏 4
    ❤️ 7
    :flyte: 1
    j
    • 2
    • 1
  • m

    Miggy

    03/30/2022, 8:39 PM
    Hey folks, trying to debug an issue. We’re seeing various tasks fail with
    terminated in the background, manually
    . I took a look at the code here and see that this occurs when the pod get deleted. I don’t see a delete request being sent to admin so don’t think there is anything there. Propeller logs haven’t shown anything either. So I’ve been looking at kube logs to try to see what is triggering the pod deleted but there’s a lot of noise in the logs so taking some time to through them. Meanwhile, I figured I’d ask here if anyone has run into something similar before
    k
    • 2
    • 6
  • k

    Ketan (kumare3)

    03/31/2022, 4:02 AM
    hIts time for the weekly office hours. I am on the Zoom link, for the next 30 minutes. Drop in, say hi!
  • f

    Fabio Grätz

    03/31/2022, 6:18 PM
    I need some help getting launchplans with a cronjob configured to work: I first register a launchplan with a
    * * * * *
    cron expression via flyteremote. After that, the flyte UI still shows
    This workflow has no schedules
    , as expected. I then execute
    flytectl update launchplan -p default -d development --version fabio-cron-2a3fb7f-dirty-BQN44 cron_hydra_workflow_cfg_flyte_workflows.collaborative_filtering.workflow_0 --activate
    which gives:
    updated launchplan successfully on cron_hydra_workflow_cfg_flyte_workflows.collaborative_filtering.workflow_0%
    In the UI, the schedule is updated as well, it now says
    Every minute
    . Problem: I’m not seeing any executions. The flyte admin shows this in the logs after I activated the launchplan:
    {"json":{"src":"launch_plan.go:108"},"level":"info","msg":"Adding resource type for unspecified value in request: [%+v]id:\u003cproject:\"default\" domain:\"development\" name:\"cron_hydra_workflow_cfg_flyte_workflows.collaborative_filtering.workflow_0\" version:\"fabio-cron-2a3fb7f-dirty-BQN44\" \u003e state:ACTIVE ","ts":"2022-03-31T18:13:01Z"}
    {"json":{"src":"log.go:42"},"level":"warning","msg":"Failed to parse authenticated client metadata when creating audit log","ts":"2022-03-31T18:13:01Z"}
    {"json":{"src":"log.go:91"},"level":"info","msg":"Recording request: [{\"Principal\":{\"Subject\":\"\",\"ClientID\":\"\",\"TokenIssuedAt\":\"0001-01-01T00:00:00Z\"},\"Client\":{\"ClientIP\":\"\"},\"Request\":{\"Method\":\"UpdateLaunchPlan\",\"Parameters\":{\"domain\":\"development\",\"name\":\"cron_hydra_workflow_cfg_flyte_workflows.collaborative_filtering.workflow_0\",\"project\":\"default\",\"version\":\"fabio-cron-2a3fb7f-dirty-BQN44\"},\"Mode\":1,\"ReceivedAt\":\"2022-03-31T18:13:01.578706629Z\"},\"Response\":{\"ResponseCode\":\"OK\",\"SentAt\":\"2022-03-31T18:13:01.597983728Z\"}}]","ts":"2022-03-31T18:13:01Z"}
    The
    flytescheduler
    doesn’t show any update in the logs. Does anyone have some guidance where to start debugging this? Thanks!
    k
    k
    p
    • 4
    • 55
  • a

    Alex Bain

    03/31/2022, 7:58 PM
    Hey Flyte Team, we run a telegraf sidecar container in our flyteadmin and flytepropeller pods to scrape Prometheus metrics (from flyteadmin and flytepropeller). Does flyteconsole support this as well or not yet?
    k
    • 2
    • 1
  • w

    William Young

    03/31/2022, 9:18 PM
    Hey folks. I was wondering if Flyte has support for writing spark jobs in Scala. I see a lot of examples for pyspark using the python flyte api, but I want to use flyte-scala to write scala spark pipelines, and documentation seems kinda sparse on this topic.
    k
    • 2
    • 24
Powered by Linen
Title
w

William Young

03/31/2022, 9:18 PM
Hey folks. I was wondering if Flyte has support for writing spark jobs in Scala. I see a lot of examples for pyspark using the python flyte api, but I want to use flyte-scala to write scala spark pipelines, and documentation seems kinda sparse on this topic.
k

Ketan (kumare3)

03/31/2022, 9:34 PM
this does not exist yet
but if you folks are interested we can probably get something in later part of april
this is completely doable
cc @Guillaume Perchais / @Babis Kiosidis / @Nelson Arapé from the Spotify team who help us drive the java/scala sdk
@William Young do you work for spotify 😄 If so we should definitely prioritize this. There are lot of folks in teh community who want this 😄.
w

William Young

03/31/2022, 9:38 PM
yes I do!
It would make a lot of folks in my part of the org very happy if this were possible. 🙂
Implementation-wise, we are looking at three possibilities: running Spark on Databricks, Dataproc, and K8. Preferrably all 3. I think the first two would just be package up a jar, and make api call. Don’t know much about K8 though.
k

Ketan (kumare3)

03/31/2022, 11:04 PM
ya the first 2 should be trivial, but databricks and dataproc plugins need to be implemented
and for k8s is also all the backend work is already done
just need a java plugin
i acutally gave instructions to someone
let me see if i can share the issue / docs?
this is pretty easy to do
w

William Young

04/01/2022, 1:19 PM
I took a quick look. I think the Databricks one might be harder as they don’t have a java api that I can find. Dataproc of course does have one though, and I’m guessing the implementation would be somewhat similar to the Dataflow one we already have. Would need guidance on K8. cc@Mark Grey
k

Ketan (kumare3)

04/01/2022, 1:24 PM
K8 should not be hard
We actually want to implement a backend plugin for data to
Databricks
Would you folks have a few minutes to sync?
w

William Young

04/01/2022, 3:19 PM
So the folks you mentioned above are probably the actual owners of the Flyte infrastructure within Spotify. I am working with Mark (mentioned above) who is closer to them organizationally than I am. I am just a motivated potential user. But given that Spark is not a “blessed” technology within Spotify yet, unsure as to their roadmap. I’d be happy to take a look into this myself if I had guidance from one of the above though. Might require an internal discussion as I don’t know the code structure well enough yet to know how much of this is a potential open source contribution, and how much is Spotify specific.
k

Ketan (kumare3)

04/01/2022, 3:21 PM
More from a point of view to understand what is the goal, how to maximize impact in short term
w

William Young

04/06/2022, 3:12 PM
Will get back in the next week or two? Have to have some internal discussions first, but I still really wanna do this.
k

Ketan (kumare3)

04/06/2022, 5:08 PM
this is fantastic
we should be able to help with this too
View count: 5