https://flyte.org logo
Join Slack
Powered by
Hey, everyone! Just a heads up: I think 1.x.x brok...
# flyte-ui-ux
s
Hey, everyone! Just a heads up: I think 1.x.x broke some things on the UI. When I enter a workflow's page, there's no description, even though there's a docstring defined on it (see first image). Also, if I click "View All" to view all the versions of the workflow, an error happens (see second image).
Also, did you guys remove the support to DataFrames? Now DF inputs/outputs are identified with "unknown" and "This type is not yet supported"
f
Thanks for reporting this @sparse-window-1536 , we are looking into it.
c
@sparse-window-1536 can you please check which FlyeConsole version do you have (click on 
i
icon in the top right corner.) I believe at least one of this two issues is fixed in flyteConsole v.1.1.0
s
wow, my version is 0.54.1, even though I'm using 1.x.x! how can I update it??
(its the sandbox btw)
t
you’re using 1.x of flytekit?
do you have anything on your sandbox you want to save? (prior executions or whatnot) if you don’t, i’d recommend just killing it and starting a new one
s
yeah, I'm using the image cr.flyte.org/flyteorg/flyte-sandbox:dind-5570eff6bd636e07e40b22c79319e46f927519a3, and deploying Flyte "locally", i.e. by using FLYTE_TEST=local
t
i am not familiar with that env var
is that the latest image?
indeed
s
I changed it because I wanted to increase the maximum amount of memory a task could use. The only way I was able to do this with the original sandbox was to change the 
/flyteorg/share/flyte/values.yaml
file and use the local helm repo to start the server
t
you didn’t want to adjust it afterwards?
s
How can I do this? I have no idea
t
just updating the admin config after installing
you’re using old sandbox right, not the new one.
so that should be pretty easy to do
you can also customize things on a project/domain basis
f
Cc @billowy-sundown-31926 can we document this better
✅ 1
b
Yes Ketan
s
Hey @thankful-minister-83577 I'm trying to configure the resources on a system-wide level, but I'm confused with that first link you sent me. On it, the memory limit is 500m, but when I try to register tasks, flytectl tells me that the limit is 1Gi. I noticed this value was at .flyte.configmap.task_resource_defaults.task_resources.limits.memory, and that's where I modified it. Shouldn't it be there that I should apply modifications?
And I'm still not sure how to configure the resources. The documentation says "In the absence of an override, the global default values in the FlyteAdmin config are used. The override values from the database are assigned at execution, rather than registration time.", but how can I override values?
Also, even removing the FLYTE_TEST variable and the initialization script, my UI version is still 0.54.1. The command I ran was:
Copy code
docker run -d --name flyte-sandbox --privileged --mount type=bind,src=$PWD,dst=/root -p 30080-30082:30080-30082 -p 30084:30084 -p 30086:30086 -p 30088:30088 -p 30089:30089 --entrypoint /bin/sh <http://cr.flyte.org/flyteorg/flyte-sandbox:dind-5570eff6bd636e07e40b22c79319e46f927519a3|cr.flyte.org/flyteorg/flyte-sandbox:dind-5570eff6bd636e07e40b22c79319e46f927519a3> -c 'tini flyte-entrypoint.sh'
(there were more commands after -c that changed the values for the max memory, but I removed them)
t
hey @sparse-window-1536 you can override can you try the following
cat > update.yaml
Copy code
defaults:
    cpu: "1"
    memory: 150Mi
limits:
    cpu: "2"
    memory: 450Mi
project: flyteexamples
domain: development
but replace it with the values you need
and then run
Copy code
flytectl update task-resource-attribute --attrFile update.yaml
this is probably better than mucking around with the admin configmaps (since they’re even harder to get to in the 
demo
version of the sandbox)
wrt the console version however, you have 
kubectl
access right?
can you 
kubectl -n flyte edit deploy flyteconsole
and change the image to 
<http://ghcr.io/flyteorg/flyteconsole:v1.1.0|ghcr.io/flyteorg/flyteconsole:v1.1.0>
s
can't I override on a cluster basis?
like, to all projects
I did the change, and now it worked. I'm on 1.1.0 now. FYI, the old version was actually flyteconsole-release:1.0.1, and it was on Flyte's repository
t
yeah you can
override for the entire cluster.
can you 
kubectl -n flyte get cm
and yeah we try to tag each individual version
and then was also add a flyte release tag
s
like that?
Copy code
bash-5.1# kubectl -n flyte get cm
NAME                               DATA   AGE
kube-root-ca.crt                   1      39m
flyte-contour                      1      39m
datacatalog-config                 4      39m
kubernetes-dashboard-settings      0      39m
flyte-propeller-config             11     39m
clusterresource-template           2      39m
flyte-clusterresourcesync-config   3      39m
flyte-console-config               3      39m
flyte-admin-config                 9      39m
flyte-scheduler-config             4      39m
propeller-leader                   0      37m
leader-elect                       0      39m
t
can you 
kubectl -n flyte edit flyte-admin-config
look for the resources section
there should be a requests and a limits key underneath it
should look like this
Copy code
task_resource_defaults.yaml: |
    task_resources:
      defaults:
        cpu: 1000m
        memory: 1000Mi
        storage: 1000Mi
      limits:
        cpu: 128
        gpu: 1
        memory: 192Gi
        storage: 2000Mi
you’ll probably have to update your admin deployment afterwards
either kill all the admin pods or i think it’s something like 
kubectl -n flyte rollout restart deploy flyteadmin
those words might be out of order
s
I was able to reconfigure! I just don't know if I'll use this in my sandbox environment (I'm setting up a script so that my team can easily set up a sandbox when they don't want to deploy to the remote server), I think it'll become too complex
t
where are they setting up the sandbox? like on their local machines?
we’re trying to make the deploy-to-remote-server step as easy as possible.
s
yeah, on their local machines. we're doing that while our remote server is not yet configured (this will be the next step!)
t
@great-school-54368 @icy-agent-73298 - whenever you guys get back on… is there a quick and easy way to provide configuration to flytectl sandbox to configure things like the default task resources in admin’s configmap?
i’ve always just edited after the fact, but maybe there’s a better way
f
I think there is a way to edit the helm
i
Nope @thankful-minister-83577 flytectl doesn’t allow to edit any kubernetes objects directly and it’s only uses admin apis to change any configs. For changing the system level resource configuration you are right with suggestion of bumping the resources in the admin config map. The one that we set from flytectl are values that admins stores as overrides but they cannot breach the system limits.
We should probably allows setting overrides for these default task resource system wide values through helm , have those values exposed as ENV variables in entry point script and pass those from flytectl sandbox/demo --env similar to FLYTE_TIMEOUT env variable .
t
there might be no end in sight for that though, but i kinda agree for this one.
this is the one thing that i’ve often needed to change too, just in running examples and such
i
Also i think this should be an advanced usecase , if the current limits in the charts are not enough for regular use cases then we should bump those in the helm chart directly and get it checkedin.
@great-school-54368 the storage values seems to be very less for sandbox https://github.com/flyteorg/flyte/blob/2762eab96c1a5c90d794e850799eec3a39f547ab/charts/flyte/values.yaml#L457
Copy code
task_resource_defaults:
      # -- Task default resources parameters
      task_resources:
        defaults:
          cpu: 100m
          memory: 200Mi
          storage: 5Mi
        limits:
          cpu: 2
          memory: 1Gi
          storage: 20Mi
          gpu: 1
Not sure if users want to bump up resources for this reason.
Also should we remove the platform config checks which act as cealing in flyteadmin. Thats a question for @acceptable-policeman-57188 once she is back . This would allow the user to just use flytectl to set there resource configuration for any project domain or workflow and not worry about the platform limits but i guess we have these platform limits for a reason to prevent abuse of k8s resources .
a
yes the platform limits were intended to act as a global ceiling since there is no authz implementation atm but we can revisit that if it no longer makes sense!
7 Views
Open in Slack
PreviousNext
Powered by