Madhur Tandon
07/03/2022, 5:19 AMGeorge Odette
07/03/2022, 10:41 AMGeorge Odette
07/03/2022, 10:41 AMGeorge Odette
07/05/2022, 10:43 AMseunggs
07/05/2022, 4:52 PMseunggs
07/05/2022, 5:00 PMseunggs
07/05/2022, 5:00 PMkubectl describe cm flyte-admin-base-config
, I see these valuesseunggs
07/05/2022, 5:01 PMdb.yaml:
----
database:
dbname: flyteadmin
host: postgres
port: 5432
username: postgres
seunggs
07/05/2022, 5:01 PMseunggs
07/05/2022, 5:01 PMserver.yaml:
----
auth:
appAuth:
thirdPartyConfig:
flyteClient:
clientId: flytectl
redirectUri: <http://localhost:53593/callback>
scopes:
- offline
- all
authorizedUris:
- <https://localhost:30081>
- <http://flyteadmin:80>
- <http://flyteadmin.flyte.svc.cluster.local:80>
userAuth:
openId:
baseUrl: <https://accounts.google.com>
clientId: <http://657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com|657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com>
scopes:
- profile
- openid
seunggs
07/05/2022, 5:01 PMseunggs
07/05/2022, 5:03 PM<https://flyte.sidetrek.com/me> (501)
not implemented error in the flyte consoleseunggs
07/05/2022, 5:03 PM"unknown service flyteidl.service.IdentityService"
krishna Yerramsetty
07/05/2022, 5:40 PM@dynamic
def wrapper_task(a: typing.List[int]) -> str:
mapped_out = map_task(a_mappable_task)(a=a).with_overrides(
retries=4,
)
coalesced = coalesce(b=mapped_out)
return coalesced
@task
def a_mappable_task(a: int) -> str:
inc = a + 2
stringified = str(inc)
return stringified
@task
def coalesce(b: typing.List[str]) -> str:
coalesced = "".join(b)
return coalesced
@workflow
def my_map_workflow(a: typing.List[int]) -> str:
return wrapper_task(a=a)
if __name__ == "__main__":
result = my_map_workflow(a=[1, 2, 3, 4, 5])
print(f"{result}")
I am seeing a type mismatch compilation error however at run-time:
User] malformed dynamic workflow, caused by: Collected Errors: 2
Error 0: Code: MismatchingTypes, Node Id: dn1, Description: Variable [o0] (type [collection_type:<collection_type:<simple:STRING > > ]) doesn't match expected type [collection_type:<simple:STRING > ].
Any ideas on why the return is a list of lists from the mapped task? And how do I correctly pass these outputs onto other tasks?Sandra Youssef
Vladimir Lialine
07/06/2022, 12:24 AMBernhard Stadlbauer
07/06/2022, 7:09 AMflyte-core
helm
chart in version 1.0.2
on one of our clusters yesterday, and it came up successfully. Whilst trying to upgrade another cluster today, we did get an an ImagePullBackoff
when trying to pull <http://cr.flyte.org/flyteorg/flyteadmin-release:v1.0.2|cr.flyte.org/flyteorg/flyteadmin-release:v1.0.2>
.
I can also confirm not being able to pull that image locally:
❯ docker pull <http://cr.flyte.org/flyteorg/flyteadmin-release:v1.0.2|cr.flyte.org/flyteorg/flyteadmin-release:v1.0.2>
Error response from daemon: manifest unknown
Has anything changed here recently?Klemens Kasseroller
07/06/2022, 1:28 PMflyte-core-v1.0.2
flyte-deps-v1.0.2
Sonja Ericsson
07/06/2022, 1:37 PMMike Zhong
07/06/2022, 1:51 PMMatheus Moreno
07/06/2022, 2:23 PMDeploying Flyte-deps...
Release "flyte-deps" does not exist. Installing it now.
Error: failed to download "flyteorg/flyte-deps" at version "v1.1.0" (hint: running helm repo update may help)
We already tried removing the .kube
, .cache
and .local
directories, and changing the sandbox version. I have no idea what else we can do.Haytham Abuelfutuh
austin
07/06/2022, 10:31 PMseunggs
07/07/2022, 4:00 AMseunggs
07/07/2022, 4:00 AMseunggs
07/07/2022, 4:01 AM/console/projects/flyteexamples/executions?duration=all
Sandra Youssef
Sandra Youssef
Evan Sadler
07/08/2022, 2:25 PMEvan Sadler
07/11/2022, 4:15 PMspark.jars.packages
to the config (see example below). There is some kind of connection refused error: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused:
.
I imagine this has to do with some configuration that I don’t quite understand in the backend. Any help is much appreciated. I tested it out with another package and it had the same error.
import datetime
import random
from operator import add
import flytekit
from flytekit import Resources, task, workflow
from flytekitplugins.spark import Spark
@task(
task_config=Spark(
# this configuration is applied to the spark cluster
spark_conf={
"spark.driver.memory": "8g",
"spark.jars.repositories": "<https://mmlspark.azureedge.net/maven>",
"spark.jars.packages": "com.microsoft.azure:synapseml_2.12:0.9.5" # adding this causes problems
}
),
limits=Resources(mem="2000M"),
cache_version="1",
)
def hello_spark(partitions: int) -> float:
print("Starting Spark with Partitions: {}".format(partitions))
n = 100000 * partitions
sess = flytekit.current_context().spark_session
count = (
sess.sparkContext.parallelize(range(1, n + 1), partitions).map(f).reduce(add)
)
pi_val = 4.0 * count / n
print("Pi val is :{}".format(pi_val))
return pi_val