https://flyte.org logo
Join Slack
Powered by
Hi, All. I am seeing an issue I have not seen befo...
# ask-the-community
l
Hi, All. I am seeing an issue I have not seen before when fast registering locally. The only difference between previous successful registration and this failure is a bump to flytekit 1.12.0 from 1.11.0 I see 
Successfully serialized 39 flyte objects
, but then 
Error: unknown type <nil>
and for the specific workflow I see
Copy code
| /tmp/workflows/04_flyte.my_package.abc.workflow_xyz_2.pb                                     
| Failed  | Error hydrating spec due to unknown type <nil> |
Any tips on how to debug this? Thanks in advance!
I looked for similar issues on github and haven’t seen anything, but I do know that 1.12 was just released a few days ago in case that is the culprit.
I’m going to try to remove @dataclass_json from the types used in that workflow since they contain a FlyteFile element to see if that works as a response to this important note
If you’re using Flytekit version >= v1.11.1, you don’t need to decorate with 
@dataclass_json
or inherit from Mashumaro’s 
DataClassJSONMixin
.
found here.
No luck with ^
k
Ohh can you downgrade
It might be 1.12
Cc @Eduardo Apolinario (eapolinario) do you know
e
@Len Strnad, can you give more details about the code you're trying to register? For example, the overall shape of the objects you're trying to register. Happy to jump on a call to debug this.
l
I have pinned us to 1.11, but obviously less ideal. I'm stoked about some changes in 1.12.
@Eduardo Apolinario (eapolinario) I tried to dive in here. My hunch for now was dataclasses with optional types and defaults of None used in an input for a task with map_task. I tried to build an mre, but ran out of time. I'll try again tomorrow and get back with you!
e
Thank you. Let's sync tomorrow.
l
Found an MRE! I am just working in minimizing the example as much as possible to narrow down.
Copy code
from flytekit import task, workflow, map_task
from dataclasses import dataclass


@dataclass
class MyOutputs:
    b: str = "anything"


@task
def test_task(i: int) -> MyOutputs:
    return MyOutputs()


@workflow
def test_workflow2() -> list[MyOutputs]:
    return map_task(test_task)(i=[1, 2, 3])
Seems to do it for me. It looks like there wasn’t a test for map_task in the pr that touched some of this here: https://github.com/flyteorg/flytekit/pull/2279?
Happy to make an issue if someone else can verify this is a valid MRE.
e
Great, thank you! I’ll take a look shortly.
@Len Strnad, I'm unable to repro this. I tried your example is working for both array node map tasks and legacy map tasks on latest flytekit. How are you registering? Can you confirm which version of the Flyte you're running in the backend too?
l
We fast register with the command below:
Copy code
CMD="
: \
&& mkdir -p ${FLYTE_TEMPDIR} \
&& pyflyte -k ${PROJECT_NAME} -k flyte.${PROJECT_NAME} serialize --image ${DOCKER_IMAGE_NAME}:${DOCKER_VERSION_TAG} fast workflows -f ${FLYTE_TEMPDIR} \
&& flytectl --admin.endpoint ${FLYTE_HOST} register files ${FLYTE_TEMPDIR}/* --project=${PROJECT_SLUG} --domain=${FLYTE_DOMAIN} --version=${FLYTE_VERSION} \
&& :
" ;
As for looking into the flyte version on the backend, finding one of the pod components and inspecting the labels in k9s I see 
<http://helm.sh/chart=flyte-core-v1.11.0|helm.sh/chart=flyte-core-v1.11.0>
so likely 1.11
Up to this point, we haven’t seen a need to keep versions matching between flyte core and flytekit if I recall correctly. Do these versions need to always match?
e
> Do these versions need to always match? nope, they don't. I was trying to narrow down the discrepancies between our test environments. Let me see what I can find out.
@Len Strnad, can you also confirm which version of 
flytectl
you're running?
l
Copy code
root@len-small-cpu:/src# flytectl version
INFO[0000] [0] Couldn't find a config file []. Relying on env vars and pflags. 

 A new release of flytectl is available: 0.6.35 → v0.8.18 
To upgrade, run: flytectl upgrade 
<https://github.com/flyteorg/flytectl/releases/tag/v0.8.18> 

{
  "App": "flytectl",
  "Build": "bd00706",
  "Version": "0.6.35",
  "BuildTime": "2024-05-10 20:53:41.628788929 +0000 UTC m=+0.019757206"
}
Upgrading to 0.8.18 now to test. Took me a sec to find where we install this--in our dockerfile.
It registers with the upgrade. I guess we fell behind there. I’ll refrain from creating an issue.
Let me know if there is any other valuable info you might want, but we will just plan to upgrade flytectl. Thanks for pointing that out!
e
great. This explains it. Map tasks were rewritten and under the covers they are now implemented as a new node type, which was only implemented in flytectl in 0.6.42.
l
I can't reproduce it too, does this related to 
FlyteFile
or something?
image.png
I've tested the example with latest flyte sandbox release and latest flytekit image.
@Len Strnad @Eduardo Apolinario (eapolinario) Hi, I can help this is there's any follow-up, does this error not related to flytekit?
e
No, wait, @L godlike, there's no followup. This was caused by an old version of flytectl that didn't have the code necessary to parse the new array node map tasks.
l
Thank you!!
s
Would anyone know how to install flytectl for arch linux?
k
@Srishti Chouhan does this not work 
curl -sL <https://ctl.flyte.org/install> | bash
s
I was able to do it with: yay -S flytectl-bin
13 Views
Open in Slack
PreviousNext
Powered by