Hi, I am a newbie to this Flyte Project, I can contribute for Flyte in documentation.

Hello Flyte team, Ive updated and added a few PRs https://github.com/flyteorg/flytekit/pull/1240 https://github.com/flyteorg/flytekit/pull/1230 and had an initial go at adding an NLP tutorial but not sure why some of the tests are failing https://github.com/flyteorg/flytesnacks/pull/911/files

@Kevin Su, could you please review https://github.com/flyteorg/flytekit/pull/1230 PR?

@Eduardo Apolinario (eapolinario) @Kevin Su, please review https://github.com/flyteorg/flytekit/pull/1236 PR.

Wanted to know that this issue https://github.com/flyteorg/flyte/issues/2908 is for this repo https://github.com/flyteorg/flyteconsole right?

Hey @Niels Bantilan! Regarding https://github.com/flyteorg/flyte/issues/2759,

tf.Module

is the parent class for

tf.keras.layers.Layer

and

tf.keras.Model

(see the note on https://www.tensorflow.org/guide/intro_to_modules page). So we need a type transformer for

tf.Module

. But we have two PRs — https://github.com/flyteorg/flytekit/pull/1242 to fix https://github.com/flyteorg/flyte/issues/2759 and https://github.com/flyteorg/flytekit/pull/1241 to fix https://github.com/flyteorg/flyte/issues/2570 that have the same functionality — a type transformer subclassing

tf.keras.Model

. We’ll need to tell one of them to modify the code. So which PR shall we retain?

Can someone review https://github.com/flyteorg/flytetools/pull/66 PR, please?

Also @Niels Bantilan, regarding https://github.com/flyteorg/flytekit/pull/1240, are there any use cases you could think of?

I am trying to run the script in my flytesnacks PR https://github.com/flyteorg/flytesnacks/pull/911 on a flyte demo cluster to check out the flytedeck plot but i am getting an error - seems like the kubernetes pods are not responding ? (see screenshot below) ive created the demo cluster as per the instructions in the docs https://docs.flyte.org/en/latest/getting_started/index.html#creating-a-demo-flyte-cluster and exported

KUBECONFIG

and

FLYTECTL_CONFIG

environment variable values which were printed out after the cluster was setup. Do i need to login to the kubernetes ui and/or configure anything else ?

Hey team!! I wanted my blog to be reviewed. Here is the link: https://dev.to/srini047/what-is-data-orchestration-4kfb-temp-slug-7953970?preview=44f2a4a274977f9272ea655b1c6839766e9d423bd8ae68c54d7223fba32443a0094b8e7455b66550cb1c5ad94e5d3eab3465b770d3b6de80b2574723

@Eduardo Apolinario (eapolinario) @Kevin Su, can https://github.com/flyteorg/flytekit/pull/1236 be merged? Also Eduardo, do you think

typing.TypeVar

needs to be replaced with

typing.Annotated

in FlyteFile even if it isn’t throwing any mypy errors?

One of the tests on GA is failing due to pytest not being recognized as a command on Windows build (Python 3.7 -spark2) (https://github.com/flyteorg/flytekit/actions/runs/3326738618/jobs/5511139310#step:6:12)

https://github.com/flyteorg/flytekit/pull/1258#issuecomment-1293083159; @Niels Bantilan @Kevin Su @Eduardo Apolinario (eapolinario), would like to know your thoughts.

@Ryan Nazareth @Kevin Su, left two comments on https://github.com/flyteorg/flytekit/pull/1230 PR. Let me know what you think.

@Kevin Su, apologies for the constant pings 😅. Can you take a look at https://github.com/flyteorg/flytekit/pull/1241#issuecomment-1293274305 comment when you find the time?

@FlyteCoreTeam Having another issue when running one of the tasks for the nlp PR. https://github.com/flyteorg/flytesnacks/pull/911 on demo cluster with custom built image I am getting this error for some of the downstream tasks which take the serialised flytefile (

.model

) as input:

Pod failed. No message received from kubernetes.

[fb8f63b27ec284bbfbc8-n3-0] terminated with exit code (137). Reason [Error]. Message:

`tar: Removing leading

/' from member names

See screenshot below. The error seems like its trying to extract something from tar archive but Im not doing this anywhere in my task logic ? Also, think someone else has reported this error as well https://flyte-org.slack.com/archives/CP2HDHKE1/p1664441568396559?thread_ts=1664433734.963769&cid=CP2HDHKE1

@Kevin Su / @Eduardo Apolinario (eapolinario), need a +1: https://github.com/flyteorg/flytekit/pull/1230

@Prafulla Mahindrakar, can I get a +1 for https://github.com/flyteorg/flyte/pull/3033 PR? It fixes the issue you filed.

@Eduardo Apolinario (eapolinario) Would it be a proper way to serialize tensors using pickle?

@Samhita Alla I have one last question, how do I fix lint errors on

make lint

? I'm sorry but I'm not quite sure about it (https://github.com/flyteorg/flytekit/actions/runs/3326738618/jobs/5511140308#step:6:360)

@Samhita Alla I'm done pushing the mentioned changes except for linting issues. I was unable to sign off properly because I messed up my pre-commit hooks, will GA run?

Just checking if this channel will still be open after Hacktoberfest ends on Monday ? I may have PRs outstanding beyond that date, so may still have queries

@Eduardo Apolinario (eapolinario) @Yee @Kevin Su Is it possible to use

LiteralCollection

with multiple literal types? If so, how should the

get_literal_type

method be defined since

collection_type

can only accept a single

LiteralType

? See https://github.com/flyteorg/flytekit/pull/1269/files PR for reference.

Just a note, I’m still working on https://github.com/flyteorg/flytekit/pull/1240/files just been a bit busy these past few weeks. I’ll hopefully get it done before this channel closes 😁

I have filled out the form but haven't received any confirmation mail yet, can i expect any kind of confirmation?

Pushed some changes to https://github.com/flyteorg/flytekit/pull/1240 and made some comments as im struggling a bit with the design (and getting the tests to pass). ☹️

I've pushed the mentioned changes to https://github.com/flyteorg/flytekit/pull/1269/files @Samhita Alla

Hi all, i have a question regarding the type annotations in workflow tasks. I am having an issue in https://github.com/flyteorg/flytekit/pull/1240, where the test for the workflow for

tests/flytekit/unit/extras/tensorflow/record/test_record.py

, which has tasks to generate TFRecordFile and TFRecordDirectory from tf.train.Example outputs and subsequently deserialise back to

tf.data.TFRecordDataset

if required. The bit thats causing an error is

@task
def consume(dataset: Annotated[TFRecordDatasetV2, TFRecordDatasetConfig(name="testing")]):

which, seems to not be recognising the Annotated TFRecordDatasetV2 type although it should be the expected type in the transformers i have registered here flytekit/extras/tensorflow/record.py. I presume this is why it is defaulting to FlytePickle as in the traceback (this is only my assumption and based on some other logic i have seen elsewhere in the codebase). Also, my other test

tests/flytekit/unit/extras/tensorflow/record/test_transformations.py

which is testing the

to_python_value

and

to_literal

logic works fine so i feel something else is an issue which I am finding difficult to spot at the moment.

expected_python_type = typing.Annotated[flytekit.types.pickle.pickle.FlytePickle.__class_getitem__.<locals>._SpecificFormatClass, TFRecordDatasetConfig(compression_type=None, buffer_size=None, num_parallel_reads=None, name='testing')]

    def to_python_value(self, ctx: FlyteContext, lv: Literal, expected_python_type: Type[T]) -> T:
        uri = lv.scalar.blob.uri
        # Deserialize the pickle, and return data in the pickle,
        # and download pickle file to local first if file is not in the local file systems.
        if ctx.file_access.is_remote(uri):
            local_path = ctx.file_access.get_random_local_path()
            ctx.file_access.get_data(uri, local_path, False)
            uri = local_path
        with open(uri, "rb") as infile:
>           data = cloudpickle.load(infile)
E           _pickle.UnpicklingError: invalid load key, '5'.

Hello Flyte core team, once this channel is archived, is there another channel/discord where contributors discuss PRs/issues etc. or is it mainly on github ? Ive noticed the other channels on slack are mainly for end users queries. I intend to tackle a few more issues further down the line (maybe early next year) and propose a few new features (mainly flyte and other cloud services integration), so was wondering which medium is used for discussion.

archived the channel