https://flyte.org logo
Join the conversationJoin Slack
Channels
announcements
ask-the-community
auth
conference-talks
contribute
databricks-integration
datahub-flyte
deployment
ecosystem-unionml
engineeringlabs
events
feature-discussions
flyte-bazel
flyte-build
flyte-console
flyte-deployment
flyte-documentation
flyte-github
flyte-ui-ux
flytekit
flytekit-java
flytelab
great-content
hacktoberfest-2022
helsing-flyte
in-flyte-conversations
introductions
jobs
konan-integration
linkedin-flyte
random
ray-integration
ray-on-flyte
release
scipy-2022-sprint
sig-large-models
workflow-building-ui-proj
writing-w-sfloris
Powered by Linen
ask-the-community
  • k

    Katrina P

    04/21/2022, 4:33 AM
    Hi! I tried joining the office hours tonight but had some connection issues unfortunately 😞. I had reached out previously here . I've tried running the sandbox with the k8s operator for spark instructions again with the most recent release, and I'm still running into the same issues. I've followed the instructions here and tried running the most recent release registered via:
    flytectl register files --config ~/.flyte/config.yaml <https://github.com/flyteorg/flytesnacks/releases/download/v0.3.70/snacks-cookbook-integrations-kubernetes-k8s_spark.tar.gz> --archive -p flytesnacks -d development --version latest
    (I can run the examples locally)
    ++ id -u
    + myuid=0
    ++ id -g
    + mygid=0
    + set +e
    ++ getent passwd 0
    + uidentry=root:x:0:0:root:/root:/bin/bash
    + set -e
    + '[' -z root:x:0:0:root:/root:/bin/bash ']'
    + SPARK_CLASSPATH=':/opt/spark/jars/*'
    + env
    + grep SPARK_JAVA_OPT_
    + sed 's/[^=]*=\(.*\)/\1/g'
    + sort -t_ -k4 -n
    + readarray -t SPARK_EXECUTOR_JAVA_OPTS
    Non-spark-on-k8s command provided, proceeding in pass-through mode...
    + '[' -n '' ']'
    + '[' '' == 2 ']'
    + '[' '' == 3 ']'
    + '[' -n '' ']'
    + '[' -z ']'
    + case "$1" in
    + echo 'Non-spark-on-k8s command provided, proceeding in pass-through mode...'
    + CMD=("$@")
    + exec /usr/bin/tini -s -- pyflyte-execute --inputs <s3://my-s3-bucket/metadata/propeller/flytesnacks-development-kdu1tzd712/n0/data/inputs.pb> --output-prefix <s3://my-s3-bucket/metadata/propeller/flytesnacks-development-kdu1tzd712/n0/data/0> --raw-output-data-prefix <s3://my-s3-bucket/72/kdu1tzd712-n0-0> --checkpoint-path <s3://my-s3-bucket/72/kdu1tzd712-n0-0/_flytecheckpoints> --prev-checkpoint '""' --resolver flytekit.core.python_auto_container.default_task_resolver -- task-module k8s_spark.dataframe_passing task-name create_spark_df
    22/04/21 04:27:54 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    {"asctime": "2022-04-21 04:28:13,716", "name": "flytekit.entrypoint", "levelname": "ERROR", "message": "!! Begin System Error Captured by Flyte !!"}
    {"asctime": "2022-04-21 04:28:13,717", "name": "flytekit.entrypoint", "levelname": "ERROR", "message": "Traceback (most recent call last):\n\n      File \"/opt/venv/lib/python3.8/site-packages/flytekit/exceptions/scopes.py\", line 165, in system_entry_point\n        return wrapped(*args, **kwargs)\n      File \"/opt/venv/lib/python3.8/site-packages/flytekit/core/base_task.py\", line 463, in dispatch_execute\n        new_user_params = self.pre_execute(ctx.user_space_params)\n      File \"/opt/venv/lib/python3.8/site-packages/flytekitplugins/spark/task.py\", line 124, in pre_execute\n        self.sess = sess_builder.getOrCreate()\n      File \"/opt/venv/lib/python3.8/site-packages/pyspark/sql/session.py\", line 228, in getOrCreate\n        sc = SparkContext.getOrCreate(sparkConf)\n      File \"/opt/venv/lib/python3.8/site-packages/pyspark/context.py\", line 392, in getOrCreate\n        SparkContext(conf=conf or SparkConf())\n      File \"/opt/venv/lib/python3.8/site-packages/pyspark/context.py\", line 144, in __init__\n        SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)\n      File \"/opt/venv/lib/python3.8/site-packages/pyspark/context.py\", line 339, in _ensure_initialized\n        SparkContext._gateway = gateway or launch_gateway(conf)\n      File \"/opt/venv/lib/python3.8/site-packages/pyspark/java_gateway.py\", line 108, in launch_gateway\n        raise RuntimeError(\"Java gateway process exited before sending its port number\")\n\nMessage:\n\n    Java gateway process exited before sending its port number\n\nSYSTEM ERROR! Contact platform administrators."}
    {"asctime": "2022-04-21 04:28:13,717", "name": "flytekit.entrypoint", "levelname": "ERROR", "message": "!! End Error Captured by Flyte !!"}
    k
    s
    • 3
    • 8
  • h

    Hampus Rosvall

    04/21/2022, 11:33 AM
    Hey, I am having troubles setting up Flyte as internal facing and use port-forwarding to register workflows to the backend? I am deploying the Ingress as internal facing. But I am getting an error when I hit
    localhost:port
    in the browser after port-forwarding to the console. However, I am able to query the host deployed by the Ingress inside the VPC and get information about e.g., projects.
    Failed to fetch data
    invalid wire type 4 at offset 1
  • h

    Han

    04/22/2022, 10:17 PM
    Quick question. we have people working on a shared server. I have written a workflow that runs locally under my env. But when my colleague tries to run the same workflow on the server, he got the following error
    $ python tasks/run_workflow.py
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/pathlib.py", line 1284, in mkdir
        self._accessor.mkdir(self, mode)
    FileNotFoundError: [Errno 2] No such file or directory: '/tmp/flyte/20220422_220444/sandbox/local_flytekit'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/pathlib.py", line 1284, in mkdir
        self._accessor.mkdir(self, mode)
    FileNotFoundError: [Errno 2] No such file or directory: '/tmp/flyte/20220422_220444/sandbox'
    
    During handling of the above exception, another exception occurred:
    ...
    ...
    PermissionError: [Errno 13] Permission denied: '/tmp/flyte/20220422_220444'
    How do I ensure reproducibility for different local environments?
    e
    k
    • 3
    • 18
  • e

    Erbene Castro

    04/23/2022, 6:57 PM
    Hi, how do I change the demo Flyte admin task resources limits? I used to change those in the kubernetes configmaps but this newer version doesn't have the same configmaps and I can't find the cluster and flyte admin configurations.
    k
    • 2
    • 5
  • r

    Rahul Mehta

    04/24/2022, 5:16 PM
    Hi, quick question -- where should the s3 credentials be set in values-eks.yaml? I see the references to `rdsHost`/`dbPassword` in the chart values (& will use an external secret instead of hard-coding them in there), but couldn't find the equivalent for S3. Also, we're looking to deploy flyte into a kops managed cluster (not quite on EKS yet). Would it be better to use values-eks.yaml or values.yaml as a starting point?
    k
    • 2
    • 2
  • r

    Rahul Mehta

    04/24/2022, 11:00 PM
    One more question that may be a bit naive: if there is a workflow in a project
    test
    , domain
    development
    & a version
    v1
    , how would I "promote" that workflow to
    staging
    in the same project w/ the same version? With what I've seen in the flytekit API, it only seems possible to register/create a workflow based on what's in the source code (ie. the decorated workflow function). Is it possible to "copy"/promote a workflow to a different domain only w/ the API? I also definitely could be misunderstanding how the domains abstraction is supposed to be applied to enable promotion/a proper CI/CD approach to pipelines
    k
    • 2
    • 12
  • h

    Hampus Rosvall

    04/25/2022, 11:29 AM
    Hey, what is the preferred way of providing project specific IAM Roles to a workflow? I am trying to run a workflow using an IAM role with permissions suitable for only that project. In my launch plan I enter some role e.g., as in the picture below but it doesn’t seem to be picked up when I describe the Pod. Does your default role e.g.,
    eks-flyte-user-rule
    assume the IAM Role you provide in the launch plan, or is it assumed by some service account that gets created?
    Environment:
          ...
          AWS_ROLE_ARN:                       arn:aws:iam::<account_id>:role/eks-flyte-user-role
    p
    k
    m
    • 4
    • 45
  • c

    Chase Grisham

    04/25/2022, 8:23 PM
    Does anyone know if the
    --pre
    flag applies to the sandbox the way to does to demo? ie: does
    flytectl sandbox start --source . --pre
    download the same flyte version as
    flytectl demo start --pre
    ?
    e
    • 2
    • 1
  • c

    Chase Grisham

    04/26/2022, 3:53 PM
    Having trouble running the
    k8s_spark.<http://dataframe_passing.my|dataframe_passing.my>_smart_structured_dataset
    example. I've been able to run the other spark example
    pyspark_pi
    . I've set up the K8s Operator, built the Docker Image based on the Dockerfile in the
    cookbook/integrations/kubernetes/k8s
    folder. Getting this error about s3 :
    [3/3] currentAttempt done. Last Error: SYSTEM::Traceback (most recent call last):
    
          File "/opt/venv/lib/python3.8/site-packages/flytekit/exceptions/scopes.py", line 165, in system_entry_point
            return wrapped(*args, **kwargs)
          File "/opt/venv/lib/python3.8/site-packages/flytekit/core/base_task.py", line 527, in dispatch_execute
            raise TypeError(
    
    Message:
    
        Failed to convert return value for var o0 for function k8s_spark.dataframe_passing.create_spark_df with error <class 'py4j.protocol.Py4JJavaError'>: An error occurred while calling o41.parquet.
    : org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "s3"
    	at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3443)
    	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3466)
    	at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)
    	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3574)
    	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3521)
    	at org.apache.hadoop.fs.FileSystem.get(FiInternal(DataFrameWriter.scala:355)
    	at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:239)
    	at org.apache.spark.sql.DataFrameWriter.parquet(DataFrameWriter.scala:781)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
    	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
    	at py4j.Gateway.invoke(Gateway.java:282)
    	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    	at py4j.commands.CallCommand.execute(CallCommand.java:79)
    	at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
    	at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
    	at java.lang.Thread.run(Thread.java:748)
    
    
    SYSTEM ERROR! Contact platform administrators.
    k
    y
    +4
    • 7
    • 100
  • h

    Han

    04/26/2022, 8:24 PM
    Couple of problems I ran into while I try to go through this env setup tutorial. 1. I am seeing
    <http://googletagmanager.com|googletagmanager.com>
    and
    <http://gstatic.com|gstatic.com>
    for flytectl local sandbox dashboard links. Are they necessary? 2. What's the purpose of
    pyflyte
    ? 3. cont'd
    y
    k
    +2
    • 5
    • 29
  • h

    Hafsa Junaid

    04/27/2022, 3:04 AM
    Hey! No idea why pods are not successfully deployed, Following doc as it is https://docs.flyte.org/en/latest/deployment/gcp/manual.html#deployment-gcp-manual
    h
    k
    p
    • 4
    • 20
  • s

    Slackbot

    04/29/2022, 2:24 AM
    This message was deleted.
    a
    • 2
    • 1
  • h

    Hafsa Junaid

    04/29/2022, 3:03 AM
    Hey Team!!! Hello @Haytham Abuelfutuh As per the https://docs.flyte.org/en/latest/deployment/gcp/manual.html#deployment-gcp-manual docs command, the connection with psql is verified. But these logs kubectl logs -n flyte svc/flyteadmin -c run-migrations shows that connection ain't built for flyteadmin. The pods are failing and UI is not accessible. Any guide?
    y
    h
    • 3
    • 8
  • s

    Stefan Avesand

    05/04/2022, 12:32 PM
    Good morning team! Trying to use the new StructuredDataset type to read data from BQ but can’t figure it out.
    sd = StructuredDataset(uri='<bq://sp-one-model.quarterly>_forecast_2022F1.premium_revenue_tab_input_vat')
    sd.open(pd.DataFrame).all()
    
    AttributeError: 'NoneType' object has no attribute 'uri'
    s
    k
    a
    • 4
    • 25
  • s

    seunggs

    05/05/2022, 7:31 AM
    Hi! I have a quick question - I’m trying to create a flyte task from a web UI. Are there any tutorials or examples I can reference to do this? Any guidance on this would be appreciated!
    k
    d
    • 3
    • 20
  • r

    Robin Kahlow

    05/06/2022, 12:32 PM
    Hey! I was following Flyte's auth setup with Google up to (not including) https://docs.flyte.org/en/latest/deployment/cluster_config/auth_setup.html#oauth2-authorization-server. In Google Cloud Console's OAuth consent screen settings, the user type is set to external, the publishing status is testing and some users are added as test users. Am I correct in assuming only those test users will be able to log in on Flyte's console and not any other users? Thank you!
    p
    • 2
    • 2
  • s

    seunggs

    05/06/2022, 9:13 PM
    Hi, is there some documentation on running a part of the workflow? If there are 5 tasks in a workflow for example and I’d like to run the first 3 tasks to view the output of task #3, how could I do that? Also, where are the intermediate outputs for each task saved? I’m trying to visualize the intermediate task outputs and not sure if it’ll be performant enough to be near real-time for large datasets?
    y
    k
    • 3
    • 44
  • m

    Michael Cheng Jan Kao

    05/11/2022, 1:15 AM
    hi, is there a recommended way of updating the resources after it has been set? I am currently doing this, wondering if this is acceptable and have any unintended side effects.
    @task(requests=Resources(cpu="1", mem="4Gi"), limits=Resources(cpu="2", mem="32Gi"))
    def hello_world():
        print('hello world')
    
    hello_world._resources = ResourceSpec(requests=Resources(cpu='1', mem='16Gi', gpu=None, storage=None, ephemeral_storage=None), limits=Resources(cpu='2', mem='32Gi', gpu=None, storage=None, ephemeral_storage=None))
    k
    • 2
    • 6
  • r

    Robin Kahlow

    05/11/2022, 12:31 PM
    Hello again, is there a simple way to install specified Python packages before running a task without creating a new docker image?
    k
    • 2
    • 7
  • s

    seunggs

    05/11/2022, 11:41 PM
    Hi, is there a way to use a different ingress service? I’m using
    emissary-ingress
    and I was wondering if you had a reference implementation for that instead of using Amazon Loadbalancer Ingress Controller?
    k
    • 2
    • 4
  • y

    yujinlee

    05/13/2022, 2:16 AM
    Hi, may I ask a question here? Is there a way to get private docker image from Docker registry to register task? I wonder how to login docker registry in flyte to download private image.
    y
    k
    a
    • 4
    • 16
  • r

    Robin Kahlow

    05/13/2022, 2:40 PM
    I want to change the log url displayed in flyte console to where our logs are stored, so I added the entry below to the values. The url does change but it seems like podName is always empty so the url is https://my.papertrailapp.com/systems//events, is there something else I need to do to make the podName variable work? I just looked at the defaults which uses podName too https://docs.flyte.org/en/stable/deployment/cluster_config/flytepropeller_config.html#logs-logs-logconfig
    task_logs:
        plugins:
          logs:
            kubernetes-enabled: true
            kubernetes-template-uri: <https://my.papertrailapp.com/systems/{{> .podName }}/events
    d
    • 2
    • 25
  • s

    Sebastia Agramunt

    05/17/2022, 7:14 PM
    Hi, newbie question… Do I really need kubernetes to run Flyte?. I need to run ML workflows on docker images for a small (but growing) project.
    y
    • 2
    • 6
  • a

    Attila Nagy

    05/18/2022, 7:51 AM
    Hi, Coming from https://docs.flyte.org/en/latest/deployment/aws/opta.html. I would like to customize the resources made by Opta (for example the parameters of AWS ASG worker nodes, like EC2 key pair name, auto public IP, or VPC ACLs), but can't see how to do that and where are they defined. Any ideas/pointers?
    k
    y
    j
    • 4
    • 35
  • y

    yujinlee

    05/20/2022, 4:02 AM
    Hi, How do I delete a registered task or workflow in a specific project-domain? First, I checked the CLI help. If you look at the
    flytectl --help
    , it says that you can delete tasks,worfklows.
    delete      Terminates/deletes various Flyte resources such as tasks, workflows, launch plans, executions, and projects.
    But in the available command of
    delete
    , I don't see an option to actually delete tasks or workflows.
    Usage:
      flytectl delete [command]
    
    Available Commands:
      cluster-resource-attribute Deletes matchable resources of cluster attributes.
      execution                  Terminates/deletes execution resources.
      execution-cluster-label    Deletes matchable resources of execution cluster label.
      execution-queue-attribute  Deletes matchable resources of execution queue attributes.
      plugin-override            Deletes matchable resources of plugin overrides.
      task-resource-attribute    Deletes matchable resources of task attributes.
      workflow-execution-config  Deletes matchable resources of workflow execution config.
    k
    j
    • 3
    • 6
  • h

    HaoboGu

    05/20/2022, 6:45 AM
    Hi, I'm trying to test my workflow on local sandbox, but after I executed `flytectl sandbox start --source .``, I got the following message in container:
  • h

    HaoboGu

    05/20/2022, 6:45 AM
    Release "flyte" does not exist. Installing it now.
    
    W0520 06:19:59.185843     736 warnings.go:70] batch/v1beta1 CronJob is deprecated in v1.21+, unavailable in v1.25+; use batch/v1 CronJob
    
    W0520 06:21:41.614777     736 warnings.go:70] batch/v1beta1 CronJob is deprecated in v1.21+, unavailable in v1.25+; use batch/v1 CronJob
    
    NAME: flyte
    
    LAST DEPLOYED: Fri May 20 06:19:57 2022
    
    NAMESPACE: flyte
    
    STATUS: deployed
    
    REVISION: 1
    
    TEST SUITE: None
    
    Waiting for Flyte to become ready...
    
    Terminated
    
    Timed out while waiting for the flyteadmin rollout to be created
    p
    k
    • 3
    • 22
  • h

    HaoboGu

    05/20/2022, 6:46 AM
    Then the container just quits.. does anybody know what's wrong with this sandbox? Thanks!
  • k

    Katrina P

    05/20/2022, 6:48 PM
    I'm still working on figuring out how to deploy Flyte into our company's k8s infrastructure. I noticed this in the docs https://github.com/flyteorg/flyte/tree/v1.0.0/kustomize/overlays/eks#how-to-build-your-overlay -- anyone here have experience doing this? I'm having to install Flyte using kustomize into a shared services cluster and trying to figure out the smartest way to do this (pretty new to kustomize!)
    k
    y
    • 3
    • 14
  • r

    Ruksana Kabealo

    05/20/2022, 7:37 PM
    Hi everyone! I'm trying to follow the "Deploying Workflows - Registration" doc here: https://docs.flyte.org/projects/cookbook/en/stable/auto/deployment/deploying_workflows.html I've built the docker image that contains my code. I'm trying to serialize it with pyflyte. I'm running the following command:
    pyflyte -c /root/.flyte/config.yaml --pkgs delaine serialize --in-container-config-path /root/config.yaml --local-source-root ./ --image delaieine:2 workflows -f _pb_output/
    And getting the following error traceback:
    Traceback (most recent call last):
     File "/root/delaieine/bin/pyflyte", line 8, in <module>
      sys.exit(main())
     File "/root/delaieine/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
      return self.main(*args, **kwargs)
     File "/root/delaieine/lib/python3.9/site-packages/click/core.py", line 1055, in main
      rv = self.invoke(ctx)
     File "/root/delaieine/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
     File "/root/delaieine/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
     File "/root/delaieine/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
      return ctx.invoke(self.callback, **ctx.params)
     File "/root/delaieine/lib/python3.9/site-packages/click/core.py", line 760, in invoke
      return __callback(*args, **kwargs)
     File "/root/delaieine/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
      return f(get_current_context(), *args, **kwargs)
     File "/root/delaieine/lib/python3.9/site-packages/flytekit/clis/sdk_in_container/serialize.py", line 139, in workflows
      serialize_all(
     File "/root/delaieine/lib/python3.9/site-packages/flytekit/exceptions/scopes.py", line 160, in system_entry_point
      return wrapped(*args, **kwargs)
     File "/root/delaieine/lib/python3.9/site-packages/flytekit/clis/sdk_in_container/serialize.py", line 60, in serialize_all
      image_config=ImageConfig.auto(config_file, img_name=image),
     File "/root/delaieine/lib/python3.9/site-packages/flytekit/configuration/__init__.py", line 250, in auto
      for k, v in _internal.Images.get_specified_images(config_file).items()
     File "/root/delaieine/lib/python3.9/site-packages/flytekit/configuration/internal.py", line 25, in get_specified_images
      image_names = cfg.legacy_config.options("images")
    AttributeError: 'NoneType' object has no attribute 'options'
    NoneType leads me to believe it's not liking either the config or the docker image. I've checked the image/tag spelling. That's fine. The config exists, the path is right, and it looks like this:
    admin:
     # For GRPC endpoints you might want to use dns:///flyte.myexample.com
     endpoint: localhost:30081
     authType: Pkce
     insecure: true
    I'm using flytekit=1.0.2 and flytectl=0.5.25 I'm not sure how to troubleshoot this. Any help would be much appreciated!
    y
    • 2
    • 13
Powered by Linen
Title
r

Ruksana Kabealo

05/20/2022, 7:37 PM
Hi everyone! I'm trying to follow the "Deploying Workflows - Registration" doc here: https://docs.flyte.org/projects/cookbook/en/stable/auto/deployment/deploying_workflows.html I've built the docker image that contains my code. I'm trying to serialize it with pyflyte. I'm running the following command:
pyflyte -c /root/.flyte/config.yaml --pkgs delaine serialize --in-container-config-path /root/config.yaml --local-source-root ./ --image delaieine:2 workflows -f _pb_output/
And getting the following error traceback:
Traceback (most recent call last):
 File "/root/delaieine/bin/pyflyte", line 8, in <module>
  sys.exit(main())
 File "/root/delaieine/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
  return self.main(*args, **kwargs)
 File "/root/delaieine/lib/python3.9/site-packages/click/core.py", line 1055, in main
  rv = self.invoke(ctx)
 File "/root/delaieine/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
  return _process_result(sub_ctx.command.invoke(sub_ctx))
 File "/root/delaieine/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
  return _process_result(sub_ctx.command.invoke(sub_ctx))
 File "/root/delaieine/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
  return ctx.invoke(self.callback, **ctx.params)
 File "/root/delaieine/lib/python3.9/site-packages/click/core.py", line 760, in invoke
  return __callback(*args, **kwargs)
 File "/root/delaieine/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
  return f(get_current_context(), *args, **kwargs)
 File "/root/delaieine/lib/python3.9/site-packages/flytekit/clis/sdk_in_container/serialize.py", line 139, in workflows
  serialize_all(
 File "/root/delaieine/lib/python3.9/site-packages/flytekit/exceptions/scopes.py", line 160, in system_entry_point
  return wrapped(*args, **kwargs)
 File "/root/delaieine/lib/python3.9/site-packages/flytekit/clis/sdk_in_container/serialize.py", line 60, in serialize_all
  image_config=ImageConfig.auto(config_file, img_name=image),
 File "/root/delaieine/lib/python3.9/site-packages/flytekit/configuration/__init__.py", line 250, in auto
  for k, v in _internal.Images.get_specified_images(config_file).items()
 File "/root/delaieine/lib/python3.9/site-packages/flytekit/configuration/internal.py", line 25, in get_specified_images
  image_names = cfg.legacy_config.options("images")
AttributeError: 'NoneType' object has no attribute 'options'
NoneType leads me to believe it's not liking either the config or the docker image. I've checked the image/tag spelling. That's fine. The config exists, the path is right, and it looks like this:
admin:
 # For GRPC endpoints you might want to use dns:///flyte.myexample.com
 endpoint: localhost:30081
 authType: Pkce
 insecure: true
I'm using flytekit=1.0.2 and flytectl=0.5.25 I'm not sure how to troubleshoot this. Any help would be much appreciated!
y

Yee

05/20/2022, 8:14 PM
very sorry
this is a bug in flytekit’s attempt to handle both the .ini style of config file that is Python’s default, as well as the yaml style config that is shared with the
flytectl
tool.
some configuration entries though don’t make sense to have in the yaml file - image is one of them. the reason is that this is something that is set at compilation time (when your code is serialized into protobuf)
so we should just exclude that here.
https://github.com/flyteorg/flytekit/pull/1016
cc @Eduardo Apolinario (eapolinario)
r

Ruksana Kabealo

05/20/2022, 9:20 PM
Just to be clear, by "excluding that here" you mean not giving pyflyte a config at all correct? So no -c and no --in-container-config-path?
y

Yee

05/20/2022, 9:46 PM
oh no sorry
gimme a bit
https://pypi.org/project/flytekit/1.0.3/
okay - patched into this version. would you mind bumping the version and trying again please? sorry for the bug.
👍 1
r

Ruksana Kabealo

05/23/2022, 1:59 PM
@Yee Will do! Thank you!
@Yee This worked! Thanks so much!
❤️ 1
View count: 6