Rahul Mehta
09/22/2022, 1:57 AM[1,2,3]
, can I pass each item to a map task along with some other fixed inputs that are common across all tasks). All the examples in the docs only include tasks w/ a single argument
Relatedly, is there a way to use a workflow as part of a map task (ie. map a sub-workflow over a sequence of inputs)
Edit: guess not
Map tasks only accept python function tasks with 0 or 1 inputs
Sanjay Chouhan
09/22/2022, 6:38 AMadmin:
# For GRPC endpoints you might want to use dns:///flyte.myexample.com
endpoint: dns:///a4ad903c61##################.<http://us-west-1.elb.amazonaws.com:80|us-west-1.elb.amazonaws.com:80>
authType: Pkce
insecure: true
logger:
show-source: true
level: 0
storage:
connection:
access-key: minio
auth-type: accesskey
disable-ssl: true
endpoint: <http://a093eb##############.us-west-1.elb.amazonaws.com:9001/>
region: us-east-1
secret-key: miniostorage
type: minio
container: "my-s3-bucket"
enable-multicontainer: true
Why it's checking in localhost when I have mentioned the AWS ingress URL?Sathish kumar Venkatesan
09/22/2022, 10:51 AMfrom flytekit import kwtypes, workflow
from flytekitplugins.snowflake import SnowflakeConfig, SnowflakeTask
snowflake_task_no_io = SnowflakeTask(
name="sql.snowflake.no_io",
inputs={},
query_template="select * from TEST_DEV.IDENTITY.TEST_123 ;",
output_schema_type=None,
task_config=SnowflakeConfig(
account="xxxxx.us-east-1",
database="TEST_DEV",
schema="IDENTITY",
warehouse="DEMO_WH",
),
)
@workflow
def no_io_wf():
return snowflake_task_no_io()
KS Tarun
09/22/2022, 12:08 PM@task(container_image="{{.image.trainer.fqn }}:{{.image.trainer.version}}")
My sandbox.config looks like this:
[images]
trainer = <http://ghcr.io/flyteorg/flytecookbook:core-latest|ghcr.io/flyteorg/flytecookbook:core-latest>
predictor = <http://ghcr.io/flyteorg/flytecookbook:pima_diabetes-d4838f0f5e39a21f845a93b9e3375a675bd75eaa|ghcr.io/flyteorg/flytecookbook:pima_diabetes-d4838f0f5e39a21f845a93b9e3375a675bd75eaa>
I'm getting this error:
raise AssertionError(f"Image Config with name {name} not hound in the configuration")
AssertionError:Image Config with name trainer not found in the configuration
Any suggestions about this ?Katrina P
09/22/2022, 9:02 PMseunggs
09/23/2022, 12:24 AMflyte-user-role
(which has full s3 access) attached as an annotation still gives me PutObject access denied error?seunggs
09/23/2022, 12:25 AMseunggs
09/23/2022, 12:25 AMRobin Kahlow
09/23/2022, 11:23 AMRobin Kahlow
09/23/2022, 11:37 AMarg: ""
), rather than storing an empty string. That causes it to use the default of the field rather than using an empty string.Rupsha Chaudhuri
09/23/2022, 4:41 PMobject [my_domainctw5x2vdnkvd-n0-0] terminated in the background, manually
Ankit Goyal
09/23/2022, 8:07 PMseunggs
09/23/2022, 10:38 PMerror: "size of name exceeded length 20 : my-project.workflows.04b2ed0b-434b-4d7b-9220-90606fdc2afc.test_wf_1"
. What’s strange is that this seems to run fine from the flyte dashboard ui?seunggs
09/23/2022, 10:40 PMseunggs
09/23/2022, 10:43 PMseunggs
09/23/2022, 10:44 PM{
project: "shelly-robotics-bidepal-robot",
domain: "development",
name: "my-project.workflows.04b2ed0b-434b-4d7b-9220-90606fdc2afc.test_wf_1",
spec: {
launch_plan: {
project: "shelly-robotics-bidepal-robot",
domain: "development",
name: "my-project.workflows.04b2ed0b-434b-4d7b-9220-90606fdc2afc.test_wf_1",
version: "0.0.1",
},
inputs: {
x: 1,
},
},
};
Maarten T
09/26/2022, 8:12 AMfloodcast_model = ContainerTask(
name="floodcast-docker",
input_data_dir="/input",
output_data_dir="/output",
inputs=kwtypes(rainfall=List[FlyteFile]),
outputs=kwtypes(floodmaps=List[FlyteFile]),
image="<myimage>",
command=["/env/bin/python", "run.py"])
Sebastian
09/26/2022, 3:03 PMflytectl update launchplan ... -- version $GITSHA --activate >> --archive-old <<
.
This seems like a basic use case but I have not found any resources on it. Please let me know if I overlooked anythingRobert Everson
09/26/2022, 7:20 PMEventSinkError
that on some workflows appears to occur occasionally, but on others happens all the time? We moved to a new k8s cluster for our flyteadmin instance (and from 3 instances to 1) and the timing of that seems to mostly line up with this error starting, but after checking the flyteadmin logs, I can see propeller sending a CreateWorkflowEvent
and it getting successfully registered, but propeller will still throw an
EventRecordingFailed: failed to record node event, caused by: EventSinkError: Error sending event, caused by [rpc error: code = Unavailable desc = upstream connect error or disconnect/reset before headers. reset reason: protocol error]
Sebastian
09/27/2022, 5:42 AMlaunch_plan.LaunchPlan.get_or_create(
workflow=wf,
name="my_wf_prod", # prod specific
schedule=CronSchedule(schedule="1 0 * * *"),
default_inputs={
"execution_date": datetime.now(), # aside: this does not work. How do I provide an execution date?
},
fixed_inputs={
"out_path": consts.OUT_S3_PATH_PROD # prod specific vars
},
)
there is also a copy of this launch plan for the dev env, so that I can plug in the dev env vars. But this is silly and requires two copies. How can I parameterize workflows with something like env vars so I can vary them between prod and dev?fei sun
09/27/2022, 6:51 AMflytectl demo start
finished successfully, it’s able to open flyte webui which is localhost:30080, but pyflyte remote run failed with the following error message:fei.sun@MacBook-Pro cookbook % pyflyte run --remote core/flyte_basics/hello_world.py my_wf
E0927 14:46:18.763713000 4366484864 <http://ssl_transport_security.cc:1495]|ssl_transport_security.cc:1495]> Handshake failed with fatal error SSL_ERROR_SSL: error:100000f7:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER.
E0927 14:46:18.765990000 4366484864 <http://ssl_transport_security.cc:1495]|ssl_transport_security.cc:1495]> Handshake failed with fatal error SSL_ERROR_SSL: error:100000f7:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER.
{"asctime": "2022-09-27 14:46:18,781", "name": "flytekit.cli", "levelname": "ERROR", "message": "Non-auth RPC error <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"failed to connect to all addresses\"\n\tdebug_error_string = \"{\"created\":\"@1664261178.780964000\",\"description\":\"Failed to pick subchannel\",\"file\":\"src/core/ext/filters/client_channel/client_channel.cc\",\"file_line\":3261,\"referenced_errors\":[{\"created\":\"@1664261178.780963000\",\"description\":\"failed to connect to all addresses\",\"file\":\"src/core/lib/transport/error_utils.cc\",\"file_line\":167,\"grpc_status\":14}]}\"\n>, sleeping 200ms and retrying"}
{"asctime": "2022-09-27 14:46:18,986", "name": "flytekit.cli", "levelname": "ERROR", "message": "Non-auth RPC error <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"failed to connect to all addresses\"\n\tdebug_error_string = \"{\"created\":\"@1664261178.985906000\",\"description\":\"Failed to pick subchannel\",\"file\":\"src/core/ext/filters/client_channel/client_channel.cc\",\"file_line\":3261,\"referenced_errors\":[{\"created\":\"@1664261178.985905000\",\"description\":\"failed to connect to all addresses\",\"file\":\"src/core/lib/transport/error_utils.cc\",\"file_line\":167,\"grpc_status\":14}]}\"\n>, sleeping 400ms and retrying"}
Traceback (most recent call last):
File "/Users/fei.sun/Library/Python/3.8/bin//pyflyte", line 8, in <module>
sys.exit(main())
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/flytekit/clis/sdk_in_container/run.py", line 539, in _run
remote_entity = remote.register_script(
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/flytekit/remote/remote.py", line 596, in register_script
upload_location, md5_bytes = fast_register_single_script(
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/flytekit/tools/script_mode.py", line 113, in fast_register_single_script
upload_location = create_upload_location_fn(content_md5=md5)
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/flytekit/clients/friendly.py", line 998, in get_upload_signed_url
return super(SynchronousFlyteClient, self).create_upload_location(
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/flytekit/clients/raw.py", line 41, in handler
return fn(*args, **kwargs)
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/flytekit/clients/raw.py", line 854, in create_upload_location
return self._dataproxy_stub.CreateUploadLocation(create_upload_location_request, metadata=self._metadata)
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/Users/fei.sun/Library/Python/3.8/lib/python/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"created":"@1664261179.392327000","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3261,"referenced_errors":[{"created":"@1664261179.392310000","description":"failed to connect to all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":167,"grpc_status":14}]}"
fei sun
09/27/2022, 6:55 AMTamis van der Laan
09/27/2022, 8:39 AMTamis van der Laan
09/27/2022, 8:40 AMTamis van der Laan
09/27/2022, 8:42 AMSamhita Alla
09/27/2022, 11:57 AMHampus Rosvall
09/28/2022, 6:17 AMAnthony
09/28/2022, 8:31 AMWorkflow[flyte-anti-fraud-ml:development:app.workflow.main_flow] failed. RuntimeExecutionError: max number of system retry attempts [31/30] exhausted. Last known status message: failed at Node[n0]. RuntimeExecutionError: failed during plugin execution, caused by: error file @[<s3://my-s3-bucket/metadata/propeller/flyte-anti-fraud-ml-development-f31c365f02c114639b00/n0/data/0/error.pb>] is too large [28775519] bytes, max allowed [10485760] bytes
I added max-output-size-bytes
params to the flyte-propeller-config
and wait to apply all changes before re-submitting a new task.
kubectl edit configmap -n flyte flyte-propeller-config
My propeller section of flyte-propeller-config
looks like:
core.yaml: |
manager:
pod-application: flytepropeller
pod-template-container-name: flytepropeller
pod-template-name: flytepropeller-template
propeller:
max-output-size-bytes: 52428800
downstream-eval-duration: 30s
enable-admin-launcher: true
leader-election:
enabled: true
lease-duration: 15s
lock-config-map:
name: propeller-leader
namespace: flyte
renew-deadline: 10s
retry-period: 2s
limit-namespace: all
max-workflow-retries: 3
metadata-prefix: metadata/propeller
metrics-prefix: flyte
prof-port: 10254
Task configuration has been set up via kubectl -n flyte edit cm flyte-admin-base-config
storage.yaml: |
storage:
type: minio
container: "my-s3-bucket"
stow:
kind: s3
config:
access_key_id: minio
auth_type: accesskey
secret_key: miniostorage
disable_ssl: true
endpoint: <http://minio.flyte.svc.cluster.local:9000>
region: us-east-1
signedUrl:
stowConfigOverride:
endpoint: <http://localhost:30084>
enable-multicontainer: false
limits:
maxDownloadMBs: 50
task_resource_defaults.yaml: |
task_resources:
defaults:
cpu: 1
memory: 3000Mi
storage: 100Mi
limits:
cpu: 4
gpu: 1
memory: 3Gi
storage: 500Mi
Also changing maxDownloadMBs
didn’t change the situation
Changing cache max_size_mbs
in flyte-propeller-config
from 0 to some custom value also not working:
cache.yaml: |
cache:
max_size_mbs: 100
target_gc_percent: 70
I ty to change different time with different params but the error was arising during each new executions.
I saw that none of max-output-size-bytes
or max-workflow-retries
(changed from 30 --> 3) are passed to the workflow execution:
RuntimeExecutionError: max number of system retry attempts [31/30] exhausted...
error file @[<s3://my-s3-bucket/metadata/propeller/flyte-anti-fraud-ml-development-f31c365f02c114639b00/n0/data/0/error.pb>] is too large [28775519] bytes, max allowed [10485760] bytes...
Hereafter are my cli steps to create a new execution:
- kubectl -n flyte edit cm flyte-admin-base-config
- kubectl edit configmap -n flyte flyte-propeller-config
- flytectl get task-resource-attribute -p flyteexamples -d development
- flytectl update project -p flyte-anti-fraud-ml -d development --storage.cache.max_size_mbs 100
- flytectl get launchplan --project flyte-anti-fraud-ml --domain development app.workflow.main_flow --latest --execFile exec_spec.yaml
- flytectl create execution --project flyte-anti-fraud-ml --domain development --execFile exec_spec.yaml
What additional steps I have to do to force flytectl to use my propeller changes and solve the problem of a max 10Mb size allowed for serialized uploads to flyte?Sanjay Chouhan
09/28/2022, 9:33 AMPadma Priya M
09/28/2022, 9:36 AM[1/1] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes.
[ff43e19ed1617402a8c6-n0-0] terminated with exit code (1). Reason [Error]. Message:
ine 286, in _handle_annotated_task
_dispatch_execute(ctx, task_def, inputs, output_prefix)
File "/opt/venv/lib/python3.8/site-packages/flytekit/bin/entrypoint.py", line 76, in _dispatch_execute
logger.debug(f"Starting _dispatch_execute for {task_def.name}")
AttributeError: 'function' object has no attribute 'name'
Traceback (most recent call last):
File "/opt/venv/bin/pyflyte-fast-execute", line 8, in <module>
sys.exit(fast_execute_task_cmd())
File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1130, in _call_
return self.main(*args, **kwargs)
File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/opt/venv/lib/python3.8/site-packages/flytekit/bin/entrypoint.py", line 502, in fast_execute_task_cmd
subprocess.run(cmd, check=True)
File "/usr/local/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['pyflyte-execute', '--inputs', '<s3://my-s3-bucket/metadata/propeller/flytesnacks-development-ff43e19ed1617402a8c6/n0/data/inputs.pb>', '--output-prefix', '<s3://my-s3-bucket/metadata/propeller/flytesnacks-development-ff43e19ed1617402a8c6/n0/data/0>', '--raw-output-data-prefix', '<s3://my-s3-bucket/qf/ff43e19ed1617402a8c6-n0-0>', '--checkpoint-path', '<s3://my-s3-bucket/qf/ff43e19ed1617402a8c6-n0-0/_flytecheckpoints>', '--prev-checkpoint', '""', '--dynamic-addl-distro', '<s3://my-s3-bucket/72/flytesnacks/development/LXFFQS6IOSM2OAH57AOZKM2SZA======/scriptmode.tar.gz>', '--dynamic-dest-dir', '/root', '--resolver', 'flytekit.core.python_auto_container.default_task_resolver', '--', 'task-module', 'flytekitplugins.papermill.task', 'task-name', '_dummy_task_func']' returned non-zero exit status 1.
Padma Priya M
09/28/2022, 9:36 AM[1/1] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes.
[ff43e19ed1617402a8c6-n0-0] terminated with exit code (1). Reason [Error]. Message:
ine 286, in _handle_annotated_task
_dispatch_execute(ctx, task_def, inputs, output_prefix)
File "/opt/venv/lib/python3.8/site-packages/flytekit/bin/entrypoint.py", line 76, in _dispatch_execute
logger.debug(f"Starting _dispatch_execute for {task_def.name}")
AttributeError: 'function' object has no attribute 'name'
Traceback (most recent call last):
File "/opt/venv/bin/pyflyte-fast-execute", line 8, in <module>
sys.exit(fast_execute_task_cmd())
File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1130, in _call_
return self.main(*args, **kwargs)
File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/opt/venv/lib/python3.8/site-packages/flytekit/bin/entrypoint.py", line 502, in fast_execute_task_cmd
subprocess.run(cmd, check=True)
File "/usr/local/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['pyflyte-execute', '--inputs', '<s3://my-s3-bucket/metadata/propeller/flytesnacks-development-ff43e19ed1617402a8c6/n0/data/inputs.pb>', '--output-prefix', '<s3://my-s3-bucket/metadata/propeller/flytesnacks-development-ff43e19ed1617402a8c6/n0/data/0>', '--raw-output-data-prefix', '<s3://my-s3-bucket/qf/ff43e19ed1617402a8c6-n0-0>', '--checkpoint-path', '<s3://my-s3-bucket/qf/ff43e19ed1617402a8c6-n0-0/_flytecheckpoints>', '--prev-checkpoint', '""', '--dynamic-addl-distro', '<s3://my-s3-bucket/72/flytesnacks/development/LXFFQS6IOSM2OAH57AOZKM2SZA======/scriptmode.tar.gz>', '--dynamic-dest-dir', '/root', '--resolver', 'flytekit.core.python_auto_container.default_task_resolver', '--', 'task-module', 'flytekitplugins.papermill.task', 'task-name', '_dummy_task_func']' returned non-zero exit status 1.
import math
import os
import sys
import pathlib
from flytekit import kwtypes, task, workflow
from flytekitplugins.papermill import NotebookTask
nb = NotebookTask(
name="simple-nb",
notebook_path=os.path.join(
sys.path[0], "notebook_sample.ipynb"
),
render_deck=True,
inputs=kwtypes(v=float),
outputs=kwtypes(square=float),
)
@task
def square_root_task(f: float) -> float:
return math.sqrt(f)
@workflow
def nb_to_python_wf(f: float) -> float:
out = nb(v=f)
return square_root_task(f=out.square)
if __name__ == "__main__":
print(nb_to_python_wf(f=3.14))
Kevin Su
09/28/2022, 11:19 AMPadma Priya M
09/28/2022, 11:25 AMSamhita Alla
09/29/2022, 12:48 PMKevin Su
09/29/2022, 3:12 PMpyflyte register --image <http://ghcr.io/flyteorg/flytecookbook:papermilltasks-latest|ghcr.io/flyteorg/flytecookbook:papermilltasks-latest> example.py
Samhita Alla
10/05/2022, 1:42 PMpip-compile <http://requirements.in|requirements.in>
), install the dependencies, and run the pyflyte register again.Padma Priya M
10/19/2022, 4:23 AM#
# This file is autogenerated by pip-compile with python 3.9
# To update, run:
#
# pip-compile <http://requirements.in|requirements.in>
#
--index-url <https://sonatype.adfdata.net/repository/pypi_adf/simple>
--extra-index-url <https://pypi.org/simple>
--trusted-host <http://sonatype.adfdata.net|sonatype.adfdata.net>
ansiwrap==0.8.4
# via papermill
arrow==1.2.2
# via jinja2-time
asttokens==2.0.5
# via stack-data
attrs==21.4.0
# via jsonschema
backcall==0.2.0
# via ipython
beautifulsoup4==4.11.1
# via nbconvert
binaryornot==0.4.4
# via cookiecutter
bleach==5.0.0
# via nbconvert
certifi==2021.10.8
# via requests
cffi==1.15.1
# via cryptography
chardet==4.0.0
# via binaryornot
charset-normalizer==2.0.12
# via requests
checksumdir==1.2.0
# via flytekit
click==8.1.2
# via
# cookiecutter
# flytekit
# papermill
cloudpickle==2.0.0
# via flytekit
cookiecutter==1.7.3
# via flytekit
croniter==1.3.4
# via flytekit
cryptography==38.0.1
# via secretstorage
dataclasses-json==0.5.7
# via flytekit
debugpy==1.6.0
# via ipykernel
decorator==5.1.1
# via
# ipython
# retry
defusedxml==0.7.1
# via nbconvert
deprecated==1.2.13
# via flytekit
diskcache==5.4.0
# via flytekit
docker-image-py==0.1.12
# via flytekit
docstring-parser==0.13
# via flytekit
entrypoints==0.4
# via
# jupyter-client
# nbconvert
# papermill
executing==0.8.3
# via stack-data
fastjsonschema==2.15.3
# via nbformat
flyteidl==0.24.21
# via flytekit
flytekit==0.32.3
# via flytekitplugins-papermill
flytekitplugins-papermill==0.32.3
# via -r <http://requirements.in|requirements.in>
googleapis-common-protos==1.56.0
# via
# flyteidl
# grpcio-status
grpcio==1.44.0
# via
# flytekit
# grpcio-status
grpcio-status==1.44.0
# via flytekit
idna==3.3
# via requests
importlib-metadata==4.11.3
# via keyring
ipykernel==6.13.0
# via flytekitplugins-papermill
ipython==8.2.0
# via ipykernel
jedi==0.18.1
# via ipython
jeepney==0.8.0
# via
# keyring
# secretstorage
jinja2==3.1.1
# via
# cookiecutter
# jinja2-time
# nbconvert
jinja2-time==0.2.0
# via cookiecutter
jsonschema==4.4.0
# via nbformat
jupyter-client==7.2.2
# via
# ipykernel
# nbclient
jupyter-core==4.9.2
# via
# jupyter-client
# nbconvert
# nbformat
jupyterlab-pygments==0.2.1
# via nbconvert
keyring==23.5.0
# via flytekit
markupsafe==2.1.1
# via
# jinja2
# nbconvert
marshmallow==3.15.0
# via
# dataclasses-json
# marshmallow-enum
# marshmallow-jsonschema
marshmallow-enum==1.5.1
# via dataclasses-json
marshmallow-jsonschema==0.13.0
# via flytekit
matplotlib-inline==0.1.3
# via
# ipykernel
# ipython
mistune==0.8.4
# via nbconvert
mypy-extensions==0.4.3
# via typing-inspect
natsort==8.1.0
# via flytekit
nbclient==0.6.0
# via
# nbconvert
# papermill
nbconvert==6.5.0
# via flytekitplugins-papermill
nbformat==5.3.0
# via
# nbclient
# nbconvert
# papermill
nest-asyncio==1.5.5
# via
# ipykernel
# jupyter-client
# nbclient
numpy==1.22.3
# via
# pandas
# pyarrow
packaging==21.3
# via
# ipykernel
# marshmallow
# nbconvert
pandas==1.4.2
# via flytekit
pandocfilters==1.5.0
# via nbconvert
papermill==2.3.4
# via flytekitplugins-papermill
parso==0.8.3
# via jedi
pexpect==4.8.0
# via ipython
pickleshare==0.7.5
# via ipython
poyo==0.5.0
# via cookiecutter
prompt-toolkit==3.0.29
# via ipython
protobuf==3.20.0
# via
# flyteidl
# flytekit
# googleapis-common-protos
# grpcio-status
# protoc-gen-swagger
protoc-gen-swagger==0.1.0
# via flyteidl
psutil==5.9.0
# via ipykernel
ptyprocess==0.7.0
# via pexpect
pure-eval==0.2.2
# via stack-data
py==1.11.0
# via retry
pyarrow==6.0.1
# via flytekit
pycparser==2.21
# via cffi
pygments==2.11.2
# via
# ipython
# nbconvert
pyparsing==3.0.8
# via packaging
pyrsistent==0.18.1
# via jsonschema
python-dateutil==2.8.2
# via
# arrow
# croniter
# flytekit
# jupyter-client
# pandas
python-json-logger==2.0.2
# via flytekit
python-slugify==6.1.1
# via cookiecutter
pytimeparse==1.1.8
# via flytekit
pytz==2022.1
# via
# flytekit
# pandas
pyyaml==6.0
# via
# flytekit
# papermill
pyzmq==22.3.0
# via jupyter-client
regex==2022.3.15
# via docker-image-py
requests==2.27.1
# via
# cookiecutter
# flytekit
# papermill
# responses
responses==0.20.0
# via flytekit
retry==0.9.2
# via flytekit
secretstorage==3.3.3
# via keyring
six==1.16.0
# via
# asttokens
# bleach
# cookiecutter
# grpcio
# python-dateutil
sortedcontainers==2.4.0
# via flytekit
soupsieve==2.3.2
# via beautifulsoup4
stack-data==0.2.0
# via ipython
statsd==3.3.0
# via flytekit
tenacity==8.0.1
# via papermill
text-unidecode==1.3
# via python-slugify
textwrap3==0.9.2
# via ansiwrap
tinycss2==1.1.1
# via nbconvert
tornado==6.1
# via
# ipykernel
# jupyter-client
tqdm==4.64.0
# via papermill
traitlets==5.1.1
# via
# ipykernel
# ipython
# jupyter-client
# jupyter-core
# matplotlib-inline
# nbclient
# nbconvert
# nbformat
typing-extensions==4.1.1
# via
# flytekit
# typing-inspect
typing-inspect==0.7.1
# via dataclasses-json
urllib3==1.26.9
# via
# flytekit
# requests
# responses
wcwidth==0.2.5
# via prompt-toolkit
webencodings==0.5.1
# via
# bleach
# tinycss2
wheel==0.37.1
# via flytekit
wrapt==1.14.0
# via
# deprecated
# flytekit
zipp==3.8.0
# via importlib-metadata
# The following packages are considered to be unsafe in a requirements file:
# setuptools
@Samhita Alla @Kevin SuSamhita Alla
10/19/2022, 11:43 AM#
# This file is autogenerated by pip-compile with python 3.9
# To update, run:
#
# /Library/Developer/CommandLineTools/usr/bin/make requirements.txt
#
ansiwrap==0.8.4
# via papermill
appnope==0.1.3
# via
# ipykernel
# ipython
arrow==1.2.3
# via jinja2-time
asttokens==2.0.8
# via stack-data
attrs==22.1.0
# via
# jsonschema
# visions
backcall==0.2.0
# via ipython
beautifulsoup4==4.11.1
# via nbconvert
binaryornot==0.4.4
# via cookiecutter
bleach==5.0.1
# via nbconvert
certifi==2022.9.24
# via requests
cffi==1.15.1
# via cryptography
chardet==5.0.0
# via binaryornot
charset-normalizer==2.1.1
# via requests
click==8.1.3
# via
# cookiecutter
# flytekit
# papermill
cloudpickle==2.2.0
# via flytekit
cookiecutter==2.1.1
# via flytekit
croniter==1.3.7
# via flytekit
cryptography==38.0.1
# via pyopenssl
cycler==0.11.0
# via matplotlib
dataclasses-json==0.5.7
# via flytekit
debugpy==1.6.3
# via ipykernel
decorator==5.1.1
# via
# ipython
# retry
defusedxml==0.7.1
# via nbconvert
deprecated==1.2.13
# via flytekit
diskcache==5.4.0
# via flytekit
docker==6.0.0
# via flytekit
docker-image-py==0.1.12
# via flytekit
docstring-parser==0.15
# via flytekit
entrypoints==0.4
# via
# jupyter-client
# papermill
executing==1.1.1
# via stack-data
fastjsonschema==2.16.2
# via nbformat
flyteidl==1.1.22
# via flytekit
flytekit==1.2.1
# via
# -r ../../../common/requirements-common.in
# flytekitplugins-deck-standard
# flytekitplugins-papermill
flytekitplugins-deck-standard==1.2.1
# via -r ../../../common/requirements-common.in
flytekitplugins-papermill==1.2.1
# via -r <http://requirements.in|requirements.in>
fonttools==4.37.4
# via matplotlib
googleapis-common-protos==1.56.4
# via
# flyteidl
# grpcio-status
grpcio==1.50.0
# via
# flytekit
# grpcio-status
grpcio-status==1.48.0
# via
# -r <http://requirements.in|requirements.in>
# flytekit
htmlmin==0.1.12
# via pandas-profiling
idna==3.4
# via requests
imagehash==4.3.1
# via visions
importlib-metadata==5.0.0
# via
# flytekit
# keyring
# markdown
# nbconvert
ipykernel==6.16.0
# via flytekitplugins-papermill
ipython==8.5.0
# via ipykernel
jaraco-classes==3.2.3
# via keyring
jedi==0.18.1
# via ipython
jinja2==3.1.2
# via
# cookiecutter
# jinja2-time
# nbconvert
# pandas-profiling
jinja2-time==0.2.0
# via cookiecutter
joblib==1.1.1
# via
# flytekit
# pandas-profiling
# phik
jsonschema==4.16.0
# via nbformat
jupyter-client==7.4.2
# via
# ipykernel
# nbclient
jupyter-core==4.11.1
# via
# jupyter-client
# nbconvert
# nbformat
jupyterlab-pygments==0.2.2
# via nbconvert
keyring==23.9.3
# via flytekit
kiwisolver==1.4.4
# via matplotlib
markdown==3.4.1
# via flytekitplugins-deck-standard
markupsafe==2.1.1
# via
# jinja2
# nbconvert
marshmallow==3.18.0
# via
# dataclasses-json
# marshmallow-enum
# marshmallow-jsonschema
marshmallow-enum==1.5.1
# via dataclasses-json
marshmallow-jsonschema==0.13.0
# via flytekit
matplotlib==3.5.3
# via
# -r ../../../common/requirements-common.in
# missingno
# pandas-profiling
# phik
# seaborn
matplotlib-inline==0.1.6
# via
# ipykernel
# ipython
missingno==0.5.1
# via pandas-profiling
mistune==2.0.4
# via nbconvert
more-itertools==9.0.0
# via jaraco-classes
multimethod==1.8
# via
# pandas-profiling
# visions
mypy-extensions==0.4.3
# via typing-inspect
natsort==8.2.0
# via flytekit
nbclient==0.7.0
# via
# nbconvert
# papermill
nbconvert==7.2.1
# via flytekitplugins-papermill
nbformat==5.7.0
# via
# nbclient
# nbconvert
# papermill
nest-asyncio==1.5.6
# via
# ipykernel
# jupyter-client
# nbclient
networkx==2.8.7
# via visions
numpy==1.23.4
# via
# imagehash
# matplotlib
# missingno
# pandas
# pandas-profiling
# patsy
# phik
# pyarrow
# pywavelets
# scipy
# seaborn
# statsmodels
# visions
packaging==21.3
# via
# docker
# ipykernel
# marshmallow
# matplotlib
# nbconvert
# statsmodels
pandas==1.4.4
# via
# flytekit
# pandas-profiling
# phik
# seaborn
# statsmodels
# visions
pandas-profiling==3.3.0
# via flytekitplugins-deck-standard
pandocfilters==1.5.0
# via nbconvert
papermill==2.4.0
# via flytekitplugins-papermill
parso==0.8.3
# via jedi
patsy==0.5.3
# via statsmodels
pexpect==4.8.0
# via ipython
phik==0.12.2
# via pandas-profiling
pickleshare==0.7.5
# via ipython
pillow==9.2.0
# via
# imagehash
# matplotlib
# visions
plotly==5.10.0
# via flytekitplugins-deck-standard
prompt-toolkit==3.0.31
# via ipython
protobuf==3.20.3
# via
# flyteidl
# flytekit
# googleapis-common-protos
# grpcio-status
# protoc-gen-swagger
protoc-gen-swagger==0.1.0
# via flyteidl
psutil==5.9.3
# via ipykernel
ptyprocess==0.7.0
# via pexpect
pure-eval==0.2.2
# via stack-data
py==1.11.0
# via retry
pyarrow==6.0.1
# via flytekit
pycparser==2.21
# via cffi
pydantic==1.9.2
# via pandas-profiling
pygments==2.13.0
# via
# ipython
# nbconvert
pyopenssl==22.1.0
# via flytekit
pyparsing==3.0.9
# via
# matplotlib
# packaging
pyrsistent==0.18.1
# via jsonschema
python-dateutil==2.8.2
# via
# arrow
# croniter
# flytekit
# jupyter-client
# matplotlib
# pandas
python-json-logger==2.0.4
# via flytekit
python-slugify==6.1.2
# via cookiecutter
pytimeparse==1.1.8
# via flytekit
pytz==2022.5
# via
# flytekit
# pandas
pywavelets==1.4.1
# via imagehash
pyyaml==6.0
# via
# cookiecutter
# flytekit
# pandas-profiling
# papermill
pyzmq==24.0.1
# via
# ipykernel
# jupyter-client
regex==2022.9.13
# via docker-image-py
requests==2.28.1
# via
# cookiecutter
# docker
# flytekit
# pandas-profiling
# papermill
# responses
responses==0.22.0
# via flytekit
retry==0.9.2
# via flytekit
scipy==1.9.2
# via
# imagehash
# missingno
# pandas-profiling
# phik
# seaborn
# statsmodels
seaborn==0.11.2
# via
# missingno
# pandas-profiling
six==1.16.0
# via
# asttokens
# bleach
# grpcio
# patsy
# python-dateutil
sortedcontainers==2.4.0
# via flytekit
soupsieve==2.3.2.post1
# via beautifulsoup4
stack-data==0.5.1
# via ipython
statsd==3.3.0
# via flytekit
statsmodels==0.13.2
# via pandas-profiling
tangled-up-in-unicode==0.2.0
# via
# pandas-profiling
# visions
tenacity==8.1.0
# via
# papermill
# plotly
text-unidecode==1.3
# via python-slugify
textwrap3==0.9.2
# via ansiwrap
tinycss2==1.2.1
# via nbconvert
toml==0.10.2
# via responses
tornado==6.2
# via
# ipykernel
# jupyter-client
tqdm==4.64.1
# via
# pandas-profiling
# papermill
traitlets==5.5.0
# via
# ipykernel
# ipython
# jupyter-client
# jupyter-core
# matplotlib-inline
# nbclient
# nbconvert
# nbformat
types-toml==0.10.8
# via responses
typing-extensions==4.4.0
# via
# flytekit
# pydantic
# typing-inspect
typing-inspect==0.8.0
# via dataclasses-json
urllib3==1.26.12
# via
# docker
# flytekit
# requests
# responses
visions[type_image_path]==0.7.5
# via pandas-profiling
wcwidth==0.2.5
# via prompt-toolkit
webencodings==0.5.1
# via
# bleach
# tinycss2
websocket-client==1.4.1
# via docker
wheel==0.37.1
# via
# -r ../../../common/requirements-common.in
# flytekit
wrapt==1.14.1
# via
# deprecated
# flytekit
zipp==3.9.0
# via importlib-metadata
Kevin Su
10/19/2022, 11:34 PMPadma Priya M
10/27/2022, 7:21 AMSamhita Alla
10/27/2022, 8:43 AMPadma Priya M
10/27/2022, 9:20 AMSamhita Alla
10/27/2022, 9:22 AMKevin Su
10/27/2022, 9:37 AMpip install git+<https://github.com/flyteorg/flytekit@>"papermill-bug-1#egg=flytekitplugins-papermill&subdirectory=plugins/flytekit-papermill"