Hello,
I'm trying to build a project using Flyte and Selenium, but I've encountered an issue. I initially developed my project using regular Python Selenium, and it works fine. The workflow is quite simple:
1. It navigates to a specified URL.
2. Clicks the login button and fills in the necessary fields.
3. Solves the CAPTCHA.
4. Executes a data search function.
5. Saves the listed data as a CSV file and moves to the next page, repeating the same process.
Everything works as expected up to this point. However, when I try to migrate these operations to Flyte as separate tasks, I run into an issue: it seems that WebDriver objects cannot be shared between tasks, which causes an error.
I don't want to write all my code in a single script; instead, I aim to build a modular structure. However, there's no continuity between tasks. For example, after the login task, I need to continue without closing the browser. Once logged in, I should be able to execute the search function and then the data extraction function. But I can't seem to establish such a workflow in Flyte.
How can I solve this issue? Any guidance would be greatly appreciated!
f
freezing-airport-6809
02/14/2025, 2:37 AM
This is available in union - you can try on union serverless using @actor.cache
p
proud-apple-49696
02/18/2025, 11:43 AM
Can I use this locally with flyte that I host on my own server?
flytectl demo start starting with this
f
freezing-airport-6809
02/18/2025, 4:12 PM
Sorry this is a union only feature
p
proud-apple-49696
02/19/2025, 12:52 PM
I get this error when trying in union
Pod failed. No message received from kubernetes.
[a7q6gqckhhqq52pkjk69-n0-0] terminated with exit code (126). Reason [Error]. Message:
[FATAL tini (7)] exec pyflyte-fast-execute failed: Permission denied
f
freezing-airport-6809
02/20/2025, 6:04 AM
where did you try?
freezing-airport-6809
02/20/2025, 6:04 AM
@proud-apple-49696?
freezing-airport-6809
02/20/2025, 6:04 AM
are you tyring on union serverless?
freezing-airport-6809
02/20/2025, 6:04 AM
if so, we can help on that can you hop on to #C075AJZSXEV