ancient-wolf-19325
09/07/2024, 1:44 AMSyntaxError: unexpected character after line continuation character (<unknown>, line 1) when I run this code as a flyte task. However, the same input works if I run in a normal jupyter notebook. I would like to know if there is any encoding issue that I should check for?
import unicodedata
@task
def preprocess(row: Dict[str, Any]):
row_text = row['text']
row_text = unicodedata.normalize('NFKD', row_text)
# Remove any remaining non-ASCII characters
row_text = row_text.encode('ASCII', 'ignore').decode('ASCII')
pattern = r'[^\w\s\-\.\,\:\;\'\"\{\}\\]'
row_text = re.sub(pattern, '', row_text).strip('"')
row_text_dic = ast.literal_eval(row_text) #Throws exception here
# Answer
I'm not able to find any data sources to answer your question.
You might want to provide the exact input string causing the issue and any differences in the environment setup between your Jupyter notebook and Flyte task execution.
If this doesn't help, please reach out to the Flyte support team for further assistance.
Slack Link
https://flyte-org.slack.com/archives/CP2HDHKE1/p1725673439.439949 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.