[DATALAB-2428] [GCP][Investigation][Dataproc][Apache Standalone cluster] Often Flights_data_Preparation/Visualisation_R playbooks fail

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: v.2.5
• Component/s: DataLab Main
• Labels: Debian DevOps GCP Known_issues(release2.5) Known_issues(release2.5.1) Known_issues(release2.6)
• Epic Link: Playbook adjusting
• Story Points: 2
• Assignee: Leonid Frolov
• Reporter: Vira Vitanska
• Created: 09/Jun/21 09:33
• Updated: 16/Sep/22 08:01

Description

Preconditions:

1. Dataproc is running on Jupyter

Steps to reproduce:

1. Run Flights_data_Preparation/Visualisation_R on Dataproc

Actual result:

1. Flights_data_Preparation/Visualisation_R running fails

Dataproc:

An error was encountered:
Spark package found in SPARK_HOME: /usr/lib/spark
[1] "Error in writeJobj(con, object): invalid jobj 1"

Apache Standalone cluster:

An error was encountered:
Spark package found in SPARK_HOME: /opt/spark
[1] "Error in writeJobj(con, object): invalid jobj 1"

Expected result:

1. Flights_data_Preparation/Visualisation_R running is running succesfully