I hava a spark script written in pyspark and I want to submit it via oozie spark action.
something like this:
The script imports pyspark module:
However, the oozie will throw the " Can not import pyspark module" exception.
This happens when I upgrade to CDH 5.5.1 from CDH 5.4.6.
The workaround would be using the shell action, but I think the spark action is better to describe the spark task.