Details
-
Question
-
Status: Resolved
-
Critical
-
Resolution: Not A Problem
-
2.1.3
-
None
-
None
-
spark1.3 ;hadoop2.6
Description
when i submit a spark application :
/spark-submit --class com.yto.log.SparkStatCleanJobYARN --name TopNStatJobYARN --master yarn --executor-memory 1G --num-executors 1 /home/hadoop/lib/sql-1.0-jar-with-dependencies.jar hdfs://hadoop001:8020/clean 20161110
the hadfs file is already existed,but not recognition
19/09/11 14:18:50 INFO mapred.FileInputFormat: Total input paths to process : 219/09/11 14:18:50 INFO mapred.FileInputFormat: Total input paths to process : 219/09/11 14:18:50 ERROR datasources.FileFormatWriter: Aborting job null.java.io.IOException: Not a file: hdfs://hadoop001:8020/clean/day= at org.apache.hadoop.mapred.FileInputFormat.getSplits