[PARQUET-529] Avoid evoking job.toString() in ParquetLoader

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.8.0, 1.8.1
• Fix Version/s: 1.9.0, 1.8.2
• Component/s: parquet-pig
• Assignee: Liwei Lin(Inactive)
• Reporter: Liwei Lin(Inactive)
• Created: 14/Feb/16 14:05
• Updated: 21/Apr/18 12:38
• Resolved: 22/Feb/16 02:37

Description

When ran under hadoop2 environment and log level setting to DEBUG, ParquetLoader would evoke job.toString() in several methods, which might cause the whole application to stop due to :

java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING

at org.apache.hadoop.mapreduce.Job.ensureState(Job.java:283)
at org.apache.hadoop.mapreduce.Job.toString(Job.java:452)
at java.lang.String.valueOf(String.java:2847)
at java.lang.StringBuilder.append(StringBuilder.java:128)
at org.apache.parquet.pig.ParquetLoader.getSchema(ParquetLoader.java:260)
at org.apache.parquet.pig.TestParquetLoader.testSchema(TestParquetLoader.java:54)
...

The reason is that in the hadoop 2.x branch, org.apache.hadoop.mapreduce.Job.toString() has added an ensureState(JobState.RUNNING) check; see map-reduce: Job.java#452. In contrast, the hadoop 1.x branch does not contain such checks, so ParquetLoader works well.

This ticket simply avoids evoking job.toString() in ParquetLoader.

Issue Links

• blocks: PARQUET-401 Deprecate Log and move to SLF4J Logger
• links to: PR#326