Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Invalid
-
1.3.0
-
None
-
None
Description
Running wordcount job with -Ddfs.client.read.shortcircuit=true/false fails to mention "hdfs.DFSClient: Short circuit read is true" or "hdfs.DFSClient: Short circuit read is false" messages in syslogs.
Attaching screen shot of syslog output for Hadoop 1.1.2. The above message was present in the logs earlier.
Syslog Output of Hadoop 1.3
2013-04-18 13:07:08,265 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library
2013-04-18 13:07:10,002 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0
2013-04-18 13:07:10,577 INFO org.apache.hadoop.mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4edc41c5
2013-04-18 13:07:10,682 INFO org.apache.hadoop.mapred.MapTask: Processing split: hdfs://node1:port1/input1.txt:0+215754
2013-04-18 13:07:10,706 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 200
2013-04-18 13:07:10,910 INFO org.apache.hadoop.mapred.MapTask: data buffer = 150994944/167772160
2013-04-18 13:07:10,910 INFO org.apache.hadoop.mapred.MapTask: record buffer = 2359296/2621440
2013-04-18 13:07:10,920 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native library is available
2013-04-18 13:07:10,920 INFO org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native library loaded
2013-04-18 13:07:10,934 INFO com.hadoop.compression.lzo.GPLNativeCodeLoader: Loaded native gpl library
2013-04-18 13:07:10,947 INFO com.hadoop.compression.lzo.LzoCodec: Successfully loaded & initialized native-lzo library [hadoop-lzo rev cf4e7cbf8ed0f0622504d008101c2729dc0c9ff3]
2013-04-18 13:07:11,414 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2013-04-18 13:07:11,586 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor
2013-04-18 13:07:11,962 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2013-04-18 13:07:12,034 INFO org.apache.hadoop.mapred.Task: Task:attempt_201304181305_0001_m_000000_0 is done. And is in the process of commiting
2013-04-18 13:07:12,106 INFO org.apache.hadoop.mapred.Task: Task 'attempt_201304181305_0001_m_000000_0' done.
2013-04-18 13:07:12,152 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
2013-04-18 13:07:12,637 INFO org.apache.hadoop.io.nativeio.NativeIO: Initialized cache for UID to User mapping with a cache timeout of 14400 seconds.
2013-04-18 13:07:12,637 INFO org.apache.hadoop.io.nativeio.NativeIO: Got UserName mapred for UID 2002 from the native implementation
