[HDFS-16318] Add exception blockinfo

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.3.1
• Component/s: hdfs
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: guophilipse
• Created: 12/Nov/21 08:41
• Updated: 07/Dec/21 10:16

Description

we may suffer `Could not obtain the last block location` exception, but we may reading more than one file, the following exception cannnot guide us to find the problem block or dn info.  we can add more info in the log to help us .

`2021-11-12 14:01:59,633 WARN [main] org.apache.hadoop.hdfs.DFSClient: Last block locations not available. Datanodes might not have reported blocks completely. Will retry for 3 times`
`2021-11-12 14:02:03,724 WARN [main] org.apache.hadoop.hdfs.DFSClient: Last block locations not available. Datanodes might not have reported blocks completely. Will retry for 2 times`
`2021-11-12 14:02:07,726 WARN [main] org.apache.hadoop.hdfs.DFSClient: Last block locations not available. Datanodes might not have reported blocks completely. Will retry for 1 times`

`Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.GeneratedConstructorAccessor19.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:251)
    ... 11 more`
`Caused by: java.io.IOException: Could not obtain the last block locations.
    at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:291)
    at org.apache.hadoop.hdfs.DFSInputStream.(DFSInputStream.java:264)
    at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1535)
    at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:304)
    at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:299)
    at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
    at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:312)
    at org.apache.hadoop.fs.FilterFileSystem.open(FilterFileSystem.java:162)
    at org.apache.hadoop.fs.viewfs.ChRootedFileSystem.open(ChRootedFileSystem.java:261)
    at org.apache.hadoop.fs.viewfs.ViewFileSystem.open(ViewFileSystem.java:463)
    at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:768)
    at org.apache.hadoop.mapred.LineRecordReader.(LineRecordReader.java:109)
    at org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:67)
    at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.(CombineHiveRecordReader.java:66)
    ... 15 more`

Issue Links

• links to: GitHub Pull Request #3649