[HBASE-22607] TestExportSnapshotNoCluster::testSnapshotWithRefsExportFileSystemState() fails intermittently

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha-1, 2.2.0, 2.0.6
• Fix Version/s: 3.0.0-alpha-1, 2.3.0, 2.2.3, 2.1.9
• Component/s: test
• Hadoop Flags: Reviewed
• Assignee: Mingliang Liu
• Reporter: Mingliang Liu
• Created: 19/Jun/19 22:16
• Updated: 04/Feb/20 20:58
• Resolved: 20/Nov/19 16:42

Description

In previous runs, test TestExportSnapshotNoCluster.testSnapshotWithRefsExportFileSystemState fails intermittently with java.net.ConnectException: Connection refused exception, see build 510, 545, and 556.

So one sample exception is like:

at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346)
at com.sun.proxy.$Proxy20.getListing(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1630)
at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1614)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:900)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:114)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:964)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:961)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:961)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1537)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1580)
at org.apache.hadoop.hbase.util.CommonFSUtils.listStatus(CommonFSUtils.java:693)
at org.apache.hadoop.hbase.util.FSTableDescriptors.getCurrentTableInfoStatus(FSTableDescriptors.java:448)
at org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:429)
at org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:410)
at org.apache.hadoop.hbase.util.FSTableDescriptors.createTableDescriptorForTableDirectory(FSTableDescriptors.java:763)
at org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils$SnapshotMock.createTable(SnapshotTestingUtils.java:675)
at org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils$SnapshotMock.createSnapshot(SnapshotTestingUtils.java:653)
at org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils$SnapshotMock.createSnapshot(SnapshotTestingUtils.java:647)
at org.apache.hadoop.hbase.snapshot.SnapshotTestingUtils$SnapshotMock.createSnapshotV2(SnapshotTestingUtils.java:637)
at org.apache.hadoop.hbase.snapshot.TestExportSnapshotNoCluster.testSnapshotWithRefsExportFileSystemState(TestExportSnapshotNoCluster.java:80)

This seems that, somehow the rootdir filesystem is not LocalFileSystem, but on HDFS. I have not dig deeper why this happens since it's failing intermittently and I can not reproduce it locally. Since this is testing export snapshot tool without cluster, we can enforce it using LocalFileSystem; no breaking change.

Issue Links

• relates to: HBASE-23792 [Flakey Test] TestExportSnapshotNoCluster.testSnapshotWithRefsExportFileSystemState