[SOLR-10360] Solr HDFS snapshot export fails due to FileNotFoundException error

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 6.3
• Fix Version/s: 6.7, 7.0

Environment

SOLR deployed along with a HADOOP cluster (HDFS +M/R 1).

• Assignee: Mark Miller
• Reporter: Hrishikesh Gadre
• Created: 24/Mar/17 18:45
• Updated: 08/Jun/19 15:32
• Resolved: 31/May/17 18:25

Description

The Solr snapshot export command (implemented in the snapshotscli.sh) uses Hadoop distcp tool to copy the Solr index files to the desired location. When the cluster is configured with MR1 framework (instead of YARN), this command fails with following error,

Err:With failures, global counters are inaccurate; consider running with -i
Copy failed: java.io.FileNotFoundException: File does not exist: /user/systest/ /backups/mysnap/copylistings/shard1
	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:2007)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1977)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1890)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:572)

During investigation I found that an extra space in the distcp command was causing this failure. For example,

This command fails with the error mentioned above,

hadoop distcp -f  ' /backups/mysnap/copylistings/shard1' /backups/mysnap/snapshot.shard1

Removing the space from the copylisting directory path fixes this issue.

hadoop distcp -f  '/backups/mysnap/copylistings/shard1' /backups/mysnap/snapshot.shard1

Issue Links

• links to: GitHub Pull Request #173