Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10360

Solr HDFS snapshot export fails due to FileNotFoundException error

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 6.3
    • Fix Version/s: 6.7, 7.0
    • Component/s: None
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None
    • Environment:

      SOLR deployed along with a HADOOP cluster (HDFS +M/R 1).

      Description

      The Solr snapshot export command (implemented in the snapshotscli.sh) uses Hadoop distcp tool to copy the Solr index files to the desired location. When the cluster is configured with MR1 framework (instead of YARN), this command fails with following error,

      Err:With failures, global counters are inaccurate; consider running with -i
      Copy failed: java.io.FileNotFoundException: File does not exist: /user/systest/ /backups/mysnap/copylistings/shard1
      	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
      	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:2007)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1977)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1890)
      	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:572)
      

      During investigation I found that an extra space in the distcp command was causing this failure. For example,

      This command fails with the error mentioned above,

      hadoop distcp -f  ' /backups/mysnap/copylistings/shard1' /backups/mysnap/snapshot.shard1
      

      Removing the space from the copylisting directory path fixes this issue.

      hadoop distcp -f  '/backups/mysnap/copylistings/shard1' /backups/mysnap/snapshot.shard1
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                markrmiller@gmail.com Mark Miller
                Reporter:
                hgadre Hrishikesh Gadre
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: