Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3626

Creating file with invalid path can corrupt edit log

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 2.0.0-alpha
    • 0.23.4, 2.0.3-alpha
    • namenode
    • None
    • Reviewed

    Description

      Joris Bontje reports the following:

      The following command results in a corrupt NN editlog (note the double slash and reading from stdin):
      $ cat /usr/share/dict/words | hadoop fs -put - hdfs://localhost:8020//path/file

      After this, restarting the namenode will result into the following fatal exception:

      2012-07-10 06:29:19,910 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Reading /var/lib/hadoop-hdfs/cache/hdfs/dfs/name/current/edits_0000000000000000173-0000000000000000188 expecting start txid #173
      2012-07-10 06:29:19,912 ERROR org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader: Encountered exception on operation MkdirOp [length=0, path=/, timestamp=1341915658216, permissions=cloudera:supergroup:rwxr-xr-x, opCode=OP_MKDIR, txid=182]
      java.lang.ArrayIndexOutOfBoundsException: -1
      

      Attachments

        1. hdfs-3626.txt
          17 kB
          Todd Lipcon
        2. hdfs-3626.txt
          17 kB
          Todd Lipcon
        3. hdfs-3626.txt
          15 kB
          Todd Lipcon
        4. hdfs-3626.txt
          14 kB
          Todd Lipcon

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            tlipcon Todd Lipcon
            tlipcon Todd Lipcon
            Votes:
            0 Vote for this issue
            Watchers:
            16 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment