Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7563

NFS gateway parseStaticMap NumberFormatException

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.7.0
    • Component/s: nfs
    • Labels:
      None
    • Environment:

      HDP 2.2

    • Hadoop Flags:
      Reviewed

      Description

      When using the new NFS UID mapping for the HDFS NFS gateway I've discovered that my Windows 7 workstation at this bank is passing UID number 4294xxxxxx but entering this in the /etc/nfs.map in order to remap that to a Hadoop UID prevents the NFS gateway from restarting with the error message:

      Exception in thread "main" java.lang.NumberFormatException: For input string: "4294xxxxxx"
              at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
              at java.lang.Integer.parseInt(Integer.java:495)
              at java.lang.Integer.parseInt(Integer.java:527)
              at org.apache.hadoop.security.ShellBasedIdMapping.parseStaticMap(ShellBasedIdMapping.java:318)
              at org.apache.hadoop.security.ShellBasedIdMapping.updateMaps(ShellBasedIdMapping.java:229)
              at org.apache.hadoop.security.ShellBasedIdMapping.<init>(ShellBasedIdMapping.java:91)
              at org.apache.hadoop.hdfs.nfs.nfs3.RpcProgramNfs3.<init>(RpcProgramNfs3.java:176)
              at org.apache.hadoop.hdfs.nfs.nfs3.Nfs3.<init>(Nfs3.java:45)
              at org.apache.hadoop.hdfs.nfs.nfs3.Nfs3.startService(Nfs3.java:66)
              at org.apache.hadoop.hdfs.nfs.nfs3.Nfs3.main(Nfs3.java:72)
      

      The /etc/nfs.map file simply contains

      uid 4294xxxxxx 1yyyy
      

      It seems that the code at

      hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ShellBasedIdMapping.java

      is expecting an integer at line 318 of the parseStaticMap method:

      int remoteId = Integer.parseInt(lineMatcher.group(2));
      int localId = Integer.parseInt(lineMatcher.group(3));

      This UID does seem very high to me but it has worked successfully on a MapR-FS NFS share and stores files created with that UID over NFS.

      The UID / GID mappings for the HDFS NFS gateway will need to be switched to using Long to accomodate this, I've attached a patch for the parsing and UID/GID HashMaps.

      Regards,

      Hari Sekhon
      http://www.linkedin.com/in/harisekhon

        Attachments

        1. UID_GID_Long_HashMaps.patch
          1 kB
          Hari Sekhon
        2. HDFS-7563.001.patch
          2 kB
          Yongjun Zhang

          Activity

            People

            • Assignee:
              yzhangal Yongjun Zhang
              Reporter:
              harisekhon Hari Sekhon
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: