Hadoop Common
  1. Hadoop Common
  2. HADOOP-10388 Pure native hadoop client
  3. HADOOP-10806

ndfs: need to implement umask, pass permission bits to hdfsCreateDirectory

    Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: HADOOP-10388
    • Fix Version/s: HADOOP-10388
    • Component/s: native
    • Labels:
      None

      Description

      We need to pass in permission bits to hdfsCreateDirectory. Also, we need to read fs.permissions.umask-mode so that we know what to mask off of the permission bits (umask is always implemented client-side)

      1. HADOOP-10806-pnative.001.patch
        5 kB
        Colin Patrick McCabe
      2. HADOOP-10806-pnative.002.patch
        6 kB
        Colin Patrick McCabe

        Activity

        Hide
        Colin Patrick McCabe added a comment -

        This patch adds some code for parsing fs.permissions.umask-mode and sending permission bits in the hdfsCreateDirectory rpc to the NameNode. This fixes a bug whereby hdfsCreateDirectory would fail with this message:

        org.apache.hadoop.native.HadoopCore.IOException: conn_process_response: error com.google.protobuf.UninitializedMessageException: Message missing required fields: masked.perm
                at com.google.protobuf.AbstractMessage$Builder.newUninitializedMessageException(AbstractMessage.java:770)
        

        Clearly we'll also need this for the file write path, when we're creating new files.

        We don't handle "fancy" fs.permissions.umask-mode values like "rw-r-r-" etc. yet. It's not too difficult to do that, but it's better to do it in a follow-on JIRA to avoid complicating this one. For now, we just deal with numeric umasks.

        Show
        Colin Patrick McCabe added a comment - This patch adds some code for parsing fs.permissions.umask-mode and sending permission bits in the hdfsCreateDirectory rpc to the NameNode. This fixes a bug whereby hdfsCreateDirectory would fail with this message: org.apache.hadoop. native .HadoopCore.IOException: conn_process_response: error com.google.protobuf.UninitializedMessageException: Message missing required fields: masked.perm at com.google.protobuf.AbstractMessage$Builder.newUninitializedMessageException(AbstractMessage.java:770) Clearly we'll also need this for the file write path, when we're creating new files. We don't handle "fancy" fs.permissions.umask-mode values like "rw-r- r -" etc. yet. It's not too difficult to do that, but it's better to do it in a follow-on JIRA to avoid complicating this one. For now, we just deal with numeric umasks.
        Hide
        Abraham Elmahrek added a comment -

        Nit: "Read configuration keys for the new native filesystem." => "Configure native file system using configuration object"?

        Nice man. +1.

        Show
        Abraham Elmahrek added a comment - Nit: "Read configuration keys for the new native filesystem." => "Configure native file system using configuration object"? Nice man. +1.
        Hide
        Wenwu Peng added a comment -

        Thanks Colin for patch, Overview,Looks good for me +1,
        There is only minor comment,
        Better to add error code type "ENOTSUP" to hadoop_err.c

        + return hadoop_lerr_alloc(ENOTSUP, "parse_permission(%s): "
        + "can't parse non-octal permissions (yet)", str);

        Show
        Wenwu Peng added a comment - Thanks Colin for patch, Overview,Looks good for me +1, There is only minor comment, Better to add error code type "ENOTSUP" to hadoop_err.c + return hadoop_lerr_alloc(ENOTSUP, "parse_permission(%s): " + "can't parse non-octal permissions (yet)", str);
        Hide
        Colin Patrick McCabe added a comment -

        Nit: "Read configuration keys for the new native filesystem." => "Configure native file system using configuration object"?

        added

        Better to add error code type "ENOTSUP" to hadoop_err.c

        added

        Show
        Colin Patrick McCabe added a comment - Nit: "Read configuration keys for the new native filesystem." => "Configure native file system using configuration object"? added Better to add error code type "ENOTSUP" to hadoop_err.c added

          People

          • Assignee:
            Colin Patrick McCabe
            Reporter:
            Colin Patrick McCabe
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development