Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-4185

Tez may skip file permission update on intermediate output

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.2, 0.10.1, 0.9.3
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Before updating file permissions TEZ check if the permission change is needed with the following conditional:

      if (!SPILL_FILE_PERMS.equals(SPILL_FILE_PERMS.applyUMask(FsPermission.getUMask(conf)))) {
        rfs.setPermission(filename, SPILL_FILE_PERMS);
      } 

      The problem is that the config object used by this conditional can be different than the config used by the file system (rfs). The reason for this is the underlaying file system instance cache in hadoop.

      Either we should remove this conditional altogether (it's a local file system, it won't generate NN calls) or modify it to use rfs.getConf().

      The end result is a file, which is not readable by the group, because the permission change was skipped due to the umask mismatch between the 2 configs.

        Attachments

        1. TEZ-4185.1.patch
          8 kB
          Attila Magyar
        2. TEZ-4185.2.patch
          10 kB
          Attila Magyar
        3. TEZ-4185.3.patch
          15 kB
          Attila Magyar

          Issue Links

            Activity

              People

              • Assignee:
                amagyar Attila Magyar
                Reporter:
                amagyar Attila Magyar
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: