Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3103

[HOD] Hadoop.tmp.dir should not be set to cluster directory

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.16.0
    • 0.16.2
    • contrib/hod
    • None

    Description

      Currently HOD generates hadoop.tmp.dir pointing to the cluster directory(hod.clusterdir), which users typically create in their home directories. This hadoop.tmp.dir is used on the client side when a jar is run(org/apache/hadoop/util/RunJar.java). The user home directories might be NFS mounted in some environments, which might result in a hit, for example when running a jar.

      So, HOD should not generate hadoop.tmp.dir pointing to cluster directory. Instead, it should use hod.temp-dir, which is the location that dfs.client.buffer.dir uses to alleviate any such NFS hit problems. Further, though currently hadoop.tmp.dir is not used on hadoop daemons side, it would be good if HOD generates it on daemons side too, and thus preclude any related problems in future.

      Attachments

        1. HADOOP-3103.1.for.17
          4 kB
          Vinod Kumar Vavilapalli
        2. HADOOP-3103.1.for.16
          3 kB
          Vinod Kumar Vavilapalli
        3. HADOOP-3103.for.17
          4 kB
          Vinod Kumar Vavilapalli
        4. HADOOP-3103.for.16
          3 kB
          Vinod Kumar Vavilapalli
        5. HADOOP-3103
          3 kB
          Vinod Kumar Vavilapalli

        Activity

          People

            vinodkv Vinod Kumar Vavilapalli
            vinodkv Vinod Kumar Vavilapalli
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: