Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-15068

Run ClearDanglingScratchDir periodically inside HS2

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      In HIVE-13429, we introduce a tool which clear dangling scratch directory. In this ticket, we want to invoke the tool automatically on a Hive cluster. Options are:
      1. cron job, which would involve manual cron job setup
      2. As a metastore thread. However, it is possible we run metastore without hdfs in the future (eg, managing s3 files). ClearDanglingScratchDir needs support which only exists in hdfs, it won't work if the above scenario happens
      3. As a HS2 thread. The downside is if no HS2 is running, the tool will not run automatically. But we expect HS2 will be a required component down the road

      Here I choose approach 3 in the implementation.

        Attachments

        1. HIVE-15068.1.patch
          17 kB
          Daniel Dai
        2. HIVE-15068.2.patch
          17 kB
          Daniel Dai
        3. HIVE-15068.3.patch
          17 kB
          Daniel Dai
        4. HIVE-15068.4.branch-1.patch
          18 kB
          Daniel Dai
        5. HIVE-15068.4.patch
          18 kB
          Daniel Dai

          Issue Links

            Activity

              People

              • Assignee:
                daijy Daniel Dai
                Reporter:
                daijy Daniel Dai
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: