Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3974

IMPALA-2843 should consider using local_library_dir instead of java.io.tmpdir

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Catalog
    • Labels:
      None

      Description

      that may take several gb and can fill up default /tmp. user may be able to set java.io.tmpdir properly but that is yet another headache..

        Issue Links

          Activity

          Hide
          bharathv bharath v added a comment -

          IMPALA-3983/IMPALA-3974: Delete function jar resources after load

          The Catalog copies the UDF jar files to the local file system to
          load the Java UDF classes for validation purposes. However we
          do not clean them up after the UDF load and hence on a deployment
          with large number of functions registered, these jar can accumulate
          over a period of time and can fill up the tmp space. We fix it by
          deleting the jar resource once the function is loaded.

          Also, this patch switches to --local_library_dir for copying these
          temporary jars instead of using the path from java.io.tmpdir.

          Change-Id: I5f9dedb5b342415380c83e61a72eb497371a8199
          Reviewed-on: http://gerrit.cloudera.org:8080/4617
          Reviewed-by: Bharath Vissapragada <bharathv@cloudera.com>
          Tested-by: Internal Jenkins

          Commit: 9f5c6a5b6230ee7c565eade38a75912edb07dd7d

          Show
          bharathv bharath v added a comment - IMPALA-3983 / IMPALA-3974 : Delete function jar resources after load The Catalog copies the UDF jar files to the local file system to load the Java UDF classes for validation purposes. However we do not clean them up after the UDF load and hence on a deployment with large number of functions registered, these jar can accumulate over a period of time and can fill up the tmp space. We fix it by deleting the jar resource once the function is loaded. Also, this patch switches to --local_library_dir for copying these temporary jars instead of using the path from java.io.tmpdir. Change-Id: I5f9dedb5b342415380c83e61a72eb497371a8199 Reviewed-on: http://gerrit.cloudera.org:8080/4617 Reviewed-by: Bharath Vissapragada <bharathv@cloudera.com> Tested-by: Internal Jenkins Commit: 9f5c6a5b6230ee7c565eade38a75912edb07dd7d
          Hide
          HuaisiXu Huaisi Xu added a comment -

          It is more important to clean them up since these jars can accumulate across catalogd restart.

          Show
          HuaisiXu Huaisi Xu added a comment - It is more important to clean them up since these jars can accumulate across catalogd restart.

            People

            • Assignee:
              bharathv bharath v
              Reporter:
              HuaisiXu Huaisi Xu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development