Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-3042

Sqoop does not clear compile directory under /tmp/sqoop-<username>/compile automatically

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.4.6
    • Fix Version/s: 3.0.0
    • Component/s: None
    • Labels:

      Description

      After running sqoop, all the temp files generated by ClassWriter are left behind on disk, so anyone can check those JAVA files to see the schema of those tables that Sqoop has been interacting with. By default, the directory is under /tmp/sqoop-<username>/compile.

      In class org.apache.sqoop.SqoopOptions, function getNonceJarDir(), I can see that we did add "deleteOnExit" on the temp dir:

          for (int attempts = 0; attempts < MAX_DIR_CREATE_ATTEMPTS; attempts++) {
            hashDir = new File(baseDir, RandomHash.generateMD5String());
            while (hashDir.exists()) {
              hashDir = new File(baseDir, RandomHash.generateMD5String());
            }
      
            if (hashDir.mkdirs()) {
              // We created the directory. Use it.
              // If this directory is not actually filled with files, delete it
              // when the JVM quits.
              hashDir.deleteOnExit();
              break;
            }
          }
      

      However, I believe it failed to delete due to directory is not empty.

        Attachments

        1. SQOOP-3042.9.patch
          10 kB
          Eric Lin
        2. SQOOP-3042.7.patch
          9 kB
          Eric Lin
        3. SQOOP-3042.6.patch
          8 kB
          Eric Lin
        4. SQOOP-3042.5.patch
          6 kB
          Eric Lin
        5. SQOOP-3042.4.patch
          7 kB
          Eric Lin
        6. SQOOP-3042.2.patch
          5 kB
          Eric Lin
        7. SQOOP-3042.1.patch
          3 kB
          Eric Lin

          Issue Links

            Activity

              People

              • Assignee:
                ericlin Eric Lin
                Reporter:
                ericlin Eric Lin
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: