Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-16967

TestSequenceFile#testRecursiveSeqFileCreate fails in subsequent run

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.2.1, 3.4.0
    • Fix Version/s: 3.4.0
    • Component/s: common, test
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      The test expects an IOException when creating a writer for file `target/test/data/recursiveCreateDir/file` with `createParent=false`. And it expects to create the writer successfully when `createParent=True`. `createParent` means `create parent directory if non-existent`.

      The test will pass if it is run for the first time, but it will fail for the second run. This is because the test did not clean the parent directory created during the first run.

      The parent directory `recursiveCreateDir` was created, but it was not deleted before the test finished. So, when the test was run again, it still treated the parent directory `recursiveCreateDir` as non-existent and expected an IOException from creating a writer with `createParent=false`. Then the test did not get the expected IOException because `recursiveCreateDir` has been created in the first test run.

      @SuppressWarnings("deprecation")
       @Test
       public void testRecursiveSeqFileCreate() throws IOException {
       FileSystem fs = FileSystem.getLocal(conf);
       Path name = new Path(new Path(GenericTestUtils.getTempPath(
       "recursiveCreateDir")), "file"); // FILE SUCCESSULLY CREATED HERE
       boolean createParent = false;
      try {
       SequenceFile.createWriter(fs, conf, name, RandomDatum.class,
       RandomDatum.class, 512, (short) 1, 4096, createParent,
       CompressionType.NONE, null, new Metadata());
       fail("Expected an IOException due to missing parent");
       } catch (IOException ioe) {
       // Expected
       }
      createParent = true;
       SequenceFile.createWriter(fs, conf, name, RandomDatum.class,
       RandomDatum.class, 512, (short) 1, 4096, createParent,
       CompressionType.NONE, null, new Metadata());
       // should succeed, fails if exception thrown
       }
      

      Suggested patch:

       

      diff --git a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFile.java b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFile.java
      index 044824356ed..1aff2936264 100644
      --- a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFile.java
      +++ b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFile.java
      @@ -649,8 +649,9 @@ public void testCreateWriterOnExistingFile() throws IOException {
       @Test
       public void testRecursiveSeqFileCreate() throws IOException {
       FileSystem fs = FileSystem.getLocal(conf);
      - Path name = new Path(new Path(GenericTestUtils.getTempPath(
      - "recursiveCreateDir")), "file");
      + Path parentDir = new Path(GenericTestUtils.getTempPath(
      + "recursiveCreateDir"));
      + Path name = new Path(parentDir, "file");
       boolean createParent = false;
       
       try {
      @@ -667,6 +668,9 @@ public void testRecursiveSeqFileCreate() throws IOException {
       RandomDatum.class, 512, (short) 1, 4096, createParent,
       CompressionType.NONE, null, new Metadata());
       // should succeed, fails if exception thrown
      +
      + fs.deleteOnExit(parentDir);
      + fs.close();
       }
       
       @Test

        Attachments

        1. HADOOP-16967.000.patch
          1 kB
          Ctest
        2. HADOOP-16967.001.patch
          2 kB
          Ctest
        3. HADOOP-16967.002.patch
          2 kB
          Ctest

          Activity

            People

            • Assignee:
              ctest.team Ctest
              Reporter:
              ctest.team Ctest
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: