Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12938

TestErasureCodigCLI testAll failing consistently.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.1.0
    • 3.1.0, 3.0.1
    • erasure-coding, hdfs
    • None

    Description

      TestErasureCodingCLI#testAll is failing consistently.
      It failed in this precommit: https://builds.apache.org/job/PreCommit-HDFS-Build/22435/testReport/org.apache.hadoop.cli/TestErasureCodingCLI/testAll/
      I ran locally on my laptop and it failed too.
      It failed with this stack trace:

      java.lang.AssertionError: One of the tests failed. See the Detailed results to identify the command that failed
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.hadoop.cli.CLITestHelper.displayResults(CLITestHelper.java:264)
	at org.apache.hadoop.cli.CLITestHelper.tearDown(CLITestHelper.java:126)
	at org.apache.hadoop.cli.TestErasureCodingCLI.tearDown(TestErasureCodingCLI.java:77)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)

      Below is the detailed report from org.apache.hadoop.cli.TestErasureCodingCLI-output.txt.

      2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(156)) - -------------------------------------------
2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(157)) -                     Test ID: [15]
2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(158)) -            Test Description: [setPolicy : set policy on non-empty directory]
2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(159)) - 
2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(163)) -               Test Commands: [-fs hdfs://localhost:52345 -mkdir /ecdir]
2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(163)) -               Test Commands: [-fs hdfs://localhost:52345 -touchz /ecdir/file1]
2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(163)) -               Test Commands: [-fs hdfs://localhost:52345 -setPolicy -policy RS-6-3-1024k -path /ecdir]
2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(167)) - 
2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(170)) -            Cleanup Commands: [-fs hdfs://localhost:52345 -rm -R /ecdir]
2017-12-18 09:25:44,821 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(174)) - 
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(178)) -                  Comparator: [SubstringComparator]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(180)) -          Comparision result:   [fail]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(182)) -             Expected output:   [Warning: setting erasure coding policy on an non-empty directory will not automatically convert existing data to RS-6-3-1024]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(184)) -               Actual output:   [Set erasure coding policy RS-6-3-1024k on /ecdir
Warning: setting erasure coding policy on a non-empty directory will not automatically convert existing files to RS-6-3-1024k
]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(187)) - 
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(156)) - -------------------------------------------
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(157)) -                     Test ID: [17]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(158)) -            Test Description: [unsetPolicy : unset policy on non-empty directory]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(159)) - 
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(163)) -               Test Commands: [-fs hdfs://localhost:52345 -mkdir /ecdir]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(163)) -               Test Commands: [-fs hdfs://localhost:52345 -setPolicy -policy RS-6-3-1024k -path /ecdir]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(163)) -               Test Commands: [-fs hdfs://localhost:52345 -touchz /ecdir/file1]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(163)) -               Test Commands: [-fs hdfs://localhost:52345 -unsetPolicy -path /ecdir]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(167)) - 
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(170)) -            Cleanup Commands: [-fs hdfs://localhost:52345 -rm -R /ecdir]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(174)) - 
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(178)) -                  Comparator: [SubstringComparator]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(180)) -          Comparision result:   [fail]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(182)) -             Expected output:   [Warning: unsetting erasure coding policy on an non-empty directory will not automatically convert existing data to replicated data]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(184)) -               Actual output:   [Unset erasure coding policy from /ecdir
Warning: unsetting erasure coding policy on a non-empty directory will not automatically convert existing files to replicated data.
]
2017-12-18 09:25:44,822 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(187)) - 
2017-12-18 09:25:44,823 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(191)) - Summary results:
2017-12-18 09:25:44,823 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(192)) - ----------------------------------

2017-12-18 09:25:44,823 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(212)) -                Testing mode: test
2017-12-18 09:25:44,823 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(213)) - 
2017-12-18 09:25:44,823 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(214)) -              Overall result: --- FAIL ---
2017-12-18 09:25:44,823 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(222)) -                # Tests pass: 51 (96%)
2017-12-18 09:25:44,823 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(224)) -                # Tests fail: 2 (3%)
2017-12-18 09:25:44,823 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(228)) -          # Validations done: 67 (each test may do multiple validations)
2017-12-18 09:25:44,823 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(231)) - 
2017-12-18 09:25:44,823 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(232)) - Failing tests:
2017-12-18 09:25:44,823 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(233)) - --------------
2017-12-18 09:25:44,823 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(239)) - 15: setPolicy : set policy on non-empty directory
2017-12-18 09:25:44,823 [Thread-0] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(239)) - 17: unsetPolicy : unset policy on non-empty directory

      setPolicy and unsetPolicy commands are failing.

      Attachments

        1. HDFS-12938.001.patch
          1 kB
          Ajay Kumar

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. HDFS-12956 TestErasureCodingCLI failing because of mismatch in expected output.

          • Major - Major loss of function.
          • Resolved
          is broken by

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-12819 Setting/Unsetting EC policy shows warning if the directory is not empty

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              ajayydv Ajay Kumar
              shahrs87 Rushabh Shah
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: