Bigtop
  1. Bigtop
  2. BIGTOP-719

Add TestTextSnappy to test hadoop fs -text with snappy compressed files

    Details

    • Type: Test Test
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.3.1, 0.5.0
    • Fix Version/s: 0.7.0
    • Component/s: Tests
    • Labels:
      None

      Description

      This test should catch problems like https://issues.apache.org/jira/browse/HADOOP-8833

      Before the HADOOP-8833 fix, the "hadoop fs -text" command did not work on snappy compressed files.

      This test runs the command on a snappy file and compares the actual output with the expected output.

      1. BIGTOP-719-trunk.patch
        3 kB
        Stephen Chu
      2. part-00001.snappy
        0.0 kB
        Stephen Chu
      3. TestTextSnappy.patch
        3 kB
        Stephen Chu

        Issue Links

          Activity

          Hide
          Bruno Mahé added a comment -

          Is it normal if part-000001.snappy seems empty?

          Show
          Bruno Mahé added a comment - Is it normal if part-000001.snappy seems empty?
          Hide
          Stephen Chu added a comment -

          Yeah, the attached part-00001.snappy file is only 50B. Very small, but enough to reproduce HADOOP-8833 (test will fail against Bigtop 0.3.0 release).

          Show
          Stephen Chu added a comment - Yeah, the attached part-00001.snappy file is only 50B. Very small, but enough to reproduce HADOOP-8833 (test will fail against Bigtop 0.3.0 release).
          Hide
          Konstantin Boudnik added a comment -

          +1

          HADOOP-8833 has been fixed against 2.x version. Shall your patch be applied against the trunk as well?

          Show
          Konstantin Boudnik added a comment - +1 HADOOP-8833 has been fixed against 2.x version. Shall your patch be applied against the trunk as well?
          Hide
          Stephen Chu added a comment -

          Yep, you're right. Should add the test to trunk as well. I'll post a patch soon.

          Show
          Stephen Chu added a comment - Yep, you're right. Should add the test to trunk as well. I'll post a patch soon.
          Hide
          Stephen Chu added a comment -

          Attaching patch for trunk.

          Committed test to branch-0.3.1

          Show
          Stephen Chu added a comment - Attaching patch for trunk. Committed test to branch-0.3.1
          Hide
          Konstantin Boudnik added a comment -

          I would also suggest not to use
          System.out.println anywhere

          Show
          Konstantin Boudnik added a comment - I would also suggest not to use System.out.println anywhere
          Hide
          Bruno Mahé added a comment -

          What about branch-0.3?

          Show
          Bruno Mahé added a comment - What about branch-0.3?
          Hide
          Konstantin Boudnik added a comment -

          Bruno, I think this was I who set the affected version to 0.3.1 because 0.3 has been released.

          Show
          Konstantin Boudnik added a comment - Bruno, I think this was I who set the affected version to 0.3.1 because 0.3 has been released.
          Hide
          Bruno Mahé added a comment -

          That's why I was talking about the branch 0.3, not the tag (it would not make sense anyway).
          As the release manager for 0.3.1, how do you see the development for 0.3.2? A branch of branch-0.3 or branching on top of branch-0.3.1?
          I don't mind either way but I was under the assumption that the development on 0.3.X would happen on branch-0.3.

          Show
          Bruno Mahé added a comment - That's why I was talking about the branch 0.3, not the tag (it would not make sense anyway). As the release manager for 0.3.1, how do you see the development for 0.3.2? A branch of branch-0.3 or branching on top of branch-0.3.1? I don't mind either way but I was under the assumption that the development on 0.3.X would happen on branch-0.3.
          Hide
          Roman Shaposhnik added a comment -

          +1 and committed!

          Show
          Roman Shaposhnik added a comment - +1 and committed!

            People

            • Assignee:
              Stephen Chu
              Reporter:
              Stephen Chu
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development