Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-784

TestBackfillingLoader test case is failing

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Unit test TestBackfillingLoader is failing. The reason is because we changed to use Parquet for storing files on HDFS. However, some unit test classes such as QueueToWriterConnector, and TestBackfillingLoader are still using Sequence file.

      1. CHUKWA-784.patch
        45 kB
        Eric Yang
      2. CHUKWA-784-2.patch
        8 kB
        Eric Yang

        Activity

        Hide
        eyang Eric Yang added a comment -

        Fix LocalWriter to use Parquet File format, and fix test case to verify parquet file format for back filling.

        Show
        eyang Eric Yang added a comment - Fix LocalWriter to use Parquet File format, and fix test case to verify parquet file format for back filling.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Chukwa-master #532 (See https://builds.apache.org/job/Chukwa-master/532/)
        CHUKWA-784. Improve CharFileTailingAdaptorUTF8NewLineEscaped and (eyang: rev a990fc54a555a409b6716e170d9acb92d56bf2c3)

        • src/main/java/org/apache/hadoop/chukwa/datacollection/writer/parquet/ChukwaParquetWriter.java
        • CHANGES.txt
        • src/test/java/org/apache/hadoop/chukwa/datacollection/writer/TestChukwaWriters.java
        • src/main/java/org/apache/hadoop/chukwa/datacollection/writer/localfs/LocalWriter.java
        • src/test/java/org/apache/hadoop/chukwa/tools/backfilling/TestBackfillingLoader.java
        • src/main/java/org/apache/hadoop/chukwa/datacollection/writer/parquet/ChukwaAvroSchema.java
        • src/main/java/org/apache/hadoop/chukwa/tools/backfilling/QueueToWriterConnector.java
        • src/main/java/org/apache/hadoop/chukwa/datacollection/adaptor/filetailer/CharFileTailingAdaptorUTF8NewLineEscaped.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Chukwa-master #532 (See https://builds.apache.org/job/Chukwa-master/532/ ) CHUKWA-784 . Improve CharFileTailingAdaptorUTF8NewLineEscaped and (eyang: rev a990fc54a555a409b6716e170d9acb92d56bf2c3) src/main/java/org/apache/hadoop/chukwa/datacollection/writer/parquet/ChukwaParquetWriter.java CHANGES.txt src/test/java/org/apache/hadoop/chukwa/datacollection/writer/TestChukwaWriters.java src/main/java/org/apache/hadoop/chukwa/datacollection/writer/localfs/LocalWriter.java src/test/java/org/apache/hadoop/chukwa/tools/backfilling/TestBackfillingLoader.java src/main/java/org/apache/hadoop/chukwa/datacollection/writer/parquet/ChukwaAvroSchema.java src/main/java/org/apache/hadoop/chukwa/tools/backfilling/QueueToWriterConnector.java src/main/java/org/apache/hadoop/chukwa/datacollection/adaptor/filetailer/CharFileTailingAdaptorUTF8NewLineEscaped.java
        Hide
        eyang Eric Yang added a comment -

        Fix license and code clean up.

        Show
        eyang Eric Yang added a comment - Fix license and code clean up.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Chukwa-master #533 (See https://builds.apache.org/job/Chukwa-master/533/)
        CHUKWA-784. Improve CharFileTailingAdaptorUTF8NewLineEscaped and (eyang: rev 93a1f75c45f7333dcba2980f5ec9ebbdc64bb9d3)

        • src/test/java/org/apache/hadoop/chukwa/datacollection/writer/TestChukwaWriters.java
        • src/main/java/org/apache/hadoop/chukwa/datacollection/writer/parquet/ChukwaAvroSchema.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Chukwa-master #533 (See https://builds.apache.org/job/Chukwa-master/533/ ) CHUKWA-784 . Improve CharFileTailingAdaptorUTF8NewLineEscaped and (eyang: rev 93a1f75c45f7333dcba2980f5ec9ebbdc64bb9d3) src/test/java/org/apache/hadoop/chukwa/datacollection/writer/TestChukwaWriters.java src/main/java/org/apache/hadoop/chukwa/datacollection/writer/parquet/ChukwaAvroSchema.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Chukwa-master #534 (See https://builds.apache.org/job/Chukwa-master/534/)
        CHUKWA-784. Improve CharFileTailingAdaptorUTF8NewLineEscaped and (eyang: rev c440a9f901a9f29b60f10047cb67e0b1cb188f89)

        • src/main/java/org/apache/hadoop/chukwa/datacollection/writer/localfs/LocalWriter.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Chukwa-master #534 (See https://builds.apache.org/job/Chukwa-master/534/ ) CHUKWA-784 . Improve CharFileTailingAdaptorUTF8NewLineEscaped and (eyang: rev c440a9f901a9f29b60f10047cb67e0b1cb188f89) src/main/java/org/apache/hadoop/chukwa/datacollection/writer/localfs/LocalWriter.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Chukwa-master #535 (See https://builds.apache.org/job/Chukwa-master/535/)
        CHUKWA-784. Improve CharFileTailingAdaptorUTF8NewLineEscaped and (eyang: rev a13bc5ae09af5b60c52b465792d33bd5fc86ad12)

        • pom.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Chukwa-master #535 (See https://builds.apache.org/job/Chukwa-master/535/ ) CHUKWA-784 . Improve CharFileTailingAdaptorUTF8NewLineEscaped and (eyang: rev a13bc5ae09af5b60c52b465792d33bd5fc86ad12) pom.xml
        Hide
        eyang Eric Yang added a comment -

        I just committed this. Unit tests are passing again. yay!

        Show
        eyang Eric Yang added a comment - I just committed this. Unit tests are passing again. yay!

          People

          • Assignee:
            eyang Eric Yang
            Reporter:
            eyang Eric Yang
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development