Uploaded image for project: 'Chukwa (retired)'
  1. Chukwa (retired)
  2. CHUKWA-686

Creating a new adaptor: FileTailingAdaptorPreserveLines

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 0.6.0
    • Data Collection
    • None

    Description

      FileTailingAdaptor is great to tail log files and send them to Hadoop.
      However, last line of the chunk is usually cut which leads to some errors.

      CharFileTailingAdaptorUTF8 can solve such problem.
      Nonetheless, this adaptor calls the MapProcessor.process() method for every
      line in each chunk, thus slowing a lot the Demux phase.

      FileTailingAdaptorPreserveLines adaptor mixes the benefits of the two
      adaptors: the (Demux) speed of FileTailingAdaptor and
      the preservation of lines from CharFileTailingAdaptorUTF8.

      Attachments

        1. chukwa-686.patch
          8 kB
          Sourygna Luangsay
        2. chukwa-686-2.patch
          8 kB
          Sourygna Luangsay

        Activity

          People

            sourygna Sourygna Luangsay
            sourygna Sourygna Luangsay
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: