Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-686

Creating a new adaptor: FileTailingAdaptorPreserveLines

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.6.0
    • Component/s: Data Collection
    • Labels:
      None

      Description

      FileTailingAdaptor is great to tail log files and send them to Hadoop.
      However, last line of the chunk is usually cut which leads to some errors.

      CharFileTailingAdaptorUTF8 can solve such problem.
      Nonetheless, this adaptor calls the MapProcessor.process() method for every
      line in each chunk, thus slowing a lot the Demux phase.

      FileTailingAdaptorPreserveLines adaptor mixes the benefits of the two
      adaptors: the (Demux) speed of FileTailingAdaptor and
      the preservation of lines from CharFileTailingAdaptorUTF8.

        Attachments

        1. chukwa-686.patch
          8 kB
          Sourygna Luangsay
        2. chukwa-686-2.patch
          8 kB
          Sourygna Luangsay

          Activity

            People

            • Assignee:
              sourygna Sourygna Luangsay
              Reporter:
              sourygna Sourygna Luangsay
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: