Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-409

Make SocketTeeWriter work in a single stage pipleine

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.3.0
    • Fix Version/s: None
    • Component/s: Data Collection
    • Labels:
      None

      Description

      I wanted to use SocketTeeWriter without going through the steps of
      writing to HDFS.
      It seems that PipelineStageWriter is designed to have any number of
      PipeLineable writers, optionally followed by a SeqFileWriter.
      So I changed my collector config :

      <property>
      <name>chukwaCollector.writerClass</name>
      <value>org.apache.hadoop.chukwa.datacollection.writer.PipelineStageWriter</value>
      </property>

      <property>
      <name>chukwaCollector.pipeline</name>
      <value>org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter</value>
      </property>

      After doing one minor change to SocketTeeWriter, I could get this to
      work. The advantage is that now I do not need to set up HDFS.

      1. CHUKWA-409.patch
        0.9 kB
        thushara wijeratna

        Activity

        Hide
        asrabkin Ari Rabkin added a comment -

        +1; I will commit this to Trunk tomorrow, barring objection.

        Priority is minor, because there's an easy workaround for 0.3:
        Use a pipeline such as org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,org.apache.hadoop.chukwa.datacollection.writer.NullWriter

        Show
        asrabkin Ari Rabkin added a comment - +1; I will commit this to Trunk tomorrow, barring objection. Priority is minor, because there's an easy workaround for 0.3: Use a pipeline such as org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,org.apache.hadoop.chukwa.datacollection.writer.NullWriter
        Hide
        asrabkin Ari Rabkin added a comment -

        I just committed this. Thanks, Thushara!

        Show
        asrabkin Ari Rabkin added a comment - I just committed this. Thanks, Thushara!

          People

          • Assignee:
            Unassigned
            Reporter:
            thushara thushara wijeratna
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development