Chukwa
  1. Chukwa
  2. CHUKWA-119

ChunkImpl requires ChukwaAgent class

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.1.2
    • Fix Version/s: 0.1.2
    • Component/s: Data Collection
    • Labels:
      None

      Description

      If ChukwaAgent class is not available on the collector side, the collector is unable to read a chunk.
      // code
      ChunkImpl()

      { this.tags = ChukwaAgent.getTags(); }

      // Stacktrace
      Java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.chukwa.datacollection.agent.ChukwaAgent
      at org.apache.hadoop.chukwa.ChunkImpl.<init>(ChunkImpl.java:46)
      at org.apache.hadoop.chukwa.ChunkImpl.read(ChunkImpl.java:229)
      at org.apache.hadoop.chukwa.datacollection.collector.servlet.ServletCollector.accept(ServletCollector.java:140)

      1. CHUKWA-119-3.patch
        8 kB
        Jerome Boulon
      2. CHUKWA-119-2.patch
        7 kB
        Jerome Boulon
      3. CHUKWA-119.patch
        5 kB
        Eric Yang

        Activity

        Hide
        Jerome Boulon added a comment -

        Mark as blocker since this will activate ChukwaAgent metrics

        Show
        Jerome Boulon added a comment - Mark as blocker since this will activate ChukwaAgent metrics
        Hide
        Eric Yang added a comment -

        The source of the problem is caused by the new agentMetrics. It prevents ChukwaAgent from initialize properly in Collector's vm. We should use hadoop MetricsContext to calculate chukwa metrics. It is better to send the stats as chucks since it's light weight and reuse our own code.

        Show
        Eric Yang added a comment - The source of the problem is caused by the new agentMetrics. It prevents ChukwaAgent from initialize properly in Collector's vm. We should use hadoop MetricsContext to calculate chukwa metrics. It is better to send the stats as chucks since it's light weight and reuse our own code.
        Hide
        Ari Rabkin added a comment -

        Hrm. I think the right fix is to restructure the code. Suppose we have some static "tags" variable in ChunkImpl, and have ChukwaAgent set it. That way, we avoid any dependency of ChunkImpl on ChukwaAgent.

        Thoughts?

        Show
        Ari Rabkin added a comment - Hrm. I think the right fix is to restructure the code. Suppose we have some static "tags" variable in ChunkImpl, and have ChukwaAgent set it. That way, we avoid any dependency of ChunkImpl on ChukwaAgent. Thoughts?
        Hide
        Jerome Boulon added a comment - - edited

        +1 on using a static Tag. That will be a good idea. reverse the link, add a static method setTag on ChunkImpl. Agent will call setTag
        I don't see why we should build yet another metrics framework.

        Show
        Jerome Boulon added a comment - - edited +1 on using a static Tag. That will be a good idea. reverse the link, add a static method setTag on ChunkImpl. Agent will call setTag I don't see why we should build yet another metrics framework.
        Hide
        Eric Yang added a comment -

        +1 on setTag in ChunkImpl.

        Show
        Eric Yang added a comment - +1 on setTag in ChunkImpl.
        Hide
        Eric Yang added a comment -

        Implemented setTags in adaptors, and remove ChukwaAgent from Chunk Implementation class.

        Show
        Eric Yang added a comment - Implemented setTags in adaptors, and remove ChukwaAgent from Chunk Implementation class.
        Hide
        Jerome Boulon added a comment -

        -1 on the patch: CHUKWA-119.patch.
        Instead of doing "c.setTags(ChukwaAgent.getTags());" all over the place, add a static block inside the agent that will set the correct tag once.

        Show
        Jerome Boulon added a comment - -1 on the patch: CHUKWA-119 .patch. Instead of doing "c.setTags(ChukwaAgent.getTags());" all over the place, add a static block inside the agent that will set the correct tag once.
        Hide
        Jerome Boulon added a comment -
        • Change Chunk interface, change the setTag method to addTag
        • ChunkImpl.getBlankChunk() return a real blank chunk
        • Chunks are initialized with default tags: dataFactory.getDefaultTags()
        • ChukwaAgent set default tags:
        Show
        Jerome Boulon added a comment - Change Chunk interface, change the setTag method to addTag ChunkImpl.getBlankChunk() return a real blank chunk Chunks are initialized with default tags: dataFactory.getDefaultTags() ChukwaAgent set default tags:
        Hide
        Cheng added a comment -

        Jerome, I applied your patch to my work copy but got error below when I run the collector.

        java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.chukwa.datacollection.DataFactory
        at org.apache.hadoop.chukwa.ChunkImpl.<init>(ChunkImpl.java:34)
        at org.apache.hadoop.chukwa.ChunkImpl.read(ChunkImpl.java:230)
        at org.apache.hadoop.chukwa.datacollection.collector.servlet.ServletCollector.accept(ServletCollector.java:140)
        at org.apache.hadoop.chukwa.datacollection.collector.servlet.ServletCollector.doPost(ServletCollector.java:180)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)

        Show
        Cheng added a comment - Jerome, I applied your patch to my work copy but got error below when I run the collector. java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.chukwa.datacollection.DataFactory at org.apache.hadoop.chukwa.ChunkImpl.<init>(ChunkImpl.java:34) at org.apache.hadoop.chukwa.ChunkImpl.read(ChunkImpl.java:230) at org.apache.hadoop.chukwa.datacollection.collector.servlet.ServletCollector.accept(ServletCollector.java:140) at org.apache.hadoop.chukwa.datacollection.collector.servlet.ServletCollector.doPost(ServletCollector.java:180) at javax.servlet.http.HttpServlet.service(HttpServlet.java:760) at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
        Hide
        Jerome Boulon added a comment -

        My fault, dataFactory initialized the InMemoryQueue ...

        Show
        Jerome Boulon added a comment - My fault, dataFactory initialized the InMemoryQueue ...
        Hide
        Jerome Boulon added a comment -

        create the in memory queue on demand.

        Show
        Jerome Boulon added a comment - create the in memory queue on demand.
        Hide
        Cheng added a comment -

        +1 tested. looks good.

        Show
        Cheng added a comment - +1 tested. looks good.
        Hide
        Eric Yang added a comment -

        I just committed this, thanks Jerome.

        Show
        Eric Yang added a comment - I just committed this, thanks Jerome.

          People

          • Assignee:
            Jerome Boulon
            Reporter:
            Jerome Boulon
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development