Chukwa
  1. Chukwa
  2. CHUKWA-119

ChunkImpl requires ChukwaAgent class

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.1.2
    • Fix Version/s: 0.1.2
    • Component/s: Data Collection
    • Labels:
      None

      Description

      If ChukwaAgent class is not available on the collector side, the collector is unable to read a chunk.
      // code
      ChunkImpl()

      { this.tags = ChukwaAgent.getTags(); }

      // Stacktrace
      Java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.chukwa.datacollection.agent.ChukwaAgent
      at org.apache.hadoop.chukwa.ChunkImpl.<init>(ChunkImpl.java:46)
      at org.apache.hadoop.chukwa.ChunkImpl.read(ChunkImpl.java:229)
      at org.apache.hadoop.chukwa.datacollection.collector.servlet.ServletCollector.accept(ServletCollector.java:140)

      1. CHUKWA-119-3.patch
        8 kB
        Jerome Boulon
      2. CHUKWA-119-2.patch
        7 kB
        Jerome Boulon
      3. CHUKWA-119.patch
        5 kB
        Eric Yang

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        5d 1h 18m 1 Jerome Boulon 15/Apr/09 21:57
        Patch Available Patch Available Resolved Resolved
        23h 7m 1 Eric Yang 16/Apr/09 21:04
        Eric Yang made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Eric Yang added a comment -

        I just committed this, thanks Jerome.

        Show
        Eric Yang added a comment - I just committed this, thanks Jerome.
        Hide
        Cheng added a comment -

        +1 tested. looks good.

        Show
        Cheng added a comment - +1 tested. looks good.
        Jerome Boulon made changes -
        Attachment CHUKWA-119-3.patch [ 12405600 ]
        Hide
        Jerome Boulon added a comment -

        create the in memory queue on demand.

        Show
        Jerome Boulon added a comment - create the in memory queue on demand.
        Hide
        Jerome Boulon added a comment -

        My fault, dataFactory initialized the InMemoryQueue ...

        Show
        Jerome Boulon added a comment - My fault, dataFactory initialized the InMemoryQueue ...
        Hide
        Cheng added a comment -

        Jerome, I applied your patch to my work copy but got error below when I run the collector.

        java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.chukwa.datacollection.DataFactory
        at org.apache.hadoop.chukwa.ChunkImpl.<init>(ChunkImpl.java:34)
        at org.apache.hadoop.chukwa.ChunkImpl.read(ChunkImpl.java:230)
        at org.apache.hadoop.chukwa.datacollection.collector.servlet.ServletCollector.accept(ServletCollector.java:140)
        at org.apache.hadoop.chukwa.datacollection.collector.servlet.ServletCollector.doPost(ServletCollector.java:180)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)

        Show
        Cheng added a comment - Jerome, I applied your patch to my work copy but got error below when I run the collector. java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.chukwa.datacollection.DataFactory at org.apache.hadoop.chukwa.ChunkImpl.<init>(ChunkImpl.java:34) at org.apache.hadoop.chukwa.ChunkImpl.read(ChunkImpl.java:230) at org.apache.hadoop.chukwa.datacollection.collector.servlet.ServletCollector.accept(ServletCollector.java:140) at org.apache.hadoop.chukwa.datacollection.collector.servlet.ServletCollector.doPost(ServletCollector.java:180) at javax.servlet.http.HttpServlet.service(HttpServlet.java:760) at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
        Jerome Boulon made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Assignee Eric Yang [ eyang ] Jerome Boulon [ jboulon ]
        Hide
        Jerome Boulon added a comment -
        • Change Chunk interface, change the setTag method to addTag
        • ChunkImpl.getBlankChunk() return a real blank chunk
        • Chunks are initialized with default tags: dataFactory.getDefaultTags()
        • ChukwaAgent set default tags:
        Show
        Jerome Boulon added a comment - Change Chunk interface, change the setTag method to addTag ChunkImpl.getBlankChunk() return a real blank chunk Chunks are initialized with default tags: dataFactory.getDefaultTags() ChukwaAgent set default tags:
        Jerome Boulon made changes -
        Attachment CHUKWA-119-2.patch [ 12405576 ]
        Jerome Boulon made changes -
        Attachment CHUKWA-119.patch [ 12405575 ]
        Jerome Boulon made changes -
        Attachment CHUKWA-119.patch [ 12405575 ]
        Hide
        Jerome Boulon added a comment -

        -1 on the patch: CHUKWA-119.patch.
        Instead of doing "c.setTags(ChukwaAgent.getTags());" all over the place, add a static block inside the agent that will set the correct tag once.

        Show
        Jerome Boulon added a comment - -1 on the patch: CHUKWA-119 .patch. Instead of doing "c.setTags(ChukwaAgent.getTags());" all over the place, add a static block inside the agent that will set the correct tag once.
        Eric Yang made changes -
        Assignee Jerome Boulon [ jboulon ] Eric Yang [ eyang ]
        Eric Yang made changes -
        Attachment CHUKWA-119.patch [ 12405464 ]
        Hide
        Eric Yang added a comment -

        Implemented setTags in adaptors, and remove ChukwaAgent from Chunk Implementation class.

        Show
        Eric Yang added a comment - Implemented setTags in adaptors, and remove ChukwaAgent from Chunk Implementation class.
        Hide
        Eric Yang added a comment -

        +1 on setTag in ChunkImpl.

        Show
        Eric Yang added a comment - +1 on setTag in ChunkImpl.
        Hide
        Jerome Boulon added a comment - - edited

        +1 on using a static Tag. That will be a good idea. reverse the link, add a static method setTag on ChunkImpl. Agent will call setTag
        I don't see why we should build yet another metrics framework.

        Show
        Jerome Boulon added a comment - - edited +1 on using a static Tag. That will be a good idea. reverse the link, add a static method setTag on ChunkImpl. Agent will call setTag I don't see why we should build yet another metrics framework.
        Hide
        Ari Rabkin added a comment -

        Hrm. I think the right fix is to restructure the code. Suppose we have some static "tags" variable in ChunkImpl, and have ChukwaAgent set it. That way, we avoid any dependency of ChunkImpl on ChukwaAgent.

        Thoughts?

        Show
        Ari Rabkin added a comment - Hrm. I think the right fix is to restructure the code. Suppose we have some static "tags" variable in ChunkImpl, and have ChukwaAgent set it. That way, we avoid any dependency of ChunkImpl on ChukwaAgent. Thoughts?
        Eric Yang made changes -
        Affects Version/s 0.1.2 [ 12313776 ]
        Fix Version/s 0.1.2 [ 12313776 ]
        Eric Yang made changes -
        Component/s data collection [ 12312723 ]
        Hide
        Eric Yang added a comment -

        The source of the problem is caused by the new agentMetrics. It prevents ChukwaAgent from initialize properly in Collector's vm. We should use hadoop MetricsContext to calculate chukwa metrics. It is better to send the stats as chucks since it's light weight and reuse our own code.

        Show
        Eric Yang added a comment - The source of the problem is caused by the new agentMetrics. It prevents ChukwaAgent from initialize properly in Collector's vm. We should use hadoop MetricsContext to calculate chukwa metrics. It is better to send the stats as chucks since it's light weight and reuse our own code.
        Jerome Boulon made changes -
        Assignee Jerome Boulon [ jboulon ]
        Jerome Boulon made changes -
        Field Original Value New Value
        Priority Major [ 3 ] Blocker [ 1 ]
        Hide
        Jerome Boulon added a comment -

        Mark as blocker since this will activate ChukwaAgent metrics

        Show
        Jerome Boulon added a comment - Mark as blocker since this will activate ChukwaAgent metrics
        Jerome Boulon created issue -

          People

          • Assignee:
            Jerome Boulon
            Reporter:
            Jerome Boulon
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development