Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-324

MetricsDataLoader should handle chukwa record with 0 matched fields

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.2.0
    • Fix Version/s: 0.2.0
    • Component/s: Data Processors
    • Labels:
      None
    • Release Note:
      Hide
      CHUKWA-324. For 0 matched field, changed MDL to skip empty SQL statement. (Eric Yang)
      Show
      CHUKWA-324 . For 0 matched field, changed MDL to skip empty SQL statement. (Eric Yang)

      Description

      As seen from postProcess.log:

      2009-06-22 12:01:17,946 TRACE pool-1-thread-2 MetricDataLoader - INSERT INTO mr_job_2059_week SET ON DUPLICATE KEY
      UPDATE ;
      2009-06-22 12:01:25,957 ERROR pool-1-thread-2 MetricDataLoader - java.sql.BatchUpdateException: You have an error in
      your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'ON
      DUPLICATE KEY UPDATE' at line 1
      java.sql.BatchUpdateException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL
      server version for the right syntax to use near 'ON DUPLICATE KEY UPDATE' at line 1
      at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:1007)
      at org.apache.hadoop.chukwa.dataloader.MetricDataLoader.run(MetricDataLoader.java:484)
      at org.apache.hadoop.chukwa.dataloader.MetricDataLoader.call(MetricDataLoader.java:561)
      at org.apache.hadoop.chukwa.dataloader.MetricDataLoader.call(MetricDataLoader.java:50)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
      at java.lang.Thread.run(Thread.java:619)
      2009-06-22 12:01:25,957 ERROR pool-1-thread-2 MetricDataLoader - SQLException: You have an error in your SQL syntax;
      check the manual that corresponds to your MySQL server version for the right syntax to use near 'ON DUPLICATE KEY
      UPDATE' at line 1

      In this case, MDL should skip the record.

        Activity

        Hide
        hudson Hudson added a comment -

        Integrated in Chukwa-trunk #61 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/61/)
        . For 0 matched field, changed MDL to skip empty SQL statement. (Eric Yang)

        Show
        hudson Hudson added a comment - Integrated in Chukwa-trunk #61 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/61/ ) . For 0 matched field, changed MDL to skip empty SQL statement. (Eric Yang)
        Hide
        eyang Eric Yang added a comment -

        I just committed this, thanks Terence.

        Show
        eyang Eric Yang added a comment - I just committed this, thanks Terence.
        Hide
        terencekwan Terence Kwan added a comment -

        look good. +1

        Show
        terencekwan Terence Kwan added a comment - look good. +1
        Hide
        eyang Eric Yang added a comment -
        • Added detection for 0 fields matched condition.
        • Fixed a bug where metric.record_type.field case sensitivity had a mix match.
        Show
        eyang Eric Yang added a comment - Added detection for 0 fields matched condition. Fixed a bug where metric.record_type.field case sensitivity had a mix match.

          People

          • Assignee:
            eyang Eric Yang
            Reporter:
            eyang Eric Yang
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development