Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-808

SQLExceptions From Batched Exports Aren't Very Helpful

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.2
    • Fix Version/s: 1.4.3
    • Component/s: None
    • Labels:
      None

      Description

      They just show the error messages from the BatchUpdateException, which has a generic message saying look at the result of getNextException() for specific details.

      1. SQOOP-808.1.patch
        1 kB
        Nick White
      2. SQOOP-808.2.patch
        2 kB
        Nick White
      3. SQOOP-808.patch
        1 kB
        Nick White
      4. TestSQLErrorMessages.java
        4 kB
        Nick White
      5. TestSQLErrorMessages.log
        80 kB
        Nick White

        Activity

        Hide
        jarcec Jarek Jarcec Cecho added a comment -

        Hi Nick,
        I'm wondering, my logs seems to always contain entire exception chain, so why not simply look at next exception? I'm worried about silently skipping top level exception as it might hide issues and actually might fail in case that getNextException() will be null.

        Jarcec

        Show
        jarcec Jarek Jarcec Cecho added a comment - Hi Nick, I'm wondering, my logs seems to always contain entire exception chain, so why not simply look at next exception? I'm worried about silently skipping top level exception as it might hide issues and actually might fail in case that getNextException() will be null. Jarcec
        Hide
        njw45 Nick White added a comment -

        I've attached a revised patch that uses the top level exception if getNextException returns null. While HSQLDB chains the SQLException as a cause of the BatchUpdateException, postgres' http://jdbc.postgresql.org/download/postgresql-8.4-703.jdbc4.jar driver doesn't (it only adds the SQLException as a getNextException).

        I agree that this incompatibility may cause problems if there's a getNextException and useful information in the BatchUpdateException's cause, but I'm not sure what to do about that...(are there any databases that do this?)

        Show
        njw45 Nick White added a comment - I've attached a revised patch that uses the top level exception if getNextException returns null. While HSQLDB chains the SQLException as a cause of the BatchUpdateException, postgres' http://jdbc.postgresql.org/download/postgresql-8.4-703.jdbc4.jar driver doesn't (it only adds the SQLException as a getNextException). I agree that this incompatibility may cause problems if there's a getNextException and useful information in the BatchUpdateException's cause, but I'm not sure what to do about that...(are there any databases that do this?)
        Hide
        jarcec Jarek Jarcec Cecho added a comment -

        Nick,
        would you mind sharing simple table with example data that will trigger issue you're facing? I always got entire stack trace in the logs, so I'm not sure how this patch actually changes that.

        Jarcec

        Show
        jarcec Jarek Jarcec Cecho added a comment - Nick, would you mind sharing simple table with example data that will trigger issue you're facing? I always got entire stack trace in the logs, so I'm not sure how this patch actually changes that. Jarcec
        Hide
        njw45 Nick White added a comment -

        One of the two tests in the attached .java file fails without my patch and succeeds with it. I've attached the console output from the failure. It connects to a local postgres instance, database "db", username "xxx" and password "xxx" - I was using a version 9.2 server and the 8.4 version driver from my previous comment.

        Show
        njw45 Nick White added a comment - One of the two tests in the attached .java file fails without my patch and succeeds with it. I've attached the console output from the failure. It connects to a local postgres instance, database "db", username "xxx" and password "xxx" - I was using a version 9.2 server and the 8.4 version driver from my previous comment.
        Hide
        jarcec Jarek Jarcec Cecho added a comment -

        Hi Nick,
        thank you very much for your examples. They indeed helped me understand your issue. I've google around and I did not found better solution.

        Before I'll commit your patch - would you mind cleaning up check style warnings (you can run check style report by "ant checkstyle") and appending comment why you're looking into the next exception (maybe linking this issue?)?

        Jarcec

        Show
        jarcec Jarek Jarcec Cecho added a comment - Hi Nick, thank you very much for your examples. They indeed helped me understand your issue. I've google around and I did not found better solution. Before I'll commit your patch - would you mind cleaning up check style warnings (you can run check style report by "ant checkstyle") and appending comment why you're looking into the next exception (maybe linking this issue?)? Jarcec
        Hide
        njw45 Nick White added a comment -

        Thanks - I've attached a new version of the patch with the fixes you suggest.

        Show
        njw45 Nick White added a comment - Thanks - I've attached a new version of the patch with the fixes you suggest.
        Hide
        jarcec Jarek Jarcec Cecho added a comment -

        It's a simple patch, so I'm giving my +1 here.

        Show
        jarcec Jarek Jarcec Cecho added a comment - It's a simple patch, so I'm giving my +1 here.
        Hide
        jarcec Jarek Jarcec Cecho added a comment -
        Show
        jarcec Jarek Jarcec Cecho added a comment - The patch is in https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=7573450e18a577a0c6c61b2eae40ff47aa17c572 Thank you for your contribution Nick! Jarcec
        Hide
        hudson Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop200 #468 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/468/)
        SQOOP-808: SQLExceptions From Batched Exports Aren't Very Helpful (Revision 7573450e18a577a0c6c61b2eae40ff47aa17c572)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=7573450e18a577a0c6c61b2eae40ff47aa17c572
        Files :

        • src/java/org/apache/sqoop/mapreduce/AsyncSqlOutputFormat.java
        Show
        hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop200 #468 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/468/ ) SQOOP-808 : SQLExceptions From Batched Exports Aren't Very Helpful (Revision 7573450e18a577a0c6c61b2eae40ff47aa17c572) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=7573450e18a577a0c6c61b2eae40ff47aa17c572 Files : src/java/org/apache/sqoop/mapreduce/AsyncSqlOutputFormat.java
        Hide
        hudson Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop20 #489 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/489/)
        SQOOP-808: SQLExceptions From Batched Exports Aren't Very Helpful (Revision 7573450e18a577a0c6c61b2eae40ff47aa17c572)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=7573450e18a577a0c6c61b2eae40ff47aa17c572
        Files :

        • src/java/org/apache/sqoop/mapreduce/AsyncSqlOutputFormat.java
        Show
        hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop20 #489 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/489/ ) SQOOP-808 : SQLExceptions From Batched Exports Aren't Very Helpful (Revision 7573450e18a577a0c6c61b2eae40ff47aa17c572) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=7573450e18a577a0c6c61b2eae40ff47aa17c572 Files : src/java/org/apache/sqoop/mapreduce/AsyncSqlOutputFormat.java
        Hide
        hudson Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop23 #682 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/682/)
        SQOOP-808: SQLExceptions From Batched Exports Aren't Very Helpful (Revision 7573450e18a577a0c6c61b2eae40ff47aa17c572)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=7573450e18a577a0c6c61b2eae40ff47aa17c572
        Files :

        • src/java/org/apache/sqoop/mapreduce/AsyncSqlOutputFormat.java
        Show
        hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop23 #682 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/682/ ) SQOOP-808 : SQLExceptions From Batched Exports Aren't Very Helpful (Revision 7573450e18a577a0c6c61b2eae40ff47aa17c572) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=7573450e18a577a0c6c61b2eae40ff47aa17c572 Files : src/java/org/apache/sqoop/mapreduce/AsyncSqlOutputFormat.java
        Hide
        hudson Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop100 #472 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/472/)
        SQOOP-808: SQLExceptions From Batched Exports Aren't Very Helpful (Revision 7573450e18a577a0c6c61b2eae40ff47aa17c572)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=7573450e18a577a0c6c61b2eae40ff47aa17c572
        Files :

        • src/java/org/apache/sqoop/mapreduce/AsyncSqlOutputFormat.java
        Show
        hudson Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop100 #472 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/472/ ) SQOOP-808 : SQLExceptions From Batched Exports Aren't Very Helpful (Revision 7573450e18a577a0c6c61b2eae40ff47aa17c572) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=7573450e18a577a0c6c61b2eae40ff47aa17c572 Files : src/java/org/apache/sqoop/mapreduce/AsyncSqlOutputFormat.java

          People

          • Assignee:
            njw45 Nick White
            Reporter:
            njw45 Nick White
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development