Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-1107

Further improve error reporting when exporting malformed data

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 1.4.3
    • Fix Version/s: 1.4.5
    • Component/s: None
    • Labels:
      None

      Description

      We've significantly improved error handling when exporting malformed data in SQOOP-720, where we started dumping a lot of state into log. Unfortunately current output do show on which column the parsing exception occurred. This is usually not an issue for primitive Java types as the underlying exception is descriptive enough:

      Caused by: java.lang.NumberFormatException: For input string: "A"
      	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
      	at java.lang.Integer.parseInt(Integer.java:449)
      	at java.lang.Integer.valueOf(Integer.java:554)
      	at text.__loadFromFields(text.java:196)
      

      However BigDecimal do not print out the value:

      Exception: 
      java.lang.NumberFormatException
      	at java.math.BigDecimal.<init>(BigDecimal.java:459)
      	at java.math.BigDecimal.<init>(BigDecimal.java:728)
      

      I would therefore suggest to start catching {{RuntimeException}}s in the parsing code and re-throwing them with value of currently parsed column.

      1. bugSQOOP-1107.patch
        1 kB
        Jarek Jarcec Cecho

        Issue Links

          Activity

          Hide
          jarcec Jarek Jarcec Cecho added a comment -

          Postponing to 1.4.5 to unblock work on 1.4.4 release.

          Show
          jarcec Jarek Jarcec Cecho added a comment - Postponing to 1.4.5 to unblock work on 1.4.4 release.
          Hide
          hshreedharan Hari Shreedharan added a comment -

          +1. Committing.

          Show
          hshreedharan Hari Shreedharan added a comment - +1. Committing.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 15ae44016d94bcb4eb1c9e965d8424d395b3b959 in branch refs/heads/trunk from Hari Shreedharan
          [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=15ae440 ]

          SQOOP-1107. Further improve error reporting when exporting malformed data

          (Jarek Jarcec Cecho via Hari Shreedharan)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 15ae44016d94bcb4eb1c9e965d8424d395b3b959 in branch refs/heads/trunk from Hari Shreedharan [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=15ae440 ] SQOOP-1107 . Further improve error reporting when exporting malformed data (Jarek Jarcec Cecho via Hari Shreedharan)
          Hide
          hshreedharan Hari Shreedharan added a comment -

          Committed. Thanks Jarcec!

          Show
          hshreedharan Hari Shreedharan added a comment - Committed. Thanks Jarcec!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #848 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/848/)
          SQOOP-1107. Further improve error reporting when exporting malformed data (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=15ae44016d94bcb4eb1c9e965d8424d395b3b959)

          • src/java/org/apache/sqoop/orm/ClassWriter.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #848 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/848/ ) SQOOP-1107 . Further improve error reporting when exporting malformed data (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=15ae44016d94bcb4eb1c9e965d8424d395b3b959 ) src/java/org/apache/sqoop/orm/ClassWriter.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #845 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/845/)
          SQOOP-1107. Further improve error reporting when exporting malformed data (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=15ae44016d94bcb4eb1c9e965d8424d395b3b959)

          • src/java/org/apache/sqoop/orm/ClassWriter.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #845 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/845/ ) SQOOP-1107 . Further improve error reporting when exporting malformed data (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=15ae44016d94bcb4eb1c9e965d8424d395b3b959 ) src/java/org/apache/sqoop/orm/ClassWriter.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1043 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1043/)
          SQOOP-1107. Further improve error reporting when exporting malformed data (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=15ae44016d94bcb4eb1c9e965d8424d395b3b959)

          • src/java/org/apache/sqoop/orm/ClassWriter.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1043 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1043/ ) SQOOP-1107 . Further improve error reporting when exporting malformed data (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=15ae44016d94bcb4eb1c9e965d8424d395b3b959 ) src/java/org/apache/sqoop/orm/ClassWriter.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #804 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/804/)
          SQOOP-1107. Further improve error reporting when exporting malformed data (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=15ae44016d94bcb4eb1c9e965d8424d395b3b959)

          • src/java/org/apache/sqoop/orm/ClassWriter.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #804 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/804/ ) SQOOP-1107 . Further improve error reporting when exporting malformed data (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=15ae44016d94bcb4eb1c9e965d8424d395b3b959 ) src/java/org/apache/sqoop/orm/ClassWriter.java
          Hide
          jarcec Jarek Jarcec Cecho added a comment -

          The failure in profile 200 is a jenkins issue.

          Show
          jarcec Jarek Jarcec Cecho added a comment - The failure in profile 200 is a jenkins issue.

            People

            • Assignee:
              jarcec Jarek Jarcec Cecho
              Reporter:
              jarcec Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development