Sqoop
  1. Sqoop
  2. SQOOP-720

Improve error handling when exporting malformed text data

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.2
    • Fix Version/s: 1.4.3
    • Component/s: None
    • Labels:
      None

      Description

      We're currently catching only one specific checked exception during data parsing in Export tool. All non-checked exceptions (NumberFormatException) are passed to the hadoop framework. We should improve that by catching everything and providing debug info to help user triangulate the malformed data.

      1. bugSQOOP-720.patch
        2 kB
        Jarek Jarcec Cecho

        Issue Links

          Activity

          Jarek Jarcec Cecho created issue -
          Jarek Jarcec Cecho made changes -
          Field Original Value New Value
          Remote Link This issue links to "Review board (Web Link)" [ 11510 ]
          Jarek Jarcec Cecho made changes -
          Attachment bugSQOOP-720.patch [ 12554631 ]
          Jarek Jarcec Cecho made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Cheolsoo Park added a comment -

          +1.

          Committed to trunk. Thanks Jarcec!

          Show
          Cheolsoo Park added a comment - +1. Committed to trunk. Thanks Jarcec!
          Cheolsoo Park made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop20 #340 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/340/)
          SQOOP-720: Improve error handling when exporting malformed text data (Revision 528f7a8bb51863b328da4a81873daa5e87fe029c)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=528f7a8bb51863b328da4a81873daa5e87fe029c
          Files :

          • src/java/org/apache/sqoop/mapreduce/TextExportMapper.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop20 #340 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/340/ ) SQOOP-720 : Improve error handling when exporting malformed text data (Revision 528f7a8bb51863b328da4a81873daa5e87fe029c) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=528f7a8bb51863b328da4a81873daa5e87fe029c Files : src/java/org/apache/sqoop/mapreduce/TextExportMapper.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop23 #510 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/510/)
          SQOOP-720: Improve error handling when exporting malformed text data (Revision 528f7a8bb51863b328da4a81873daa5e87fe029c)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=528f7a8bb51863b328da4a81873daa5e87fe029c
          Files :

          • src/java/org/apache/sqoop/mapreduce/TextExportMapper.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop23 #510 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/510/ ) SQOOP-720 : Improve error handling when exporting malformed text data (Revision 528f7a8bb51863b328da4a81873daa5e87fe029c) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=528f7a8bb51863b328da4a81873daa5e87fe029c Files : src/java/org/apache/sqoop/mapreduce/TextExportMapper.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop200 #350 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/350/)
          SQOOP-720: Improve error handling when exporting malformed text data (Revision 528f7a8bb51863b328da4a81873daa5e87fe029c)

          Result = FAILURE
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=528f7a8bb51863b328da4a81873daa5e87fe029c
          Files :

          • src/java/org/apache/sqoop/mapreduce/TextExportMapper.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop200 #350 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/350/ ) SQOOP-720 : Improve error handling when exporting malformed text data (Revision 528f7a8bb51863b328da4a81873daa5e87fe029c) Result = FAILURE cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=528f7a8bb51863b328da4a81873daa5e87fe029c Files : src/java/org/apache/sqoop/mapreduce/TextExportMapper.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop-ant-jdk-1.6-hadoop100 #342 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/342/)
          SQOOP-720: Improve error handling when exporting malformed text data (Revision 528f7a8bb51863b328da4a81873daa5e87fe029c)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=528f7a8bb51863b328da4a81873daa5e87fe029c
          Files :

          • src/java/org/apache/sqoop/mapreduce/TextExportMapper.java
          Show
          Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop100 #342 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/342/ ) SQOOP-720 : Improve error handling when exporting malformed text data (Revision 528f7a8bb51863b328da4a81873daa5e87fe029c) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=528f7a8bb51863b328da4a81873daa5e87fe029c Files : src/java/org/apache/sqoop/mapreduce/TextExportMapper.java
          Hide
          Jarek Jarcec Cecho added a comment -

          Failure on hadoop200 profile is expected due to SQOOP-731, will be fixed soon.

          Show
          Jarek Jarcec Cecho added a comment - Failure on hadoop200 profile is expected due to SQOOP-731 , will be fixed soon.
          Jarek Jarcec Cecho made changes -
          Fix Version/s 1.4.3 [ 12322644 ]

            People

            • Assignee:
              Jarek Jarcec Cecho
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development