Sqoop
  1. Sqoop
  2. SQOOP-652

Improve error message generated in DataDrivenDBInputFormat when no splitter is available

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 1.4.2
    • Fix Version/s: 1.4.3
    • Component/s: None
    • Labels:

      Description

      Currently DataDrivenDBInputFormat will throw following exception in case that there is no available Splitter:

      DBSplitter splitter = getSplitter(sqlDataType);
      if (null == splitter) {
        throw new IOException("Unknown SQL data type: " + sqlDataType);
      }
      

      Text of the exception is very confusing as it's not describing what has actually happened. Would be great if this message would be improved by saying that Sqoop do not have splitter for given sql type and therefore user should either use different split column (argument --split-by) or lower number of mappers to 1 (splitters are not used in such case).

        Activity

        Jarek Jarcec Cecho created issue -
        Linden Hillenbrand made changes -
        Field Original Value New Value
        Assignee Linden Hillenbrand [ lhillenbrand ]
        Linden Hillenbrand made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Linden Hillenbrand made changes -
        Attachment DataDrivenDBInputFormat.patch [ 12560175 ]
        Hide
        Linden Hillenbrand added a comment -

        Added a more detailed description of what the exception is.

        Show
        Linden Hillenbrand added a comment - Added a more detailed description of what the exception is.
        Linden Hillenbrand made changes -
        Status In Progress [ 3 ] Patch Available [ 10002 ]
        Hide
        Jarek Jarcec Cecho added a comment -

        Hi Linden,
        your patch is excellent! Thank you very much for your time and effort. Would you mind just tweaking the exception ending little bit? I would propose something like:

        ...or lower the number of mappers to 1. Unknown SQL data type:" + sqlDataType);
        

        My motivation is that current exception will simply generate something like following line:

        ...or lower the number of mappers to 1. -1
        

        I'm afraid that the "random" number at the end might confuse our users.

        Jarcec

        Show
        Jarek Jarcec Cecho added a comment - Hi Linden, your patch is excellent! Thank you very much for your time and effort. Would you mind just tweaking the exception ending little bit? I would propose something like: ...or lower the number of mappers to 1. Unknown SQL data type:" + sqlDataType); My motivation is that current exception will simply generate something like following line: ...or lower the number of mappers to 1. -1 I'm afraid that the "random" number at the end might confuse our users. Jarcec
        Hide
        Linden Hillenbrand added a comment -

        Completely agreed, will be adding immediately!

        Show
        Linden Hillenbrand added a comment - Completely agreed, will be adding immediately!
        Linden Hillenbrand made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Linden Hillenbrand made changes -
        Attachment DataDrivenDBInputFormat.patch [ 12560175 ]
        Linden Hillenbrand made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Linden Hillenbrand made changes -
        Attachment DataDrivenDBInputFormat.patch [ 12561632 ]
        Hide
        Linden Hillenbrand added a comment -

        Re-factored to include Jarcec's suggestions.

        Show
        Linden Hillenbrand added a comment - Re-factored to include Jarcec's suggestions.
        Linden Hillenbrand made changes -
        Status In Progress [ 3 ] Patch Available [ 10002 ]
        Hide
        Jarek Jarcec Cecho added a comment -

        It's simple patch, so I'm giving my +1 here.

        Show
        Jarek Jarcec Cecho added a comment - It's simple patch, so I'm giving my +1 here.
        Hide
        Jarek Jarcec Cecho added a comment -
        Show
        Jarek Jarcec Cecho added a comment - The patch is in: https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17 Thank you Linden for your contribution! Jarcec
        Jarek Jarcec Cecho made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop23 #583 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/583/)
        SQOOP-652: Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17
        Files :

        • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop23 #583 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/583/ ) SQOOP-652 : Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17 Files : src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop20 #395 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/395/)
        SQOOP-652: Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17
        Files :

        • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop20 #395 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/395/ ) SQOOP-652 : Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17 Files : src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop200 #386 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/386/)
        SQOOP-652: Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17
        Files :

        • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop200 #386 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/386/ ) SQOOP-652 : Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17 Files : src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop100 #397 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/397/)
        SQOOP-652: Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17
        Files :

        • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop100 #397 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/397/ ) SQOOP-652 : Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17 Files : src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        8d 14h 59m 1 Linden Hillenbrand 19/Dec/12 04:11
        Open Open In Progress In Progress
        44d 14h 58m 2 Linden Hillenbrand 19/Dec/12 04:12
        In Progress In Progress Patch Available Patch Available
        3m 2s 2 Linden Hillenbrand 19/Dec/12 04:13
        Patch Available Patch Available Resolved Resolved
        19m 44s 1 Jarek Jarcec Cecho 19/Dec/12 04:33

          People

          • Assignee:
            Linden Hillenbrand
            Reporter:
            Jarek Jarcec Cecho
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development