Sqoop
  1. Sqoop
  2. SQOOP-652

Improve error message generated in DataDrivenDBInputFormat when no splitter is available

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 1.4.2
    • Fix Version/s: 1.4.3
    • Component/s: None
    • Labels:

      Description

      Currently DataDrivenDBInputFormat will throw following exception in case that there is no available Splitter:

      DBSplitter splitter = getSplitter(sqlDataType);
      if (null == splitter) {
        throw new IOException("Unknown SQL data type: " + sqlDataType);
      }
      

      Text of the exception is very confusing as it's not describing what has actually happened. Would be great if this message would be improved by saying that Sqoop do not have splitter for given sql type and therefore user should either use different split column (argument --split-by) or lower number of mappers to 1 (splitters are not used in such case).

        Activity

        Hide
        Linden Hillenbrand added a comment -

        Added a more detailed description of what the exception is.

        Show
        Linden Hillenbrand added a comment - Added a more detailed description of what the exception is.
        Hide
        Jarek Jarcec Cecho added a comment -

        Hi Linden,
        your patch is excellent! Thank you very much for your time and effort. Would you mind just tweaking the exception ending little bit? I would propose something like:

        ...or lower the number of mappers to 1. Unknown SQL data type:" + sqlDataType);
        

        My motivation is that current exception will simply generate something like following line:

        ...or lower the number of mappers to 1. -1
        

        I'm afraid that the "random" number at the end might confuse our users.

        Jarcec

        Show
        Jarek Jarcec Cecho added a comment - Hi Linden, your patch is excellent! Thank you very much for your time and effort. Would you mind just tweaking the exception ending little bit? I would propose something like: ...or lower the number of mappers to 1. Unknown SQL data type:" + sqlDataType); My motivation is that current exception will simply generate something like following line: ...or lower the number of mappers to 1. -1 I'm afraid that the "random" number at the end might confuse our users. Jarcec
        Hide
        Linden Hillenbrand added a comment -

        Completely agreed, will be adding immediately!

        Show
        Linden Hillenbrand added a comment - Completely agreed, will be adding immediately!
        Hide
        Linden Hillenbrand added a comment -

        Re-factored to include Jarcec's suggestions.

        Show
        Linden Hillenbrand added a comment - Re-factored to include Jarcec's suggestions.
        Hide
        Jarek Jarcec Cecho added a comment -

        It's simple patch, so I'm giving my +1 here.

        Show
        Jarek Jarcec Cecho added a comment - It's simple patch, so I'm giving my +1 here.
        Hide
        Jarek Jarcec Cecho added a comment -
        Show
        Jarek Jarcec Cecho added a comment - The patch is in: https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17 Thank you Linden for your contribution! Jarcec
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop23 #583 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/583/)
        SQOOP-652: Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17
        Files :

        • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop23 #583 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/583/ ) SQOOP-652 : Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17 Files : src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop20 #395 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/395/)
        SQOOP-652: Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17
        Files :

        • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop20 #395 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/395/ ) SQOOP-652 : Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17 Files : src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop200 #386 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/386/)
        SQOOP-652: Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17
        Files :

        • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop200 #386 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/386/ ) SQOOP-652 : Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17 Files : src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop100 #397 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/397/)
        SQOOP-652: Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17
        Files :

        • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop100 #397 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/397/ ) SQOOP-652 : Improve error message generated in DataDrivenDBInputFormat when no splitter is available (Revision 0e8a7dc73e4b8fa64a8c222384fe0027045b8a17) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0e8a7dc73e4b8fa64a8c222384fe0027045b8a17 Files : src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java

          People

          • Assignee:
            Linden Hillenbrand
            Reporter:
            Jarek Jarcec Cecho
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development