Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-585

Bug when sqoop a join of two tables with the same column name with mysql backend

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.4
    • Fix Version/s: 1.4.5
    • Component/s: connectors/sqlserver
    • Labels:
    • Environment:

      Description

      I get this error when I run the command:

      INFO: Executing SQL statement: SELECT events.*, square_applications.name as square_application, user_agents.name as user_agent from events LEFT JOIN square_applications on events.square_application_id=square_applications.id LEFT JOIN user_agents on events.user_agent_id=user_agents.id where (1 = 0)
      Aug 23, 2012 10:10:56 AM org.apache.sqoop.tool.ImportTool run
      SEVERE: Imported Failed: Duplicate Column identifier specified: 'name'

      Calling "getColumnLabel" instead of "getColumnName" in SqlManager.java fixes this problem and this still works in postgres as well as mysql. Here is my patch:

      Index: src/java/org/apache/sqoop/manager/SqlManager.java
      ===================================================================
      — src/java/org/apache/sqoop/manager/SqlManager.java (revision 1375124)
      +++ src/java/org/apache/sqoop/manager/SqlManager.java (working copy)
      @@ -135,9 +135,9 @@
      ArrayList<String> columns = new ArrayList<String>();
      ResultSetMetaData metadata = results.getMetaData();
      for (int i = 1; i < cols + 1; i++) {

      • String colName = metadata.getColumnName;
        + String colName = metadata.getColumnLabel;
        if (colName == null || colName.equals("")) {
      • colName = metadata.getColumnLabel;
        + colName = metadata.getColumnName;
        if (null == colName) { colName = "_RESULT_" + i; }

        @@ -208,9 +208,9 @@
        typeId = Types.BIGINT;
        }

      • String colName = metadata.getColumnName;
        + String colName = metadata.getColumnLabel;
        if (colName == null || colName.equals("")) { - colName = metadata.getColumnLabel(i); + colName = metadata.getColumnName(i); }

      colTypes.put(colName, Integer.valueOf(typeId));

      1. columnLabel.patch
        1 kB
        Vida Ha
      2. 585-1.4.4.txt
        3 kB
        Amit Sela
      3. 585-trunk.txt
        3 kB
        Amit Sela

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1097 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1097/)
          SQOOP-585: Bug when sqoop a join of two tables with the same column name with mysql backend (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e2d6499da8567888989908ddcd078f046e859679)

          • src/java/org/apache/sqoop/util/ResultSetPrinter.java
          • src/java/org/apache/sqoop/manager/SqlManager.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1097 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1097/ ) SQOOP-585 : Bug when sqoop a join of two tables with the same column name with mysql backend (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e2d6499da8567888989908ddcd078f046e859679 ) src/java/org/apache/sqoop/util/ResultSetPrinter.java src/java/org/apache/sqoop/manager/SqlManager.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #894 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/894/)
          SQOOP-585: Bug when sqoop a join of two tables with the same column name with mysql backend (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e2d6499da8567888989908ddcd078f046e859679)

          • src/java/org/apache/sqoop/manager/SqlManager.java
          • src/java/org/apache/sqoop/util/ResultSetPrinter.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #894 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/894/ ) SQOOP-585 : Bug when sqoop a join of two tables with the same column name with mysql backend (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e2d6499da8567888989908ddcd078f046e859679 ) src/java/org/apache/sqoop/manager/SqlManager.java src/java/org/apache/sqoop/util/ResultSetPrinter.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #900 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/900/)
          SQOOP-585: Bug when sqoop a join of two tables with the same column name with mysql backend (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e2d6499da8567888989908ddcd078f046e859679)

          • src/java/org/apache/sqoop/manager/SqlManager.java
          • src/java/org/apache/sqoop/util/ResultSetPrinter.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #900 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/900/ ) SQOOP-585 : Bug when sqoop a join of two tables with the same column name with mysql backend (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e2d6499da8567888989908ddcd078f046e859679 ) src/java/org/apache/sqoop/manager/SqlManager.java src/java/org/apache/sqoop/util/ResultSetPrinter.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #859 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/859/)
          SQOOP-585: Bug when sqoop a join of two tables with the same column name with mysql backend (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e2d6499da8567888989908ddcd078f046e859679)

          • src/java/org/apache/sqoop/util/ResultSetPrinter.java
          • src/java/org/apache/sqoop/manager/SqlManager.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #859 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/859/ ) SQOOP-585 : Bug when sqoop a join of two tables with the same column name with mysql backend (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=e2d6499da8567888989908ddcd078f046e859679 ) src/java/org/apache/sqoop/util/ResultSetPrinter.java src/java/org/apache/sqoop/manager/SqlManager.java
          Hide
          jarcec Jarek Jarcec Cecho added a comment -

          The patch is in, thank you for your contribution Amit Sela, greatly appreciated!

          Show
          jarcec Jarek Jarcec Cecho added a comment - The patch is in, thank you for your contribution Amit Sela , greatly appreciated!
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit e2d6499da8567888989908ddcd078f046e859679 in sqoop's branch refs/heads/trunk from Jarek Jarcec Cecho
          [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=e2d6499 ]

          SQOOP-585: Bug when sqoop a join of two tables with the same column name with mysql backend

          (Amit Sela via Jarek Jarcec Cecho)

          Show
          jira-bot ASF subversion and git services added a comment - Commit e2d6499da8567888989908ddcd078f046e859679 in sqoop's branch refs/heads/trunk from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=e2d6499 ] SQOOP-585 : Bug when sqoop a join of two tables with the same column name with mysql backend (Amit Sela via Jarek Jarcec Cecho)
          Hide
          amitsela Amit Sela added a comment -

          Added patches for 1.4.4 branch and trunk. Fixed the getColumnLable()/getColumnName() order in both SqlManager and ResultSetPrinter.

          Show
          amitsela Amit Sela added a comment - Added patches for 1.4.4 branch and trunk. Fixed the getColumnLable()/getColumnName() order in both SqlManager and ResultSetPrinter.
          Hide
          amitsela Amit Sela added a comment -

          Patch for SqlManager and ResultSetPrinter

          Show
          amitsela Amit Sela added a comment - Patch for SqlManager and ResultSetPrinter
          Hide
          jarcec Jarek Jarcec Cecho added a comment -

          Hi Vida,
          thank you for your bug report and patch. We appreciate your time.

          Could you please do similar change in file "ResulSetPrinter.java" in method printResultSet? It's not crucial for import/export functionality, but it would be great to be consistent.

          I'm also little bit worried about backward compatibility, but this seems to be worth fixing anyway.

          Jarcec

          Show
          jarcec Jarek Jarcec Cecho added a comment - Hi Vida, thank you for your bug report and patch. We appreciate your time. Could you please do similar change in file "ResulSetPrinter.java" in method printResultSet? It's not crucial for import/export functionality, but it would be great to be consistent. I'm also little bit worried about backward compatibility, but this seems to be worth fixing anyway. Jarcec
          Hide
          abhijeet_gaikwad Abhijeet Gaikwad added a comment -

          Thanks Vida.

          I could reproduce the scenario on current trunk.

          Patch looks good to me.

          ant test - success.
          ant checkstyle - 0 errors
          Basic tests + tests for the scenario - Success with MySql

          Show
          abhijeet_gaikwad Abhijeet Gaikwad added a comment - Thanks Vida. I could reproduce the scenario on current trunk. Patch looks good to me. ant test - success. ant checkstyle - 0 errors Basic tests + tests for the scenario - Success with MySql
          Hide
          vida@squareup.com Vida Ha added a comment -

          Done. Thanks.

          Show
          vida@squareup.com Vida Ha added a comment - Done. Thanks.
          Hide
          abhijeet_gaikwad Abhijeet Gaikwad added a comment -

          As a formality:
          1. Can you please change affect-version to the sqoop version you found this error on.
          2. Please attach patch file rather pasting patch in the description. If can you see "More Actions" drop down at the top, then you have the privileges.

          Thanks.

          Show
          abhijeet_gaikwad Abhijeet Gaikwad added a comment - As a formality: 1. Can you please change affect-version to the sqoop version you found this error on. 2. Please attach patch file rather pasting patch in the description. If can you see "More Actions" drop down at the top, then you have the privileges. Thanks.

            People

            • Assignee:
              amitsela Amit Sela
              Reporter:
              vida@squareup.com Vida Ha
            • Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development