Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-3287

Import to HDFS not working in parquet not working for LDAP SQL Server

    XMLWordPrintableJSON

    Details

      Description

      I am importing data from SQL Server to HDFS. Seems there is a bug when autnetication mode is LDAP and format is parquet. Please let me know if this is a bug or I am doing incorrectly. I am using Cloudera distribution.

      Below different scenarios I have tested

      Command Used are.

      1) SQL Server (Native Connection) -> HDFS Text Format (WORKS FINE)

      sqoop import \
      --connect "jdbc:sqlserver://<<ServerName>>;databasename=<<DB_Name>>"
      --username xxxx \
      --password xxxx \
      --table 'table1' \
      --split-by col1 \
      --target-dir /user/user1/01/

      2) SQL Server (Native Connection) -> HDFS Parquet Format (WORKS FINE)

      sqoop import \
      --connect "jdbc:sqlserver://<<ServerName>>;databasename=<<DB_Name>>"
      --username xxxx \
      --password xxxx \
      --table 'table1' \
      --split-by col1 \
      --target-dir /user/user1/01/ \
      --as-parquetfile

       

      3) SQL Server (LDAP Connection) -> HDFS Text format (WORKS FINE)

      sqoop import \
      --connect "jdbc:jtds:sqlserver://<<Server2>>;useNTLMv2=true;domain=XXXX;databaseName=<<DB_name>>" \
      --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver \
      --username xxxx \
      --password xxxx \
      --table table2 \
      --target-dir /user/user1/02/ \
      – --schema=s1

      4) SQL Server (LDAP Connection) -> HDFS Parquet format (Executes successfully, but the exported files are not parquet, they are in TEXT format)

      sqoop import \
      --connect "jdbc:jtds:sqlserver://<<Server2>>;useNTLMv2=true;domain=XXXX;databaseName=<<DB_name>>" \
      --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver \
      --username xxxx \
      --password xxxx \
      --table table2 \
      --target-dir /user/user1/02/ \
      – --schema=s1 \
      --as-parquetfile

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              ashishprem Ashish Kumar Sinha
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: