Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-3180

when mysql one column must not null but insert has null map hang on close method

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.4.4
    • None
    • None
    • None

    Description

      2017-05-05 14:35:55,301 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
      2017-05-05 14:35:55,377 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
      2017-05-05 14:35:55,377 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
      2017-05-05 14:35:55,385 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:
      2017-05-05 14:35:55,385 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1493953508663_14532, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@692b53b4)
      2017-05-05 14:35:55,478 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
      2017-05-05 14:35:55,806 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /data1/nodemanager/usercache/master/appcache/application_1493953508663_14532,/data2/nodemanager/usercache/master/appcache/application_1493953508663_14532,/data3/nodemanager/usercache/master/appcache/application_1493953508663_14532,/data4/nodemanager/usercache/master/appcache/application_1493953508663_14532,/data5/nodemanager/usercache/master/appcache/application_1493953508663_14532,/data6/nodemanager/usercache/master/appcache/application_1493953508663_14532,/data7/nodemanager/usercache/master/appcache/application_1493953508663_14532,/data8/nodemanager/usercache/master/appcache/application_1493953508663_14532,/data9/nodemanager/usercache/master/appcache/application_1493953508663_14532,/data10/nodemanager/usercache/master/appcache/application_1493953508663_14532,/data11/nodemanager/usercache/master/appcache/application_1493953508663_14532,/data12/nodemanager/usercache/master/appcache/application_1493953508663_14532
      2017-05-05 14:35:56,388 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
      2017-05-05 14:35:56,919 INFO [main] org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
      2017-05-05 14:35:57,229 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: Paths:/data/sqoop_buffer/export_buffer/tmp_st_tms_typecode_bod_hummerteam_sale_20170505_143426_5424/000000_0:0+136738
      2017-05-05 14:35:57,235 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: map.input.file is deprecated. Instead, use mapreduce.map.input.file
      2017-05-05 14:35:57,235 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: map.input.start is deprecated. Instead, use mapreduce.map.input.start
      2017-05-05 14:35:57,235 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: map.input.length is deprecated. Instead, use mapreduce.map.input.length
      2017-05-05 14:35:57,685 INFO [Thread-12] org.apache.sqoop.mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
      2017-05-05 14:35:57,695 INFO [Thread-11] org.apache.sqoop.mapreduce.AsyncSqlOutputFormat: Start AsyncSqlExecThread thread to insert data
      2017-05-05 14:35:57,695 INFO [main] org.apache.sqoop.mapreduce.AsyncSqlRecordWriter: Start execUpdate,commit is true,stopThread is true,records length is 98batch is false
      2017-05-05 14:35:57,923 INFO [main] org.apache.sqoop.mapreduce.AsyncSqlOutputFormat: Init AsyncDBOperation: isBatch is false,commit is true,stopThread is true
      2017-05-05 14:35:57,987 INFO [Thread-11] org.apache.sqoop.mapreduce.AsyncSqlOutputFormat: Insert thread stmt is com.mysql.jdbc.JDBC4PreparedStatement@51072bc2: INSERT INTO st_tms_typecode_bod_hummerteam_sale (column)values(my data)
      2017-05-05 14:35:57,988 INFO [Thread-11] org.apache.sqoop.mapreduce.AsyncSqlOutputFormat: Start try block
      2017-05-05 14:35:57,988 INFO [Thread-11] org.apache.sqoop.mapreduce.AsyncSqlOutputFormat: Execute without batch mode
      2017-05-05 14:35:58,052 ERROR [Thread-11] org.apache.sqoop.mapreduce.AsyncSqlOutputFormat: ERROR 3
      2017-05-05 14:35:58,052 ERROR [Thread-11] org.apache.sqoop.mapreduce.AsyncSqlOutputFormat: Got exception in update thread: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Column 'total_pleased_rate_count' cannot be null
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
      	at com.mysql.jdbc.Util.handleNewInstance(Util.java:408)
      	at com.mysql.jdbc.Util.getInstance(Util.java:383)
      	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1049)
      	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4226)
      	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4158)
      	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2615)
      	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2776)
      	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2840)
      	at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2082)
      	at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1302)
      	at org.apache.sqoop.mapreduce.AsyncSqlOutputFormat$AsyncSqlExecThread.run(AsyncSqlOutputFormat.java:243)
      
      2017-05-05 14:35:58,056 INFO [main] org.apache.sqoop.mapreduce.AsyncSqlRecordWriter: Start execUpdate,commit is true,stopThread is true,records length is 0batch is false
      2017-05-05 14:35:58,056 INFO [main] org.apache.sqoop.mapreduce.AsyncSqlOutputFormat: Init AsyncDBOperation: isBatch is false,commit is true,stopThread is true
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            zhaiyuyong scott.zhai
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: