[SQOOP-1746] Sqoop2: Mapreduce job is running in DEBUG mode by default

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.99.4
• Fix Version/s: 1.99.6
• Labels: newbie
• Assignee: Sankara Reddy Telukutla
• Reporter: Jarek Jarcec Cecho
• Created: 17/Nov/14 17:27
• Updated: 06/May/15 20:45
• Resolved: 20/Feb/15 17:52

Description

I've gave it a spin for 1.99.4 and noticed that the mapreduce job is printing out a lot of DEBUG statements by default. The biggest issue is that Sqoop will print out all data that it transfers which is generating a lot of IO and taking significant amount of space:

2014-11-17 09:00:18,529 DEBUG [pool-5-thread-1] org.apache.sqoop.job.mr.SqoopProgressRunnable: Auto-progress thread reporting progress
2014-11-17 09:00:18,960 INFO [main] org.apache.sqoop.connector.hdfs.HdfsExtractor: Extracting text file
2014-11-17 09:00:19,007 INFO [main] org.apache.sqoop.connector.hdfs.HdfsExtractor: Start position: 0
2014-11-17 09:00:19,022 DEBUG [main] org.apache.sqoop.job.mr.SqoopMapper: Extracted data: 300001,'673l78hi9fb9p5m9k3advjlv61lqnd7uvp1on6g0i5qs65ojc7tddlq0qu3akcq3qfbl03gub4h8a43kcg4lp6heg7jbq2q1fopae22srbmotirv9jdjtko6qqrhnll9ridqljjlflc0c15n4b66d9gagd8pc5jkdhtndgbb2c4ujev9lf7fd53lhev4sheim6jvb53nphatm122cstfl6r93erguc2r8lo641jcbmc41vrmppuubg1l1ufpju'
2014-11-17 09:00:19,032 DEBUG [main] org.apache.sqoop.job.mr.SqoopMapper: Extracted data: 300002,'k6oul3abrfmnaskaag4hdtjtm9cinh9v6k6sg4rpp515ubeam91c7pfug958pmggmsiu8m7gcsj35ap5stm9q0v2m5pftavcigjtduf3jst28tevbmddolh8hcn99859edd8142h8rp5mvhcj6enma390vcrd7e8rolpi0o2acaufp416lruc6956ridnr13kbkvftp8bgao7i5p35d1jm0iel35c2d42o3rc2umobvclsaajtttnkaf4ma3k4'
2014-11-17 09:00:19,395 DEBUG [main] org.apache.sqoop.job.mr.SqoopMapper: Extracted data: 300003,'u73vfgluis91dii9i9g8shfbulgeb5npjp1gknbogirrlfdco196grkhitpadp6evkl16r6favi38nvtgk7vq24qpgqgr8udj7pirkd78tm16hivq3lh8v9a0n4eqb5ffoon302g482ri7rj6odjujuh3mph87hrq1csd9cgab6rn2embnet813gr9434vkruv7smch9vacgnlg077lvdoac7ot8adfmg5repc0rotcm4836t6nbus2u81dbhp'
2014-11-17 09:00:19,412 DEBUG [main] org.apache.sqoop.job.mr.SqoopMapper: Extracted data: 300004,'8bi8r95n22sqsm9e9gj13rd2qcb4ebtfjdoleajm1cmpoonmhc3h4uecf851csejp4n4113cp0j4qhnftc0rg19di4tg8ps9t7cdopc1nv8jd4ekue1v94jb9bfcht4l65b919kh35838ebbv7cluj5ji961ir0chogbk0bvabo7tl9bn5stvtv8dqtnma5r5p7uh8r7ui4f5miev94m4dghgntf2890usnnru4kuhm5qo1jpbaseto2u6vbs8'
2014-11-17 09:00:19,414 DEBUG [main] org.apache.sqoop.job.mr.SqoopMapper: Extracted data: 300005,'82q57u5p8dc333dct523t09dubfae7da4usnoelnopsqj5j355g9hfjt0g1vp4t4dg9lebgk662d4alib6i5ckm2u7285ect12vegt2qn09tcgmiqg6d97h0v0qdpipf4up0ubquglnpp18h59r5n04d4sv9r9cujkufo6vh3gp1dnmoqvj2vu3l7j9jfbaphav2jlk6pj91h7sos9de6527tlo8q0b3g9kjoc45dfmdr7efp9uulvbr5ea2mo'
2014-11-17 09:00:19,417 DEBUG [main] org.apache.sqoop.job.mr.SqoopMapper: Extracted data: 300006,'6t8gcv9lp0447ttc1bi74he6c9p3gueiehbbapsjiokcrgffbhaj04gufjtgg0jngslj92905953mf6vlvldsgsfmemgrldt7gvs3fa0p5jcre5s3bodjmeovmbti6m96p6jj0tccs85am1g427ner2uukcaekm7l2l321gaj9meb52efgtbf0hho52uhndge7ik204jdkou6u3mis6emlnj8btcntj01mi438ls83adds76jqb5s8csuo7crh'
2014-11-17 09:00:19,418 DEBUG [main] org.apache.sqoop.job.mr.SqoopMapper: Extracted data: 300007,'9kjf7i6pduk3ivv7lunmbffl8nk67pt7bebaig1t08bk9tuuvstbe9hpqheopgrhsttqpt32uu40lt1q7fugfl8pdcqj4l9o5nbnl9ft2f4gcg4gc4f7c7f0vuln7jif4afcpbhinir4hoq6913u9bgkcf3oqvborok25266raj8i0mq0gfmbtui7ern0rmqgb96dkt102veda1uhstkl20bv9d40qu06hgkqbmlpipefit4h1fn32f8hb9u7p'
2014-11-17 09:00:19,422 DEBUG [main] org.apache.sqoop.job.mr.SqoopMapper: Extracted data: 300008,'nrcslsm2031gpkmupc3a28gm7pclgrmg4ja928qvjsrlepbfgpere1d3vssf94h770rred5gpmteeibf6n1mi754og1ohc0lt8h3crb2vhica9bu92cppe86nuh3hvskg2oddrg6fira7tc73cefgk7mmladsrji7m2eqclqj40k288q0v8rrfcjakosgnusl4tmseh7k3hede22edrev1gs7jsmcahchn3gc48f9rt50vpl5j6koh32go3haj'
2014-11-17 09:00:19,423 DEBUG [main] org.apache.sqoop.job.mr.SqoopMapper: Extracted data: 300009,'6tjr7qekakr2mo1tllo0m5q0v6rjk0ehgij7mop760l3bp35oemjt1uvhpbjrpgms1be94ud4kd0h0vsv505altb20a1j84so52pguc2f1dohs9shpfd5onr4hjv6ihj729a5s5v5nq8u3rlb46aq6aajrhb295bgodnkn40d4t18gmftsahm6u2l77p28m0aic3ktcsa3jrfbk8aamipc523avl1oodk5ciodpcml13446gknt3fpne32an18'

Issue Links

• is related to: SQOOP-2350 Sqoop2: log4j properties file should be configurable