Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1781

Join condition is still not found when it exists in OR clause

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0
    • Fix Version/s: 0.11.0
    • Component/s: None
    • Labels:

      Description

      This is TPC-DS q13.

      select avg(ss_quantity)
             ,avg(ss_ext_sales_price)
             ,avg(ss_ext_wholesale_cost)
             ,sum(ss_ext_wholesale_cost)
       from store_sales
           ,store
           ,customer_demographics
           ,household_demographics
           ,customer_address
           ,date_dim
       where s_store_sk = ss_store_sk
       and  ss_sold_date_sk = d_date_sk and d_year = 2001
       and((ss_hdemo_sk=hd_demo_sk
        and cd_demo_sk = ss_cdemo_sk
        and cd_marital_status = 'M'
        and cd_education_status = 'Advanced Degree'
        and ss_sales_price between 100.00 and 150.00
        and hd_dep_count = 3   
           )or
           (ss_hdemo_sk=hd_demo_sk
        and cd_demo_sk = ss_cdemo_sk
        and cd_marital_status = 'S'
        and cd_education_status = 'College'
        and ss_sales_price between 50.00 and 100.00   
        and hd_dep_count = 1
           ) or 
           (ss_hdemo_sk=hd_demo_sk
        and cd_demo_sk = ss_cdemo_sk
        and cd_marital_status = 'W'
        and cd_education_status = '2 yr Degree'
        and ss_sales_price between 150.00 and 200.00 
        and hd_dep_count = 1  
           ))
       and((ss_addr_sk = ca_address_sk
        and ca_country = 'United States'
        and ca_state in ('TX', 'OH', 'TX')
        and ss_net_profit between 100 and 200  
           ) or
           (ss_addr_sk = ca_address_sk
        and ca_country = 'United States'
        and ca_state in ('OR', 'NM', 'KY')
        and ss_net_profit between 150 and 300  
           ) or
           (ss_addr_sk = ca_address_sk
        and ca_country = 'United States'
        and ca_state in ('VA', 'TX', 'MS')
        and ss_net_profit between 50 and 250  
           ));
      

      after (almost) 10 minutes, this error log is written. (in worker log)

      2015-08-18 14:29:41,220 INFO org.apache.tajo.worker.TaskImpl: One local chunk is added to listTablets
      2015-08-18 14:29:41,221 INFO org.apache.tajo.worker.TaskImpl: One local chunk is added to listTablets
      2015-08-18 14:40:26,950 INFO org.apache.tajo.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 2309ms
      GC pool 'PS MarkSweep' had collection(s): count=1 time=1988ms
      GC pool 'PS Scavenge' had collection(s): count=1 time=3ms
      2015-08-18 15:02:56,295 ERROR org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec: 장치에 남은 공간이 없음
      java.io.IOException: 장치에 남은 공간이 없음
      	at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
      	at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60)
      	at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
      	at sun.nio.ch.IOUtil.write(IOUtil.java:51)
      	at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:205)
      	at org.apache.tajo.storage.RawFile$RawFileAppender.flushBuffer(RawFile.java:532)
      	at org.apache.tajo.storage.RawFile$RawFileAppender.addTuple(RawFile.java:640)
      	at org.apache.tajo.storage.HashShuffleAppender.addTuples(HashShuffleAppender.java:92)
      	at org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:117)
      	at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:403)
      	at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      2015-08-18 15:02:56,347 ERROR org.apache.tajo.worker.TaskImpl: java.io.IOException: 장치에 남은 공간이 없음
      java.io.IOException: java.io.IOException: 장치에 남은 공간이 없음
      	at org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:147)
      	at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:403)
      	at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.io.IOException: 장치에 남은 공간이 없음
      	at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
      	at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60)
      	at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
      	at sun.nio.ch.IOUtil.write(IOUtil.java:51)
      	at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:205)
      	at org.apache.tajo.storage.RawFile$RawFileAppender.flushBuffer(RawFile.java:532)
      	at org.apache.tajo.storage.RawFile$RawFileAppender.addTuple(RawFile.java:640)
      	at org.apache.tajo.storage.HashShuffleAppender.addTuples(HashShuffleAppender.java:92)
      	at org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:117)
      	... 7 more
      2015-08-18 15:02:56,348 INFO org.apache.tajo.worker.TaskAttemptContext: Query status of ta_1439875670149_0001_000007_000000_00 is changed to TA_FAILED
      2015-08-18 15:02:56,411 INFO org.apache.tajo.worker.TaskImpl: ta_1439875670149_0001_000007_000000_00 completed. Worker's task counter - total:1, succeeded: 0, killed: 0, failed: 1
      2015-08-18 15:02:56,413 ERROR org.apache.tajo.querymaster.TaskAttempt: ta_1439875670149_0001_000007_000000_00 FROM dkhwangbo >> java.io.IOException: 장치에 남은 공간이 없음
      2015-08-18 15:02:56,413 INFO org.apache.tajo.querymaster.Task: ====================================================================================
      2015-08-18 15:02:56,413 INFO org.apache.tajo.querymaster.Task: >>> Task Failed: ta_1439875670149_0001_000007_000000_00, retry:true, attempts:1 <<<
      2015-08-18 15:02:56,413 INFO org.apache.tajo.querymaster.Task: ====================================================================================
      

      I think tajo run with inefficient way to run this query.

        Attachments

          Activity

            People

            • Assignee:
              jihoonson Jihoon Son
              Reporter:
              dkhwangbo Dongkyu Hwangbo
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: