Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-653

RCFileAppender throws IOException

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0, 0.9.0
    • Fix Version/s: 0.8.0, 0.9.0
    • Component/s: None
    • Labels:
      None

      Description

      I ran a bellow query. This query throws IOException in worker

      tajo> create table orders_rc (
      o_orderkey      INT8,
      o_custkey       INT8,
      o_orderstatus   TEXT,
      o_totalprice    FLOAT8,
      o_orderdate     TEXT,
      o_orderpriority TEXT,
      o_clerk TEXT,
      o_shippriority  INT4,
      o_comment       TEXT)
      using rcfile WITH ('compression.codec'='org.apache.hadoop.io.compress.GzipCodec')
      
      tajo> insert overwrite into orders_rc select * from orders;
      
      2014-03-05 10:29:18,763 INFO  zlib.ZlibFactory (ZlibFactory.java:<clinit>(48)) - Successfully loaded & initialized native-zlib library
      2014-03-05 10:29:18,765 INFO  compress.CodecPool (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz]
      2014-03-05 10:29:18,936 INFO  compress.CodecPool (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz]
      2014-03-05 10:29:18,954 INFO  compress.CodecPool (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz]
      2014-03-05 10:29:18,963 INFO  compress.CodecPool (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz]
      2014-03-05 10:29:18,975 INFO  compress.CodecPool (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz]
      2014-03-05 10:29:18,979 INFO  compress.CodecPool (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz]
      2014-03-05 10:29:18,993 INFO  compress.CodecPool (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz]
      2014-03-05 10:29:18,998 INFO  compress.CodecPool (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz]
      2014-03-05 10:29:19,014 INFO  compress.CodecPool (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz]
      2014-03-05 10:29:19,028 INFO  compress.CodecPool (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz]
      2014-03-05 10:29:19,039 INFO  compress.CodecPool (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz]
      2014-03-05 10:29:19,080 INFO  compress.CodecPool (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz]
      2014-03-05 10:29:19,094 INFO  compress.CodecPool (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz]
      2014-03-05 10:29:22,484 ERROR worker.Task (Task.java:run(383)) - java.io.IOException: write beyond end of stream
              at org.apache.hadoop.io.compress.CompressorStream.write(CompressorStream.java:66)
              at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122)
              at java.io.DataOutputStream.write(DataOutputStream.java:107)
              at org.apache.tajo.storage.rcfile.NonSyncByteArrayOutputStream.writeTo(NonSyncByteArrayOutputStream.java:142)
              at org.apache.tajo.storage.rcfile.RCFile$RCFileAppender.WriteKeyBuffer(RCFile.java:947)
              at org.apache.tajo.storage.rcfile.RCFile$RCFileAppender.writeKey(RCFile.java:1046)
              at org.apache.tajo.storage.rcfile.RCFile$RCFileAppender.flushRecords(RCFile.java:1005)
              at org.apache.tajo.storage.rcfile.RCFile$RCFileAppender.append(RCFile.java:894)
              at org.apache.tajo.storage.rcfile.RCFile$RCFileAppender.addTuple(RCFile.java:851)
              at org.apache.tajo.engine.planner.physical.StoreTableExec.next(StoreTableExec.java:77)
              at org.apache.tajo.worker.Task.run(Task.java:374)
              at org.apache.tajo.worker.TaskRunner$1.run(TaskRunner.java:391)
              at java.lang.Thread.run(Thread.java:744)
      

        Attachments

          Activity

            People

            • Assignee:
              jhkim Jinho Kim
              Reporter:
              hjkim Hyoungjun Kim
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: