Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8833

Check failed: bit_width <= sizeof(T) * 8 (40 vs. 32) in BatchedBitReader::UnpackBatch()

    XMLWordPrintableJSON

    Details

      Description

      F0801 21:24:10.571285 15993 bit-stream-utils.inline.h:126] d04ba69d5da8ffd1:a9045b8200000001] Check failed: bit_width <= sizeof(T) * 8 (40 vs. 32) 
      *** Check failure stack trace: ***
          @          0x52f63ac  google::LogMessage::Fail()
          @          0x52f7c51  google::LogMessage::SendToLog()
          @          0x52f5d86  google::LogMessage::Flush()
          @          0x52f934d  google::LogMessageFatal::~LogMessageFatal()
          @          0x2b265b5  impala::BatchedBitReader::UnpackBatch<>()
          @          0x2ae8623  impala::RleBatchDecoder<>::FillLiteralBuffer()
          @          0x2b2cadb  impala::RleBatchDecoder<>::DecodeLiteralValues<>()
          @          0x2b27bfb  impala::DictDecoder<>::DecodeNextValue()
          @          0x2b16fed  impala::ScalarColumnReader<>::ReadSlotsNoConversion()
          @          0x2ac7252  impala::ScalarColumnReader<>::ReadSlots()
          @          0x2a76cef  impala::ScalarColumnReader<>::MaterializeValueBatchRepeatedDefLevel()
          @          0x2a58faa  impala::ScalarColumnReader<>::ReadValueBatch<>()
          @          0x2a20e8e  impala::ScalarColumnReader<>::ReadNonRepeatedValueBatch()
          @          0x29b189c  impala::HdfsParquetScanner::AssembleRows()
          @          0x29ac6de  impala::HdfsParquetScanner::GetNextInternal()
          @          0x29aa656  impala::HdfsParquetScanner::ProcessSplit()
          @          0x249172d  impala::HdfsScanNode::ProcessSplit()
          @          0x2490902  impala::HdfsScanNode::ScannerThread()
          @          0x248fc8b  _ZZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS_18ThreadResourcePoolEENKUlvE_clEv
          @          0x2492253  
      

      https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/6915

      Log lines around the failure:

      [gw5] PASSED query_test/test_scanners.py::TestParquet::test_bad_compression_codec[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
      query_test/test_nested_types.py::TestMaxNestingDepth::test_load_hive_table[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
      query_test/test_scanners.py::TestParquet::test_bad_compression_codec[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error': 1, 'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@0.5', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
      [gw1] PASSED query_test/test_tpcds_queries.py::TestTpcdsQuery::test_tpcds_q7[protocol: beeswax | exec_option: {'decimal_v2': 0, 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
      query_test/test_tpcds_queries.py::TestTpcdsQuery::test_tpcds_q8[protocol: beeswax | exec_option: {'decimal_v2': 0, 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
      [gw1] PASSED query_test/test_tpcds_queries.py::TestTpcdsQuery::test_tpcds_q8[protocol: beeswax | exec_option: {'decimal_v2': 0, 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
      query_test/test_tpcds_queries.py::TestTpcdsQuery::test_tpcds_q10a[protocol: beeswax | exec_option: {'decimal_v2': 0, 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
      [gw10] PASSED query_test/test_scanners_fuzz.py::TestScannersFuzzing::test_fuzz_decimal_tbl[protocol: beeswax | exec_option: {'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0', 'abort_on_error': False, 'mem_limit': '512m', 'num_nodes': 0} | table_format: parquet/none]
      [gw6] FAILED query_test/test_decimal_fuzz.py::TestDecimalFuzz::test_decimal_ops[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0}]
      [gw2] FAILED query_test/test_scanners.py::TestParquet::test_type_widening[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': 'HDFS_SCANNER_THREAD_CHECK_SOFT_MEM_LIMIT:FAIL@0.5', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
      [gw4] FAILED query_test/test_scanners_fuzz.py::TestScannersFuzzing::test_fuzz_alltypes[protocol: beeswax | exec_option: {'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0', 'abort_on_error': False, 'mem_limit': '512m', 'num_nodes': 0} | table_format: parquet/none]
      [gw12] FAILED query_test/test_scanners.py::TestParquet::test_huge_num_rows[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error': 1, 'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@0.5', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
      [gw8] ERROR query_test/test_nested_types.py::TestMaxNestingDepth::test_load_hive_table[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
      [gw13] FAILED query_test/test_spilling.py::TestSpillingDebugActionDimensions::test_spilling_naaj[protocol: beeswax | exec_option: {'debug_action': None, 'default_spillable_buffer_size': '256k'} | table_format: parquet/none]
      [gw5] ERROR query_test/test_scanners.py::TestParquet::test_bad_compression_codec[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error': 1, 'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@0.5', 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
      [gw11] FAILED query_test/test_scanners.py::TestParquet::test_corrupt_footer_len_decr[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
      [gw10] ERROR query_test/test_scanners_fuzz.py::TestScannersFuzzing::test_fuzz_decimal_tbl[protocol: beeswax | exec_option: {'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0', 'abort_on_error': False, 'mem_limit': '512m', 'num_nodes': 0} | table_format: parquet/none]
      query_test/test_scanners_fuzz.py::TestScannersFuzzing::test_fuzz_decimal_tbl[protocol: beeswax | exec_option: {'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@0.5', 'abort_on_error': False, 'mem_limit': '512m', 'num_nodes': 0} | table_format: text/none]
      [gw9] FAILED query_test/test_insert_parquet.py::TestInsertParquetQueries::test_insert_parquet[compression_codec: snappy | protocol: beeswax | exec_option: {'sync_ddl': 1, 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none | file_size: 0-unique_database0]
      query_test/test_spilling.py::TestSpillingDebugActionDimensions::test_spilling_naaj[protocol: beeswax | exec_option: {'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0', 'default_spillable_buffer_size': '256k'} | table_format: parquet/none]
      query_test/test_decimal_fuzz.py::TestDecimalFuzz::test_width_bucket[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0}]
      

      I'm guessing this is the scanner fuzzing.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                daniel.becker Daniel Becker
                Reporter:
                tarmstrong Tim Armstrong
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: