Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4322

test_scanners_fuzz.py hits a DCHECK

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • Impala 2.8.0
    • Impala 2.8.0
    • Backend

    Description

      Nightly exhaustive builds are failing. From the Jenkis console:

      ERROR query_test/test_scanners_fuzz.py::TestScannersFuzzing::()::test_fuzz_alltypes[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/lzo/block]
      

      It looks like this has something to do with the lzo scanner.
      From impalad.FATAL:

      F1018 15:45:14.758010  7262 hdfs-lzo-text-scanner.cc:236] Check failed: read_until >       unprocessed_bytes (0 vs. ^A) 
      F1018 15:45:14.781767  7254 hdfs-lzo-text-scanner.cc:236] Check failed: read_until >       unprocessed_bytes (0 vs. ^E)
      

      http://sandbox.jenkins.cloudera.com/view/Impala/view/Evergreen-asf-master/job/impala-asf-master-exhaustive/98/

      Another strange thing is that 3 minidumps get produced and saved in our logs (1 for every Impalad), but a core dump does not get generated. A core dump should be generated if a DCHECK is hit.

      Attachments

        Activity

          tarmstrong, I think you've dealt with issues uncovered by fuzz testing before, so I'll assign this to you.

          tarasbob Taras Bobrovytsky added a comment - tarmstrong , I think you've dealt with issues uncovered by fuzz testing before, so I'll assign this to you.
          alex.behm Alexander Behm added a comment -

          sailesh, might this be related to your recent LZO fix for S3?

          alex.behm Alexander Behm added a comment - sailesh , might this be related to your recent LZO fix for S3?
          sailesh Sailesh Mukil added a comment -

          alex.behm Yes, this looks like it's from my recent patch. I'll take this over.

          sailesh Sailesh Mukil added a comment - alex.behm Yes, this looks like it's from my recent patch. I'll take this over.
          tarmstrong Tim Armstrong added a comment -

          taras I think the coredump issue was http://gerrit.cloudera.org:8080/4785

          tarmstrong Tim Armstrong added a comment - taras I think the coredump issue was http://gerrit.cloudera.org:8080/4785
          sailesh Sailesh Mukil added a comment - https://github.com/cloudera/impala-lzo/commit/4862cf29d2167f84a6c35c8f2467a4ca0fe34c3d

          People

            sailesh Sailesh Mukil
            tarasbob Taras Bobrovytsky
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: