Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Backend
    • Labels:

      Description

      Nightly exhaustive builds are failing. From the Jenkis console:

      ERROR query_test/test_scanners_fuzz.py::TestScannersFuzzing::()::test_fuzz_alltypes[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/lzo/block]
      

      It looks like this has something to do with the lzo scanner.
      From impalad.FATAL:

      F1018 15:45:14.758010  7262 hdfs-lzo-text-scanner.cc:236] Check failed: read_until >       unprocessed_bytes (0 vs. ^A) 
      F1018 15:45:14.781767  7254 hdfs-lzo-text-scanner.cc:236] Check failed: read_until >       unprocessed_bytes (0 vs. ^E)
      

      http://sandbox.jenkins.cloudera.com/view/Impala/view/Evergreen-asf-master/job/impala-asf-master-exhaustive/98/

      Another strange thing is that 3 minidumps get produced and saved in our logs (1 for every Impalad), but a core dump does not get generated. A core dump should be generated if a DCHECK is hit.

        Activity

        Hide
        tarasbob Taras Bobrovytsky added a comment -

        Tim Armstrong, I think you've dealt with issues uncovered by fuzz testing before, so I'll assign this to you.

        Show
        tarasbob Taras Bobrovytsky added a comment - Tim Armstrong , I think you've dealt with issues uncovered by fuzz testing before, so I'll assign this to you.
        Hide
        alex.behm Alexander Behm added a comment -

        Sailesh Mukil, might this be related to your recent LZO fix for S3?

        Show
        alex.behm Alexander Behm added a comment - Sailesh Mukil , might this be related to your recent LZO fix for S3?
        Hide
        sailesh Sailesh Mukil added a comment -

        Alexander Behm Yes, this looks like it's from my recent patch. I'll take this over.

        Show
        sailesh Sailesh Mukil added a comment - Alexander Behm Yes, this looks like it's from my recent patch. I'll take this over.
        Hide
        tarmstrong Tim Armstrong added a comment -
        Show
        tarmstrong Tim Armstrong added a comment - Taras Murzenkov I think the coredump issue was http://gerrit.cloudera.org:8080/4785
        Show
        sailesh Sailesh Mukil added a comment - https://github.com/cloudera/impala-lzo/commit/4862cf29d2167f84a6c35c8f2467a4ca0fe34c3d

          People

          • Assignee:
            sailesh Sailesh Mukil
            Reporter:
            tarasbob Taras Bobrovytsky
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development