Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3200 Replace BufferedBlockMgr with new buffer pool
  3. IMPALA-5124

Fix BufferPool handling of scratch read errors

        Details

        • Type: Sub-task
        • Status: Resolved
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: Impala 2.9.0
        • Fix Version/s: Impala 2.9.0
        • Component/s: Backend
        • Labels:
          None

          Description

          We are missing test coverage of the BufferPool error path where the integrity check of on-disk data fails. We should add a backend test that injects read errors of the following kinds:

          • Truncated files
          • Corrupted files
          • Inaccessible/removed files

          I wrote a prototype of the tests and confirmed that there are some bugs in this area.

            Activity

            Hide
            Permalink
            tarmstrong Tim Armstrong added a comment -

            IMPALA-5124: add tests for scratch read errors

            Adds tests for read errors from permissions (i.e. open() fails),
            corrupt data (integrity check fails) and truncated files (read() fails).

            Fixes a couple of bugs:

            • Truncated reads were not detected in TmpFilemgr
            • IoMgr buffers weren't returned on error paths (this isn't a true leak
              but results in DCHECKs being hit).

            Change-Id: I3f2b93588dd47f70a4863ecad3b5556c3634ccb4
            Reviewed-on: http://gerrit.cloudera.org:8080/6562
            Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
            Tested-by: Impala Public Jenkins
            —

            Show
            tarmstrong Tim Armstrong added a comment - IMPALA-5124 : add tests for scratch read errors Adds tests for read errors from permissions (i.e. open() fails), corrupt data (integrity check fails) and truncated files (read() fails). Fixes a couple of bugs: Truncated reads were not detected in TmpFilemgr IoMgr buffers weren't returned on error paths (this isn't a true leak but results in DCHECKs being hit). Change-Id: I3f2b93588dd47f70a4863ecad3b5556c3634ccb4 Reviewed-on: http://gerrit.cloudera.org:8080/6562 Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com> Tested-by: Impala Public Jenkins —

              People

              • Assignee:
                tarmstrong Tim Armstrong
                Reporter:
                tarmstrong Tim Armstrong
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development