Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Kudu_Impala
    • Fix Version/s: Impala 2.8.0
    • Component/s: Backend
    • Labels:

      Description

      When Kudu scans are assigned to remote impalads (e.g. the kudu tabletservers do not exist), Impala should indicate that there will be remote reads as we do for HDFS. The information should already be available in the planner and scheduler, the KuduScanNode can put information in the profile. These are "expected" remote reads.

      The other case to consider is "unexpected" remote reads where Impala schedules the scan node where there should be a tserver (according to the metadata gathered during planning), but it isn't there (e.g. tablet moved). For this we'd need support from the Kudu client which doesn't exist. I suspect it's a somewhat uncommon case but worth adding when it's possible: https://issues.apache.org/jira/browse/KUDU-1598

        Activity

        Hide
        mjacobs Matthew Jacobs added a comment -

        "Expected" remote reads will show up in the profile, e.g.

              KUDU_SCAN_NODE (id=0):(Total: 14.043ms, non-child: 14.043ms, % non-child: 100.00%)
                 - BytesRead: 0
                 - KuduRemoteScanTokens: 0 (0)
        

        commit a1c9cb364655d0a65c4b00cb37757d55fc3131a1
        Author: Matthew Jacobs <mj@cloudera.com>
        Date: Thu Sep 8 17:30:14 2016 -0700

        IMPALA-4102: Remote Kudu reads should be reported

        Adds a profile counter for the number of kudu scan tokens
        (ranges) that are "expected" to be remote.

        Testing: Manual; Have been running with this on the Kudu
        cluster. Cannot easily simulate this in the minicluster
        because the scheduler considers multiple impalads on the
        same host to be local for the purposes of determining
        locality. See BackendConfig::LookUpBackendIp().

        Change-Id: I74fd5773c4ae10267de80b6572d93197a4131696
        Reviewed-on: http://gerrit.cloudera.org:8080/4687
        Reviewed-by: Alex Behm <alex.behm@cloudera.com>
        Tested-by: Internal Jenkins

        Show
        mjacobs Matthew Jacobs added a comment - "Expected" remote reads will show up in the profile, e.g. KUDU_SCAN_NODE (id=0):(Total: 14.043ms, non-child: 14.043ms, % non-child: 100.00%) - BytesRead: 0 - KuduRemoteScanTokens: 0 (0) commit a1c9cb364655d0a65c4b00cb37757d55fc3131a1 Author: Matthew Jacobs <mj@cloudera.com> Date: Thu Sep 8 17:30:14 2016 -0700 IMPALA-4102 : Remote Kudu reads should be reported Adds a profile counter for the number of kudu scan tokens (ranges) that are "expected" to be remote. Testing: Manual; Have been running with this on the Kudu cluster. Cannot easily simulate this in the minicluster because the scheduler considers multiple impalads on the same host to be local for the purposes of determining locality. See BackendConfig::LookUpBackendIp(). Change-Id: I74fd5773c4ae10267de80b6572d93197a4131696 Reviewed-on: http://gerrit.cloudera.org:8080/4687 Reviewed-by: Alex Behm <alex.behm@cloudera.com> Tested-by: Internal Jenkins

          People

          • Assignee:
            mjacobs Matthew Jacobs
            Reporter:
            mjacobs Matthew Jacobs
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development