Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 0.89-fb
    • Component/s: Scanners
    • Labels:

      Description

      A new scanner API for reducing unnecessary RPC calls:
      Motivation:

      1. RPC is expensive to both client and server.
      2. The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last next, and close, I want to remove them all (for most of the situation)

      Solution:

      1. a new scanner API (scanOpen) which has an option of transfer data along with the scannerID back in this call
      2. a new scanner API (scanNext) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner.
      3. scanClose is still useful when you want to close the scanner before reach the end.

      For most of the meta-scan, only one RPC will fetch all lines.

        Activity

        @deprecated Yi Deng created issue -
        @deprecated Yi Deng made changes -
        Field Original Value New Value
        Description A new scanner API for reducing unnecessary RPC calls:
        Motivation:
        # RPC is expensive to both client and server.
        # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last next, and close, I want to remove them all (for most of the situation)
        Solution:
        # a new scanner API (scannerOpen) which has an option of transfer data along with the scannerID back in this call
        # a new scanner API (scannerNext) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner.
        # the current scannerClose is still useful when you want to close the scanner before reach the end.
        A new scanner API for reducing unnecessary RPC calls:
        Motivation:
        # RPC is expensive to both client and server.
        # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last nextRows, and close, I want to remove them all (for most of the situation)

        Solution:
        # a new scanner API (scannerOpen) which has an option of transfer data along with the scannerID back in this call
        # a new scanner API (scannerNext) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner.
        # the current scannerClose is still useful when you want to close the scanner before reach the end.
        @deprecated Yi Deng made changes -
        Description A new scanner API for reducing unnecessary RPC calls:
        Motivation:
        # RPC is expensive to both client and server.
        # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last nextRows, and close, I want to remove them all (for most of the situation)

        Solution:
        # a new scanner API (scannerOpen) which has an option of transfer data along with the scannerID back in this call
        # a new scanner API (scannerNext) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner.
        # the current scannerClose is still useful when you want to close the scanner before reach the end.
        A new scanner API for reducing unnecessary RPC calls:
        Motivation:
        # RPC is expensive to both client and server.
        # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last nextRows, and close, I want to remove them all (for most of the situation)

        Solution:
        # a new scanner API ({code}scanOpen{code}) which has an option of transfer data along with the scannerID back in this call
        # a new scanner API (scanNext) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner.
        # the current scannClose is still useful when you want to close the scanner before reach the end.
        @deprecated Yi Deng made changes -
        Description A new scanner API for reducing unnecessary RPC calls:
        Motivation:
        # RPC is expensive to both client and server.
        # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last nextRows, and close, I want to remove them all (for most of the situation)

        Solution:
        # a new scanner API ({code}scanOpen{code}) which has an option of transfer data along with the scannerID back in this call
        # a new scanner API (scanNext) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner.
        # the current scannClose is still useful when you want to close the scanner before reach the end.
        A new scanner API for reducing unnecessary RPC calls:
        Motivation:
        # RPC is expensive to both client and server.
        # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last nextRows, and close, I want to remove them all (for most of the situation)

        Solution:
        # a new scanner API (*scanOpen*) which has an option of transfer data along with the scannerID back in this call
        # a new scanner API (*scanNext*) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner.
        # *scanClose* is still useful when you want to close the scanner before reach the end.
        @deprecated Yi Deng made changes -
        Description A new scanner API for reducing unnecessary RPC calls:
        Motivation:
        # RPC is expensive to both client and server.
        # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last nextRows, and close, I want to remove them all (for most of the situation)

        Solution:
        # a new scanner API (*scanOpen*) which has an option of transfer data along with the scannerID back in this call
        # a new scanner API (*scanNext*) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner.
        # *scanClose* is still useful when you want to close the scanner before reach the end.
        A new scanner API for reducing unnecessary RPC calls:
        Motivation:
        # RPC is expensive to both client and server.
        # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last next, and close, I want to remove them all (for most of the situation)

        Solution:
        # a new scanner API (*scanOpen*) which has an option of transfer data along with the scannerID back in this call
        # a new scanner API (*scanNext*) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner.
        # *scanClose* is still useful when you want to close the scanner before reach the end.
        @deprecated Yi Deng made changes -
        Description A new scanner API for reducing unnecessary RPC calls:
        Motivation:
        # RPC is expensive to both client and server.
        # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last next, and close, I want to remove them all (for most of the situation)

        Solution:
        # a new scanner API (*scanOpen*) which has an option of transfer data along with the scannerID back in this call
        # a new scanner API (*scanNext*) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner.
        # *scanClose* is still useful when you want to close the scanner before reach the end.
        A new scanner API for reducing unnecessary RPC calls:
        Motivation:
        # RPC is expensive to both client and server.
        # The most important function for scanning is getting data, but for each scanning process within a region, there are 3 times of RPC that doesn't transfer data: open, last next, and close, I want to remove them all (for most of the situation)

        Solution:
        # a new scanner API (*scanOpen*) which has an option of transfer data along with the scannerID back in this call
        # a new scanner API (*scanNext*) which is similar to current next, but returns flags of whether more data is available and whether need to scan next region. If no data left, automatically close the scanner.
        # *scanClose* is still useful when you want to close the scanner before reach the end.

        For most of the meta-scan, only one RPC will fetch all lines.

          People

          • Assignee:
            Unassigned
            Reporter:
            @deprecated Yi Deng
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development