HBase
  1. HBase
  2. HBASE-11083

ExportSnapshot should provide capability to limit bandwidth consumption

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.0
    • Component/s: snapshots
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This capability was first brought up in this thread:
      http://search-hadoop.com/m/DHED4Td8Xb1

      The rewritten distcp already provides this capability.
      See MAPREDUCE-2765

      distcp implementation utilizes ThrottledInputStream which provides bandwidth throttling on a specified InputStream.

      As a first step, we can

      • add an option to ExportSnapshot which expresses bandwidth per map in MB
      • utilize ThrottledInputStream in ExportSnapshot#ExportMapper#copyFile().

        Activity

        Enis Soztutar made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Ted Yu made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Ted Yu made changes -
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 0.99.0 [ 12325675 ]
        Ted Yu made changes -
        Attachment 11083-v1.txt [ 12642060 ]
        Ted Yu made changes -
        Attachment 11083-v2.txt [ 12642082 ]
        Ted Yu made changes -
        Attachment 11083-v2.txt [ 12642081 ]
        Ted Yu made changes -
        Attachment 11083-v2.txt [ 12642081 ]
        Ted Yu made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Ted Yu made changes -
        Attachment 11083-v1.txt [ 12642060 ]
        Ted Yu made changes -
        Field Original Value New Value
        Assignee Ted Yu [ yuzhihong@gmail.com ]
        Ted Yu created issue -

          People

          • Assignee:
            Ted Yu
            Reporter:
            Ted Yu
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development