Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-19432

Roll the specified writer in HFileOutputFormat2

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-alpha-1, 2.0.0-beta-1
    • 2.0.0-beta-1, 2.0.0
    • None
    • None
    • Reviewed

    Description

      	// If any of the HFiles for the column families has reached
      	// maxsize, we need to roll all the writers
      	if (wl != null && wl.written + length >= maxsize) {
      	  this.rollRequested = true;
      	}
      

      If we always roll all the writers, a large number of small files will be generated in the multi family or multi table scene.
      So we should only roll the specified writer which HFile has reached maxsize.

      Attachments

        1. HBASE-19432.master.001.patch
          3 kB
          Guangxu Cheng
        2. HBASE-19432.master.002.patch
          3 kB
          Guangxu Cheng

        Activity

          People

            gxcheng Guangxu Cheng
            gxcheng Guangxu Cheng
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: