Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-6455

org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.94.0, 0.95.2
    • 0.95.0
    • Performance
    • Reviewed

    Description

      I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path.

      The attached patch addresses both the issues.

      1. Creates separate input and output path rooted under a single folder e.g. "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/inputs" and "<user_home>/performance_evaluation/<yyyyMMddHHmmss>/outputs", and
      2. The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files.

      Attachments

        1. HBASE-6455_trunk.patch
          2 kB
          Aditya Kishore

        Activity

          People

            adityakishore Aditya Kishore
            adityakishore Aditya Kishore
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: