Details
-
Bug
-
Status: Closed
-
Minor
-
Resolution: Fixed
-
None
-
None
-
None
-
None
Description
Default mapfile index interval is every 128 entries. Basic tests show PerformanceEvaluation mapfile test random reading 100k records in 60plus seconds. If index interval is set to 1 so we don't have to next around looking for our record, then 100k random reads take 7 seconds. This is using local filesystem. If I set it to 16, then takes 12 seconds.
Testing doing PerformanceEvaluation random reads against hbase, with interval set to 16, we run 50% faster (hdfs is in the picture).