Hama
  1. Hama
  2. HAMA-154

[Performance]Combine multi-mapreduce jobs into a single mapreduce job while computing such as 'C=alpha*B + A'

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: math
    • Labels:
      None

      Description

      now, In Hama we use two map-reduce jobs to do 'C=alpha*B + A'.
      first job is to generate a matrix B' = alpha * B
      second job is to generate the result matrix C = B' + A
      I think the first map-reduce job is unneccesary for we can do the constant-multiplication during Addition-MapReducing.

      And we also encount the same problem during computing 'A1 + A2 + ... + An'.
      we need to generate (n-1) hbase-tables for the temp results during addition. Can we just combine n-1 addition-mapreduce jobs into only a simple one?

      1. HAMA-154_v02.patch
        7 kB
        Sijie Guo
      2. HAMA-154_v03.patch
        12 kB
        Edward J. Yoon
      3. HAMA-154.patch
        3 kB
        Sijie Guo

        Activity

        Hide
        Sijie Guo added a comment -

        attach a simple patch to change 'C = alpha * B + A'.
        can anyone review it first and give some suggestion?

        Show
        Sijie Guo added a comment - attach a simple patch to change 'C = alpha * B + A'. can anyone review it first and give some suggestion?
        Hide
        Edward J. Yoon added a comment -

        Oh yes, It'll be improved.

        Show
        Edward J. Yoon added a comment - Oh yes, It'll be improved.
        Hide
        Edward J. Yoon added a comment -

        >> And we also encount the same problem during computing 'A1 + A2 + ... + An'.
        >> we need to generate (n-1) hbase-tables for the temp results during addition. Can we just combine n-1 addition-mapreduce jobs into only a simple one?

        Did you mean the addition of multiple matrices? Then, I guess an scanners of each table can be opened at once.

        Show
        Edward J. Yoon added a comment - >> And we also encount the same problem during computing 'A1 + A2 + ... + An'. >> we need to generate (n-1) hbase-tables for the temp results during addition. Can we just combine n-1 addition-mapreduce jobs into only a simple one? Did you mean the addition of multiple matrices? Then, I guess an scanners of each table can be opened at once.
        Hide
        Sijie Guo added a comment -

        >> Did you mean the addition of multiple matrices? Then, I guess an scanners of each table can be opened at once.

        Yes. I will try it tonight and attach a new patch for reviewing.

        Show
        Sijie Guo added a comment - >> Did you mean the addition of multiple matrices? Then, I guess an scanners of each table can be opened at once. Yes. I will try it tonight and attach a new patch for reviewing.
        Hide
        Sijie Guo added a comment -

        attach a new patch.

        it deals with 'C = A + alpha1 * B1 + alpha2 * B2 + ... + alphan * Bn' in a map-reduce job.
        the RowCylicAdditionMap will receive tow parameters during InitJob.
        first one is summands list which is a summands list composed by the matrix paths seperated by comma. such as 'B1,B2,...,Bn';
        second one is alphas list wich is a list composed by the alphas used in the addition seperated by comma. such as 'alpha1, alpha2,...,alphan';

        Maybe I need a detail performance test to eveluate this method latter.

        Welcome for comments.

        Show
        Sijie Guo added a comment - attach a new patch. it deals with 'C = A + alpha1 * B1 + alpha2 * B2 + ... + alphan * Bn' in a map-reduce job. the RowCylicAdditionMap will receive tow parameters during InitJob. first one is summands list which is a summands list composed by the matrix paths seperated by comma. such as 'B1,B2,...,Bn'; second one is alphas list wich is a list composed by the alphas used in the addition seperated by comma. such as 'alpha1, alpha2,...,alphan'; Maybe I need a detail performance test to eveluate this method latter. Welcome for comments.
        Hide
        Edward J. Yoon added a comment -

        Looks great to me. +1

        Show
        Edward J. Yoon added a comment - Looks great to me. +1
        Hide
        Edward J. Yoon added a comment -

        It's my test result.

        Matrix a, b : 5000 * 5000
        c = a.add(0.343, b);

        turnk version : 20 minutes
        patch applied version : 8 minutes

        Show
        Edward J. Yoon added a comment - It's my test result. Matrix a, b : 5000 * 5000 c = a.add(0.343, b); turnk version : 20 minutes patch applied version : 8 minutes
        Hide
        Edward J. Yoon added a comment -

        Full log:

        [d8g053:/root/hama-trunk]# bin/hama examples add fiveA fiveB
        09/03/04 17:35:15 INFO hama.AbstractMatrix: Initializing the matrix storage.
        09/03/04 17:35:21 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randnomnp
        09/03/04 17:35:21 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
        09/03/04 17:35:21 WARN mapred.JobClient: Use genericOptions for the option -libjars
        09/03/04 17:35:21 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
        09/03/04 17:35:21 INFO mapred.TableInputFormatBase: split: 0->d8g053.nhncorp.com:,000000000001394
        09/03/04 17:35:21 INFO mapred.TableInputFormatBase: split: 1->d8g053.nhncorp.com:000000000001394,000000000003146
        09/03/04 17:35:21 INFO mapred.TableInputFormatBase: split: 2->d8g054.nhncorp.com:000000000003146,
        09/03/04 17:35:21 INFO mapred.JobClient: Running job: job_200902131649_0036
        09/03/04 17:35:22 INFO mapred.JobClient: map 0% reduce 0%
        09/03/04 17:37:44 INFO mapred.JobClient: map 33% reduce 0%
        09/03/04 17:37:51 INFO mapred.JobClient: map 33% reduce 3%
        09/03/04 17:37:54 INFO mapred.JobClient: map 33% reduce 7%
        09/03/04 17:38:00 INFO mapred.JobClient: map 33% reduce 11%
        09/03/04 17:38:26 INFO mapred.JobClient: map 66% reduce 11%
        09/03/04 17:38:32 INFO mapred.JobClient: map 66% reduce 14%
        09/03/04 17:38:33 INFO mapred.JobClient: map 100% reduce 14%
        09/03/04 17:38:40 INFO mapred.JobClient: map 100% reduce 18%
        09/03/04 17:38:43 INFO mapred.JobClient: map 100% reduce 37%
        09/03/04 17:38:48 INFO mapred.JobClient: map 100% reduce 53%
        09/03/04 17:38:50 INFO mapred.JobClient: map 100% reduce 68%
        09/03/04 17:38:55 INFO mapred.JobClient: map 100% reduce 69%
        09/03/04 17:39:00 INFO mapred.JobClient: map 100% reduce 70%
        09/03/04 17:39:08 INFO mapred.JobClient: map 100% reduce 71%
        09/03/04 17:39:13 INFO mapred.JobClient: map 100% reduce 72%
        09/03/04 17:39:18 INFO mapred.JobClient: map 100% reduce 73%
        09/03/04 17:39:23 INFO mapred.JobClient: map 100% reduce 74%
        09/03/04 17:39:28 INFO mapred.JobClient: map 100% reduce 75%
        09/03/04 17:39:35 INFO mapred.JobClient: map 100% reduce 76%
        09/03/04 17:39:40 INFO mapred.JobClient: map 100% reduce 77%
        09/03/04 17:39:53 INFO mapred.JobClient: map 100% reduce 78%
        09/03/04 17:39:58 INFO mapred.JobClient: map 100% reduce 79%
        09/03/04 17:40:03 INFO mapred.JobClient: map 100% reduce 80%
        09/03/04 17:40:08 INFO mapred.JobClient: map 100% reduce 81%
        09/03/04 17:40:13 INFO mapred.JobClient: map 100% reduce 82%
        09/03/04 17:40:20 INFO mapred.JobClient: map 100% reduce 83%
        09/03/04 17:40:27 INFO mapred.JobClient: map 100% reduce 84%
        09/03/04 17:40:32 INFO mapred.JobClient: map 100% reduce 85%
        09/03/04 17:40:38 INFO mapred.JobClient: map 100% reduce 86%
        09/03/04 17:40:43 INFO mapred.JobClient: map 100% reduce 87%
        09/03/04 17:40:57 INFO mapred.JobClient: map 100% reduce 88%
        09/03/04 17:41:02 INFO mapred.JobClient: map 100% reduce 89%
        09/03/04 17:41:08 INFO mapred.JobClient: map 100% reduce 90%
        09/03/04 17:41:13 INFO mapred.JobClient: map 100% reduce 91%
        09/03/04 17:41:19 INFO mapred.JobClient: map 100% reduce 92%
        09/03/04 17:41:25 INFO mapred.JobClient: map 100% reduce 93%
        09/03/04 17:41:33 INFO mapred.JobClient: map 100% reduce 94%
        09/03/04 17:41:38 INFO mapred.JobClient: map 100% reduce 95%
        09/03/04 17:41:43 INFO mapred.JobClient: map 100% reduce 96%
        09/03/04 17:41:50 INFO mapred.JobClient: map 100% reduce 97%
        09/03/04 17:41:55 INFO mapred.JobClient: map 100% reduce 98%
        09/03/04 17:42:02 INFO mapred.JobClient: map 100% reduce 99%
        09/03/04 17:42:05 INFO mapred.JobClient: map 100% reduce 100%
        09/03/04 17:42:10 INFO mapred.JobClient: Job complete: job_200902131649_0036
        09/03/04 17:42:10 INFO mapred.JobClient: Counters: 15
        09/03/04 17:42:10 INFO mapred.JobClient: File Systems
        09/03/04 17:42:10 INFO mapred.JobClient: Local bytes read=1239653931
        09/03/04 17:42:10 INFO mapred.JobClient: Local bytes written=1859371053
        09/03/04 17:42:10 INFO mapred.JobClient: Job Counters
        09/03/04 17:42:10 INFO mapred.JobClient: Launched reduce tasks=3
        09/03/04 17:42:10 INFO mapred.JobClient: Rack-local map tasks=2
        09/03/04 17:42:10 INFO mapred.JobClient: Launched map tasks=5
        09/03/04 17:42:10 INFO mapred.JobClient: Data-local map tasks=3
        09/03/04 17:42:10 INFO mapred.JobClient: Map-Reduce Framework
        09/03/04 17:42:10 INFO mapred.JobClient: Reduce input groups=5001
        09/03/04 17:42:10 INFO mapred.JobClient: Combine output records=0
        09/03/04 17:42:10 INFO mapred.JobClient: Map input records=5001
        09/03/04 17:42:10 INFO mapred.JobClient: Reduce output records=5001
        09/03/04 17:42:10 INFO mapred.JobClient: Map output bytes=619765182
        09/03/04 17:42:10 INFO mapred.JobClient: Map input bytes=0
        09/03/04 17:42:10 INFO mapred.JobClient: Combine input records=0
        09/03/04 17:42:10 INFO mapred.JobClient: Map output records=5001
        09/03/04 17:42:10 INFO mapred.JobClient: Reduce input records=5001
        09/03/04 17:42:13 INFO hama.AbstractMatrix: Initializing the matrix storage.
        09/03/04 17:42:17 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randeeiww
        09/03/04 17:42:17 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
        09/03/04 17:42:17 WARN mapred.JobClient: Use genericOptions for the option -libjars
        09/03/04 17:42:18 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
        09/03/04 17:42:18 INFO mapred.HTableInputFormatBase: split: 0->d8g054.nhncorp.com:,000000000001367
        09/03/04 17:42:18 INFO mapred.HTableInputFormatBase: split: 1->d8g053.nhncorp.com:000000000001367,000000000003424
        09/03/04 17:42:18 INFO mapred.HTableInputFormatBase: split: 2->d8g055.nhncorp.com:000000000003424,
        09/03/04 17:42:18 INFO mapred.JobClient: Running job: job_200902131649_0037
        09/03/04 17:42:19 INFO mapred.JobClient: map 0% reduce 0%
        09/03/04 17:42:43 INFO mapred.JobClient: map 1% reduce 0%
        09/03/04 17:42:50 INFO mapred.JobClient: map 2% reduce 0%
        09/03/04 17:42:53 INFO mapred.JobClient: map 3% reduce 0%
        09/03/04 17:42:58 INFO mapred.JobClient: map 5% reduce 0%
        09/03/04 17:43:04 INFO mapred.JobClient: map 6% reduce 0%
        09/03/04 17:43:08 INFO mapred.JobClient: map 7% reduce 0%
        09/03/04 17:43:13 INFO mapred.JobClient: map 8% reduce 0%
        09/03/04 17:43:18 INFO mapred.JobClient: map 9% reduce 0%
        09/03/04 17:43:23 INFO mapred.JobClient: map 10% reduce 0%
        09/03/04 17:43:24 INFO mapred.JobClient: map 11% reduce 0%
        09/03/04 17:43:28 INFO mapred.JobClient: map 12% reduce 0%
        09/03/04 17:43:33 INFO mapred.JobClient: map 13% reduce 0%
        09/03/04 17:43:38 INFO mapred.JobClient: map 14% reduce 0%
        09/03/04 17:43:40 INFO mapred.JobClient: map 15% reduce 0%
        09/03/04 17:43:44 INFO mapred.JobClient: map 16% reduce 0%
        09/03/04 17:43:50 INFO mapred.JobClient: map 17% reduce 0%
        09/03/04 17:43:53 INFO mapred.JobClient: map 18% reduce 0%
        09/03/04 17:43:58 INFO mapred.JobClient: map 19% reduce 0%
        09/03/04 17:44:03 INFO mapred.JobClient: map 20% reduce 0%
        09/03/04 17:44:13 INFO mapred.JobClient: map 21% reduce 0%
        09/03/04 17:44:18 INFO mapred.JobClient: map 22% reduce 0%
        09/03/04 17:44:23 INFO mapred.JobClient: map 23% reduce 0%
        09/03/04 17:44:33 INFO mapred.JobClient: map 24% reduce 0%
        09/03/04 17:44:35 INFO mapred.JobClient: map 25% reduce 0%
        09/03/04 17:44:38 INFO mapred.JobClient: map 26% reduce 0%
        09/03/04 17:44:45 INFO mapred.JobClient: map 27% reduce 0%
        09/03/04 17:44:48 INFO mapred.JobClient: map 28% reduce 0%
        09/03/04 17:44:53 INFO mapred.JobClient: map 29% reduce 0%
        09/03/04 17:44:58 INFO mapred.JobClient: map 30% reduce 0%
        09/03/04 17:45:03 INFO mapred.JobClient: map 31% reduce 0%
        09/03/04 17:45:04 INFO mapred.JobClient: map 32% reduce 0%
        09/03/04 17:45:08 INFO mapred.JobClient: map 33% reduce 0%
        09/03/04 17:45:14 INFO mapred.JobClient: map 34% reduce 0%
        09/03/04 17:45:15 INFO mapred.JobClient: map 35% reduce 0%
        09/03/04 17:45:20 INFO mapred.JobClient: map 36% reduce 0%
        09/03/04 17:45:24 INFO mapred.JobClient: map 37% reduce 0%
        09/03/04 17:45:30 INFO mapred.JobClient: map 38% reduce 0%
        09/03/04 17:45:39 INFO mapred.JobClient: map 39% reduce 0%
        09/03/04 17:45:44 INFO mapred.JobClient: map 40% reduce 0%
        09/03/04 17:45:49 INFO mapred.JobClient: map 41% reduce 0%
        09/03/04 17:45:54 INFO mapred.JobClient: map 42% reduce 0%
        09/03/04 17:45:59 INFO mapred.JobClient: map 43% reduce 0%
        09/03/04 17:46:08 INFO mapred.JobClient: map 45% reduce 0%
        09/03/04 17:46:25 INFO mapred.JobClient: map 45% reduce 3%
        09/03/04 17:46:26 INFO mapred.JobClient: map 45% reduce 7%
        09/03/04 17:46:28 INFO mapred.JobClient: map 45% reduce 11%
        09/03/04 17:46:55 INFO mapred.JobClient: map 46% reduce 11%
        09/03/04 17:47:05 INFO mapred.JobClient: map 47% reduce 11%
        09/03/04 17:47:21 INFO mapred.JobClient: map 48% reduce 11%
        09/03/04 17:47:30 INFO mapred.JobClient: map 49% reduce 11%
        09/03/04 17:47:35 INFO mapred.JobClient: map 50% reduce 11%
        09/03/04 17:47:50 INFO mapred.JobClient: map 51% reduce 11%
        09/03/04 17:48:00 INFO mapred.JobClient: map 52% reduce 11%
        09/03/04 17:48:05 INFO mapred.JobClient: map 53% reduce 11%
        09/03/04 17:48:15 INFO mapred.JobClient: map 54% reduce 11%
        09/03/04 17:48:30 INFO mapred.JobClient: map 55% reduce 11%
        09/03/04 17:48:35 INFO mapred.JobClient: map 56% reduce 11%
        09/03/04 17:48:45 INFO mapred.JobClient: map 57% reduce 11%
        09/03/04 17:48:55 INFO mapred.JobClient: map 58% reduce 11%
        09/03/04 17:49:40 INFO mapred.JobClient: map 59% reduce 11%
        09/03/04 17:49:50 INFO mapred.JobClient: map 60% reduce 11%
        09/03/04 17:50:00 INFO mapred.JobClient: map 61% reduce 11%
        09/03/04 17:50:10 INFO mapred.JobClient: map 62% reduce 11%
        09/03/04 17:50:21 INFO mapred.JobClient: map 63% reduce 11%
        09/03/04 17:50:31 INFO mapred.JobClient: map 64% reduce 11%
        09/03/04 17:50:41 INFO mapred.JobClient: map 65% reduce 11%
        09/03/04 17:50:51 INFO mapred.JobClient: map 66% reduce 11%
        09/03/04 17:51:04 INFO mapred.JobClient: map 66% reduce 14%
        09/03/04 17:51:09 INFO mapred.JobClient: map 66% reduce 18%
        09/03/04 17:51:12 INFO mapred.JobClient: map 66% reduce 22%
        09/03/04 17:51:47 INFO mapred.JobClient: map 100% reduce 22%
        09/03/04 17:51:59 INFO mapred.JobClient: map 100% reduce 52%
        09/03/04 17:52:02 INFO mapred.JobClient: map 100% reduce 67%
        09/03/04 17:52:04 INFO mapred.JobClient: map 100% reduce 68%
        09/03/04 17:52:09 INFO mapred.JobClient: map 100% reduce 69%
        09/03/04 17:52:14 INFO mapred.JobClient: map 100% reduce 70%
        09/03/04 17:52:19 INFO mapred.JobClient: map 100% reduce 71%
        09/03/04 17:52:27 INFO mapred.JobClient: map 100% reduce 72%
        09/03/04 17:52:32 INFO mapred.JobClient: map 100% reduce 73%
        09/03/04 17:52:37 INFO mapred.JobClient: map 100% reduce 74%
        09/03/04 17:52:44 INFO mapred.JobClient: map 100% reduce 75%
        09/03/04 17:52:49 INFO mapred.JobClient: map 100% reduce 76%
        09/03/04 17:52:57 INFO mapred.JobClient: map 100% reduce 77%
        09/03/04 17:53:12 INFO mapred.JobClient: map 100% reduce 78%
        09/03/04 17:53:14 INFO mapred.JobClient: map 100% reduce 79%
        09/03/04 17:53:19 INFO mapred.JobClient: map 100% reduce 80%
        09/03/04 17:53:27 INFO mapred.JobClient: map 100% reduce 81%
        09/03/04 17:53:32 INFO mapred.JobClient: map 100% reduce 82%
        09/03/04 17:53:39 INFO mapred.JobClient: map 100% reduce 83%
        09/03/04 17:53:44 INFO mapred.JobClient: map 100% reduce 84%
        09/03/04 17:53:49 INFO mapred.JobClient: map 100% reduce 85%
        09/03/04 17:53:57 INFO mapred.JobClient: map 100% reduce 86%
        09/03/04 17:54:02 INFO mapred.JobClient: map 100% reduce 87%
        09/03/04 17:54:19 INFO mapred.JobClient: map 100% reduce 88%
        09/03/04 17:54:28 INFO mapred.JobClient: map 100% reduce 89%
        09/03/04 17:54:33 INFO mapred.JobClient: map 100% reduce 90%
        09/03/04 17:54:39 INFO mapred.JobClient: map 100% reduce 91%
        09/03/04 17:54:44 INFO mapred.JobClient: map 100% reduce 92%
        09/03/04 17:54:49 INFO mapred.JobClient: map 100% reduce 93%
        09/03/04 17:54:58 INFO mapred.JobClient: map 100% reduce 94%
        09/03/04 17:55:04 INFO mapred.JobClient: map 100% reduce 95%
        09/03/04 17:55:09 INFO mapred.JobClient: map 100% reduce 96%
        09/03/04 17:55:14 INFO mapred.JobClient: map 100% reduce 97%
        09/03/04 17:55:19 INFO mapred.JobClient: map 100% reduce 98%
        09/03/04 17:55:28 INFO mapred.JobClient: map 100% reduce 99%
        09/03/04 17:55:31 INFO mapred.JobClient: map 100% reduce 100%
        09/03/04 17:55:36 INFO mapred.JobClient: Job complete: job_200902131649_0037
        09/03/04 17:55:36 INFO mapred.JobClient: Counters: 15
        09/03/04 17:55:36 INFO mapred.JobClient: File Systems
        09/03/04 17:55:36 INFO mapred.JobClient: Local bytes read=1350551934
        09/03/04 17:55:36 INFO mapred.JobClient: Local bytes written=2025705516
        09/03/04 17:55:36 INFO mapred.JobClient: Job Counters
        09/03/04 17:55:36 INFO mapred.JobClient: Launched reduce tasks=3
        09/03/04 17:55:36 INFO mapred.JobClient: Rack-local map tasks=1
        09/03/04 17:55:36 INFO mapred.JobClient: Launched map tasks=4
        09/03/04 17:55:36 INFO mapred.JobClient: Data-local map tasks=3
        09/03/04 17:55:36 INFO mapred.JobClient: Map-Reduce Framework
        09/03/04 17:55:36 INFO mapred.JobClient: Reduce input groups=5000
        09/03/04 17:55:36 INFO mapred.JobClient: Combine output records=0
        09/03/04 17:55:36 INFO mapred.JobClient: Map input records=5000
        09/03/04 17:55:36 INFO mapred.JobClient: Reduce output records=5000
        09/03/04 17:55:36 INFO mapred.JobClient: Map output bytes=675210000
        09/03/04 17:55:36 INFO mapred.JobClient: Map input bytes=0
        09/03/04 17:55:36 INFO mapred.JobClient: Combine input records=0
        09/03/04 17:55:36 INFO mapred.JobClient: Map output records=5000
        09/03/04 17:55:36 INFO mapred.JobClient: Reduce input records=5000

        ---------------------------------------

        [d8g053:/root/hama-trunk]# bin/hama examples add fiveA fiveB
        09/03/04 17:59:06 INFO hama.AbstractMatrix: Initializing the matrix storage.
        09/03/04 17:59:12 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randxfeud
        09/03/04 17:59:13 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
        09/03/04 17:59:13 WARN mapred.JobClient: Use genericOptions for the option -libjars
        09/03/04 17:59:13 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
        09/03/04 17:59:13 INFO mapred.HTableInputFormatBase: split: 0->d8g054.nhncorp.com:,000000000001367
        09/03/04 17:59:13 INFO mapred.HTableInputFormatBase: split: 1->d8g053.nhncorp.com:000000000001367,000000000003424
        09/03/04 17:59:13 INFO mapred.HTableInputFormatBase: split: 2->d8g055.nhncorp.com:000000000003424,
        09/03/04 17:59:13 INFO mapred.JobClient: Running job: job_200902131649_0038
        09/03/04 17:59:14 INFO mapred.JobClient: map 0% reduce 0%
        09/03/04 17:59:41 INFO mapred.JobClient: map 1% reduce 0%
        09/03/04 17:59:44 INFO mapred.JobClient: map 2% reduce 0%
        09/03/04 17:59:46 INFO mapred.JobClient: map 3% reduce 0%
        09/03/04 17:59:54 INFO mapred.JobClient: map 4% reduce 0%
        09/03/04 17:59:56 INFO mapred.JobClient: map 5% reduce 0%
        09/03/04 18:00:01 INFO mapred.JobClient: map 6% reduce 0%
        09/03/04 18:00:04 INFO mapred.JobClient: map 7% reduce 0%
        09/03/04 18:00:09 INFO mapred.JobClient: map 8% reduce 0%
        09/03/04 18:00:11 INFO mapred.JobClient: map 9% reduce 0%
        09/03/04 18:00:16 INFO mapred.JobClient: map 10% reduce 0%
        09/03/04 18:00:19 INFO mapred.JobClient: map 11% reduce 0%
        09/03/04 18:00:26 INFO mapred.JobClient: map 12% reduce 0%
        09/03/04 18:00:29 INFO mapred.JobClient: map 13% reduce 0%
        09/03/04 18:00:31 INFO mapred.JobClient: map 14% reduce 0%
        09/03/04 18:00:34 INFO mapred.JobClient: map 15% reduce 0%
        09/03/04 18:00:41 INFO mapred.JobClient: map 16% reduce 0%
        09/03/04 18:00:44 INFO mapred.JobClient: map 17% reduce 0%
        09/03/04 18:00:46 INFO mapred.JobClient: map 18% reduce 0%
        09/03/04 18:00:51 INFO mapred.JobClient: map 19% reduce 0%
        09/03/04 18:00:56 INFO mapred.JobClient: map 20% reduce 0%
        09/03/04 18:00:59 INFO mapred.JobClient: map 21% reduce 0%
        09/03/04 18:01:04 INFO mapred.JobClient: map 22% reduce 0%
        09/03/04 18:01:06 INFO mapred.JobClient: map 23% reduce 0%
        09/03/04 18:01:11 INFO mapred.JobClient: map 24% reduce 0%
        09/03/04 18:01:16 INFO mapred.JobClient: map 25% reduce 0%
        09/03/04 18:01:18 INFO mapred.JobClient: map 26% reduce 0%
        09/03/04 18:01:24 INFO mapred.JobClient: map 27% reduce 0%
        09/03/04 18:01:27 INFO mapred.JobClient: map 28% reduce 0%
        09/03/04 18:01:32 INFO mapred.JobClient: map 29% reduce 0%
        09/03/04 18:01:34 INFO mapred.JobClient: map 30% reduce 0%
        09/03/04 18:01:42 INFO mapred.JobClient: map 31% reduce 0%
        09/03/04 18:01:44 INFO mapred.JobClient: map 32% reduce 0%
        09/03/04 18:01:47 INFO mapred.JobClient: map 33% reduce 0%
        09/03/04 18:01:52 INFO mapred.JobClient: map 34% reduce 0%
        09/03/04 18:01:57 INFO mapred.JobClient: map 35% reduce 0%
        09/03/04 18:01:59 INFO mapred.JobClient: map 36% reduce 0%
        09/03/04 18:02:02 INFO mapred.JobClient: map 37% reduce 0%
        09/03/04 18:02:07 INFO mapred.JobClient: map 38% reduce 0%
        09/03/04 18:02:12 INFO mapred.JobClient: map 39% reduce 0%
        09/03/04 18:02:14 INFO mapred.JobClient: map 40% reduce 0%
        09/03/04 18:02:17 INFO mapred.JobClient: map 41% reduce 0%
        09/03/04 18:02:22 INFO mapred.JobClient: map 42% reduce 0%
        09/03/04 18:02:27 INFO mapred.JobClient: map 43% reduce 0%
        09/03/04 18:02:29 INFO mapred.JobClient: map 44% reduce 0%
        09/03/04 18:02:32 INFO mapred.JobClient: map 45% reduce 0%
        09/03/04 18:02:37 INFO mapred.JobClient: map 46% reduce 0%
        09/03/04 18:02:42 INFO mapred.JobClient: map 47% reduce 0%
        09/03/04 18:02:47 INFO mapred.JobClient: map 49% reduce 0%
        09/03/04 18:02:52 INFO mapred.JobClient: map 50% reduce 0%
        09/03/04 18:02:57 INFO mapred.JobClient: map 51% reduce 0%
        09/03/04 18:02:59 INFO mapred.JobClient: map 52% reduce 0%
        09/03/04 18:03:02 INFO mapred.JobClient: map 53% reduce 0%
        09/03/04 18:03:07 INFO mapred.JobClient: map 54% reduce 0%
        09/03/04 18:03:11 INFO mapred.JobClient: map 55% reduce 0%
        09/03/04 18:03:19 INFO mapred.JobClient: map 56% reduce 0%
        09/03/04 18:03:20 INFO mapred.JobClient: map 56% reduce 3%
        09/03/04 18:03:21 INFO mapred.JobClient: map 56% reduce 7%
        09/03/04 18:03:24 INFO mapred.JobClient: map 56% reduce 11%
        09/03/04 18:03:29 INFO mapred.JobClient: map 57% reduce 11%
        09/03/04 18:03:34 INFO mapred.JobClient: map 58% reduce 11%
        09/03/04 18:03:44 INFO mapred.JobClient: map 59% reduce 11%
        09/03/04 18:03:50 INFO mapred.JobClient: map 92% reduce 11%
        09/03/04 18:03:54 INFO mapred.JobClient: map 93% reduce 11%
        09/03/04 18:04:01 INFO mapred.JobClient: map 93% reduce 14%
        09/03/04 18:04:04 INFO mapred.JobClient: map 94% reduce 18%
        09/03/04 18:04:05 INFO mapred.JobClient: map 94% reduce 22%
        09/03/04 18:04:14 INFO mapred.JobClient: map 95% reduce 22%
        09/03/04 18:04:19 INFO mapred.JobClient: map 96% reduce 22%
        09/03/04 18:04:29 INFO mapred.JobClient: map 97% reduce 22%
        09/03/04 18:04:34 INFO mapred.JobClient: map 98% reduce 22%
        09/03/04 18:04:44 INFO mapred.JobClient: map 99% reduce 22%
        09/03/04 18:04:53 INFO mapred.JobClient: map 100% reduce 22%
        09/03/04 18:05:02 INFO mapred.JobClient: map 100% reduce 37%
        09/03/04 18:05:03 INFO mapred.JobClient: map 100% reduce 52%
        09/03/04 18:05:05 INFO mapred.JobClient: map 100% reduce 67%
        09/03/04 18:05:06 INFO mapred.JobClient: map 100% reduce 68%
        09/03/04 18:05:13 INFO mapred.JobClient: map 100% reduce 69%
        09/03/04 18:05:18 INFO mapred.JobClient: map 100% reduce 70%
        09/03/04 18:05:23 INFO mapred.JobClient: map 100% reduce 71%
        09/03/04 18:05:30 INFO mapred.JobClient: map 100% reduce 72%
        09/03/04 18:05:35 INFO mapred.JobClient: map 100% reduce 73%
        09/03/04 18:05:41 INFO mapred.JobClient: map 100% reduce 74%
        09/03/04 18:05:48 INFO mapred.JobClient: map 100% reduce 75%
        09/03/04 18:05:51 INFO mapred.JobClient: map 100% reduce 76%
        09/03/04 18:05:58 INFO mapred.JobClient: map 100% reduce 77%
        09/03/04 18:06:05 INFO mapred.JobClient: map 100% reduce 78%
        09/03/04 18:06:10 INFO mapred.JobClient: map 100% reduce 79%
        09/03/04 18:06:17 INFO mapred.JobClient: map 100% reduce 80%
        09/03/04 18:06:21 INFO mapred.JobClient: map 100% reduce 81%
        09/03/04 18:06:36 INFO mapred.JobClient: map 100% reduce 82%
        09/03/04 18:06:43 INFO mapred.JobClient: map 100% reduce 83%
        09/03/04 18:06:50 INFO mapred.JobClient: map 100% reduce 84%
        09/03/04 18:06:55 INFO mapred.JobClient: map 100% reduce 85%
        09/03/04 18:07:01 INFO mapred.JobClient: map 100% reduce 86%
        09/03/04 18:07:07 INFO mapred.JobClient: map 100% reduce 87%
        09/03/04 18:07:14 INFO mapred.JobClient: map 100% reduce 88%
        09/03/04 18:07:20 INFO mapred.JobClient: map 100% reduce 89%
        09/03/04 18:07:27 INFO mapred.JobClient: map 100% reduce 90%
        09/03/04 18:07:34 INFO mapred.JobClient: map 100% reduce 91%
        09/03/04 18:07:39 INFO mapred.JobClient: map 100% reduce 92%
        09/03/04 18:07:45 INFO mapred.JobClient: map 100% reduce 93%
        09/03/04 18:07:51 INFO mapred.JobClient: map 100% reduce 94%
        09/03/04 18:07:58 INFO mapred.JobClient: map 100% reduce 95%
        09/03/04 18:08:10 INFO mapred.JobClient: map 100% reduce 96%
        09/03/04 18:08:16 INFO mapred.JobClient: map 100% reduce 97%
        09/03/04 18:08:21 INFO mapred.JobClient: map 100% reduce 98%
        09/03/04 18:08:27 INFO mapred.JobClient: map 100% reduce 99%
        09/03/04 18:08:32 INFO mapred.JobClient: map 100% reduce 100%
        09/03/04 18:08:38 INFO mapred.JobClient: Job complete: job_200902131649_0038
        09/03/04 18:08:38 INFO mapred.JobClient: Counters: 15
        09/03/04 18:08:38 INFO mapred.JobClient: File Systems
        09/03/04 18:08:38 INFO mapred.JobClient: Local bytes read=1350551934
        09/03/04 18:08:38 INFO mapred.JobClient: Local bytes written=2025705516
        09/03/04 18:08:38 INFO mapred.JobClient: Job Counters
        09/03/04 18:08:38 INFO mapred.JobClient: Launched reduce tasks=3
        09/03/04 18:08:38 INFO mapred.JobClient: Rack-local map tasks=1
        09/03/04 18:08:38 INFO mapred.JobClient: Launched map tasks=4
        09/03/04 18:08:38 INFO mapred.JobClient: Data-local map tasks=3
        09/03/04 18:08:38 INFO mapred.JobClient: Map-Reduce Framework
        09/03/04 18:08:38 INFO mapred.JobClient: Reduce input groups=5000
        09/03/04 18:08:38 INFO mapred.JobClient: Combine output records=0
        09/03/04 18:08:38 INFO mapred.JobClient: Map input records=5000
        09/03/04 18:08:38 INFO mapred.JobClient: Reduce output records=5000
        09/03/04 18:08:38 INFO mapred.JobClient: Map output bytes=675210000
        09/03/04 18:08:38 INFO mapred.JobClient: Map input bytes=0
        09/03/04 18:08:38 INFO mapred.JobClient: Combine input records=0
        09/03/04 18:08:38 INFO mapred.JobClient: Map output records=5000
        09/03/04 18:08:38 INFO mapred.JobClient: Reduce input records=5000

        Show
        Edward J. Yoon added a comment - Full log: [d8g053:/root/hama-trunk] # bin/hama examples add fiveA fiveB 09/03/04 17:35:15 INFO hama.AbstractMatrix: Initializing the matrix storage. 09/03/04 17:35:21 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randnomnp 09/03/04 17:35:21 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 09/03/04 17:35:21 WARN mapred.JobClient: Use genericOptions for the option -libjars 09/03/04 17:35:21 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String). 09/03/04 17:35:21 INFO mapred.TableInputFormatBase: split: 0->d8g053.nhncorp.com:,000000000001394 09/03/04 17:35:21 INFO mapred.TableInputFormatBase: split: 1->d8g053.nhncorp.com:000000000001394,000000000003146 09/03/04 17:35:21 INFO mapred.TableInputFormatBase: split: 2->d8g054.nhncorp.com:000000000003146, 09/03/04 17:35:21 INFO mapred.JobClient: Running job: job_200902131649_0036 09/03/04 17:35:22 INFO mapred.JobClient: map 0% reduce 0% 09/03/04 17:37:44 INFO mapred.JobClient: map 33% reduce 0% 09/03/04 17:37:51 INFO mapred.JobClient: map 33% reduce 3% 09/03/04 17:37:54 INFO mapred.JobClient: map 33% reduce 7% 09/03/04 17:38:00 INFO mapred.JobClient: map 33% reduce 11% 09/03/04 17:38:26 INFO mapred.JobClient: map 66% reduce 11% 09/03/04 17:38:32 INFO mapred.JobClient: map 66% reduce 14% 09/03/04 17:38:33 INFO mapred.JobClient: map 100% reduce 14% 09/03/04 17:38:40 INFO mapred.JobClient: map 100% reduce 18% 09/03/04 17:38:43 INFO mapred.JobClient: map 100% reduce 37% 09/03/04 17:38:48 INFO mapred.JobClient: map 100% reduce 53% 09/03/04 17:38:50 INFO mapred.JobClient: map 100% reduce 68% 09/03/04 17:38:55 INFO mapred.JobClient: map 100% reduce 69% 09/03/04 17:39:00 INFO mapred.JobClient: map 100% reduce 70% 09/03/04 17:39:08 INFO mapred.JobClient: map 100% reduce 71% 09/03/04 17:39:13 INFO mapred.JobClient: map 100% reduce 72% 09/03/04 17:39:18 INFO mapred.JobClient: map 100% reduce 73% 09/03/04 17:39:23 INFO mapred.JobClient: map 100% reduce 74% 09/03/04 17:39:28 INFO mapred.JobClient: map 100% reduce 75% 09/03/04 17:39:35 INFO mapred.JobClient: map 100% reduce 76% 09/03/04 17:39:40 INFO mapred.JobClient: map 100% reduce 77% 09/03/04 17:39:53 INFO mapred.JobClient: map 100% reduce 78% 09/03/04 17:39:58 INFO mapred.JobClient: map 100% reduce 79% 09/03/04 17:40:03 INFO mapred.JobClient: map 100% reduce 80% 09/03/04 17:40:08 INFO mapred.JobClient: map 100% reduce 81% 09/03/04 17:40:13 INFO mapred.JobClient: map 100% reduce 82% 09/03/04 17:40:20 INFO mapred.JobClient: map 100% reduce 83% 09/03/04 17:40:27 INFO mapred.JobClient: map 100% reduce 84% 09/03/04 17:40:32 INFO mapred.JobClient: map 100% reduce 85% 09/03/04 17:40:38 INFO mapred.JobClient: map 100% reduce 86% 09/03/04 17:40:43 INFO mapred.JobClient: map 100% reduce 87% 09/03/04 17:40:57 INFO mapred.JobClient: map 100% reduce 88% 09/03/04 17:41:02 INFO mapred.JobClient: map 100% reduce 89% 09/03/04 17:41:08 INFO mapred.JobClient: map 100% reduce 90% 09/03/04 17:41:13 INFO mapred.JobClient: map 100% reduce 91% 09/03/04 17:41:19 INFO mapred.JobClient: map 100% reduce 92% 09/03/04 17:41:25 INFO mapred.JobClient: map 100% reduce 93% 09/03/04 17:41:33 INFO mapred.JobClient: map 100% reduce 94% 09/03/04 17:41:38 INFO mapred.JobClient: map 100% reduce 95% 09/03/04 17:41:43 INFO mapred.JobClient: map 100% reduce 96% 09/03/04 17:41:50 INFO mapred.JobClient: map 100% reduce 97% 09/03/04 17:41:55 INFO mapred.JobClient: map 100% reduce 98% 09/03/04 17:42:02 INFO mapred.JobClient: map 100% reduce 99% 09/03/04 17:42:05 INFO mapred.JobClient: map 100% reduce 100% 09/03/04 17:42:10 INFO mapred.JobClient: Job complete: job_200902131649_0036 09/03/04 17:42:10 INFO mapred.JobClient: Counters: 15 09/03/04 17:42:10 INFO mapred.JobClient: File Systems 09/03/04 17:42:10 INFO mapred.JobClient: Local bytes read=1239653931 09/03/04 17:42:10 INFO mapred.JobClient: Local bytes written=1859371053 09/03/04 17:42:10 INFO mapred.JobClient: Job Counters 09/03/04 17:42:10 INFO mapred.JobClient: Launched reduce tasks=3 09/03/04 17:42:10 INFO mapred.JobClient: Rack-local map tasks=2 09/03/04 17:42:10 INFO mapred.JobClient: Launched map tasks=5 09/03/04 17:42:10 INFO mapred.JobClient: Data-local map tasks=3 09/03/04 17:42:10 INFO mapred.JobClient: Map-Reduce Framework 09/03/04 17:42:10 INFO mapred.JobClient: Reduce input groups=5001 09/03/04 17:42:10 INFO mapred.JobClient: Combine output records=0 09/03/04 17:42:10 INFO mapred.JobClient: Map input records=5001 09/03/04 17:42:10 INFO mapred.JobClient: Reduce output records=5001 09/03/04 17:42:10 INFO mapred.JobClient: Map output bytes=619765182 09/03/04 17:42:10 INFO mapred.JobClient: Map input bytes=0 09/03/04 17:42:10 INFO mapred.JobClient: Combine input records=0 09/03/04 17:42:10 INFO mapred.JobClient: Map output records=5001 09/03/04 17:42:10 INFO mapred.JobClient: Reduce input records=5001 09/03/04 17:42:13 INFO hama.AbstractMatrix: Initializing the matrix storage. 09/03/04 17:42:17 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randeeiww 09/03/04 17:42:17 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 09/03/04 17:42:17 WARN mapred.JobClient: Use genericOptions for the option -libjars 09/03/04 17:42:18 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String). 09/03/04 17:42:18 INFO mapred.HTableInputFormatBase: split: 0->d8g054.nhncorp.com:,000000000001367 09/03/04 17:42:18 INFO mapred.HTableInputFormatBase: split: 1->d8g053.nhncorp.com:000000000001367,000000000003424 09/03/04 17:42:18 INFO mapred.HTableInputFormatBase: split: 2->d8g055.nhncorp.com:000000000003424, 09/03/04 17:42:18 INFO mapred.JobClient: Running job: job_200902131649_0037 09/03/04 17:42:19 INFO mapred.JobClient: map 0% reduce 0% 09/03/04 17:42:43 INFO mapred.JobClient: map 1% reduce 0% 09/03/04 17:42:50 INFO mapred.JobClient: map 2% reduce 0% 09/03/04 17:42:53 INFO mapred.JobClient: map 3% reduce 0% 09/03/04 17:42:58 INFO mapred.JobClient: map 5% reduce 0% 09/03/04 17:43:04 INFO mapred.JobClient: map 6% reduce 0% 09/03/04 17:43:08 INFO mapred.JobClient: map 7% reduce 0% 09/03/04 17:43:13 INFO mapred.JobClient: map 8% reduce 0% 09/03/04 17:43:18 INFO mapred.JobClient: map 9% reduce 0% 09/03/04 17:43:23 INFO mapred.JobClient: map 10% reduce 0% 09/03/04 17:43:24 INFO mapred.JobClient: map 11% reduce 0% 09/03/04 17:43:28 INFO mapred.JobClient: map 12% reduce 0% 09/03/04 17:43:33 INFO mapred.JobClient: map 13% reduce 0% 09/03/04 17:43:38 INFO mapred.JobClient: map 14% reduce 0% 09/03/04 17:43:40 INFO mapred.JobClient: map 15% reduce 0% 09/03/04 17:43:44 INFO mapred.JobClient: map 16% reduce 0% 09/03/04 17:43:50 INFO mapred.JobClient: map 17% reduce 0% 09/03/04 17:43:53 INFO mapred.JobClient: map 18% reduce 0% 09/03/04 17:43:58 INFO mapred.JobClient: map 19% reduce 0% 09/03/04 17:44:03 INFO mapred.JobClient: map 20% reduce 0% 09/03/04 17:44:13 INFO mapred.JobClient: map 21% reduce 0% 09/03/04 17:44:18 INFO mapred.JobClient: map 22% reduce 0% 09/03/04 17:44:23 INFO mapred.JobClient: map 23% reduce 0% 09/03/04 17:44:33 INFO mapred.JobClient: map 24% reduce 0% 09/03/04 17:44:35 INFO mapred.JobClient: map 25% reduce 0% 09/03/04 17:44:38 INFO mapred.JobClient: map 26% reduce 0% 09/03/04 17:44:45 INFO mapred.JobClient: map 27% reduce 0% 09/03/04 17:44:48 INFO mapred.JobClient: map 28% reduce 0% 09/03/04 17:44:53 INFO mapred.JobClient: map 29% reduce 0% 09/03/04 17:44:58 INFO mapred.JobClient: map 30% reduce 0% 09/03/04 17:45:03 INFO mapred.JobClient: map 31% reduce 0% 09/03/04 17:45:04 INFO mapred.JobClient: map 32% reduce 0% 09/03/04 17:45:08 INFO mapred.JobClient: map 33% reduce 0% 09/03/04 17:45:14 INFO mapred.JobClient: map 34% reduce 0% 09/03/04 17:45:15 INFO mapred.JobClient: map 35% reduce 0% 09/03/04 17:45:20 INFO mapred.JobClient: map 36% reduce 0% 09/03/04 17:45:24 INFO mapred.JobClient: map 37% reduce 0% 09/03/04 17:45:30 INFO mapred.JobClient: map 38% reduce 0% 09/03/04 17:45:39 INFO mapred.JobClient: map 39% reduce 0% 09/03/04 17:45:44 INFO mapred.JobClient: map 40% reduce 0% 09/03/04 17:45:49 INFO mapred.JobClient: map 41% reduce 0% 09/03/04 17:45:54 INFO mapred.JobClient: map 42% reduce 0% 09/03/04 17:45:59 INFO mapred.JobClient: map 43% reduce 0% 09/03/04 17:46:08 INFO mapred.JobClient: map 45% reduce 0% 09/03/04 17:46:25 INFO mapred.JobClient: map 45% reduce 3% 09/03/04 17:46:26 INFO mapred.JobClient: map 45% reduce 7% 09/03/04 17:46:28 INFO mapred.JobClient: map 45% reduce 11% 09/03/04 17:46:55 INFO mapred.JobClient: map 46% reduce 11% 09/03/04 17:47:05 INFO mapred.JobClient: map 47% reduce 11% 09/03/04 17:47:21 INFO mapred.JobClient: map 48% reduce 11% 09/03/04 17:47:30 INFO mapred.JobClient: map 49% reduce 11% 09/03/04 17:47:35 INFO mapred.JobClient: map 50% reduce 11% 09/03/04 17:47:50 INFO mapred.JobClient: map 51% reduce 11% 09/03/04 17:48:00 INFO mapred.JobClient: map 52% reduce 11% 09/03/04 17:48:05 INFO mapred.JobClient: map 53% reduce 11% 09/03/04 17:48:15 INFO mapred.JobClient: map 54% reduce 11% 09/03/04 17:48:30 INFO mapred.JobClient: map 55% reduce 11% 09/03/04 17:48:35 INFO mapred.JobClient: map 56% reduce 11% 09/03/04 17:48:45 INFO mapred.JobClient: map 57% reduce 11% 09/03/04 17:48:55 INFO mapred.JobClient: map 58% reduce 11% 09/03/04 17:49:40 INFO mapred.JobClient: map 59% reduce 11% 09/03/04 17:49:50 INFO mapred.JobClient: map 60% reduce 11% 09/03/04 17:50:00 INFO mapred.JobClient: map 61% reduce 11% 09/03/04 17:50:10 INFO mapred.JobClient: map 62% reduce 11% 09/03/04 17:50:21 INFO mapred.JobClient: map 63% reduce 11% 09/03/04 17:50:31 INFO mapred.JobClient: map 64% reduce 11% 09/03/04 17:50:41 INFO mapred.JobClient: map 65% reduce 11% 09/03/04 17:50:51 INFO mapred.JobClient: map 66% reduce 11% 09/03/04 17:51:04 INFO mapred.JobClient: map 66% reduce 14% 09/03/04 17:51:09 INFO mapred.JobClient: map 66% reduce 18% 09/03/04 17:51:12 INFO mapred.JobClient: map 66% reduce 22% 09/03/04 17:51:47 INFO mapred.JobClient: map 100% reduce 22% 09/03/04 17:51:59 INFO mapred.JobClient: map 100% reduce 52% 09/03/04 17:52:02 INFO mapred.JobClient: map 100% reduce 67% 09/03/04 17:52:04 INFO mapred.JobClient: map 100% reduce 68% 09/03/04 17:52:09 INFO mapred.JobClient: map 100% reduce 69% 09/03/04 17:52:14 INFO mapred.JobClient: map 100% reduce 70% 09/03/04 17:52:19 INFO mapred.JobClient: map 100% reduce 71% 09/03/04 17:52:27 INFO mapred.JobClient: map 100% reduce 72% 09/03/04 17:52:32 INFO mapred.JobClient: map 100% reduce 73% 09/03/04 17:52:37 INFO mapred.JobClient: map 100% reduce 74% 09/03/04 17:52:44 INFO mapred.JobClient: map 100% reduce 75% 09/03/04 17:52:49 INFO mapred.JobClient: map 100% reduce 76% 09/03/04 17:52:57 INFO mapred.JobClient: map 100% reduce 77% 09/03/04 17:53:12 INFO mapred.JobClient: map 100% reduce 78% 09/03/04 17:53:14 INFO mapred.JobClient: map 100% reduce 79% 09/03/04 17:53:19 INFO mapred.JobClient: map 100% reduce 80% 09/03/04 17:53:27 INFO mapred.JobClient: map 100% reduce 81% 09/03/04 17:53:32 INFO mapred.JobClient: map 100% reduce 82% 09/03/04 17:53:39 INFO mapred.JobClient: map 100% reduce 83% 09/03/04 17:53:44 INFO mapred.JobClient: map 100% reduce 84% 09/03/04 17:53:49 INFO mapred.JobClient: map 100% reduce 85% 09/03/04 17:53:57 INFO mapred.JobClient: map 100% reduce 86% 09/03/04 17:54:02 INFO mapred.JobClient: map 100% reduce 87% 09/03/04 17:54:19 INFO mapred.JobClient: map 100% reduce 88% 09/03/04 17:54:28 INFO mapred.JobClient: map 100% reduce 89% 09/03/04 17:54:33 INFO mapred.JobClient: map 100% reduce 90% 09/03/04 17:54:39 INFO mapred.JobClient: map 100% reduce 91% 09/03/04 17:54:44 INFO mapred.JobClient: map 100% reduce 92% 09/03/04 17:54:49 INFO mapred.JobClient: map 100% reduce 93% 09/03/04 17:54:58 INFO mapred.JobClient: map 100% reduce 94% 09/03/04 17:55:04 INFO mapred.JobClient: map 100% reduce 95% 09/03/04 17:55:09 INFO mapred.JobClient: map 100% reduce 96% 09/03/04 17:55:14 INFO mapred.JobClient: map 100% reduce 97% 09/03/04 17:55:19 INFO mapred.JobClient: map 100% reduce 98% 09/03/04 17:55:28 INFO mapred.JobClient: map 100% reduce 99% 09/03/04 17:55:31 INFO mapred.JobClient: map 100% reduce 100% 09/03/04 17:55:36 INFO mapred.JobClient: Job complete: job_200902131649_0037 09/03/04 17:55:36 INFO mapred.JobClient: Counters: 15 09/03/04 17:55:36 INFO mapred.JobClient: File Systems 09/03/04 17:55:36 INFO mapred.JobClient: Local bytes read=1350551934 09/03/04 17:55:36 INFO mapred.JobClient: Local bytes written=2025705516 09/03/04 17:55:36 INFO mapred.JobClient: Job Counters 09/03/04 17:55:36 INFO mapred.JobClient: Launched reduce tasks=3 09/03/04 17:55:36 INFO mapred.JobClient: Rack-local map tasks=1 09/03/04 17:55:36 INFO mapred.JobClient: Launched map tasks=4 09/03/04 17:55:36 INFO mapred.JobClient: Data-local map tasks=3 09/03/04 17:55:36 INFO mapred.JobClient: Map-Reduce Framework 09/03/04 17:55:36 INFO mapred.JobClient: Reduce input groups=5000 09/03/04 17:55:36 INFO mapred.JobClient: Combine output records=0 09/03/04 17:55:36 INFO mapred.JobClient: Map input records=5000 09/03/04 17:55:36 INFO mapred.JobClient: Reduce output records=5000 09/03/04 17:55:36 INFO mapred.JobClient: Map output bytes=675210000 09/03/04 17:55:36 INFO mapred.JobClient: Map input bytes=0 09/03/04 17:55:36 INFO mapred.JobClient: Combine input records=0 09/03/04 17:55:36 INFO mapred.JobClient: Map output records=5000 09/03/04 17:55:36 INFO mapred.JobClient: Reduce input records=5000 --------------------------------------- [d8g053:/root/hama-trunk] # bin/hama examples add fiveA fiveB 09/03/04 17:59:06 INFO hama.AbstractMatrix: Initializing the matrix storage. 09/03/04 17:59:12 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randxfeud 09/03/04 17:59:13 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 09/03/04 17:59:13 WARN mapred.JobClient: Use genericOptions for the option -libjars 09/03/04 17:59:13 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String). 09/03/04 17:59:13 INFO mapred.HTableInputFormatBase: split: 0->d8g054.nhncorp.com:,000000000001367 09/03/04 17:59:13 INFO mapred.HTableInputFormatBase: split: 1->d8g053.nhncorp.com:000000000001367,000000000003424 09/03/04 17:59:13 INFO mapred.HTableInputFormatBase: split: 2->d8g055.nhncorp.com:000000000003424, 09/03/04 17:59:13 INFO mapred.JobClient: Running job: job_200902131649_0038 09/03/04 17:59:14 INFO mapred.JobClient: map 0% reduce 0% 09/03/04 17:59:41 INFO mapred.JobClient: map 1% reduce 0% 09/03/04 17:59:44 INFO mapred.JobClient: map 2% reduce 0% 09/03/04 17:59:46 INFO mapred.JobClient: map 3% reduce 0% 09/03/04 17:59:54 INFO mapred.JobClient: map 4% reduce 0% 09/03/04 17:59:56 INFO mapred.JobClient: map 5% reduce 0% 09/03/04 18:00:01 INFO mapred.JobClient: map 6% reduce 0% 09/03/04 18:00:04 INFO mapred.JobClient: map 7% reduce 0% 09/03/04 18:00:09 INFO mapred.JobClient: map 8% reduce 0% 09/03/04 18:00:11 INFO mapred.JobClient: map 9% reduce 0% 09/03/04 18:00:16 INFO mapred.JobClient: map 10% reduce 0% 09/03/04 18:00:19 INFO mapred.JobClient: map 11% reduce 0% 09/03/04 18:00:26 INFO mapred.JobClient: map 12% reduce 0% 09/03/04 18:00:29 INFO mapred.JobClient: map 13% reduce 0% 09/03/04 18:00:31 INFO mapred.JobClient: map 14% reduce 0% 09/03/04 18:00:34 INFO mapred.JobClient: map 15% reduce 0% 09/03/04 18:00:41 INFO mapred.JobClient: map 16% reduce 0% 09/03/04 18:00:44 INFO mapred.JobClient: map 17% reduce 0% 09/03/04 18:00:46 INFO mapred.JobClient: map 18% reduce 0% 09/03/04 18:00:51 INFO mapred.JobClient: map 19% reduce 0% 09/03/04 18:00:56 INFO mapred.JobClient: map 20% reduce 0% 09/03/04 18:00:59 INFO mapred.JobClient: map 21% reduce 0% 09/03/04 18:01:04 INFO mapred.JobClient: map 22% reduce 0% 09/03/04 18:01:06 INFO mapred.JobClient: map 23% reduce 0% 09/03/04 18:01:11 INFO mapred.JobClient: map 24% reduce 0% 09/03/04 18:01:16 INFO mapred.JobClient: map 25% reduce 0% 09/03/04 18:01:18 INFO mapred.JobClient: map 26% reduce 0% 09/03/04 18:01:24 INFO mapred.JobClient: map 27% reduce 0% 09/03/04 18:01:27 INFO mapred.JobClient: map 28% reduce 0% 09/03/04 18:01:32 INFO mapred.JobClient: map 29% reduce 0% 09/03/04 18:01:34 INFO mapred.JobClient: map 30% reduce 0% 09/03/04 18:01:42 INFO mapred.JobClient: map 31% reduce 0% 09/03/04 18:01:44 INFO mapred.JobClient: map 32% reduce 0% 09/03/04 18:01:47 INFO mapred.JobClient: map 33% reduce 0% 09/03/04 18:01:52 INFO mapred.JobClient: map 34% reduce 0% 09/03/04 18:01:57 INFO mapred.JobClient: map 35% reduce 0% 09/03/04 18:01:59 INFO mapred.JobClient: map 36% reduce 0% 09/03/04 18:02:02 INFO mapred.JobClient: map 37% reduce 0% 09/03/04 18:02:07 INFO mapred.JobClient: map 38% reduce 0% 09/03/04 18:02:12 INFO mapred.JobClient: map 39% reduce 0% 09/03/04 18:02:14 INFO mapred.JobClient: map 40% reduce 0% 09/03/04 18:02:17 INFO mapred.JobClient: map 41% reduce 0% 09/03/04 18:02:22 INFO mapred.JobClient: map 42% reduce 0% 09/03/04 18:02:27 INFO mapred.JobClient: map 43% reduce 0% 09/03/04 18:02:29 INFO mapred.JobClient: map 44% reduce 0% 09/03/04 18:02:32 INFO mapred.JobClient: map 45% reduce 0% 09/03/04 18:02:37 INFO mapred.JobClient: map 46% reduce 0% 09/03/04 18:02:42 INFO mapred.JobClient: map 47% reduce 0% 09/03/04 18:02:47 INFO mapred.JobClient: map 49% reduce 0% 09/03/04 18:02:52 INFO mapred.JobClient: map 50% reduce 0% 09/03/04 18:02:57 INFO mapred.JobClient: map 51% reduce 0% 09/03/04 18:02:59 INFO mapred.JobClient: map 52% reduce 0% 09/03/04 18:03:02 INFO mapred.JobClient: map 53% reduce 0% 09/03/04 18:03:07 INFO mapred.JobClient: map 54% reduce 0% 09/03/04 18:03:11 INFO mapred.JobClient: map 55% reduce 0% 09/03/04 18:03:19 INFO mapred.JobClient: map 56% reduce 0% 09/03/04 18:03:20 INFO mapred.JobClient: map 56% reduce 3% 09/03/04 18:03:21 INFO mapred.JobClient: map 56% reduce 7% 09/03/04 18:03:24 INFO mapred.JobClient: map 56% reduce 11% 09/03/04 18:03:29 INFO mapred.JobClient: map 57% reduce 11% 09/03/04 18:03:34 INFO mapred.JobClient: map 58% reduce 11% 09/03/04 18:03:44 INFO mapred.JobClient: map 59% reduce 11% 09/03/04 18:03:50 INFO mapred.JobClient: map 92% reduce 11% 09/03/04 18:03:54 INFO mapred.JobClient: map 93% reduce 11% 09/03/04 18:04:01 INFO mapred.JobClient: map 93% reduce 14% 09/03/04 18:04:04 INFO mapred.JobClient: map 94% reduce 18% 09/03/04 18:04:05 INFO mapred.JobClient: map 94% reduce 22% 09/03/04 18:04:14 INFO mapred.JobClient: map 95% reduce 22% 09/03/04 18:04:19 INFO mapred.JobClient: map 96% reduce 22% 09/03/04 18:04:29 INFO mapred.JobClient: map 97% reduce 22% 09/03/04 18:04:34 INFO mapred.JobClient: map 98% reduce 22% 09/03/04 18:04:44 INFO mapred.JobClient: map 99% reduce 22% 09/03/04 18:04:53 INFO mapred.JobClient: map 100% reduce 22% 09/03/04 18:05:02 INFO mapred.JobClient: map 100% reduce 37% 09/03/04 18:05:03 INFO mapred.JobClient: map 100% reduce 52% 09/03/04 18:05:05 INFO mapred.JobClient: map 100% reduce 67% 09/03/04 18:05:06 INFO mapred.JobClient: map 100% reduce 68% 09/03/04 18:05:13 INFO mapred.JobClient: map 100% reduce 69% 09/03/04 18:05:18 INFO mapred.JobClient: map 100% reduce 70% 09/03/04 18:05:23 INFO mapred.JobClient: map 100% reduce 71% 09/03/04 18:05:30 INFO mapred.JobClient: map 100% reduce 72% 09/03/04 18:05:35 INFO mapred.JobClient: map 100% reduce 73% 09/03/04 18:05:41 INFO mapred.JobClient: map 100% reduce 74% 09/03/04 18:05:48 INFO mapred.JobClient: map 100% reduce 75% 09/03/04 18:05:51 INFO mapred.JobClient: map 100% reduce 76% 09/03/04 18:05:58 INFO mapred.JobClient: map 100% reduce 77% 09/03/04 18:06:05 INFO mapred.JobClient: map 100% reduce 78% 09/03/04 18:06:10 INFO mapred.JobClient: map 100% reduce 79% 09/03/04 18:06:17 INFO mapred.JobClient: map 100% reduce 80% 09/03/04 18:06:21 INFO mapred.JobClient: map 100% reduce 81% 09/03/04 18:06:36 INFO mapred.JobClient: map 100% reduce 82% 09/03/04 18:06:43 INFO mapred.JobClient: map 100% reduce 83% 09/03/04 18:06:50 INFO mapred.JobClient: map 100% reduce 84% 09/03/04 18:06:55 INFO mapred.JobClient: map 100% reduce 85% 09/03/04 18:07:01 INFO mapred.JobClient: map 100% reduce 86% 09/03/04 18:07:07 INFO mapred.JobClient: map 100% reduce 87% 09/03/04 18:07:14 INFO mapred.JobClient: map 100% reduce 88% 09/03/04 18:07:20 INFO mapred.JobClient: map 100% reduce 89% 09/03/04 18:07:27 INFO mapred.JobClient: map 100% reduce 90% 09/03/04 18:07:34 INFO mapred.JobClient: map 100% reduce 91% 09/03/04 18:07:39 INFO mapred.JobClient: map 100% reduce 92% 09/03/04 18:07:45 INFO mapred.JobClient: map 100% reduce 93% 09/03/04 18:07:51 INFO mapred.JobClient: map 100% reduce 94% 09/03/04 18:07:58 INFO mapred.JobClient: map 100% reduce 95% 09/03/04 18:08:10 INFO mapred.JobClient: map 100% reduce 96% 09/03/04 18:08:16 INFO mapred.JobClient: map 100% reduce 97% 09/03/04 18:08:21 INFO mapred.JobClient: map 100% reduce 98% 09/03/04 18:08:27 INFO mapred.JobClient: map 100% reduce 99% 09/03/04 18:08:32 INFO mapred.JobClient: map 100% reduce 100% 09/03/04 18:08:38 INFO mapred.JobClient: Job complete: job_200902131649_0038 09/03/04 18:08:38 INFO mapred.JobClient: Counters: 15 09/03/04 18:08:38 INFO mapred.JobClient: File Systems 09/03/04 18:08:38 INFO mapred.JobClient: Local bytes read=1350551934 09/03/04 18:08:38 INFO mapred.JobClient: Local bytes written=2025705516 09/03/04 18:08:38 INFO mapred.JobClient: Job Counters 09/03/04 18:08:38 INFO mapred.JobClient: Launched reduce tasks=3 09/03/04 18:08:38 INFO mapred.JobClient: Rack-local map tasks=1 09/03/04 18:08:38 INFO mapred.JobClient: Launched map tasks=4 09/03/04 18:08:38 INFO mapred.JobClient: Data-local map tasks=3 09/03/04 18:08:38 INFO mapred.JobClient: Map-Reduce Framework 09/03/04 18:08:38 INFO mapred.JobClient: Reduce input groups=5000 09/03/04 18:08:38 INFO mapred.JobClient: Combine output records=0 09/03/04 18:08:38 INFO mapred.JobClient: Map input records=5000 09/03/04 18:08:38 INFO mapred.JobClient: Reduce output records=5000 09/03/04 18:08:38 INFO mapred.JobClient: Map output bytes=675210000 09/03/04 18:08:38 INFO mapred.JobClient: Map input bytes=0 09/03/04 18:08:38 INFO mapred.JobClient: Combine input records=0 09/03/04 18:08:38 INFO mapred.JobClient: Map output records=5000 09/03/04 18:08:38 INFO mapred.JobClient: Reduce input records=5000
        Hide
        Sijie Guo added a comment -

        Thanks Edward for the test

        Show
        Sijie Guo added a comment - Thanks Edward for the test
        Hide
        Edward J. Yoon added a comment -

        Attach applyable patch.

        Show
        Edward J. Yoon added a comment - Attach applyable patch.
        Hide
        Edward J. Yoon added a comment -

        Submit to hudson.

        Show
        Edward J. Yoon added a comment - Submit to hudson.
        Hide
        Hudson added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12401492/HAMA-154_v03.patch
        against trunk revision 750303.

        @author +1. The patch does not contain any @author tags.

        tests included +1. The patch appears to include 3 new or modified tests.

        javadoc +1. The javadoc tool did not generate any warning messages.

        javac +1. The applied patch does not generate any new javac compiler warnings.

        release audit +1. The applied patch does not generate any new release audit warnings.

        findbugs +1. The patch does not introduce any new Findbugs warnings.

        core tests +1. The patch passed core unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hama-Patch/163/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hama-Patch/163/artifact/trunk/build/reports/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hama-Patch/163/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hama-Patch/163/console

        This message is automatically generated.

        Show
        Hudson added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12401492/HAMA-154_v03.patch against trunk revision 750303. @author +1. The patch does not contain any @author tags. tests included +1. The patch appears to include 3 new or modified tests. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new javac compiler warnings. release audit +1. The applied patch does not generate any new release audit warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests +1. The patch passed core unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hama-Patch/163/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hama-Patch/163/artifact/trunk/build/reports/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hama-Patch/163/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hama-Patch/163/console This message is automatically generated.
        Hide
        Edward J. Yoon added a comment -

        I just committed this. Thanks Samuel!

        Show
        Edward J. Yoon added a comment - I just committed this. Thanks Samuel!

          People

          • Assignee:
            Unassigned
            Reporter:
            Sijie Guo
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 1h
              1h
              Remaining:
              Remaining Estimate - 1h
              1h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development