Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-637

Check in the codes that compute the 10^15+1st bit of π

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: examples
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We have an improved version of the current BaileyBorwinPlouffe example, which able to compute at least the10^15+1st bit of π. See also http://developer.yahoo.net/blogs/hadoop/2009/05/hadoop_computes_the_10151st_bi.html

      1. D3611.tar.gz
        1.62 MB
        Tsz Wo Nicholas Sze
      2. m637_20090630.patch
        115 kB
        Tsz Wo Nicholas Sze
      3. m637_20090624.patch
        113 kB
        Tsz Wo Nicholas Sze
      4. 5819_20090527.patch
        110 kB
        Tsz Wo Nicholas Sze
      5. 5819_20090523.patch
        105 kB
        Tsz Wo Nicholas Sze
      6. 1e15log.zip
        549 kB
        Tsz Wo Nicholas Sze
      7. 5819_20090515.patch
        97 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Hide
          Tsz Wo Nicholas Sze added a comment -

          5819_20090515.patch: a preview patch which

          • needs to be cleaned up;
          • needs more code comments; and
          • is not user friendly.

          But it has computed the 10^15+1st bit of π.

          Show
          Tsz Wo Nicholas Sze added a comment - 5819_20090515.patch: a preview patch which needs to be cleaned up; needs more code comments; and is not user friendly. But it has computed the 10^15+1st bit of π.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          1e15.zip: log files for the computation of the 10^15+1st bit of π.

          Show
          Tsz Wo Nicholas Sze added a comment - 1e15.zip: log files for the computation of the 10^15+1st bit of π.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Deleted 1e15.zip. The logs need to be cleaned up.

          Show
          Tsz Wo Nicholas Sze added a comment - Deleted 1e15.zip. The logs need to be cleaned up.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          1e15log.zip: log files for the 1e15 computation.

          Show
          Tsz Wo Nicholas Sze added a comment - 1e15log.zip: log files for the 1e15 computation.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Computed the following table for the bits of π:

            Position π bits (in hex)
          0 1 243F6A8885A3
          1 11 FDAA22168C23
          2 101 3707344A409
          3 1,001 574E69A458F
          4 10,001 44EC5716F2B
          5 100,001 944F7A204
          6 1,000,001 6FFFA4103
          7 10,000,001 6CFDD54E3
          8 100,000,001 A306CFA7
          9 1,000,000,001 3E08FF2B
          10 10,000,000,001 0A8BD8C0
          11 100,000,000,001 B2238C1
          12 1,000,000,000,001 0FEE563
          13 10,000,000,000,001 896DC3
          14 100,000,000,000,001 C216EC
          15 1,000,000,000,000,001 6216B06
          • For each position n, except the first and the last, also computed π bits at position n-4 and n+4 for correctness checking.
          • Row 0 to Row 7 were run in a single computer. A single run of Row 7 took several seconds.
          • Row 8 to Row 14 were run in a 7600-cpu-core cluster. A single run of Row 14 took 27 hours.
          Show
          Tsz Wo Nicholas Sze added a comment - Computed the following table for the bits of π:   Position π bits (in hex) 0 1 243F6A8885A3 1 11 FDAA22168C23 2 101 3707344A409 3 1,001 574E69A458F 4 10,001 44EC5716F2B 5 100,001 944F7A204 6 1,000,001 6FFFA4103 7 10,000,001 6CFDD54E3 8 100,000,001 A306CFA7 9 1,000,000,001 3E08FF2B 10 10,000,000,001 0A8BD8C0 11 100,000,000,001 B2238C1 12 1,000,000,000,001 0FEE563 13 10,000,000,000,001 896DC3 14 100,000,000,000,001 C216EC 15 1,000,000,000,000,001 6216B06 For each position n, except the first and the last, also computed π bits at position n-4 and n+4 for correctness checking. Row 0 to Row 7 were run in a single computer. A single run of Row 7 took several seconds. Row 8 to Row 14 were run in a 7600-cpu-core cluster. A single run of Row 14 took 27 hours. Row 15 is the result reported in the YDN blog .
          Hide
          Tsz Wo Nicholas Sze added a comment -

          5819_20090523.patch:

          • cleaned up codes
          • added some comments
          Show
          Tsz Wo Nicholas Sze added a comment - 5819_20090523.patch: cleaned up codes added some comments
          Hide
          Tsz Wo Nicholas Sze added a comment -

          5819_20090527.patch: this patch is ready to be reviewed.

          Show
          Tsz Wo Nicholas Sze added a comment - 5819_20090527.patch: this patch is ready to be reviewed.
          Hide
          Chris Douglas added a comment -

          Brilliant. +1

          It doesn't need to delay committing this, but a document describing some of the strategies employed would help others navigate the code. The comments help, but it's hard to pick salient, general details out of the broader work (e.g. use of ClusterStatus in MixMachine, how results are combined, etc.).

          Show
          Chris Douglas added a comment - Brilliant. +1 It doesn't need to delay committing this, but a document describing some of the strategies employed would help others navigate the code. The comments help, but it's hard to pick salient, general details out of the broader work (e.g. use of ClusterStatus in MixMachine, how results are combined, etc.).
          Hide
          Tsz Wo Nicholas Sze added a comment -

          m637_20090624.patch: add some comments in DistBbp and DistSum headers to give better pictures of the programs.

          Show
          Tsz Wo Nicholas Sze added a comment - m637_20090624.patch: add some comments in DistBbp and DistSum headers to give better pictures of the programs.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          m637_20090630.patch: fix some findbugs warnings.

          Show
          Tsz Wo Nicholas Sze added a comment - m637_20090630.patch: fix some findbugs warnings.
          Hide
          Tsz Wo Nicholas Sze added a comment -
               [exec] +1 overall.  
               [exec] 
               [exec]     +1 @author.  The patch does not contain any @author tags.
               [exec] 
               [exec]     +1 tests included.  The patch appears to include 9 new or modified tests.
               [exec] 
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
               [exec] 
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
               [exec] 
               [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
               [exec] 
               [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
          
          Show
          Tsz Wo Nicholas Sze added a comment - [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 9 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          The last row of the table should be updated to

          15 1,000,000,000,000,001 6216B06 ... D3611
          • The second part (D3611)
            • Starting at position 1,000,000,000,000,053.
            • Totally 20 bits.
            • The last bit, the 1,000,000,000,000,072nd bit, probably is the highest bits computed ever in the history.
            • Computed by totally 14,000 jobs, 7,000,000 tasks, 208 years of cpu-time and 12 days of cluster time.
          Show
          Tsz Wo Nicholas Sze added a comment - The last row of the table should be updated to 15 1,000,000,000,000,001 6216B06 ... D3611 The second part (D3611) Starting at position 1,000,000,000,000,053. Totally 20 bits. The last bit, the 1,000,000,000,000,072nd bit, probably is the highest bits computed ever in the history. Computed by totally 14,000 jobs, 7,000,000 tasks, 208 years of cpu-time and 12 days of cluster time.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this.

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          D3611.zip: logs for the D3611 computation.

          Show
          Tsz Wo Nicholas Sze added a comment - D3611.zip: logs for the D3611 computation.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          D3611.tar.gz: same as D3611.zip but use .tar.gz for better compression ratio. D3611.zip was deleted.

          Show
          Tsz Wo Nicholas Sze added a comment - D3611.tar.gz: same as D3611.zip but use .tar.gz for better compression ratio. D3611.zip was deleted.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Will add more documentation in MAPREDUCE-691.

          Show
          Tsz Wo Nicholas Sze added a comment - Will add more documentation in MAPREDUCE-691 .
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #15 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/15/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #15 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/15/ )
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Just have posted some new pi computation results in MAPREDUCE-1923.

          Show
          Tsz Wo Nicholas Sze added a comment - Just have posted some new pi computation results in MAPREDUCE-1923 .

            People

            • Assignee:
              Tsz Wo Nicholas Sze
              Reporter:
              Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development