Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3683

Edit log replay progress indicator shows >100% complete

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: namenode
    • Labels:
      None

      Description

      When loading a large edit log, the NameNode prints a percentage-complete progress indicator:

                    int percent = Math.round((float)lastAppliedTxId / numTxns * 100);
                    LOG.info("replaying edit log: " + lastAppliedTxId + "/" + numTxns
                        + " transactions completed. (" + percent + "%)");
      

      But the percentage is calculated incorrectly, since it divides the transaction ID by the number of expected transactions. This only works when starting at txid 1. (eg if we are loading 1000 transaction starting at txid 1000, the indicator will start at 100% and go to 200%)

      1. HDFS_3683.patch
        1.0 kB
        Plamen Jeliazkov
      2. HDFS_3683.patch
        1 kB
        Plamen Jeliazkov
      3. HDFS_3683.patch
        1 kB
        Plamen Jeliazkov

        Activity

        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Plamen Jeliazkov added a comment -

        Thank you Aaron for the catches and the help.

        Show
        Plamen Jeliazkov added a comment - Thank you Aaron for the catches and the help.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1177 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1177/)
        HDFS-3683. Edit log replay progress indicator shows >100% complete. Contributed by Plamen Jeliazkov. (Revision 1377367)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377367
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1177 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1177/ ) HDFS-3683 . Edit log replay progress indicator shows >100% complete. Contributed by Plamen Jeliazkov. (Revision 1377367) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377367 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1146 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1146/)
        HDFS-3683. Edit log replay progress indicator shows >100% complete. Contributed by Plamen Jeliazkov. (Revision 1377367)

        Result = FAILURE
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377367
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1146 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1146/ ) HDFS-3683 . Edit log replay progress indicator shows >100% complete. Contributed by Plamen Jeliazkov. (Revision 1377367) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377367 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2669 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2669/)
        HDFS-3683. Edit log replay progress indicator shows >100% complete. Contributed by Plamen Jeliazkov. (Revision 1377367)

        Result = FAILURE
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377367
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2669 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2669/ ) HDFS-3683 . Edit log replay progress indicator shows >100% complete. Contributed by Plamen Jeliazkov. (Revision 1377367) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377367 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2705 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2705/)
        HDFS-3683. Edit log replay progress indicator shows >100% complete. Contributed by Plamen Jeliazkov. (Revision 1377367)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377367
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2705 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2705/ ) HDFS-3683 . Edit log replay progress indicator shows >100% complete. Contributed by Plamen Jeliazkov. (Revision 1377367) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377367 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2641 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2641/)
        HDFS-3683. Edit log replay progress indicator shows >100% complete. Contributed by Plamen Jeliazkov. (Revision 1377367)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377367
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2641 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2641/ ) HDFS-3683 . Edit log replay progress indicator shows >100% complete. Contributed by Plamen Jeliazkov. (Revision 1377367) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1377367 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
        Aaron T. Myers made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 2.2.0-alpha [ 12322472 ]
        Resolution Fixed [ 1 ]
        Hide
        Aaron T. Myers added a comment -

        I've just committed this to trunk and branch-2. Thanks a lot for the contribution, Plamen.

        Show
        Aaron T. Myers added a comment - I've just committed this to trunk and branch-2. Thanks a lot for the contribution, Plamen.
        Hide
        Aaron T. Myers added a comment -

        The latest patch looks good to me. I tested it in the same manner I did yesterday, and got this output:

        12/08/25 17:28:11 INFO namenode.FSEditLogLoader: replaying edit log: 1/9 transactions completed. (11%)
        12/08/25 17:28:12 INFO namenode.FSEditLogLoader: replaying edit log: 2/9 transactions completed. (22%)
        12/08/25 17:28:13 INFO namenode.FSEditLogLoader: replaying edit log: 3/9 transactions completed. (33%)
        12/08/25 17:28:15 INFO namenode.FSEditLogLoader: replaying edit log: 5/9 transactions completed. (56%)
        12/08/25 17:28:16 INFO namenode.FSEditLogLoader: replaying edit log: 6/9 transactions completed. (67%)
        12/08/25 17:28:17 INFO namenode.FSEditLogLoader: replaying edit log: 7/9 transactions completed. (78%)
        12/08/25 17:28:18 INFO namenode.FSEditLogLoader: replaying edit log: 8/9 transactions completed. (89%)
        12/08/25 17:28:19 INFO namenode.FSEditLogLoader: replaying edit log: 9/9 transactions completed. (100%)
        

        +1, I'll commit this momentarily.

        Show
        Aaron T. Myers added a comment - The latest patch looks good to me. I tested it in the same manner I did yesterday, and got this output: 12/08/25 17:28:11 INFO namenode.FSEditLogLoader: replaying edit log: 1/9 transactions completed. (11%) 12/08/25 17:28:12 INFO namenode.FSEditLogLoader: replaying edit log: 2/9 transactions completed. (22%) 12/08/25 17:28:13 INFO namenode.FSEditLogLoader: replaying edit log: 3/9 transactions completed. (33%) 12/08/25 17:28:15 INFO namenode.FSEditLogLoader: replaying edit log: 5/9 transactions completed. (56%) 12/08/25 17:28:16 INFO namenode.FSEditLogLoader: replaying edit log: 6/9 transactions completed. (67%) 12/08/25 17:28:17 INFO namenode.FSEditLogLoader: replaying edit log: 7/9 transactions completed. (78%) 12/08/25 17:28:18 INFO namenode.FSEditLogLoader: replaying edit log: 8/9 transactions completed. (89%) 12/08/25 17:28:19 INFO namenode.FSEditLogLoader: replaying edit log: 9/9 transactions completed. (100%) +1, I'll commit this momentarily.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12542421/HDFS_3683.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.TestHftpDelegationToken

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3097//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/3097//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3097//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542421/HDFS_3683.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestHftpDelegationToken +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3097//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/3097//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3097//console This message is automatically generated.
        Plamen Jeliazkov made changes -
        Attachment HDFS_3683.patch [ 12542421 ]
        Hide
        Plamen Jeliazkov added a comment -

        And here is the patch.

        Show
        Plamen Jeliazkov added a comment - And here is the patch.
        Hide
        Plamen Jeliazkov added a comment -

        I think there is also a subtle "off by one" error in my use. This new patch should account for it. Basically "numOfTxns" is LAST_TXID - START_TXID + 1, so to get X/N = 1, X should be LAST_APPLIED_TXID - START_TXID + 1.

        Show
        Plamen Jeliazkov added a comment - I think there is also a subtle "off by one" error in my use. This new patch should account for it. Basically "numOfTxns" is LAST_TXID - START_TXID + 1, so to get X/N = 1, X should be LAST_APPLIED_TXID - START_TXID + 1.
        Hide
        Aaron T. Myers added a comment -

        Yes, that was my thinking.

        Show
        Aaron T. Myers added a comment - Yes, that was my thinking.
        Hide
        Plamen Jeliazkov added a comment -

        Oh, haha. Yes I see the issue now. That same fix for the percent is supposed to also be applied for the "X/N transactions completed" part so that it shows up as 1/22 ... 12/22 .. 22/22, correct? I will post up another patch to fix that. Thanks for the catch.

        Show
        Plamen Jeliazkov added a comment - Oh, haha. Yes I see the issue now. That same fix for the percent is supposed to also be applied for the "X/N transactions completed" part so that it shows up as 1/22 ... 12/22 .. 22/22, correct? I will post up another patch to fix that. Thanks for the catch.
        Hide
        Aaron T. Myers added a comment -

        Hi Plamen, though this patch does indeed correct the percentage in the log message, it doesn't fix the other portion, which is supposed to show the raw number of transactions loaded. I tested this by putting a one second sleep per transaction in the edit log loading code, and then restarting an NN with a few edits. This displays the following:

        12/08/24 17:20:58 INFO namenode.FSEditLogLoader: replaying edit log: 2673/22 transactions completed. (0%)
        12/08/24 17:20:59 INFO namenode.FSEditLogLoader: replaying edit log: 2674/22 transactions completed. (5%)
        12/08/24 17:21:00 INFO namenode.FSEditLogLoader: replaying edit log: 2675/22 transactions completed. (9%)
        12/08/24 17:21:01 INFO namenode.FSEditLogLoader: replaying edit log: 2676/22 transactions completed. (14%)
        12/08/24 17:21:03 INFO namenode.FSEditLogLoader: replaying edit log: 2678/22 transactions completed. (23%)
        12/08/24 17:21:05 INFO namenode.FSEditLogLoader: replaying edit log: 2680/22 transactions completed. (32%)
        12/08/24 17:21:07 INFO namenode.FSEditLogLoader: replaying edit log: 2682/22 transactions completed. (41%)
        12/08/24 17:21:09 INFO namenode.FSEditLogLoader: replaying edit log: 2684/22 transactions completed. (50%)
        12/08/24 17:21:11 INFO namenode.FSEditLogLoader: replaying edit log: 2686/22 transactions completed. (59%)
        12/08/24 17:21:13 INFO namenode.FSEditLogLoader: replaying edit log: 2688/22 transactions completed. (68%)
        12/08/24 17:21:15 INFO namenode.FSEditLogLoader: replaying edit log: 2690/22 transactions completed. (77%)
        12/08/24 17:21:17 INFO namenode.FSEditLogLoader: replaying edit log: 2692/22 transactions completed. (86%)
        12/08/24 17:21:19 INFO namenode.FSEditLogLoader: replaying edit log: 2694/22 transactions completed. (95%)
        
        Show
        Aaron T. Myers added a comment - Hi Plamen, though this patch does indeed correct the percentage in the log message, it doesn't fix the other portion, which is supposed to show the raw number of transactions loaded. I tested this by putting a one second sleep per transaction in the edit log loading code, and then restarting an NN with a few edits. This displays the following: 12/08/24 17:20:58 INFO namenode.FSEditLogLoader: replaying edit log: 2673/22 transactions completed. (0%) 12/08/24 17:20:59 INFO namenode.FSEditLogLoader: replaying edit log: 2674/22 transactions completed. (5%) 12/08/24 17:21:00 INFO namenode.FSEditLogLoader: replaying edit log: 2675/22 transactions completed. (9%) 12/08/24 17:21:01 INFO namenode.FSEditLogLoader: replaying edit log: 2676/22 transactions completed. (14%) 12/08/24 17:21:03 INFO namenode.FSEditLogLoader: replaying edit log: 2678/22 transactions completed. (23%) 12/08/24 17:21:05 INFO namenode.FSEditLogLoader: replaying edit log: 2680/22 transactions completed. (32%) 12/08/24 17:21:07 INFO namenode.FSEditLogLoader: replaying edit log: 2682/22 transactions completed. (41%) 12/08/24 17:21:09 INFO namenode.FSEditLogLoader: replaying edit log: 2684/22 transactions completed. (50%) 12/08/24 17:21:11 INFO namenode.FSEditLogLoader: replaying edit log: 2686/22 transactions completed. (59%) 12/08/24 17:21:13 INFO namenode.FSEditLogLoader: replaying edit log: 2688/22 transactions completed. (68%) 12/08/24 17:21:15 INFO namenode.FSEditLogLoader: replaying edit log: 2690/22 transactions completed. (77%) 12/08/24 17:21:17 INFO namenode.FSEditLogLoader: replaying edit log: 2692/22 transactions completed. (86%) 12/08/24 17:21:19 INFO namenode.FSEditLogLoader: replaying edit log: 2694/22 transactions completed. (95%)
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12542225/HDFS_3683.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.TestDatanodeBlockScanner
        org.apache.hadoop.hdfs.TestHftpDelegationToken
        org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3085//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/3085//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3085//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542225/HDFS_3683.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestDatanodeBlockScanner org.apache.hadoop.hdfs.TestHftpDelegationToken org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3085//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/3085//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3085//console This message is automatically generated.
        Hide
        Plamen Jeliazkov added a comment -

        Aaron, I checked all of the Test*-output.txt files in the surefire-reports that I called the loadEditRecords() method and in each one I spotted either 0% or 100%.

        Show
        Plamen Jeliazkov added a comment - Aaron, I checked all of the Test*-output.txt files in the surefire-reports that I called the loadEditRecords() method and in each one I spotted either 0% or 100%.
        Plamen Jeliazkov made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Plamen Jeliazkov made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Plamen Jeliazkov made changes -
        Attachment HDFS_3683.patch [ 12542225 ]
        Hide
        Plamen Jeliazkov added a comment -

        Patch with appropriate index.

        Show
        Plamen Jeliazkov added a comment - Patch with appropriate index.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12542195/HDFS_3683.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3082//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542195/HDFS_3683.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3082//console This message is automatically generated.
        Aaron T. Myers made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Target Version/s 2.1.0-alpha, 3.0.0 [ 12321440, 12320356 ] 2.2.0-alpha [ 12322472 ]
        Hide
        Aaron T. Myers added a comment -

        Marking patch available so that Jenkins runs.

        Show
        Aaron T. Myers added a comment - Marking patch available so that Jenkins runs.
        Aaron T. Myers made changes -
        Assignee Linden Hillenbrand [ lhillenbrand ] Plamen Jeliazkov [ zero45 ]
        Hide
        Aaron T. Myers added a comment -

        The patch looks pretty good to me. Plamen, can you comment on what testing you did of this patch? Were you able to confirm that the log message appears correct now when loading an edit log whose first transaction is not 1?

        Show
        Aaron T. Myers added a comment - The patch looks pretty good to me. Plamen, can you comment on what testing you did of this patch? Were you able to confirm that the log message appears correct now when loading an edit log whose first transaction is not 1?
        Hide
        Plamen Jeliazkov added a comment -

        Sorry thats supposed to be (LATEST_ID - STARTING_ID)...

        Show
        Plamen Jeliazkov added a comment - Sorry thats supposed to be (LATEST_ID - STARTING_ID)...
        Plamen Jeliazkov made changes -
        Attachment HDFS_3683.patch [ 12542195 ]
        Hide
        Plamen Jeliazkov added a comment -

        If we do (STARTING_ID - LATEST_ID) / (# of Transactions), we should accurately get the percentage assuming TxIds are in increasing order and expectedStartingTxId is the correct starting TxId.

        Let me know how it looks.

        Show
        Plamen Jeliazkov added a comment - If we do (STARTING_ID - LATEST_ID) / (# of Transactions), we should accurately get the percentage assuming TxIds are in increasing order and expectedStartingTxId is the correct starting TxId. Let me know how it looks.
        Linden Hillenbrand made changes -
        Field Original Value New Value
        Assignee Linden Hillenbrand [ lhillenbrand ]
        Todd Lipcon created issue -

          People

          • Assignee:
            Plamen Jeliazkov
            Reporter:
            Todd Lipcon
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development