Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9408

Add solr commit data in TreeMergeRecordWriter

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.2.1, 7.0
    • Component/s: contrib - MapReduce
    • Security Level: Public (Default Security Level. Issues are Public)
    • Flags:
      Patch

      Description

      The lucene index produced by TreeMergeRecordWriter when the segments are merged doesn't contain Solr's commit data, specifically, commitTimeMsec.

      This means that when this index is subsequently loaded into SolrCloud and if the index stays unchanged so no newer commits occurs, ADDREPLICA will appear to succeed but will not actually do any full sync due to SOLR-9369, resulting in adding an empty index as a replica.

      1. SOLR-9408.patch
        10 kB
        Varun Thacker
      2. SOLR-9408.patch
        6 kB
        Jessica Cheng Mallet
      3. SOLR-9408.patch
        6 kB
        Jessica Cheng Mallet

        Activity

        Hide
        varunthacker Varun Thacker added a comment -

        Patch looks good to me!

        I think we should commit this irrespective of SOLR-9369 .

        Mark Miller Do you have any thoughts on this?

        Show
        varunthacker Varun Thacker added a comment - Patch looks good to me! I think we should commit this irrespective of SOLR-9369 . Mark Miller Do you have any thoughts on this?
        Hide
        varunthacker Varun Thacker added a comment -

        I ran into this failure a couple of times in my testing but it doesn't fail all the time . I'll dig into it this week to see whats the issue

        ant test -Dtestcase=SoftAutoCommitTest -Dtests.method=testSoftAndHardCommitMaxTimeMixedAdds -Dtests.seed=F1E9CC578C23E178 -Dtests.slow=true -Dtests.locale=sr-Latn-RS -Dtests.timezone=America/Argentina/Salta -Dtests.asserts=true -Dtests.file.encoding=UTF-8

        Show
        varunthacker Varun Thacker added a comment - I ran into this failure a couple of times in my testing but it doesn't fail all the time . I'll dig into it this week to see whats the issue ant test -Dtestcase=SoftAutoCommitTest -Dtests.method=testSoftAndHardCommitMaxTimeMixedAdds -Dtests.seed=F1E9CC578C23E178 -Dtests.slow=true -Dtests.locale=sr-Latn-RS -Dtests.timezone=America/Argentina/Salta -Dtests.asserts=true -Dtests.file.encoding=UTF-8
        Hide
        shalinmangar Shalin Shekhar Mangar added a comment -

        This seems unrelated to the change here. I have seen this test fail on jenkins as well with the message "soft wasn't fast enough". I think we can safely ignore it. The rest of the patch looks good and we should commit it for 6.2.1

        Show
        shalinmangar Shalin Shekhar Mangar added a comment - This seems unrelated to the change here. I have seen this test fail on jenkins as well with the message "soft wasn't fast enough". I think we can safely ignore it. The rest of the patch looks good and we should commit it for 6.2.1
        Hide
        varunthacker Varun Thacker added a comment -

        Attaching updated patch against master. I'll commit this soon

        Show
        varunthacker Varun Thacker added a comment - Attaching updated patch against master. I'll commit this soon
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit ef3057e43b6c3783f1324b2893eeb8702c86487c in lucene-solr's branch refs/heads/master from Varun Thacker
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ef3057e ]

        SOLR-9408: Fix TreeMergeOutputFormat to add timestamp metadata to commits

        Show
        jira-bot ASF subversion and git services added a comment - Commit ef3057e43b6c3783f1324b2893eeb8702c86487c in lucene-solr's branch refs/heads/master from Varun Thacker [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ef3057e ] SOLR-9408 : Fix TreeMergeOutputFormat to add timestamp metadata to commits
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 08453fb7f000342352c6c08dcdf83cdbda1694c6 in lucene-solr's branch refs/heads/branch_6x from Varun Thacker
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=08453fb ]

        SOLR-9408: Fix TreeMergeOutputFormat to add timestamp metadata to commits

        Show
        jira-bot ASF subversion and git services added a comment - Commit 08453fb7f000342352c6c08dcdf83cdbda1694c6 in lucene-solr's branch refs/heads/branch_6x from Varun Thacker [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=08453fb ] SOLR-9408 : Fix TreeMergeOutputFormat to add timestamp metadata to commits
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 2335cf7cd52323c02041f28ebdbf7f8c5bb5bb4e in lucene-solr's branch refs/heads/branch_6_2 from Varun Thacker
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2335cf7 ]

        SOLR-9408: Fix TreeMergeOutputFormat to add timestamp metadata to commits

        Show
        jira-bot ASF subversion and git services added a comment - Commit 2335cf7cd52323c02041f28ebdbf7f8c5bb5bb4e in lucene-solr's branch refs/heads/branch_6_2 from Varun Thacker [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2335cf7 ] SOLR-9408 : Fix TreeMergeOutputFormat to add timestamp metadata to commits
        Hide
        varunthacker Varun Thacker added a comment -

        Thanks Jessica for the patch and Shalin for the review!

        Show
        varunthacker Varun Thacker added a comment - Thanks Jessica for the patch and Shalin for the review!
        Hide
        shalinmangar Shalin Shekhar Mangar added a comment -

        Closing after 6.2.1 release

        Show
        shalinmangar Shalin Shekhar Mangar added a comment - Closing after 6.2.1 release

          People

          • Assignee:
            varunthacker Varun Thacker
            Reporter:
            mewmewball Jessica Cheng Mallet
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development