Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.1, 7.0
    • Component/s: None
    • Labels:
      None

      Description

      I want to migrate the following classes to SolrCloudTestCase

      • DistribJoinFromCollectionTest
      • TestSubQueryTransformerDistrib
      • BlockJoinFacetDistribTest
      1. SOLR-9110.patch
        21 kB
        Mikhail Khludnev
      2. SOLR-9110.patch
        20 kB
        Mikhail Khludnev
      3. SOLR-9110.patch
        20 kB
        Mikhail Khludnev
      4. SOLR-9110.patch
        13 kB
        Mikhail Khludnev
      5. SOLR-9110.patch
        5 kB
        Mikhail Khludnev
      6. SOLR-9110-text-fix.patch
        7 kB
        Mikhail Khludnev

        Activity

        Hide
        mkhludnev Mikhail Khludnev added a comment -

        Ok. got the first breakthrough after all. I migrated SubQueries test.
        Alan Woodward, you might be interested in noob's feedback.

        Copying config folder under the given name to Zookeeper, is not obvious at all:

            String configName = "solrCloudCollectionConfig";
            int nodeCount = 5;
            configureCluster(nodeCount)
               .addConfig(configName, configDir)
               .configure();
        

        if it's possible, I ask to simplify it if possible, perhaps more descriptive names or perhaps implicit behavior, or so.
        Also, I experienced a typical leg-shooting: the test had solrconfig-basic.xml w/o update log. Somewhere deep in log it was reported somehow like ERROR ... RecoveryStrategy No UpdateLog found cannot recover. It's not easier to figure out the reason looking at log tail. So, ideally there should be a circuit-breaker preventing cloud launch without updatelogs.
        These two thoughts is just FYI, not really need to act.

        Show
        mkhludnev Mikhail Khludnev added a comment - Ok. got the first breakthrough after all. I migrated SubQueries test. Alan Woodward , you might be interested in noob's feedback. Copying config folder under the given name to Zookeeper, is not obvious at all: String configName = "solrCloudCollectionConfig" ; int nodeCount = 5; configureCluster(nodeCount) .addConfig(configName, configDir) .configure(); if it's possible, I ask to simplify it if possible, perhaps more descriptive names or perhaps implicit behavior, or so. Also, I experienced a typical leg-shooting: the test had solrconfig-basic.xml w/o update log. Somewhere deep in log it was reported somehow like ERROR ... RecoveryStrategy No UpdateLog found cannot recover . It's not easier to figure out the reason looking at log tail. So, ideally there should be a circuit-breaker preventing cloud launch without updatelogs. These two thoughts is just FYI, not really need to act.
        Hide
        romseygeek Alan Woodward added a comment -

        Copying config folder

        Agreed, this is a bit of a pain at the moment. You need to do something like {{{TEST_PATH().resolve("configsets").resolve(name).resolve("conf") - I'll add a method to SolrCloudTestCase to abstract this away.

        So, ideally there should be a circuit-breaker preventing cloud launch without updatelogs

        This would be a nice to have, yes! There are a few things that are required, both in solrconfig and schema, for cloud to work, and it would be good to have some kind of checking at core-load time - do you want to open a separate JIRA for that?

        Show
        romseygeek Alan Woodward added a comment - Copying config folder Agreed, this is a bit of a pain at the moment. You need to do something like {{{TEST_PATH().resolve("configsets").resolve(name).resolve("conf") - I'll add a method to SolrCloudTestCase to abstract this away. So, ideally there should be a circuit-breaker preventing cloud launch without updatelogs This would be a nice to have, yes! There are a few things that are required, both in solrconfig and schema, for cloud to work, and it would be good to have some kind of checking at core-load time - do you want to open a separate JIRA for that?
        Hide
        mkhludnev Mikhail Khludnev added a comment -

        spawned SOLR-9130

        Show
        mkhludnev Mikhail Khludnev added a comment - spawned SOLR-9130
        Hide
        mkhludnev Mikhail Khludnev added a comment -

        Alan Woodward, is it worth to migrate non-Zk test {{BlockJoinFacetDistribTest extends BaseDistributedSearchTestCase }} ?

        Show
        mkhludnev Mikhail Khludnev added a comment - Alan Woodward , is it worth to migrate non-Zk test {{BlockJoinFacetDistribTest extends BaseDistributedSearchTestCase }} ?
        Hide
        romseygeek Alan Woodward added a comment -

        The main reason I have for migrating the tests is to get off the legacy cloud formats, which don't apply to anything built on BaseDistributedSearchTestCase, so it's up to you really. You may find the test runs faster, as MiniSolrCloudCluster starts up its cores in parallel.

        Show
        romseygeek Alan Woodward added a comment - The main reason I have for migrating the tests is to get off the legacy cloud formats, which don't apply to anything built on BaseDistributedSearchTestCase, so it's up to you really. You may find the test runs faster, as MiniSolrCloudCluster starts up its cores in parallel.
        Hide
        mkhludnev Mikhail Khludnev added a comment -

        moved DistribJoinFromCollectionTest.
        One detail:

        • collection "to" is 2 shards x 2 replicas
        • collection "from" is 1 shard x 4 replicas, not fewer.
          Timothy Potter, do you have any concerns with this amendments for SOLR-4905?
        Show
        mkhludnev Mikhail Khludnev added a comment - moved DistribJoinFromCollectionTest. One detail: collection "to" is 2 shards x 2 replicas collection "from" is 1 shard x 4 replicas, not fewer. Timothy Potter , do you have any concerns with this amendments for SOLR-4905 ?
        Hide
        mkhludnev Mikhail Khludnev added a comment -

        migrated the 3rd test.
        One question: is it really necessary to mention tests only commit at CHANGES.txt ?

        Show
        mkhludnev Mikhail Khludnev added a comment - migrated the 3rd test. One question: is it really necessary to mention tests only commit at CHANGES.txt ?
        Hide
        steve_rowe Steve Rowe added a comment -

        One question: is it really necessary to mention tests only commit at CHANGES.txt ?

        It isn't necessary, but it can be quite helpful, since CHANGES.txt is often used as a way of understanding the project. For example, for the 6.0.1 release, I'm using CHANGES.txt to decide which issues to backport.

        If we're more careful in the future with JIRA fix version setting, it's possible that CHANGES.txt could be auto-generated from JIRA - I know some projects do that, and it's been discussed in the past for Lucene/Solr.

        Show
        steve_rowe Steve Rowe added a comment - One question: is it really necessary to mention tests only commit at CHANGES.txt ? It isn't necessary, but it can be quite helpful, since CHANGES.txt is often used as a way of understanding the project. For example, for the 6.0.1 release, I'm using CHANGES.txt to decide which issues to backport. If we're more careful in the future with JIRA fix version setting, it's possible that CHANGES.txt could be auto-generated from JIRA - I know some projects do that, and it's been discussed in the past for Lucene/Solr.
        Hide
        mkhludnev Mikhail Khludnev added a comment -

        it seems ready. But yesterday I've got non-reproducible failure on BlockJoinFacet test. Will check some more time.

        Show
        mkhludnev Mikhail Khludnev added a comment - it seems ready. But yesterday I've got non-reproducible failure on BlockJoinFacet test. Will check some more time.
        Hide
        mkhludnev Mikhail Khludnev added a comment -

        fixed failures by clearing indexes. I'm going to commit it quite soon.

        Show
        mkhludnev Mikhail Khludnev added a comment - fixed failures by clearing indexes. I'm going to commit it quite soon.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit ffa2d65efe5ed2eea04cfe60a6d97828c98da7b0 in lucene-solr's branch refs/heads/master from Mikhail Khludnev
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ffa2d65 ]

        SOLR-9110: moving JoinFromCollection- SubQueryTransformer- BlockJoinFacet- Distrib Tests to SolrCloudTestCase

        Show
        jira-bot ASF subversion and git services added a comment - Commit ffa2d65efe5ed2eea04cfe60a6d97828c98da7b0 in lucene-solr's branch refs/heads/master from Mikhail Khludnev [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ffa2d65 ] SOLR-9110 : moving JoinFromCollection- SubQueryTransformer- BlockJoinFacet- Distrib Tests to SolrCloudTestCase
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit aec3654fb8e6530494d909d57dada19642db1128 in lucene-solr's branch refs/heads/branch_6x from Mikhail Khludnev
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=aec3654 ]

        SOLR-9110: moving JoinFromCollection- SubQueryTransformer- BlockJoinFacet- Distrib Tests to SolrCloudTestCase

        Show
        jira-bot ASF subversion and git services added a comment - Commit aec3654fb8e6530494d909d57dada19642db1128 in lucene-solr's branch refs/heads/branch_6x from Mikhail Khludnev [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=aec3654 ] SOLR-9110 : moving JoinFromCollection- SubQueryTransformer- BlockJoinFacet- Distrib Tests to SolrCloudTestCase
        Hide
        mkhludnev Mikhail Khludnev added a comment -

        attaching test fix: removing static refs.

        Show
        mkhludnev Mikhail Khludnev added a comment - attaching test fix: removing static refs.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 2391b5b81dfade413d70d26da63e85cbd7245bf2 in lucene-solr's branch refs/heads/master from Mikhail Khludnev
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2391b5b ]

        SOLR-9110: removing static members from tests.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 2391b5b81dfade413d70d26da63e85cbd7245bf2 in lucene-solr's branch refs/heads/master from Mikhail Khludnev [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2391b5b ] SOLR-9110 : removing static members from tests.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 9863eea256c649dd035103128f95410bd85b8771 in lucene-solr's branch refs/heads/branch_6x from Mikhail Khludnev
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9863eea ]

        SOLR-9110: removing static members from tests.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 9863eea256c649dd035103128f95410bd85b8771 in lucene-solr's branch refs/heads/branch_6x from Mikhail Khludnev [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9863eea ] SOLR-9110 : removing static members from tests.

          People

          • Assignee:
            mkhludnev Mikhail Khludnev
            Reporter:
            mkhludnev Mikhail Khludnev
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development