Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-8546

TestLazyCores is failing a lot on the Jenkins cluster.

    Details

    • Type: Test
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.2, master (7.0)
    • Component/s: None
    • Labels:
      None

      Description

      Looks like two issues:

      • A thread leak due to 3DsearcherExecutor
      • An ObjectTracker fail because a SolrCore is left unclosed.
      1. SOLR-8546.patch
        1 kB
        Erick Erickson

        Activity

        Hide
        erickerickson Erick Erickson added a comment -

        1,500 runs with the beasting script and no failures. I'm reluctant to make changes here that I can't show, you know, actually make a difference, but I don't see much other choice except to make stabs at it from looking at Jenkins logs, check code in and then wait and see if they go away.

        Show
        erickerickson Erick Erickson added a comment - 1,500 runs with the beasting script and no failures. I'm reluctant to make changes here that I can't show, you know, actually make a difference, but I don't see much other choice except to make stabs at it from looking at Jenkins logs, check code in and then wait and see if they go away.
        Hide
        markrmiller@gmail.com Mark Miller added a comment -

        I'll see what I can find with some beasting tomorrow.

        Show
        markrmiller@gmail.com Mark Miller added a comment - I'll see what I can find with some beasting tomorrow.
        Hide
        erickerickson Erick Erickson added a comment -

        First, it's kind of weird that there are a bunch of errors that it couldn't parse the config file, don't quite know whether that's germane or not.

        My real question is how to specify the "reproduce with" in the beast.sh script I stole from you. Just define JAVA_OPTS as

        -Dtests.seed=373B7C68661497FF -Dtests.slow=true -Dtests.locale=de_AT -Dtests.timezone=Pacific/Norfolk -Dtests.asserts=true -Dtests.file.encoding=US-ASCII

        ?

        Show
        erickerickson Erick Erickson added a comment - First, it's kind of weird that there are a bunch of errors that it couldn't parse the config file, don't quite know whether that's germane or not. My real question is how to specify the "reproduce with" in the beast.sh script I stole from you. Just define JAVA_OPTS as -Dtests.seed=373B7C68661497FF -Dtests.slow=true -Dtests.locale=de_AT -Dtests.timezone=Pacific/Norfolk -Dtests.asserts=true -Dtests.file.encoding=US-ASCII ?
        Hide
        markrmiller@gmail.com Mark Miller added a comment -

        Just define JAVA_OPTS

        Yup.

        don't quite know whether that's germane or not.

        Doesn't seem related to the common jenkins fails.

        Show
        markrmiller@gmail.com Mark Miller added a comment - Just define JAVA_OPTS Yup. don't quite know whether that's germane or not. Doesn't seem related to the common jenkins fails.
        Hide
        erickerickson Erick Erickson added a comment -

        OK, 1,000 iterations beasting with the failing seed as above and... no failures. Siiiggggghhhh. I really, Really, REALLY hate things that don't reproduce.

        Show
        erickerickson Erick Erickson added a comment - OK, 1,000 iterations beasting with the failing seed as above and... no failures. Siiiggggghhhh. I really, Really, REALLY hate things that don't reproduce.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit bc1237a646066706a027ee42b975cf3aea82a37f in lucene-solr's branch refs/heads/master from Erick
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bc1237a ]

        SOLR-8546: TestLazyCores is failing a lot on the Jenkins cluster.

        Show
        jira-bot ASF subversion and git services added a comment - Commit bc1237a646066706a027ee42b975cf3aea82a37f in lucene-solr's branch refs/heads/master from Erick [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bc1237a ] SOLR-8546 : TestLazyCores is failing a lot on the Jenkins cluster.
        Hide
        erickerickson Erick Erickson added a comment -

        Oh my this is embarrassing, how long this has lingered. I finally got it to fail locally and...

        The good news is it's a test problem, not the code.
        The good news is that the test fix is trivial.

        The bad news is it's so stupid. There are two calls to random.nextint() that look like this
        int blah = random.nextInt(10000);
        some stuff
        int blort = random.nextInt(blah).

        Whenever blah == 0 it throws an error since blah must be positive.

        Checking this in, but I'll keep this JIRA open for a while to see if Jenkins is happy as well as beasting it a lot locally.

        Show
        erickerickson Erick Erickson added a comment - Oh my this is embarrassing, how long this has lingered. I finally got it to fail locally and... The good news is it's a test problem, not the code. The good news is that the test fix is trivial. The bad news is it's so stupid. There are two calls to random.nextint() that look like this int blah = random.nextInt(10000); some stuff int blort = random.nextInt(blah). Whenever blah == 0 it throws an error since blah must be positive. Checking this in, but I'll keep this JIRA open for a while to see if Jenkins is happy as well as beasting it a lot locally.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit caec6b40fd4645cc9184085c1c8742e94726ee72 in lucene-solr's branch refs/heads/branch_6x from Erick
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=caec6b4 ]

        SOLR-8546: TestLazyCores is failing a lot on the Jenkins cluster.

        Show
        jira-bot ASF subversion and git services added a comment - Commit caec6b40fd4645cc9184085c1c8742e94726ee72 in lucene-solr's branch refs/heads/branch_6x from Erick [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=caec6b4 ] SOLR-8546 : TestLazyCores is failing a lot on the Jenkins cluster.
        Hide
        erickerickson Erick Erickson added a comment -

        No failures for since the last patch so closling.

        Show
        erickerickson Erick Erickson added a comment - No failures for since the last patch so closling.
        Hide
        mikemccand Michael McCandless added a comment -

        Bulk close resolved issues after 6.2.0 release.

        Show
        mikemccand Michael McCandless added a comment - Bulk close resolved issues after 6.2.0 release.

          People

          • Assignee:
            erickerickson Erick Erickson
            Reporter:
            markrmiller@gmail.com Mark Miller
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development