Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-15702

Stabilize S3 Tests - Fix S3Repository to follow BackupRepository.createDirectory API contract

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 8.10
    • 8.11
    • None

    Description

      Currently all 3 tests in the S3IncrementalBackupTest fail sporadically (between 1-5% of the time on the jenkins instances).

      From my investigations, these failures mainly happen because S3Repository will create a directory node, and then assume it exists later on. However, the S3Mock may be too slow and later on when the assumption is made that this directory node exists, S3Mock is returning that the directory node does not exist.

      The first fix for this was to just always check twice to see if a node is there, kind of hacky but it gives the S3Repository one more chance to find the right information.

      However in the v2 S3 API, there is a concept of waiters, which will allow us to wait until S3 verifies the state we are looking for (i.e. that the directory node exists). We can either put this waiter in after creating the node, and not return until the waiter says the node is created. Or we can put it in when checking whether the node exists. I think the former is preferable, but we can do testing to see which actually preforms better.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            houston Houston Putman
            houston Houston Putman
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 20m
                20m

                Slack

                  Issue deployment