Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-5150

HdfsIndexInput may not fully read requested bytes.

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 4.4
    • 4.5, 6.0
    • None
    • None

    Description

      Patrick Hunt noticed that our HdfsDirectory code was a bit behind Blur here - the read call we are using may not read all of the requested bytes - it returns the number of bytes actually written - which we ignore.

      Blur moved to using a seek and then readFully call - synchronizing across the two calls to deal with clones.

      We have seen that really kills performance, and using the readFully call that lets you pass the position rather than first doing a seek, performs much better and does not require the synchronization.

      I also noticed that the seekInternal impl should not seek but be a no op since we are seeking on the read.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            markrmiller@gmail.com Mark Miller
            markrmiller@gmail.com Mark Miller
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment