Solr
  1. Solr
  2. SOLR-5693

Running on HDFS does work correctly with NRT search.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.7, 6.0
    • Component/s: None
    • Labels:
      None

      Description

      Like NFS, HDFS has different file delete semantics than Windows and Unix. For non NRT cases, you can work around this by reserving commit points, but NRT counts on delete on last close semantics (unix), or delete fails, try again later (windows). This is because files can be merged away before they even become part of a commit point. Meanwhile, real time readers can reference those files.

        Issue Links

          Activity

          Hide
          Mark Miller added a comment -

          Symptom is that sometimes searches will fail with file not found exceptions. Depending on the velocity of the index, this will likely be intermittent.

          Show
          Mark Miller added a comment - Symptom is that sometimes searches will fail with file not found exceptions. Depending on the velocity of the index, this will likely be intermittent.
          Hide
          ASF subversion and git services added a comment -

          Commit 1565344 from Mark Miller in branch 'dev/trunk'
          [ https://svn.apache.org/r1565344 ]

          LUCENE-5434: NRT support for file systems that do no have delete on last close or cannot delete while referenced semantics.
          SOLR-5693: Running on HDFS does work correctly with NRT search.

          Show
          ASF subversion and git services added a comment - Commit 1565344 from Mark Miller in branch 'dev/trunk' [ https://svn.apache.org/r1565344 ] LUCENE-5434 : NRT support for file systems that do no have delete on last close or cannot delete while referenced semantics. SOLR-5693 : Running on HDFS does work correctly with NRT search.
          Hide
          ASF subversion and git services added a comment -

          Commit 1565347 from Mark Miller in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1565347 ]

          LUCENE-5434: NRT support for file systems that do no have delete on last close or cannot delete while referenced semantics.
          SOLR-5693: Running on HDFS does work correctly with NRT search.

          Show
          ASF subversion and git services added a comment - Commit 1565347 from Mark Miller in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1565347 ] LUCENE-5434 : NRT support for file systems that do no have delete on last close or cannot delete while referenced semantics. SOLR-5693 : Running on HDFS does work correctly with NRT search.

            People

            • Assignee:
              Mark Miller
              Reporter:
              Mark Miller
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development