Solr
  1. Solr
  2. SOLR-4751

The replication problem of the file in a subdirectory.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 4.2.1
    • Fix Version/s: 4.3.1, 4.4, 5.0
    • Component/s: replication (java)
    • Labels:
      None

      Description

      When set lang/stopwords_ja.txt to confFiles in replication settings,

        <requestHandler name="/replication" class="solr.ReplicationHandler" >
             <lst name="master">
               <str name="replicateAfter">commit</str>
               <str name="replicateAfter">startup</str>
               <str name="confFiles">schema.xml,stopwords.txt,lang/stopwords_ja.txt</str>
             </lst>
        </requestHandler>
      

      Only stopwords_ja.txt exists in solr/collection1/conf/lang directory on slave node.

      1. SOLR-4751.patch
        2 kB
        Minoru Osuka
      2. SOLR-4751.patch
        3 kB
        Minoru Osuka

        Activity

        Hide
        Shalin Shekhar Mangar added a comment -

        Bulk close after 4.3.1 release

        Show
        Shalin Shekhar Mangar added a comment - Bulk close after 4.3.1 release
        Hide
        Minoru Osuka added a comment -

        Sekiguchi-san,

        Thank you so much for your help!

        Show
        Minoru Osuka added a comment - Sekiguchi-san, Thank you so much for your help!
        Hide
        Koji Sekiguchi added a comment -

        Committed on trunk, branch_4x and lucene_solr_4_3. Thanks!

        Show
        Koji Sekiguchi added a comment - Committed on trunk, branch_4x and lucene_solr_4_3. Thanks!
        Hide
        Koji Sekiguchi added a comment -

        Looks good! I'll commit shortly.

        Show
        Koji Sekiguchi added a comment - Looks good! I'll commit shortly.
        Hide
        Minoru Osuka added a comment -

        When the directory of a renaming place did not exist, it modified so that a directory will be created.

        The test code was satisfactory.
        Ran ant test -Dtestcase=TestReplicationHandler

        Show
        Minoru Osuka added a comment - When the directory of a renaming place did not exist, it modified so that a directory will be created. The test code was satisfactory. Ran ant test -Dtestcase=TestReplicationHandler
        Hide
        Minoru Osuka added a comment -

        Sorry for the inconvenience.

        I'll modify test code.

        Show
        Minoru Osuka added a comment - Sorry for the inconvenience. I'll modify test code.
        Hide
        Commit Tag Bot added a comment -

        [lucene_solr_4_3 commit] koji
        http://svn.apache.org/viewvc?view=revision&revision=1481094

        revert SOLR-4751 change

        Show
        Commit Tag Bot added a comment - [lucene_solr_4_3 commit] koji http://svn.apache.org/viewvc?view=revision&revision=1481094 revert SOLR-4751 change
        Hide
        Koji Sekiguchi added a comment -

        Sorry for the inconvenience.

        I'll contact the reporter and see if we can fix the tests.

        Show
        Koji Sekiguchi added a comment - Sorry for the inconvenience. I'll contact the reporter and see if we can fix the tests.
        Hide
        Mark Miller added a comment -

        Seems a test fail is exposing an issue with this. I've reverted this for the moment.

        Show
        Mark Miller added a comment - Seems a test fail is exposing an issue with this. I've reverted this for the moment.
        Hide
        Commit Tag Bot added a comment -

        [branch_4x commit] markrmiller
        http://svn.apache.org/viewvc?view=revision&revision=1481031

        SOLR-4751: revert for now

        Show
        Commit Tag Bot added a comment - [branch_4x commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1481031 SOLR-4751 : revert for now
        Hide
        Commit Tag Bot added a comment -
        Show
        Commit Tag Bot added a comment - [trunk commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1481030 SOLR-4751 : revert for now
        Hide
        Koji Sekiguchi added a comment -

        I committed the patch on trunk, branch_4x and 4.3.

        Osuka-san, can you check one of them and see that your fix can solve the problem?

        Show
        Koji Sekiguchi added a comment - I committed the patch on trunk, branch_4x and 4.3. Osuka-san, can you check one of them and see that your fix can solve the problem?
        Hide
        Commit Tag Bot added a comment -

        [lucene_solr_4_3 commit] koji
        http://svn.apache.org/viewvc?view=revision&revision=1481008

        SOLR-4751: fix replication problem of files in sub directory of conf

        Show
        Commit Tag Bot added a comment - [lucene_solr_4_3 commit] koji http://svn.apache.org/viewvc?view=revision&revision=1481008 SOLR-4751 : fix replication problem of files in sub directory of conf
        Hide
        Commit Tag Bot added a comment -

        [branch_4x commit] koji
        http://svn.apache.org/viewvc?view=revision&revision=1481004

        SOLR-4751: fix replication problem of files in sub directory of conf

        Show
        Commit Tag Bot added a comment - [branch_4x commit] koji http://svn.apache.org/viewvc?view=revision&revision=1481004 SOLR-4751 : fix replication problem of files in sub directory of conf
        Hide
        Commit Tag Bot added a comment -

        [trunk commit] koji
        http://svn.apache.org/viewvc?view=revision&revision=1480988

        SOLR-4751: fix replication problem of files in sub directory of conf

        Show
        Commit Tag Bot added a comment - [trunk commit] koji http://svn.apache.org/viewvc?view=revision&revision=1480988 SOLR-4751 : fix replication problem of files in sub directory of conf
        Hide
        Koji Sekiguchi added a comment -

        Need to find files recursively.

        The other day, I talked Osuka-san and I got the problem. It've come into being there since 3.6 as subdirectory (lang/) was created under conf directory.

        Although it'd better to have test cases for replication, but as I don't have time and the patch looks simple, I'd like to commit in a few days if no one objects.

        Meanwhile, updates of the patch are welcome.

        Show
        Koji Sekiguchi added a comment - Need to find files recursively. The other day, I talked Osuka-san and I got the problem. It've come into being there since 3.6 as subdirectory (lang/) was created under conf directory. Although it'd better to have test cases for replication, but as I don't have time and the patch looks simple, I'd like to commit in a few days if no one objects. Meanwhile, updates of the patch are welcome.
        Hide
        Minoru Osuka added a comment -

        Hi Sekiguchi-san,

        Please see following code,
        core/src/java/org/apache/solr/handler/SnapPuller.java

          private void copyTmpConfFiles2Conf(File tmpconfDir) {
            File confDir = new File(solrCore.getResourceLoader().getConfigDir());
            for (File file : tmpconfDir.listFiles()) {
              File oldFile = new File(confDir, file.getName());
              if (oldFile.exists()) {
                File backupFile = new File(confDir, oldFile.getName() + "." + getDateAsStr(new Date(oldFile.lastModified())));
                boolean status = oldFile.renameTo(backupFile);
                if (!status) {
                  throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
                          "Unable to rename: " + oldFile + " to: " + backupFile);
                }
              }
              boolean status = file.renameTo(oldFile);
              if (status) {
              } else {
                throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
                        "Unable to rename: " + file + " to: " + oldFile);
              }
            }
          }
        

        This code does not assume subdirectory exists in conf directory.
        Only the files specified by confFiles exists in temporary directory. All files are not gathered in lang directory in temporary directory.
        This code will rename incomplete lang directory to conf directory.
        Need to find files recursively.

        Show
        Minoru Osuka added a comment - Hi Sekiguchi-san, Please see following code, core/src/java/org/apache/solr/handler/SnapPuller.java private void copyTmpConfFiles2Conf(File tmpconfDir) { File confDir = new File(solrCore.getResourceLoader().getConfigDir()); for (File file : tmpconfDir.listFiles()) { File oldFile = new File(confDir, file.getName()); if (oldFile.exists()) { File backupFile = new File(confDir, oldFile.getName() + "." + getDateAsStr( new Date(oldFile.lastModified()))); boolean status = oldFile.renameTo(backupFile); if (!status) { throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Unable to rename: " + oldFile + " to: " + backupFile); } } boolean status = file.renameTo(oldFile); if (status) { } else { throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Unable to rename: " + file + " to: " + oldFile); } } } This code does not assume subdirectory exists in conf directory. Only the files specified by confFiles exists in temporary directory. All files are not gathered in lang directory in temporary directory. This code will rename incomplete lang directory to conf directory. Need to find files recursively.
        Hide
        Koji Sekiguchi added a comment -

        Hi Osuka-san, would you elaborate the problem a little more, please?

        Show
        Koji Sekiguchi added a comment - Hi Osuka-san, would you elaborate the problem a little more, please?

          People

          • Assignee:
            Koji Sekiguchi
            Reporter:
            Minoru Osuka
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development