Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Not a Problem
    • Affects Version/s: 1.9
    • Fix Version/s: None
    • Component/s: core/store
    • Labels:
      None
    • Environment:

      MySql (version 4.1 an up), Oracle (version 8.1.7 and up)

      Description

      I've written an extension for the Directory object called DBDirectory, that allows you to read and write a Lucene index to a database instead of a file system.

      This is done using blobs. Each blob represents a "file". Also, each blob has a name which is equivalent to the filename and a prefix, which is equivalent to a directory on a file system. This allows you to create multiple Lucene indexes in a single database schema.

      The solution uses two tables:
      LUCENE_INDEX - which holds the index files as blobs
      LUCENE_LOCK - holds the different locks

      Attached is my proposed solution. This solution is still very basic, but it does the job.
      The solution supports Oracle and mysql

      To use this solution:

      1. Place the files:

      • DBDirectory in src/java/org/apache/lucene/store
      • TestDBIndex in src/test/org/apache/lucene/index
      • objects-mysql.sql in src/db
      • objects-oracle.sql in src/db

      2. Edit the parameters for the database connection in TestDBIndex

      3. Create the database tables using the objects-mysql.sql script (assuming you're using mysql)

      4. Build Lucene

      5. Run TestDBIndex with the database driver in the classpath

      I've tested the solution on mysql, but it should work on Oracle, I will test that in a few days.

      Amir

      1. files.zip
        6 kB
        Amir Kibbar

        Activity

        Amir Kibbar created issue -
        Amir Kibbar made changes -
        Field Original Value New Value
        Attachment files.zip [ 12321874 ]
        Hide
        Christophe Bégot added a comment -

        I believe that I detected a bug in this extension
        The case of file containing only one character does not index correctly. The correction seems to work.

        DBDirectory.java
        Line 426

        Original:

        Blob blob=rs.getBlob("DATA");
        byte[] buffer=null;
        long pos=1;
        int length=0;
        while(pos<blob.length())

        { length=BUFFER_SIZE>blob.length()-pos?(int)(blob.length()-pos+1):BUFFER_SIZE; buffer=blob.getBytes(pos,length); file.addData(buffer); pos+=BUFFER_SIZE>blob.length()-pos?(int)(blob.length()-pos+1):BUFFER_SIZE; }

        Correction:

        Blob blob=rs.getBlob("DATA");
        byte[] buffer=null;
        long pos=1;
        int length=0;
        while(pos<=blob.length())

        { length=BUFFER_SIZE>blob.length()-pos?(int)(blob.length()-pos+1):BUFFER_SIZE; buffer=blob.getBytes(pos,length); file.addData(buffer); pos+=BUFFER_SIZE>blob.length()-pos?(int)(blob.length()-pos+1):BUFFER_SIZE; }

        Christophe

        Show
        Christophe Bégot added a comment - I believe that I detected a bug in this extension The case of file containing only one character does not index correctly. The correction seems to work. DBDirectory.java Line 426 Original: Blob blob=rs.getBlob("DATA"); byte[] buffer=null; long pos=1; int length=0; while(pos<blob.length()) { length=BUFFER_SIZE>blob.length()-pos?(int)(blob.length()-pos+1):BUFFER_SIZE; buffer=blob.getBytes(pos,length); file.addData(buffer); pos+=BUFFER_SIZE>blob.length()-pos?(int)(blob.length()-pos+1):BUFFER_SIZE; } Correction: Blob blob=rs.getBlob("DATA"); byte[] buffer=null; long pos=1; int length=0; while(pos<=blob.length()) { length=BUFFER_SIZE>blob.length()-pos?(int)(blob.length()-pos+1):BUFFER_SIZE; buffer=blob.getBytes(pos,length); file.addData(buffer); pos+=BUFFER_SIZE>blob.length()-pos?(int)(blob.length()-pos+1):BUFFER_SIZE; } Christophe
        Otis Gospodnetic made changes -
        Comment [ I believe that I detected a bug in this extention
        The case of file containing only one caracter is not well indexed. The correction seams to work.

        DBDirectory.java
        Line 426

        Original:

                    Blob blob=rs.getBlob("DATA");
                    byte[] buffer=null;
                    long pos=1;
                    int length=0;
                    while(pos<blob.length()) {
                        length=BUFFER_SIZE>blob.length()-pos?(int)(blob.length()-pos+1):BUFFER_SIZE;
                        buffer=blob.getBytes(pos,length);
                        file.addData(buffer);
                        pos+=BUFFER_SIZE>blob.length()-pos?(int)(blob.length()-pos+1):BUFFER_SIZE;
                    }


        Corrected:

                    Blob blob=rs.getBlob("DATA");
                    byte[] buffer=null;
                    long pos=1;
                    int length=0;
                    while(pos<=blob.length()) {
                        length=BUFFER_SIZE>blob.length()-pos?(int)(blob.length()-pos+1):BUFFER_SIZE;
                        buffer=blob.getBytes(pos,length);
                        file.addData(buffer);
                        pos+=BUFFER_SIZE>blob.length()-pos?(int)(blob.length()-pos+1):BUFFER_SIZE;
                    }

        Thanks you Amir for this extension
        Christophe ]
        Otis Gospodnetic made changes -
        Comment [ A comment with security level 'jira-users' was removed ]
        Hide
        Shai Erera added a comment -

        Not active since 2006 and we already have DBDirectory.

        Show
        Shai Erera added a comment - Not active since 2006 and we already have DBDirectory.
        Shai Erera made changes -
        Status Open [ 1 ] Closed [ 6 ]
        Resolution Not A Problem [ 8 ]
        Mark Thomas made changes -
        Workflow jira [ 12345622 ] Default workflow, editable Closed status [ 12564032 ]
        Mark Thomas made changes -
        Workflow Default workflow, editable Closed status [ 12564032 ] jira [ 12585504 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Amir Kibbar
          • Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development