Derby
  1. Derby
  2. DERBY-1958

improve XSDG3 error to print container, actual i/o operation, and file name.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 10.1.3.1, 10.2.1.6, 10.3.1.4
    • Fix Version/s: 10.7.1.1
    • Component/s: Store
    • Labels:
      None
    • Issue & fix info:
      Newcomer

      Description

      The current error does not give enough information to know what container is the problem:
      ERROR XSDG3: Meta-data for Container
      org.apache.derby.impl.store.raw.data.RAFContainer@10632cd could not be
      accessed

      1. Derby-1958.diff
        7 kB
        Eranda Sooriyabandara
      2. Derby-1958.diff
        2 kB
        Eranda Sooriyabandara
      3. Derby-1958.diff
        3 kB
        Eranda Sooriyabandara
      4. reproduce.diff
        3 kB
        Eranda Sooriyabandara
      5. reviewedPatch.diff
        7 kB
        Bryan Pendleton

        Activity

        Hide
        Knut Anders Hatlen added a comment -

        [bulk update] Close all resolved issues that haven't been updated for more than one year.

        Show
        Knut Anders Hatlen added a comment - [bulk update] Close all resolved issues that haven't been updated for more than one year.
        Hide
        Knut Anders Hatlen added a comment -

        One little thing... The new message will be virtually impossible to translate to other languages, since the verb in the sentence ("open", "read", "encrypt") is passed as a parameter that's always in English.

        Show
        Knut Anders Hatlen added a comment - One little thing... The new message will be virtually impossible to translate to other languages, since the verb in the sentence ("open", "read", "encrypt") is passed as a parameter that's always in English.
        Hide
        Bryan Pendleton added a comment -

        Committed to the trunk as revision 989696.

        Thanks for the patch, Eranda!

        Although we don't hit this message frequently, the next time it
        happens it will be good to have this extra information.

        Show
        Bryan Pendleton added a comment - Committed to the trunk as revision 989696. Thanks for the patch, Eranda! Although we don't hit this message frequently, the next time it happens it will be good to have this extra information.
        Hide
        Eranda Sooriyabandara added a comment -

        Hi Bryan,
        Thanks for the updated patch and I got the changed which you did. Also I successfully ran the regression tests in my machine after applying it. I think this can be committed now.
        Thanks

        Show
        Eranda Sooriyabandara added a comment - Hi Bryan, Thanks for the updated patch and I got the changed which you did. Also I successfully ran the regression tests in my machine after applying it. I think this can be committed now. Thanks
        Hide
        Bryan Pendleton added a comment -

        Hi Eranda,

        I took your patch and made some additional changes, and have attached
        the revised patch as 'reviewedPatch.diff'.

        It would be great if you could have a look and proof-read the updated patch.

        The regression tests run clean for me with this patch.

        I also planted fake IOExceptions into the code in a couple cases
        and verified that the message is printed cleanly to derby.log.

        Let me know if you see anything else that we need to attend to
        prior to committing this patch.

        Thanks!

        Show
        Bryan Pendleton added a comment - Hi Eranda, I took your patch and made some additional changes, and have attached the revised patch as 'reviewedPatch.diff'. It would be great if you could have a look and proof-read the updated patch. The regression tests run clean for me with this patch. I also planted fake IOExceptions into the code in a couple cases and verified that the message is printed cleanly to derby.log. Let me know if you see anything else that we need to attend to prior to committing this patch. Thanks!
        Hide
        Eranda Sooriyabandara added a comment -

        Hi Bryan,
        I came up with a patch. While creating this I faced a problem with the type variable in which we define read/write. So I came up with a class variables for both type and file (openType and fileName) where they are initialized at readPage/writePage methods. So what you think, can we have such variable or does it need changes?

        Show
        Eranda Sooriyabandara added a comment - Hi Bryan, I came up with a patch. While creating this I faced a problem with the type variable in which we define read/write. So I came up with a class variables for both type and file (openType and fileName) where they are initialized at readPage/writePage methods. So what you think, can we have such variable or does it need changes?
        Hide
        Bryan Pendleton added a comment -

        Hi Eranda,

        I don't think we can commit the current Derby-1958.diff, because it changes
        messages.xml to add new arguments, but it does not change the users of
        the XSDG3 message to pass the new arguments.

        In addition to the changes to messages.xml and ErrorCodeTest.java, the
        patch needs to ALSO modify InputStreamContainer.java and RAFContainer.java
        to pass the getIdentity(), type, and file arguments when the XSDG3 message
        (which is called FILE_CONTAINER_EXCEPTION in the code) is thrown.

        I think the patch should contain about 7 modified calls to newException()
        in those two source files.

        Show
        Bryan Pendleton added a comment - Hi Eranda, I don't think we can commit the current Derby-1958.diff, because it changes messages.xml to add new arguments, but it does not change the users of the XSDG3 message to pass the new arguments. In addition to the changes to messages.xml and ErrorCodeTest.java, the patch needs to ALSO modify InputStreamContainer.java and RAFContainer.java to pass the getIdentity(), type, and file arguments when the XSDG3 message (which is called FILE_CONTAINER_EXCEPTION in the code) is thrown. I think the patch should contain about 7 modified calls to newException() in those two source files.
        Hide
        Eranda Sooriyabandara added a comment - - edited

        Hi Bryan,
        Yes Bryan I understood and I introduced them to see the model for such exception. Are there any documents to change or can we commit the patch?
        Thanks

        Show
        Eranda Sooriyabandara added a comment - - edited Hi Bryan, Yes Bryan I understood and I introduced them to see the model for such exception. Are there any documents to change or can we commit the patch? Thanks
        Hide
        Bryan Pendleton added a comment -

        I think that the changes to messages.xml and ErrorCodeTest.java look fine.

        I think what we need is a patch which includes the changes to messages.xml and
        ErrorCodeTest.java, and also includes changes to InputStreamContainer.java and
        RAFContainer.java which adjust all of the places where FILE_CONTAINER_EXCEPTION
        is generated, to use the new invocation

        throw StandardException.newException(SQLState.FILE_CONTAINER_EXCEPTION,
        new Object[]

        {getIdentity().toString(),type,file}

        ) ;

        I don't think we need the 'reproduction' changes to BaseDataFileFactoryJ4.java,
        nor the "if(pageNumber>1)" section in RAFContainer, in the final patch, as those
        were just present to enable you to verify the proper formatting of the error message.

        Does that make sense?

        Show
        Bryan Pendleton added a comment - I think that the changes to messages.xml and ErrorCodeTest.java look fine. I think what we need is a patch which includes the changes to messages.xml and ErrorCodeTest.java, and also includes changes to InputStreamContainer.java and RAFContainer.java which adjust all of the places where FILE_CONTAINER_EXCEPTION is generated, to use the new invocation throw StandardException.newException(SQLState.FILE_CONTAINER_EXCEPTION, new Object[] {getIdentity().toString(),type,file} ) ; I don't think we need the 'reproduction' changes to BaseDataFileFactoryJ4.java, nor the "if(pageNumber>1)" section in RAFContainer, in the final patch, as those were just present to enable you to verify the proper formatting of the error message. Does that make sense?
        Hide
        Eranda Sooriyabandara added a comment -

        Hi Bryan,
        Here I am attaching the actual patch file and the patch with the error reproduction for your concern.
        thanks

        Show
        Eranda Sooriyabandara added a comment - Hi Bryan, Here I am attaching the actual patch file and the patch with the error reproduction for your concern. thanks
        Hide
        Bryan Pendleton added a comment -

        Yes, I think that would be a considerable improvement over the current behavior,
        and would certainly address this issue appropriately.

        Can you attach your proposed patch which creates the message as you suggest above?

        Also, you should mark this issue as assigned to you.

        Show
        Bryan Pendleton added a comment - Yes, I think that would be a considerable improvement over the current behavior, and would certainly address this issue appropriately. Can you attach your proposed patch which creates the message as you suggest above? Also, you should mark this issue as assigned to you.
        Hide
        Eranda Sooriyabandara added a comment -

        Hi Bryan,
        I throw exception,
        throw StandardException.newException(SQLState.FILE_CONTAINER_EXCEPTION, new Object[]

        {getIdentity().toString(),type,file}

        )
        and the message format of the XSDG3 error as,
        Meta-data for

        {0}

        could not be accessed to

        {1}

        {2}

        in message.xml
        and got the exception error message as
        ERROR XSDG3: Meta-data for Container(0, 161) could not be accessed to read /home/eranda/Desktop/DERBY/Derby-1958/trunk/test/MyDB/seg0/ca1.dat
        Is it good enough for now?
        thanks

        Show
        Eranda Sooriyabandara added a comment - Hi Bryan, I throw exception, throw StandardException.newException(SQLState.FILE_CONTAINER_EXCEPTION, new Object[] {getIdentity().toString(),type,file} ) and the message format of the XSDG3 error as, Meta-data for {0} could not be accessed to {1} {2} in message.xml and got the exception error message as ERROR XSDG3: Meta-data for Container(0, 161) could not be accessed to read /home/eranda/Desktop/DERBY/Derby-1958/trunk/test/MyDB/seg0/ca1.dat Is it good enough for now? thanks
        Hide
        Bryan Pendleton added a comment -

        I think that using the getIdentity() method to provide container information
        would be a good resolution for now. Eranda, what do you think about
        providing a toString() method which returns getIdentity()? What would
        the message look like in that case?

        Show
        Bryan Pendleton added a comment - I think that using the getIdentity() method to provide container information would be a good resolution for now. Eranda, what do you think about providing a toString() method which returns getIdentity()? What would the message look like in that case?
        Hide
        Eranda Sooriyabandara added a comment -

        Hi Mike,
        getIdentity() method only print "Container(0, 161)" for me. Other than that we can over right the toString() to print all stuff as Bryan mentioned.
        Here is a sample I made,
        public String toString()

        { String str = this.identity+" Version="+this.containerVersion+ " estRows="+this.estimatedRowCount+" inBackup="+this.inBackup+ " inRemove="+this.inRemove+" isDirty="+this.isDirty+ " pageSize="+this.pageSize+" fileData="+this.fileData+ " actionCode="+this.actionCode; return str; }

        and its results,

        ERROR XSDG3: Meta-data for Container(0, 161) Version=0 estRows=149 inBackup=false inRemove=false isDirty=false pageSize=4096 fileData=org.apache.derby.impl.io.DirRandomAccessFile4@1690ab actionCode=1 could not be accessed to read /home/eranda/Desktop/DERBY/Derby-1958/trunk/test/MyDB/seg0/ca1.dat

        Also I did not find a variable called FileContainer.estimatedPageCount. So I miss it in the toString method.
        Thanks

        Show
        Eranda Sooriyabandara added a comment - Hi Mike, getIdentity() method only print "Container(0, 161)" for me. Other than that we can over right the toString() to print all stuff as Bryan mentioned. Here is a sample I made, public String toString() { String str = this.identity+" Version="+this.containerVersion+ " estRows="+this.estimatedRowCount+" inBackup="+this.inBackup+ " inRemove="+this.inRemove+" isDirty="+this.isDirty+ " pageSize="+this.pageSize+" fileData="+this.fileData+ " actionCode="+this.actionCode; return str; } and its results, ERROR XSDG3: Meta-data for Container(0, 161) Version=0 estRows=149 inBackup=false inRemove=false isDirty=false pageSize=4096 fileData=org.apache.derby.impl.io.DirRandomAccessFile4@1690ab actionCode=1 could not be accessed to read /home/eranda/Desktop/DERBY/Derby-1958/trunk/test/MyDB/seg0/ca1.dat Also I did not find a variable called FileContainer.estimatedPageCount. So I miss it in the toString method. Thanks
        Hide
        Mike Matrigali added a comment -

        I am not sure it is worth it for this internal error, but you should not be passing english words as parameters to the error message. ie. passing "read" in. By doing this it is impossible to have the
        error message correctly translated. The options are either to not have the parameter and have
        different error messages, or there is a facility to pass in a message id that can be translated. This
        is heavily used for stuff like query plan printing. Again as an internal message it may be ok, but in general it should not be used.

        If you go down the route of adding to/creating a toString() I would give up on translation as that would
        fall more into a "dump" and probably be ok to not translate.

        The usual info printed for a container in an error message I think comes from a ContainerKey. I think you can get this from a getIdentity() call. See the SQLState.DATA_UNKNOWN_CONTAINER_FORMAT error thrown in the FileContainer class.

        Historically (pre-open source) not a lot of "toString()" were implemented, and if they were they were often only implemented in SANE mode with the following type of construct:
        public String toString()
        {
        if (SanityManager.DEBUG)

        { String str = "*** Alloc page ***\n" + "nextAllocPageNumber = " + nextAllocPageNumber + "\nnextAllocPageOffset = " + nextAllocPageOffset + "\nreserved1 = " + reserved1 + "\nreserved2 = " + reserved2 + "\nreserved3 = " + reserved3 + "\nreserved4 = " + reserved4 + "\nborrowedSpace = " + borrowedSpace + "\nextent = " + extent.toDebugString() + "\n" + super.toString(); return str; }

        else

        { return null; }

        }

        This was to keep the foot print as small as possible, and recognizing that they were not translatable.

        Show
        Mike Matrigali added a comment - I am not sure it is worth it for this internal error, but you should not be passing english words as parameters to the error message. ie. passing "read" in. By doing this it is impossible to have the error message correctly translated. The options are either to not have the parameter and have different error messages, or there is a facility to pass in a message id that can be translated. This is heavily used for stuff like query plan printing. Again as an internal message it may be ok, but in general it should not be used. If you go down the route of adding to/creating a toString() I would give up on translation as that would fall more into a "dump" and probably be ok to not translate. The usual info printed for a container in an error message I think comes from a ContainerKey. I think you can get this from a getIdentity() call. See the SQLState.DATA_UNKNOWN_CONTAINER_FORMAT error thrown in the FileContainer class. Historically (pre-open source) not a lot of "toString()" were implemented, and if they were they were often only implemented in SANE mode with the following type of construct: public String toString() { if (SanityManager.DEBUG) { String str = "*** Alloc page ***\n" + "nextAllocPageNumber = " + nextAllocPageNumber + "\nnextAllocPageOffset = " + nextAllocPageOffset + "\nreserved1 = " + reserved1 + "\nreserved2 = " + reserved2 + "\nreserved3 = " + reserved3 + "\nreserved4 = " + reserved4 + "\nborrowedSpace = " + borrowedSpace + "\nextent = " + extent.toDebugString() + "\n" + super.toString(); return str; } else { return null; } } This was to keep the foot print as small as possible, and recognizing that they were not translatable.
        Hide
        Bryan Pendleton added a comment -

        What do you think about adding a toString() method to RAFContainer? Then instead of getting

        Container org.apache.derby.impl.store.raw.data.RAFContainer@1318b

        the message might contain something like:

        Container (seg=1,ctr=113) version=4, estRows=104679 estPages=645 file=ca1.dat inBackup=false

        The best way to do this is probably to put toString() methods on BaseContainer,
        FileContainer, and RAFContainer, and make sure that each toString() method
        invokes super.toString() as part of its processing, since the printable information
        about a container is contained in classes up and down the hierarchy.

        Mike can probably help us decide what information is most important to
        include in the toString method so that the container state will be preserved in
        the error message, but you could start by just looking at the member fields
        of each of the classes and including them in the message.

        From a quick scan, I suspect that the most important fields to include in the
        toString are:

        • BaseContainer.identity
        • FileContainer.pageSize
        • FileContainer.containerVersion
        • FileContainer.estimatedRowCount
        • FileContainer.estimatedPageCount
        • FileContainer.isDirty
        • RAFContainer.fileData
        • RAFContainer.actionCode
        • RAFContainer.inBackup
        • RAFContainer.inRemove
        Show
        Bryan Pendleton added a comment - What do you think about adding a toString() method to RAFContainer? Then instead of getting Container org.apache.derby.impl.store.raw.data.RAFContainer@1318b the message might contain something like: Container (seg=1,ctr=113) version=4, estRows=104679 estPages=645 file=ca1.dat inBackup=false The best way to do this is probably to put toString() methods on BaseContainer, FileContainer, and RAFContainer, and make sure that each toString() method invokes super.toString() as part of its processing, since the printable information about a container is contained in classes up and down the hierarchy. Mike can probably help us decide what information is most important to include in the toString method so that the container state will be preserved in the error message, but you could start by just looking at the member fields of each of the classes and including them in the message. From a quick scan, I suspect that the most important fields to include in the toString are: BaseContainer.identity FileContainer.pageSize FileContainer.containerVersion FileContainer.estimatedRowCount FileContainer.estimatedPageCount FileContainer.isDirty RAFContainer.fileData RAFContainer.actionCode RAFContainer.inBackup RAFContainer.inRemove
        Hide
        Eranda Sooriyabandara added a comment -

        Hi Bryan,
        Here I reproduce the error and do the following changes to the code and get the following result.

        First create database
        ij> connect 'jdbc:derby:MyDB;create=true';
        ij> create table t1(a int);
        0 rows inserted/updated/deleted
        ij> insert into t1 values(1),(2);
        2 rows inserted/updated/deleted
        ij> select a from t1;
        A
        -----------
        1
        2

        2 rows selected
        ij> exit;

        then restart

        ij> connect 'jdbc:derby:MyDB;create=true';
        ij> select a from t1;
        ERROR XSDG3: Meta-data for Container org.apache.derby.impl.store.raw.data.RAFContainer@1318b could not be accessed to read /home/eranda/Desktop/DERBY/Derby-1958/trunk/test/MyDB/seg0/ca1.dat
        ij>

        what you think about the message?

        Show
        Eranda Sooriyabandara added a comment - Hi Bryan, Here I reproduce the error and do the following changes to the code and get the following result. First create database ij> connect 'jdbc:derby:MyDB;create=true'; ij> create table t1(a int); 0 rows inserted/updated/deleted ij> insert into t1 values(1),(2); 2 rows inserted/updated/deleted ij> select a from t1; A ----------- 1 2 2 rows selected ij> exit; then restart ij> connect 'jdbc:derby:MyDB;create=true'; ij> select a from t1; ERROR XSDG3: Meta-data for Container org.apache.derby.impl.store.raw.data.RAFContainer@1318b could not be accessed to read /home/eranda/Desktop/DERBY/Derby-1958/trunk/test/MyDB/seg0/ca1.dat ij> what you think about the message?
        Hide
        Bryan Pendleton added a comment -

        > But still I don't understand how to get the parameters here

        {1}

        and

        {2}

        .

        StandardException.newException() has a whole bunch of overloads,
        each one is a different way to pass arguments to an error message.

        To create an exception with a message with 3 parameters, try using
        the overload which takes (messageID, Object[]), and in the Object array
        put your three arguments.

        Something like:

        throw StandardException.newException(
        SQLState.FILE_CONTAINER_EXCEPTION,
        new Object[]

        { this.toString(), "read", file }

        );

        Show
        Bryan Pendleton added a comment - > But still I don't understand how to get the parameters here {1} and {2} . StandardException.newException() has a whole bunch of overloads, each one is a different way to pass arguments to an error message. To create an exception with a message with 3 parameters, try using the overload which takes (messageID, Object[]), and in the Object array put your three arguments. Something like: throw StandardException.newException( SQLState.FILE_CONTAINER_EXCEPTION, new Object[] { this.toString(), "read", file } );
        Hide
        Eranda Sooriyabandara added a comment -

        Hi,
        How can I pass parameters to a Error message.

        I use the exception as
        throw StandardException.newException(SQLState.FILE_CONTAINER_EXCEPTION,new IOException("FAKE FAKE FAKE"),this.toString(),"read",file);

        Here is my change to modify the error message,
        Index: java/engine/org/apache/derby/loc/messages.xml
        ===================================================================
        — java/engine/org/apache/derby/loc/messages.xml (revision 961244)
        +++ java/engine/org/apache/derby/loc/messages.xml (working copy)
        @@ -5724,7 +5724,7 @@

        <msg>
        <name>XSDG3.D</name>

        • <text>Meta-data for Container {0} could not be accessed</text>
          + <text>Meta-data for Container {0}

          could not be accessed to

          {1} file {2}</text>
          <arg>containerName</arg>
          </msg>

          But still I don't understand how to get the parameters here {1}

          and

          {2}

          .

        With this change I am not able to build, The error was
        Could not generate English properties from message descriptors.

        Thanks
        Eranda

        Show
        Eranda Sooriyabandara added a comment - Hi, How can I pass parameters to a Error message. I use the exception as throw StandardException.newException(SQLState.FILE_CONTAINER_EXCEPTION,new IOException("FAKE FAKE FAKE"),this.toString(),"read",file); Here is my change to modify the error message, Index: java/engine/org/apache/derby/loc/messages.xml =================================================================== — java/engine/org/apache/derby/loc/messages.xml (revision 961244) +++ java/engine/org/apache/derby/loc/messages.xml (working copy) @@ -5724,7 +5724,7 @@ <msg> <name>XSDG3.D</name> <text>Meta-data for Container {0} could not be accessed</text> + <text>Meta-data for Container {0} could not be accessed to {1} file {2}</text> <arg>containerName</arg> </msg> But still I don't understand how to get the parameters here {1} and {2} . With this change I am not able to build, The error was Could not generate English properties from message descriptors. Thanks Eranda
        Hide
        Mike Matrigali added a comment -

        As others have suggested, in order to go through those routines you need to "avoid" the page cache.
        Here are some suggestions:
        o for read the easiest thing is probably to shut down the database
        and restart. On restart there is nothing in the cache so it is
        necessary to read pages from disk to the cache.
        o to cause both read and write I would suggest changing the
        size of the page cache from it's default of 1000 pages to
        something much smaller, like 40 pages using derby.storage.pageCacheSize. Then create a table that has actual data in it that is bigger than 40*32k. Maybe a row with a blob that has actual data in it bigger than this number. Updates to that blob should cause both reads and writes since the pages should not fit in the cache.

        Show
        Mike Matrigali added a comment - As others have suggested, in order to go through those routines you need to "avoid" the page cache. Here are some suggestions: o for read the easiest thing is probably to shut down the database and restart. On restart there is nothing in the cache so it is necessary to read pages from disk to the cache. o to cause both read and write I would suggest changing the size of the page cache from it's default of 1000 pages to something much smaller, like 40 pages using derby.storage.pageCacheSize. Then create a table that has actual data in it that is bigger than 40*32k. Maybe a row with a blob that has actual data in it bigger than this number. Updates to that blob should cause both reads and writes since the pages should not fit in the cache.
        Hide
        Bryan Pendleton added a comment -

        I think you may be seeing the page cache at work. I think that
        pages may only when the database is checkpointed or shut down.

        And to cause a page to be read you may need to shut down and
        restart, since the page you wrote remains in the cache.

        Show
        Bryan Pendleton added a comment - I think you may be seeing the page cache at work. I think that pages may only when the database is checkpointed or shut down. And to cause a page to be read you may need to shut down and restart, since the page you wrote remains in the cache.
        Hide
        Eranda Sooriyabandara added a comment -

        Hi Bryan,
        writePage is used only when create the connection, create a table and shutdown the connection. When I insert a value to a table both methods were not called. Also I didn't see any use of the readPage method. Is that possible?
        Thanks

        Show
        Eranda Sooriyabandara added a comment - Hi Bryan, writePage is used only when create the connection, create a table and shutdown the connection. When I insert a value to a table both methods were not called. Also I didn't see any use of the readPage method. Is that possible? Thanks
        Hide
        Eranda Sooriyabandara added a comment -

        If RAFContainer is in use I can't see any problem of doing so. I'll start the working on it.
        Thanks for the ideas.

        Show
        Eranda Sooriyabandara added a comment - If RAFContainer is in use I can't see any problem of doing so. I'll start the working on it. Thanks for the ideas.
        Hide
        Bryan Pendleton added a comment -

        Thanks Knut, that makes sense.

        It seems to me that the best approach for this particular issue, then, is:
        1) Using a custom version of BaseDataFileFactoryJ4, as described
        above, force the code through RAFContainer.readPage/writePage
        2) Using a custom version of RAFContainer, force readPage/writePage
        to generate the XSDG3 error
        3) Verify that the new message is formatted as expected, and
        contains the expected information
        4) Commit the changes with the improved error message, and
        declare this issue complete.

        Eranda, does this strategy seem reasonable?

        Show
        Bryan Pendleton added a comment - Thanks Knut, that makes sense. It seems to me that the best approach for this particular issue, then, is: 1) Using a custom version of BaseDataFileFactoryJ4, as described above, force the code through RAFContainer.readPage/writePage 2) Using a custom version of RAFContainer, force readPage/writePage to generate the XSDG3 error 3) Verify that the new message is formatted as expected, and contains the expected information 4) Commit the changes with the improved error message, and declare this issue complete. Eranda, does this strategy seem reasonable?
        Hide
        Knut Anders Hatlen added a comment -

        RAFContainer is still used on J2ME, since java.nio.* is not part of
        the CDC Foundation Profile.

        I think RAFContainer4 can end up throwing this exception too, since it
        inherits many methods from RAFContainer, and some of the overridden
        methods also call methods in the super-class. One example is the stack
        trace in DERBY-3688, where RAFContainer4.openContainer() calls
        RAFContainer.openContainer() and ends up raising this error.

        It may be difficult to predict exactly which SQL statements end up
        calling readPage() and writePage() because of the page cache. If the
        database is small, you'll probably only see calls to readPage() when
        you access data for the first time. writePage() should normally only
        be called during checkpoint/shutdown, when a page is evicted from the
        cache to give room for another page, or when a new page is created.

        Show
        Knut Anders Hatlen added a comment - RAFContainer is still used on J2ME, since java.nio.* is not part of the CDC Foundation Profile. I think RAFContainer4 can end up throwing this exception too, since it inherits many methods from RAFContainer, and some of the overridden methods also call methods in the super-class. One example is the stack trace in DERBY-3688 , where RAFContainer4.openContainer() calls RAFContainer.openContainer() and ends up raising this error. It may be difficult to predict exactly which SQL statements end up calling readPage() and writePage() because of the page cache. If the database is small, you'll probably only see calls to readPage() when you access data for the first time. writePage() should normally only be called during checkpoint/shutdown, when a page is evicted from the cache to give room for another page, or when a new page is created.
        Hide
        Bryan Pendleton added a comment -

        Hi Eranda, Sorry about that, I should have tried it first not just guessed.

        It looks like readPage and writePage in RAFContainer are overridden
        by readPage and writePage in RAFContainer4, and it looks like we always
        use RAFContainer4 for any JDK 1.4+ configuration. Mike or Knut Anders or somebody
        else who's familiar with the store can probably confirm this for sure.

        There is a class called BaseDataFileFactoryJ4 which arranges for this
        overriding, though, and so I was able to re-point the store to use the
        RAFContainer rather than the RAFContainer4 by simply making this change
        to BaseDataFileFactoryJ4:

        — java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactoryJ4.java(revision 958595)
        +++ java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactoryJ4.java(working copy)
        @@ -42,6 +42,6 @@

        • objects capable of exploiting the NIO API available in Java 1.4+
          */
          protected Cacheable newRAFContainer(BaseDataFileFactory factory) { - return new RAFContainer4(factory); + return new RAFContainer(factory); }

          }

        Once I did this, I started to hit the readPage and writePage methods in RAFContainer.

        Now, I'm not totally sure what the implications of this are. Since (I believe) Derby
        no longer supports any JDK version prior to JDK 1.4 (I think Derby 10.2 was the
        last version that supported JDK 1.3 – see DERBY-1982), if RAFContainer is ONLY
        used for JDK 1.3, then perhaps we should just remove RAFContainer, rather
        than trying to fix it.

        Does the problematic XSDG3 message arise in RAFContainer4 as well?

        I did a quick search for FILE_CONTAINER_EXCEPTION in the code, and maybe
        the message only occurs in RAFContainer, but it would be nice if somebody else
        could confirm these analyses.

        Basically, the question is: can we just remove RAFContainer (or, at least,
        RAFContainer.readPage and RAFContainer.writePage), now that we no longer
        support JDK 1.3?

        Show
        Bryan Pendleton added a comment - Hi Eranda, Sorry about that, I should have tried it first not just guessed. It looks like readPage and writePage in RAFContainer are overridden by readPage and writePage in RAFContainer4, and it looks like we always use RAFContainer4 for any JDK 1.4+ configuration. Mike or Knut Anders or somebody else who's familiar with the store can probably confirm this for sure. There is a class called BaseDataFileFactoryJ4 which arranges for this overriding, though, and so I was able to re-point the store to use the RAFContainer rather than the RAFContainer4 by simply making this change to BaseDataFileFactoryJ4: — java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactoryJ4.java(revision 958595) +++ java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactoryJ4.java(working copy) @@ -42,6 +42,6 @@ objects capable of exploiting the NIO API available in Java 1.4+ */ protected Cacheable newRAFContainer(BaseDataFileFactory factory) { - return new RAFContainer4(factory); + return new RAFContainer(factory); } } Once I did this, I started to hit the readPage and writePage methods in RAFContainer. Now, I'm not totally sure what the implications of this are. Since (I believe) Derby no longer supports any JDK version prior to JDK 1.4 (I think Derby 10.2 was the last version that supported JDK 1.3 – see DERBY-1982 ), if RAFContainer is ONLY used for JDK 1.3, then perhaps we should just remove RAFContainer, rather than trying to fix it. Does the problematic XSDG3 message arise in RAFContainer4 as well? I did a quick search for FILE_CONTAINER_EXCEPTION in the code, and maybe the message only occurs in RAFContainer, but it would be nice if somebody else could confirm these analyses. Basically, the question is: can we just remove RAFContainer (or, at least, RAFContainer.readPage and RAFContainer.writePage), now that we no longer support JDK 1.3?
        Hide
        Eranda Sooriyabandara added a comment -

        Hi Bryan,
        I put breakpoints at the beginning of the pageRead and pageWrite methods and execute those commands but it did not hit. Do I need to set something to get it work?
        thanks

        Show
        Eranda Sooriyabandara added a comment - Hi Bryan, I put breakpoints at the beginning of the pageRead and pageWrite methods and execute those commands but it did not hit. Do I need to set something to get it work? thanks
        Hide
        Bryan Pendleton added a comment -

        Almost all SQL commands will end up calling readPage, and almost all
        update commands will end up calling writePage. Try:

        create table t (a i)
        insert into t values (1), (2), (3)

        This should call both readPage and writePage a number of times.

        Show
        Bryan Pendleton added a comment - Almost all SQL commands will end up calling readPage, and almost all update commands will end up calling writePage. Try: create table t (a i) insert into t values (1), (2), (3) This should call both readPage and writePage a number of times.
        Hide
        Eranda Sooriyabandara added a comment -

        Hi Bryan,
        Can I know for what sql commands this readPage and writePage methods in RAFContainer class are called?
        thanks
        Eranda

        Show
        Eranda Sooriyabandara added a comment - Hi Bryan, Can I know for what sql commands this readPage and writePage methods in RAFContainer class are called? thanks Eranda
        Hide
        Bryan Pendleton added a comment -

        Hi Eranda,
        I did a somewhat similar private alteration to Derby for DERBY-3729,
        so hopefully looking at that work will be a good place to start.

        If you look through the comments and attachments to DERBY-3729,
        and in particular this comment:
        https://issues.apache.org/jira/browse/DERBY-3729?focusedCommentId=12829808&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12829808
        I think you should be able to get some ideas about how to make a
        similar modification in order to expose this XSDG3 error message.

        Of course, whereas I modified CachedPage.java, you'll want to make
        a modification somewhere in RAFContainer.java instead. But the basic
        principles of the technique should be the same.

        Show
        Bryan Pendleton added a comment - Hi Eranda, I did a somewhat similar private alteration to Derby for DERBY-3729 , so hopefully looking at that work will be a good place to start. If you look through the comments and attachments to DERBY-3729 , and in particular this comment: https://issues.apache.org/jira/browse/DERBY-3729?focusedCommentId=12829808&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12829808 I think you should be able to get some ideas about how to make a similar modification in order to expose this XSDG3 error message. Of course, whereas I modified CachedPage.java, you'll want to make a modification somewhere in RAFContainer.java instead. But the basic principles of the technique should be the same.
        Hide
        Eranda Sooriyabandara added a comment -

        Hi Mike,
        Yes I like to have a try. If you can provide me a code which trows that exception that would be much appreciated.
        Thanks

        Show
        Eranda Sooriyabandara added a comment - Hi Mike, Yes I like to have a try. If you can provide me a code which trows that exception that would be much appreciated. Thanks
        Hide
        Mike Matrigali added a comment -

        This is an issue that "should not happen." You can search JIRA for XSDG3 to see the reported issues where this exception has been thrown, but I believe all the reproducible cases have been fixed. I reported this so that if we get another bug in the future the error at least the error will be more descriptive.

        If you are interested in fixing, I would find somewhere in the code that throws this
        exception and in your private codeline change it so that the code always throws this
        error so that you can hand test the error message change. I don't know of any way
        to cause this in a correctly running current system.

        Show
        Mike Matrigali added a comment - This is an issue that "should not happen." You can search JIRA for XSDG3 to see the reported issues where this exception has been thrown, but I believe all the reproducible cases have been fixed. I reported this so that if we get another bug in the future the error at least the error will be more descriptive. If you are interested in fixing, I would find somewhere in the code that throws this exception and in your private codeline change it so that the code always throws this error so that you can hand test the error message change. I don't know of any way to cause this in a correctly running current system.
        Hide
        Eranda Sooriyabandara added a comment -

        Hi,
        What is the reason for this exception? and how can I reproduce it?
        thanks
        Eranda

        Show
        Eranda Sooriyabandara added a comment - Hi, What is the reason for this exception? and how can I reproduce it? thanks Eranda

          People

          • Assignee:
            Eranda Sooriyabandara
            Reporter:
            Mike Matrigali
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development