Derby
  1. Derby
  2. DERBY-4525

Document the in-memory storage back end

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.6.1.0
    • Fix Version/s: 10.6.1.0
    • Component/s: Documentation
    • Labels:
      None

      Description

      The in-memory back end isn't considered experimental anymore, we have to
      write user documentation for the feature(s).
      I'm not sure how it should be structured, and where the content should be added.

      Just as a rough cut, here are a few possible topics (I'm not sure if all should be included or not):

      • documenting the new protocol name ('memory')
      • documenting the new 'drop' JDBC connection URL attribute
      • describing the limitations of the feature (all your data will be lost if..., how to use it with the client driver and the data sources)
      • "advanced use" (pull dbs on disk into memory, backup in-memory dbs to disk)
      • tuning tips (there are some issues with extreme page cache sizes, maybe the existing content on page size is valid)
      • known problems (nothing concrete here yet, but we have one inquiry about disappearing databases - the current theory is that different class loaders are used)

      Some more information is available at http://wiki.apache.org/db-derby/InMemoryBackEndPrimer

      1. DERBY-4525.diff
        31 kB
        Kim Haase
      2. DERBY-4525.stat
        0.9 kB
        Kim Haase
      3. DERBY-4525.zip
        54 kB
        Kim Haase
      4. ASF.LICENSE.NOT.GRANTED--DERBY-4525-2.diff
        34 kB
        Kim Haase
      5. ASF.LICENSE.NOT.GRANTED--DERBY-4525-2.stat
        0.9 kB
        Kim Haase
      6. ASF.LICENSE.NOT.GRANTED--DERBY-4525-2.zip
        55 kB
        Kim Haase
      7. DERBY-4525-3.diff
        36 kB
        Kim Haase
      8. DERBY-4525-3.zip
        56 kB
        Kim Haase

        Issue Links

          Activity

          Hide
          Kim Haase added a comment -

          Closing, since changes now appear in Latest Alpha Manuals.

          Show
          Kim Haase added a comment - Closing, since changes now appear in Latest Alpha Manuals.
          Hide
          Kim Haase added a comment -

          After getting verbal okay (thanks, Kristian), committed patch DERBY-4525-3.diff to documentation trunk at revision 936885.

          Show
          Kim Haase added a comment - After getting verbal okay (thanks, Kristian), committed patch DERBY-4525 -3.diff to documentation trunk at revision 936885.
          Hide
          Kim Haase added a comment -

          Thanks so much, Kristian, for the careful review. I'm attaching a final patch, DERBY-4525-3.diff and DERBY-4525-3.zip. I hope I have incorporated all your edits. A couple of notes:

          • src/ref/rrefattrib17246.dita
            In the example:
            "+<codeblock><b>jdbc:derby:toursDB;databaseName=flightsDB</b></codeblock>" -
            specifies two database names

          Actually, this is intentional – the description above is of what happens if you specify two names this way. But perhaps it is an obscure enough case that it should come after the in-memory db info – so I swapped the order.

          I hope the wording in the Tuning Guide topic is okay – I didn't add all the supporting detail, just changed the emphasis. Let me know if it still needs work.

          I added a small section on persisting databases to the Dev Guide topic. If anyone objects, we can change it.

          I changed the wording of the note in "Derby system" (src/devguide/cdevdvlp27610.dita) slightly.

          Show
          Kim Haase added a comment - Thanks so much, Kristian, for the careful review. I'm attaching a final patch, DERBY-4525 -3.diff and DERBY-4525 -3.zip. I hope I have incorporated all your edits. A couple of notes: src/ref/rrefattrib17246.dita In the example: "+<codeblock><b>jdbc:derby:toursDB;databaseName=flightsDB</b></codeblock>" - specifies two database names Actually, this is intentional – the description above is of what happens if you specify two names this way. But perhaps it is an obscure enough case that it should come after the in-memory db info – so I swapped the order. I hope the wording in the Tuning Guide topic is okay – I didn't add all the supporting detail, just changed the emphasis. Let me know if it still needs work. I added a small section on persisting databases to the Dev Guide topic. If anyone objects, we can change it. I changed the wording of the note in "Derby system" (src/devguide/cdevdvlp27610.dita) slightly.
          Hide
          Kristian Waagan added a comment -

          Hi Kim,

          Thanks for writing these docs. I think they look good, and they can be committed
          as they are if that is required to get them included in 10.6.
          I have made some comments and suggestions below. Please use you own
          judgment regarding whether they are worth implementing or not.

          • src/devguide/cdevdvlp19700.dita
            Should we add a few words to clearify that you can use the in-memory back end in
            other cases than testing and devlopment, for instance when you are processing
            transient/reproducible data?
            If you have the required memory avaialble, the benefits are (or may be) faster
            processing (no disk IO) and simplicity when it comes to discarding the data. If
            changed, revisit cdevdvlpinmemdb.html.
          • src/devguide/cdevdvlp40724.dita
            Should we mention that for some of the limits the maximum value is determined by
            the available main memory instead of disk space / file system, or is this too
            obvious to mention?
          • src/devguide/cdevdvlpinmemdb.dita
            Removing an in-memory database: Does it make sense to swap the order of the
            current content, i.e. to describe how you would actively remove an in-memory db
            in a running JVM first?
            To me, the fact that in-memory databases are lost when the JVM exits is more
            like a side-effect. We do need to mention this, but maybe more like a word of
            caution to make it clear to users that in-memory databases are not persisted.

          Mention that if authentication and SQL authorization is enabled, only the
          database owner can drop a database. I'm not sure if we have done this for other
          actions, like shutting down the database.

          • src/devguide/rdevdvlp22102.dita
            Add a link to a page describing, or just a note, that using the in-memory back
            end with the client driver requires a "special" URL (include the subsubprotocol
            as part of the name)?
          • src/ref/rrefattrib16471.dita, src/ref/rrefattrib26867.dita
            Use the same database name for embedded and client/server for consistency in the
            examples? (currently 'myInMemDB' and 'MyInMemDB')
          • src/ref/rrefattrib17246.dita
            In the example:
            "+<codeblock><b>jdbc:derby:toursDB;databaseName=flightsDB</b></codeblock>" - specifies two database names
          • src/ref/rrefattribdrop.dita
            "This attribute, like shutdown=true, cannot be combined with other attributes." - with the exception of user credentials?
            Mention that using the drop attribute with other back ends results in an
            exception with SQLState XBM0I to be thrown?
          • src/tools/rtoolsijcomref22318.dita and src/tools/rtoolsijcomref27997.dita
            Would it make sense to show that you can set the memory subsubprotocol as part
            of the protocol to create in-memory databases by default in ij?
            I.e:
            ij> protocol 'jdbc:derby:memory:';
            ij> connect 'testDB;create=true';
          • src/tuning/ctunperfinmemdb.dita
            I would maybe change the wording a bit for the page cache size. There is nothing
            wrong with running with a smaller than default page cache, but the user may see
            poor performance. You don't get the penalty for going to the disk to fetch
            pages, but if the pressure on the page cache gets too high you will get a lot of
            contention and increased CPU usage. To me, the current wording suggests that I
            should increase the page cache size when using the in-memory db. In many cases
            this is not necessary.
            Can we say that we don't recommend to run with a smaller than default page
            cache? (keep the sentence about data passing through the cache)

          Finally, should we mention that users can use the backup routines to persist an
          in-memory database? This backup can then be booted as an in-memory database
          again at a later time, or simply be used as a normal disk based db. Or does
          suggesting this use case make some people uncomfortable?

          Thanks,

          Show
          Kristian Waagan added a comment - Hi Kim, Thanks for writing these docs. I think they look good, and they can be committed as they are if that is required to get them included in 10.6. I have made some comments and suggestions below. Please use you own judgment regarding whether they are worth implementing or not. src/devguide/cdevdvlp19700.dita Should we add a few words to clearify that you can use the in-memory back end in other cases than testing and devlopment, for instance when you are processing transient/reproducible data? If you have the required memory avaialble, the benefits are (or may be) faster processing (no disk IO) and simplicity when it comes to discarding the data. If changed, revisit cdevdvlpinmemdb.html. src/devguide/cdevdvlp40724.dita Should we mention that for some of the limits the maximum value is determined by the available main memory instead of disk space / file system, or is this too obvious to mention? src/devguide/cdevdvlpinmemdb.dita Removing an in-memory database: Does it make sense to swap the order of the current content, i.e. to describe how you would actively remove an in-memory db in a running JVM first? To me, the fact that in-memory databases are lost when the JVM exits is more like a side-effect. We do need to mention this, but maybe more like a word of caution to make it clear to users that in-memory databases are not persisted. Mention that if authentication and SQL authorization is enabled, only the database owner can drop a database. I'm not sure if we have done this for other actions, like shutting down the database. src/devguide/rdevdvlp22102.dita Add a link to a page describing, or just a note, that using the in-memory back end with the client driver requires a "special" URL (include the subsubprotocol as part of the name)? src/ref/rrefattrib16471.dita, src/ref/rrefattrib26867.dita Use the same database name for embedded and client/server for consistency in the examples? (currently 'myInMemDB' and 'MyInMemDB') src/ref/rrefattrib17246.dita In the example: "+<codeblock><b>jdbc:derby:toursDB;databaseName=flightsDB</b></codeblock>" - specifies two database names src/ref/rrefattribdrop.dita "This attribute, like shutdown=true, cannot be combined with other attributes." - with the exception of user credentials? Mention that using the drop attribute with other back ends results in an exception with SQLState XBM0I to be thrown? src/tools/rtoolsijcomref22318.dita and src/tools/rtoolsijcomref27997.dita Would it make sense to show that you can set the memory subsubprotocol as part of the protocol to create in-memory databases by default in ij? I.e: ij> protocol 'jdbc:derby:memory:'; ij> connect 'testDB;create=true'; src/tuning/ctunperfinmemdb.dita I would maybe change the wording a bit for the page cache size. There is nothing wrong with running with a smaller than default page cache, but the user may see poor performance. You don't get the penalty for going to the disk to fetch pages, but if the pressure on the page cache gets too high you will get a lot of contention and increased CPU usage. To me, the current wording suggests that I should increase the page cache size when using the in-memory db. In many cases this is not necessary. Can we say that we don't recommend to run with a smaller than default page cache? (keep the sentence about data passing through the cache) Finally, should we mention that users can use the backup routines to persist an in-memory database? This backup can then be booted as an in-memory database again at a later time, or simply be used as a normal disk based db. Or does suggesting this use case make some people uncomfortable? Thanks,
          Hide
          Kim Haase added a comment -

          Attaching a revised patch, DERBY-4525-2.diff, DERBY-4525-2.stat, and DERBY-4525-2.zip. This includes changes to one additional file, rrefattrib17246.dita, on the databaseName attribute, now that we know how to use it to specify an in-memory database (see DERBY-4581). I also added an example of the use of the databaseName attribute to the create attribute topic, and corrected the dev guide map file to reflect the title change of one of the dev guide topics.

          Show
          Kim Haase added a comment - Attaching a revised patch, DERBY-4525 -2.diff, DERBY-4525 -2.stat, and DERBY-4525 -2.zip. This includes changes to one additional file, rrefattrib17246.dita, on the databaseName attribute, now that we know how to use it to specify an in-memory database (see DERBY-4581 ). I also added an example of the use of the databaseName attribute to the create attribute topic, and corrected the dev guide map file to reflect the title change of one of the dev guide topics.
          Hide
          Kim Haase added a comment -

          Here's a first pass at a patch. Attaching DERBY-4525.diff, DERBY-4525.stat, and DERBY-4525.zip, with changes to 24 files (including the map files) and 5 of the 6 manuals. Most changes are relatively small. Here's a summary.

          Admin Guide:

          cadminappsclient.dita: Accessing the Network Server by using the network client driver

          Add memory to syntax statement and explain (w. xref to cdevdvlpinmemdb.dita).

          Dev Guide:

          cdevdvlp17453.dita: Derby JDBC database connection URL

          Add memory to subsubprotocol list

          cdevdvlp27610.dita: Derby system

          Mention that use of in-memory db means that no databases exist in the directory.

          cdevdvlp96597.dita: One Derby instance for each Java Virtual Machine

          Revise.

          Mention that if you specify the "same" in-memory db in two different instances of Derby it is not a problem because you actually get two different dbs in two different JVMs.

          cdevdvlp19297.dita: Recommended practices

          Fix next-to-last bullet to remove the recommendation against doing things that are actually impossible to do.

          cdevdvlp40724.dita: The database directory

          Mention that the directory info doesn't apply to in-memory dbs. I'm guessing the size limits still apply – let me know if not.

          cdevdvlp42173.dita: Creating, dropping, and backing up databases

          You can drop an in-memory db.

          cdevdvlp18166.dita: Storage and recovery

          Doesn't apply to in-memory dbs, does it?

          cdevdvlp846369.dita: Connecting to databases within the system

          Move the verb "is" to make it clear that this info is about file-system dbs only.

          cdevdvlp40350.dita: Conventions for specifying the database path

          Add memory to list of non-filesystem access methods.

          cdevdvlp19700.dita: Special database access

          Add note on in-memory and xref.

          rdevdvlp22102.dita: Database connection examples

          Add example of in-memory.

          cdevdvlpinmemdb.dita: Using in-memory databases

          New topic. Does placement seem okay?

          Ref Man:

          rrefattrib24612.dita: Setting attributes for the database connection URL

          Add mention of subsubprotocol

          rrefattrib26867.dita: create=true attribute

          Add examples of creating in-memory db.

          rrefattribdrop.dita: drop=true attribute

          New topic.

          rrefattrib16471.dita: shutdown=true attribute

          Add examples of shutting down in-memory db.

          Tools Guide:

          rtoolsijcomref27997.dita: Protocol command

          Add example of connecting to in-memory db.

          rtoolsijcomref22318.dita: Connect command

          Add example of connecting to in-memory db. Also add example of using PROTOCOL in command

          Tuning Guide:

          ctunperfinmemdb.dita: Configure Derby to use an in-memory database

          New topic.

          ctunperf25864.dita: The tips

          Add mention of tuning for in-memory db. Also corrected title of table-function topic, here and in map file.

          Show
          Kim Haase added a comment - Here's a first pass at a patch. Attaching DERBY-4525 .diff, DERBY-4525 .stat, and DERBY-4525 .zip, with changes to 24 files (including the map files) and 5 of the 6 manuals. Most changes are relatively small. Here's a summary. Admin Guide: cadminappsclient.dita: Accessing the Network Server by using the network client driver Add memory to syntax statement and explain (w. xref to cdevdvlpinmemdb.dita). Dev Guide: cdevdvlp17453.dita: Derby JDBC database connection URL Add memory to subsubprotocol list cdevdvlp27610.dita: Derby system Mention that use of in-memory db means that no databases exist in the directory. cdevdvlp96597.dita: One Derby instance for each Java Virtual Machine Revise. Mention that if you specify the "same" in-memory db in two different instances of Derby it is not a problem because you actually get two different dbs in two different JVMs. cdevdvlp19297.dita: Recommended practices Fix next-to-last bullet to remove the recommendation against doing things that are actually impossible to do. cdevdvlp40724.dita: The database directory Mention that the directory info doesn't apply to in-memory dbs. I'm guessing the size limits still apply – let me know if not. cdevdvlp42173.dita: Creating, dropping, and backing up databases You can drop an in-memory db. cdevdvlp18166.dita: Storage and recovery Doesn't apply to in-memory dbs, does it? cdevdvlp846369.dita: Connecting to databases within the system Move the verb "is" to make it clear that this info is about file-system dbs only. cdevdvlp40350.dita: Conventions for specifying the database path Add memory to list of non-filesystem access methods. cdevdvlp19700.dita: Special database access Add note on in-memory and xref. rdevdvlp22102.dita: Database connection examples Add example of in-memory. cdevdvlpinmemdb.dita: Using in-memory databases New topic. Does placement seem okay? Ref Man: rrefattrib24612.dita: Setting attributes for the database connection URL Add mention of subsubprotocol rrefattrib26867.dita: create=true attribute Add examples of creating in-memory db. rrefattribdrop.dita: drop=true attribute New topic. rrefattrib16471.dita: shutdown=true attribute Add examples of shutting down in-memory db. Tools Guide: rtoolsijcomref27997.dita: Protocol command Add example of connecting to in-memory db. rtoolsijcomref22318.dita: Connect command Add example of connecting to in-memory db. Also add example of using PROTOCOL in command Tuning Guide: ctunperfinmemdb.dita: Configure Derby to use an in-memory database New topic. ctunperf25864.dita: The tips Add mention of tuning for in-memory db. Also corrected title of table-function topic, here and in map file.
          Hide
          Kim Haase added a comment -

          Thanks for the advice, Knut. I will file an issue on the databaseName problem and, in general, not say anything about the in-memory database and connection attributes (except to add examples for the most common ones like create and shutdown).

          Show
          Kim Haase added a comment - Thanks for the advice, Knut. I will file an issue on the databaseName problem and, in general, not say anything about the in-memory database and connection attributes (except to add examples for the most common ones like create and shutdown).
          Hide
          Knut Anders Hatlen added a comment -

          The problem with databaseName looks like a bug.

          As to the replication properties, I think you are right that we haven't done anything actively to support replication of in-memory databases. That said, I was able to set up replication between two memory databases and issue a fail-over between them, and all the operations appeared to work as expected, so I'm not sure if the docs need to say anything special about the combination memory/replication.

          Show
          Knut Anders Hatlen added a comment - The problem with databaseName looks like a bug. As to the replication properties, I think you are right that we haven't done anything actively to support replication of in-memory databases. That said, I was able to set up replication between two memory databases and issue a fail-over between them, and all the operations appeared to work as expected, so I'm not sure if the docs need to say anything special about the combination memory/replication.
          Hide
          Kim Haase added a comment -

          I wonder if we need to say anything about what connection attributes work with the in-memory db and which don't. I think in most cases it would be intuitive – attributes having to do with replication wouldn't, would they? I would think most others would.

          One that I would expect to work doesn't though – databaseName. When I'm in ij, I can create an in-memory db, disconnect, and then connect again, and the db is still there. I can connect to it the usual way, but not using the databaseName attribute – I tried a couple of different ways:

          ij> connect 'jdbc:derby:memory:myDB;create=true';
          ij> create table t(c int);
          0 rows inserted/updated/deleted
          ij> disconnect;
          ij> connect 'jdbc:derby:memory:myDB';
          ij> insert into t values(1), (2);
          2 rows inserted/updated/deleted
          ij> select * from t;
          C
          -----------
          1
          2

          2 rows selected
          ij> disconnect;
          ij> connect 'jdbc:derby:memory;databaseName=myDB';
          ERROR XJ004: Database 'memory' not found.
          ij> connect 'jdbc:derby:memory:;databaseName=myDB';
          ERROR XJ004: Database 'memory:' not found.

          Show
          Kim Haase added a comment - I wonder if we need to say anything about what connection attributes work with the in-memory db and which don't. I think in most cases it would be intuitive – attributes having to do with replication wouldn't, would they? I would think most others would. One that I would expect to work doesn't though – databaseName. When I'm in ij, I can create an in-memory db, disconnect, and then connect again, and the db is still there. I can connect to it the usual way, but not using the databaseName attribute – I tried a couple of different ways: ij> connect 'jdbc:derby:memory:myDB;create=true'; ij> create table t(c int); 0 rows inserted/updated/deleted ij> disconnect; ij> connect 'jdbc:derby:memory:myDB'; ij> insert into t values(1), (2); 2 rows inserted/updated/deleted ij> select * from t; C ----------- 1 2 2 rows selected ij> disconnect; ij> connect 'jdbc:derby:memory;databaseName=myDB'; ERROR XJ004: Database 'memory' not found. ij> connect 'jdbc:derby:memory:;databaseName=myDB'; ERROR XJ004: Database 'memory:' not found.
          Hide
          Kim Haase added a comment -

          Well, the wrong-format error message may not be that serious an issue. But it does seem to be the case that you can drop an on-disk database, at least if you create it in a directory called "memory".

          I (accidentally, because of a punctuation error) create a db called myDB in the directory memory:

          ij> connect 'jdbc:derby://localhost:1527/memory/myDB;create=true';
          ij> create table t(c int);
          0 rows inserted/updated/deleted

          The database is visible in the file system:

          ls ~/DERBYTUTOR/*
          /home/chaase/DERBYTUTOR/derby.log

          /home/chaase/DERBYTUTOR/firstdb:
          log/ seg0/ service.properties

          /home/chaase/DERBYTUTOR/memory:
          myDB/

          /home/chaase/DERBYTUTOR/seconddb:
          log/ seg0/ service.properties

          The database is actually there:

          ls ~/DERBYTUTOR/memory/myDB
          db.lck log/ service.properties
          dbex.lck seg0/ tmp/

          I am then able to drop this database even though it's a file system db:

          ij> connect 'jdbc:derby://localhost:1527/memory/myDB;drop=true';
          ERROR 08006: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: Database 'memory/myDB' dropped.

          The database is gone:

          ls ~/DERBYTUTOR/*
          /home/chaase/DERBYTUTOR/derby.log

          /home/chaase/DERBYTUTOR/firstdb:
          log/ seg0/ service.properties

          /home/chaase/DERBYTUTOR/memory:

          /home/chaase/DERBYTUTOR/seconddb:
          log/ seg0/ service.properties

          I then try to drop another filesystem db, one that doesn't have "memory" in the name. This fails, as it should:

          ij> connect 'jdbc:derby://localhost:1527/seconddb;drop=true';
          ERROR XBM0I: DERBY SQL error: SQLCODE: -1, SQLSTATE: XBM0I, SQLERRMC: Directory seconddb cannot be removed.

          I then use the correct syntax to create the in-memory db. It is created, but in a new connection.

          ij> connect 'jdbc:derby://localhost:1527/memory:myDB;create=true';
          ij(CONNECTION1)> show connections;
          CONNECTION0 - jdbc:derby://localhost:1527/memory/myDB;create=true
          CONNECTION1* - jdbc:derby://localhost:1527/memory:myDB;create=true

          • = current connection

          I find that if I start over and create the filesystem db, I can shut it down and it stays there:

          ij> connect 'jdbc:derby://localhost:1527/memory/myDB;create=true';
          ij> connect 'jdbc:derby://localhost:1527/memory/myDB;shutdown=true';
          ERROR 08006: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: Database 'memory/myDB' shutdown.

          ls ~/DERBYTUTOR/memory
          myDB/

          This is as it should be. I shouldn't be able to drop it, though.

          Show
          Kim Haase added a comment - Well, the wrong-format error message may not be that serious an issue. But it does seem to be the case that you can drop an on-disk database, at least if you create it in a directory called "memory". I (accidentally, because of a punctuation error) create a db called myDB in the directory memory: ij> connect 'jdbc:derby://localhost:1527/memory/myDB;create=true'; ij> create table t(c int); 0 rows inserted/updated/deleted The database is visible in the file system: ls ~/DERBYTUTOR/* /home/chaase/DERBYTUTOR/derby.log /home/chaase/DERBYTUTOR/firstdb: log/ seg0/ service.properties /home/chaase/DERBYTUTOR/memory: myDB/ /home/chaase/DERBYTUTOR/seconddb: log/ seg0/ service.properties The database is actually there: ls ~/DERBYTUTOR/memory/myDB db.lck log/ service.properties dbex.lck seg0/ tmp/ I am then able to drop this database even though it's a file system db: ij> connect 'jdbc:derby://localhost:1527/memory/myDB;drop=true'; ERROR 08006: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: Database 'memory/myDB' dropped. The database is gone: ls ~/DERBYTUTOR/* /home/chaase/DERBYTUTOR/derby.log /home/chaase/DERBYTUTOR/firstdb: log/ seg0/ service.properties /home/chaase/DERBYTUTOR/memory: /home/chaase/DERBYTUTOR/seconddb: log/ seg0/ service.properties I then try to drop another filesystem db, one that doesn't have "memory" in the name. This fails, as it should: ij> connect 'jdbc:derby://localhost:1527/seconddb;drop=true'; ERROR XBM0I: DERBY SQL error: SQLCODE: -1, SQLSTATE: XBM0I, SQLERRMC: Directory seconddb cannot be removed. I then use the correct syntax to create the in-memory db. It is created, but in a new connection. ij> connect 'jdbc:derby://localhost:1527/memory:myDB;create=true'; ij(CONNECTION1)> show connections; CONNECTION0 - jdbc:derby://localhost:1527/memory/myDB;create=true CONNECTION1* - jdbc:derby://localhost:1527/memory:myDB;create=true = current connection I find that if I start over and create the filesystem db, I can shut it down and it stays there: ij> connect 'jdbc:derby://localhost:1527/memory/myDB;create=true'; ij> connect 'jdbc:derby://localhost:1527/memory/myDB;shutdown=true'; ERROR 08006: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: Database 'memory/myDB' shutdown. ls ~/DERBYTUTOR/memory myDB/ This is as it should be. I shouldn't be able to drop it, though.
          Hide
          Knut Anders Hatlen added a comment -

          It sounds like a bug to me if you're able to drop a file-system database on the server. (I didn't get that message myself when I tried the same commands on trunk. Instead, it looked like the drop attribute was ignored, and a new connection was opened. Still sounds like a bug, though.)

          The difference in the error messages comes from the way we process error messages on the client. In the normal case, the server sends the message id, the message translated to the server's locale and the arguments used to generate the message. The client passes this information and information about its own locale back to the server by invoking a stored procedure. The server then translates the message to the client's locale so that it can be displayed correctly. However, when CONNECT fails, there is no connection on which the stored procedure can be invoked in order to translate the message, so the client just dumps whatever information it got from the server in the first run.

          So, yes, the difference is expected, but probably not desirable. I'm not sure if we have a JIRA tracking this problem.

          Show
          Knut Anders Hatlen added a comment - It sounds like a bug to me if you're able to drop a file-system database on the server. (I didn't get that message myself when I tried the same commands on trunk. Instead, it looked like the drop attribute was ignored, and a new connection was opened. Still sounds like a bug, though.) The difference in the error messages comes from the way we process error messages on the client. In the normal case, the server sends the message id, the message translated to the server's locale and the arguments used to generate the message. The client passes this information and information about its own locale back to the server by invoking a stored procedure. The server then translates the message to the client's locale so that it can be displayed correctly. However, when CONNECT fails, there is no connection on which the stored procedure can be invoked in order to translate the message, so the client just dumps whatever information it got from the server in the first run. So, yes, the difference is expected, but probably not desirable. I'm not sure if we have a JIRA tracking this problem.
          Hide
          Kim Haase added a comment -

          Actually, that syntax was wrong – I accidentally put a slash instead of a colon, which created a file-system database named myDB in the directory memory. I shouldn't have been able to drop it, should I? I thought drop only worked on in-memory databases.

          The same error results if I do it right.

          ij> connect 'jdbc:derby://localhost:1527/memory:myDB;create=true';
          ij> connect 'jdbc:derby://localhost:1527/memory:myDB;drop=true';
          ERROR 08006: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: Database 'memory:myDB' dropped.

          Show
          Kim Haase added a comment - Actually, that syntax was wrong – I accidentally put a slash instead of a colon, which created a file-system database named myDB in the directory memory. I shouldn't have been able to drop it, should I? I thought drop only worked on in-memory databases. The same error results if I do it right. ij> connect 'jdbc:derby://localhost:1527/memory:myDB;create=true'; ij> connect 'jdbc:derby://localhost:1527/memory:myDB;drop=true'; ERROR 08006: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: Database 'memory:myDB' dropped.
          Hide
          Kim Haase added a comment -

          I find that if I use the Network Server and I drop an in-memory database, the error is a bit different from the embedded one:

          ij> connect 'jdbc:derby://localhost:1527/memory/myDB;create=true';
          ij> create table t(c int);
          0 rows inserted/updated/deleted
          ij> connect 'jdbc:derby://localhost:1527/memory/myDB;drop=true';
          ERROR 08006: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: Database 'memory/myDB' dropped.

          Is this expected?

          Show
          Kim Haase added a comment - I find that if I use the Network Server and I drop an in-memory database, the error is a bit different from the embedded one: ij> connect 'jdbc:derby://localhost:1527/memory/myDB;create=true'; ij> create table t(c int); 0 rows inserted/updated/deleted ij> connect 'jdbc:derby://localhost:1527/memory/myDB;drop=true'; ERROR 08006: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: Database 'memory/myDB' dropped. Is this expected?
          Hide
          Kim Haase added a comment -

          Thanks, Knut! I've linked to DERBY-4428 to have that info handy.

          It is true that the issues in the two topics are different – thanks for pointing that out. "One Derby instance for each Java Virtual Machine" is not the same thing as "One Derby instance for each database". So they should both stay. Maybe I'll file and fix the separate issue about removing the obsolete info from that topic first, before I deal with the in-memory DB doc.

          One question about DERBY-4428 – I notice when I try out the drop attribute in ij I get the following error:

          ij> CONNECT 'jdbc:derby:memory:dummy;drop=true';
          ERROR 08006: Database 'memory:dummy' dropped.

          This is the expected not-really-error when you shut down a database, so I guess it is getting yet one more overload – currently there are 8 messages associated with 08006 and now I guess there will be one more. It seems to appear both when you shut the db down and when you drop it. Is that correct?

          It doesn't seem to be necessary to shut the db down before you drop it – the drop seems to accomplish both.

          It appears that you can only use the drop attribute to drop an in-memory database, not a file-system one. If I try the latter I get

          ij> connect 'jdbc:derby:firstdb;drop=true';
          ERROR XBM0I: Directory firstdb cannot be removed.

          That seems like a useful safeguard.

          Show
          Kim Haase added a comment - Thanks, Knut! I've linked to DERBY-4428 to have that info handy. It is true that the issues in the two topics are different – thanks for pointing that out. "One Derby instance for each Java Virtual Machine" is not the same thing as "One Derby instance for each database". So they should both stay. Maybe I'll file and fix the separate issue about removing the obsolete info from that topic first, before I deal with the in-memory DB doc. One question about DERBY-4428 – I notice when I try out the drop attribute in ij I get the following error: ij> CONNECT 'jdbc:derby:memory:dummy;drop=true'; ERROR 08006: Database 'memory:dummy' dropped. This is the expected not-really-error when you shut down a database, so I guess it is getting yet one more overload – currently there are 8 messages associated with 08006 and now I guess there will be one more. It seems to appear both when you shut the db down and when you drop it. Is that correct? It doesn't seem to be necessary to shut the db down before you drop it – the drop seems to accomplish both. It appears that you can only use the drop attribute to drop an in-memory database, not a file-system one. If I try the latter I get ij> connect 'jdbc:derby:firstdb;drop=true'; ERROR XBM0I: Directory firstdb cannot be removed. That seems like a useful safeguard.
          Hide
          Knut Anders Hatlen added a comment -

          Hi Kim,

          DERBY-4428 contains more information about the new 'drop' attribute.

          +1 to remove the sections that only apply to JVMs that we no longer support, but that's probably better to address in a separate issue. (We may want to keep the first two paragraphs in some form to make sure that it's documented that you cannot boot two instances of the same database simultaneously.)

          Show
          Knut Anders Hatlen added a comment - Hi Kim, DERBY-4428 contains more information about the new 'drop' attribute. +1 to remove the sections that only apply to JVMs that we no longer support, but that's probably better to address in a separate issue. (We may want to keep the first two paragraphs in some form to make sure that it's documented that you cannot boot two instances of the same database simultaneously.)
          Hide
          Kim Haase added a comment -

          I have a couple of questions – first, about the list of topics above: What is "the new 'drop' JDBC connection URL attribute"? I cannot find it in either DERBY-646 or http://wiki.apache.org/db-derby/InMemoryBackEndPrimer.

          Also, I have been going through the Developer's Guide (http://db.apache.org/derby/docs/dev/devguide/) checking for topics that need to have information about the in-memory back end added. I think something should be said in "Derby system" and some of its subtopics. "One Derby instance for each Java Virtual Machine" (http://db.apache.org/derby/docs/dev/devguide/cdevdvlp96597.html) has a link to "Double-booting system behavior" (http://db.apache.org/derby/docs/dev/devguide/cdevdvlp20458.html). This second topic seems to be entirely obsolete, since it describes problems that can happen with a JVM earlier than 1.4.2, and Derby no longer supports such JVMs. Only the very last paragraph, about using the Network Server if you want more than one Derby instance to be able to access a database, is still current. Would it make sense to remove this topic from the manual and move this last paragraph to the topic "One Derby instance for each Java Virtual Machine"?

          Show
          Kim Haase added a comment - I have a couple of questions – first, about the list of topics above: What is "the new 'drop' JDBC connection URL attribute"? I cannot find it in either DERBY-646 or http://wiki.apache.org/db-derby/InMemoryBackEndPrimer . Also, I have been going through the Developer's Guide ( http://db.apache.org/derby/docs/dev/devguide/ ) checking for topics that need to have information about the in-memory back end added. I think something should be said in "Derby system" and some of its subtopics. "One Derby instance for each Java Virtual Machine" ( http://db.apache.org/derby/docs/dev/devguide/cdevdvlp96597.html ) has a link to "Double-booting system behavior" ( http://db.apache.org/derby/docs/dev/devguide/cdevdvlp20458.html ). This second topic seems to be entirely obsolete, since it describes problems that can happen with a JVM earlier than 1.4.2, and Derby no longer supports such JVMs. Only the very last paragraph, about using the Network Server if you want more than one Derby instance to be able to access a database, is still current. Would it make sense to remove this topic from the manual and move this last paragraph to the topic "One Derby instance for each Java Virtual Machine"?

            People

            • Assignee:
              Kim Haase
              Reporter:
              Kristian Waagan
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development