Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Tools
    • Labels:
    • Issue & fix info:
      Workaround attached

      Description

      Using the command line tool

      org.apache.derby.tools.ij

      is user unfriendly.

      1. DERBY-1447-1a.patch
        6 kB
        Sylvain Leroux
      2. jline-0_9_5.jar
        45 kB
        Gary Orser
      3. jline.diff
        2 kB
        Gary Orser

        Activity

        Hide
        Gary Orser added a comment -

        jline has a BSD license. I think that is compatible with the Apache license.

        Show
        Gary Orser added a comment - jline has a BSD license. I think that is compatible with the Apache license.
        Hide
        John H. Embretsen added a comment -

        This would be a nice feature to add to ij. However...

        I know that emacs users already have a work-around for this, and I consider using JLine to be another work-around, being an extension/wrapper around Derby/ij. I think it would be better to have some command line history functionality in ij itself, than to include an external jar in the derby repository. IMHO, we should rather document this work-around somewhere (wiki / manuals), along with the emacs work-around and possibly others.

        I know that implementing this functionality in ij requires some more work, but just adding a command line history should be easier than implementing all the features provided by JLine (such as tab completion, line editing, custom keybindings, character masking), while adding great value to ij.

        Besides, JLine is not 100% Java...
        http://jline.sourceforge.net/#faq_pure_java

        Show
        John H. Embretsen added a comment - This would be a nice feature to add to ij. However... I know that emacs users already have a work-around for this, and I consider using JLine to be another work-around, being an extension/wrapper around Derby/ij. I think it would be better to have some command line history functionality in ij itself, than to include an external jar in the derby repository. IMHO, we should rather document this work-around somewhere (wiki / manuals), along with the emacs work-around and possibly others. I know that implementing this functionality in ij requires some more work, but just adding a command line history should be easier than implementing all the features provided by JLine (such as tab completion, line editing, custom keybindings, character masking), while adding great value to ij. Besides, JLine is not 100% Java... http://jline.sourceforge.net/#faq_pure_java
        Hide
        Kathey Marsden added a comment -

        I'll uncheck patch available based on John's review.
        I think instructions on how to do this would be a great addition to the Wiki but agree it is not good to add the jline dependency to the derby scripts.

        Show
        Kathey Marsden added a comment - I'll uncheck patch available based on John's review. I think instructions on how to do this would be a great addition to the Wiki but agree it is not good to add the jline dependency to the derby scripts.
        Hide
        Sylvain Leroux added a comment -

        The total lack of editing capabilities in ij is really annoying.

        To come back with a suggestion by John H. Embretsen above:
        > I know that implementing this functionality in ij requires some more work,
        > but just adding a command line history should be easier than implementing
        > all the features provided by JLine

        We could have just an history of the last few (all?) entered statements - and some kind of command to edit (and re-send) one of these statements using an external editor.

        Something like that:
        ij> HISTORY; – display the history
        1 CONNECT 'jdbc:derby:dummy';
        2 CREATE TABLE T(A INT, B int);
        3 INSERT INTO T(a,b) VALUE (1, 2),
        (3,4),
        (5,6);
        ij> EDIT 2; – edit the second statement using an external editor.
        – Re-runs the statement after successfull edit

        I'm not sure HISTORY and EDIT have to really be "statements". I think they should be handled before reaching the ij parser.

        Some command line utilities distinguish between "real" commands and pseudo-commands like that by using a special notation, like backslash-letter. It's a little bit esoteric, but could be an option to help the user remember those are "interactive commands", and such, couldn't/shouldn't appear in a batch script:
        ij> \h
        1 CONNECT 'jdbc:derby:dummy';
        2 CREATE TABLE T(A INT, B int);
        3 INSERT INTO T(a,b) VALUE (1, 2),
        (3,4),
        (5,6);
        ij> \e 2

        Show
        Sylvain Leroux added a comment - The total lack of editing capabilities in ij is really annoying. To come back with a suggestion by John H. Embretsen above: > I know that implementing this functionality in ij requires some more work, > but just adding a command line history should be easier than implementing > all the features provided by JLine We could have just an history of the last few (all?) entered statements - and some kind of command to edit (and re-send) one of these statements using an external editor. Something like that: ij> HISTORY; – display the history 1 CONNECT 'jdbc:derby:dummy'; 2 CREATE TABLE T(A INT, B int); 3 INSERT INTO T(a,b) VALUE (1, 2), (3,4), (5,6); ij> EDIT 2; – edit the second statement using an external editor. – Re-runs the statement after successfull edit I'm not sure HISTORY and EDIT have to really be "statements". I think they should be handled before reaching the ij parser. Some command line utilities distinguish between "real" commands and pseudo-commands like that by using a special notation, like backslash-letter. It's a little bit esoteric, but could be an option to help the user remember those are "interactive commands", and such, couldn't/shouldn't appear in a batch script: ij> \h 1 CONNECT 'jdbc:derby:dummy'; 2 CREATE TABLE T(A INT, B int); 3 INSERT INTO T(a,b) VALUE (1, 2), (3,4), (5,6); ij> \e 2
        Hide
        Kristian Waagan added a comment -

        For reference: http://wiki.apache.org/db-derby/CommandHistoryInIj

        I agree the missing history functionality, and no line editing, is annoying! I've learned to live with using JLine on *nix and Windows has the functionality (not sure at which level - is it Java or at a lower level?).
        If you go ahead implementing something, make sure wrapped lines (and multi-line commands) are handled... JLine doesn't handle that very nicely.

        Show
        Kristian Waagan added a comment - For reference: http://wiki.apache.org/db-derby/CommandHistoryInIj I agree the missing history functionality, and no line editing, is annoying! I've learned to live with using JLine on *nix and Windows has the functionality (not sure at which level - is it Java or at a lower level?). If you go ahead implementing something, make sure wrapped lines (and multi-line commands) are handled... JLine doesn't handle that very nicely.
        Hide
        Sylvain Leroux added a comment -

        Hi Kristian,

        and thanks for the comments!

        Here, I was targeting environment without JLine-like software available. For those cases, ij should provide some rudimentary editing capabilities available everywhere.

        But, for "advanced" editing features, JLine will still be the best option - however, it requires manual setup (due to incompatibilities between BSD and Apache licenses). And it is not 100% Java (it uses platform specific native code).


        At functional level, I agree with you: the history should handle properly multi-line commands an multi-command lines:
        ij> insert into T(a,b)
        values(1,2),
        (3,4);

        ij> insert into T(a,b) values(1,2); insert into T(a,b) values(3,4);

        Concerning parameters substitution as suggested in DERBY-4557 ($1, $2, ...), I think the history should hold the value before substitution.

        Show
        Sylvain Leroux added a comment - Hi Kristian, and thanks for the comments! Here, I was targeting environment without JLine-like software available. For those cases, ij should provide some rudimentary editing capabilities available everywhere. But, for "advanced" editing features, JLine will still be the best option - however, it requires manual setup (due to incompatibilities between BSD and Apache licenses). And it is not 100% Java (it uses platform specific native code). At functional level, I agree with you: the history should handle properly multi-line commands an multi-command lines: ij> insert into T(a,b) values(1,2), (3,4); ij> insert into T(a,b) values(1,2); insert into T(a,b) values(3,4); Concerning parameters substitution as suggested in DERBY-4557 ($1, $2, ...), I think the history should hold the value before substitution.
        Hide
        Sylvain Leroux added a comment -

        Attaching a version "1a" of a patch for this issue.

        I'm rather unsatisfied by what I have done here:
        First, ij has a contrived way of handling its input(s). So I finally had to hook the history/edit in the main ij parser. This has the disadvantage of adding two more tokens to the parser. And, since it works at statement-level, it made appear two statements on the same line as two different entries in the history (see below). A more robust approach could have been to use a specialized InputStream or to modify on org.apache.derby.impl.tools.ij.StatementFinder. But it seems such solutions would require a deep rewrite of ij's input handling facilities.

        Next, I use an external program to perform the edit. This works with a GUI editor (tested on Linux with gedit), but /not/ with console-based editors. The problem is that Runtime.getRuntime().exec() redirects the standard streams of the launched process. So terminal-based editors will not have direct access to the console.


        The patch introduces two new commands in ij:
        HISTORY – display the history of all the statements handled by ij
        EDIT n – Allow to edit a previous statement by its number

        The editor to use is defined by the ij.editor property.
        Please note it should be a /GUI/ editor - or use some 'launcher' to reset the standard I/O descriptor to the console.

        Here is an example (defining "gedit" as editor):
        sh$ java -Dij.editor=gedit org.apache.derby.tools.ij
        ij version 10.6
        ij> connect 'jdbc:derby:memory:dummy;create=true';
        ij> create table t(a int, bint);
        ERROR 42XA9: Column 'BINT' needs an explicit datatype. The datatype can be omitted only for columns with generation clauses.
        ij> history;
        0 connect 'jdbc:derby:memory:dummy;create=true';
        1 create table t(a int, bint);
        2 history;
        ij> edit 1;

            • At this point, the external editor is launched.
            • I made the required changes, save and quit
            • ij will now execute the modified statement
              ij> create table t(a int, b int);
              0 rows inserted/updated/deleted

        In the example below, you will see that two statements on the same line appear as two distinct statements in the history:
        ij> insert into t values (1, 100); insert into t values (2, 200);
        ij> history;
        0 connect 'jdbc:derby:memory:dummy;create=true';
        1 create table t(a int, bint);
        2 history;
        3 edit 1;
        4 create table t(a int, b int);
        5 insert into t values (1, 100);
        6 insert into t values (2, 200);
        7 history;

        Show
        Sylvain Leroux added a comment - Attaching a version "1a" of a patch for this issue. I'm rather unsatisfied by what I have done here: First, ij has a contrived way of handling its input(s). So I finally had to hook the history/edit in the main ij parser. This has the disadvantage of adding two more tokens to the parser. And, since it works at statement-level, it made appear two statements on the same line as two different entries in the history (see below). A more robust approach could have been to use a specialized InputStream or to modify on org.apache.derby.impl.tools.ij.StatementFinder. But it seems such solutions would require a deep rewrite of ij's input handling facilities. Next, I use an external program to perform the edit. This works with a GUI editor (tested on Linux with gedit), but /not/ with console-based editors. The problem is that Runtime.getRuntime().exec() redirects the standard streams of the launched process. So terminal-based editors will not have direct access to the console. The patch introduces two new commands in ij: HISTORY – display the history of all the statements handled by ij EDIT n – Allow to edit a previous statement by its number The editor to use is defined by the ij.editor property. Please note it should be a /GUI/ editor - or use some 'launcher' to reset the standard I/O descriptor to the console. Here is an example (defining "gedit" as editor): sh$ java -Dij.editor=gedit org.apache.derby.tools.ij ij version 10.6 ij> connect 'jdbc:derby:memory:dummy;create=true'; ij> create table t(a int, bint); ERROR 42XA9: Column 'BINT' needs an explicit datatype. The datatype can be omitted only for columns with generation clauses. ij> history; 0 connect 'jdbc:derby:memory:dummy;create=true'; 1 create table t(a int, bint); 2 history; ij> edit 1; At this point, the external editor is launched. I made the required changes, save and quit ij will now execute the modified statement ij> create table t(a int, b int); 0 rows inserted/updated/deleted In the example below, you will see that two statements on the same line appear as two distinct statements in the history: ij> insert into t values (1, 100); insert into t values (2, 200); ij> history; 0 connect 'jdbc:derby:memory:dummy;create=true'; 1 create table t(a int, bint); 2 history; 3 edit 1; 4 create table t(a int, b int); 5 insert into t values (1, 100); 6 insert into t values (2, 200); 7 history;
        Hide
        Sylvain Leroux added a comment -

        Currently, the patch is usable, but is more a workaround, than a real feature. And I'm not sure it is desirable to pursue in that direction.
        Since I have no more idea for this issue, I resign for now.

        Show
        Sylvain Leroux added a comment - Currently, the patch is usable, but is more a workaround, than a real feature. And I'm not sure it is desirable to pursue in that direction. Since I have no more idea for this issue, I resign for now.
        Hide
        Kristian Waagan added a comment -

        Found this old thread by coincidence: https://groups.google.com/forum/?fromgroups#!topic/jvm-languages/0uxWJdR79Xs[1-25]

        Haven't looked into this, but I was wondering if instead of bundling JLine with Derby we could make ij use it automatically if it's on the classpath. However, I think this will still require some environment setup, which makes the manual wrapper solution just as viable. Maybe it can be seen as a security issue too?

        Show
        Kristian Waagan added a comment - Found this old thread by coincidence: https://groups.google.com/forum/?fromgroups#!topic/jvm-languages/0uxWJdR79Xs[1-25 ] Haven't looked into this, but I was wondering if instead of bundling JLine with Derby we could make ij use it automatically if it's on the classpath. However, I think this will still require some environment setup, which makes the manual wrapper solution just as viable. Maybe it can be seen as a security issue too?
        Hide
        Glen Mazza added a comment -

        Incidentally, Apache Karaf's command line shell has this functionality, both up/down arrow and history--being not only Apache-licensed but Apache-owned code, there should be no legal problem with effectively taking it as-is and placing it in ij. However, while I like the up/down arrow functionality to repeat and/or edit recently made SQL statements, keeping a history file of commands which Karaf does would probably be ill-advisable, as sometimes information being inserted via SELECT/UPDATE, etc., statements can be sensitive/private and it wouldn't be good to keep that data in an unsecured file.

        Show
        Glen Mazza added a comment - Incidentally, Apache Karaf's command line shell has this functionality, both up/down arrow and history--being not only Apache-licensed but Apache-owned code, there should be no legal problem with effectively taking it as-is and placing it in ij. However, while I like the up/down arrow functionality to repeat and/or edit recently made SQL statements, keeping a history file of commands which Karaf does would probably be ill-advisable, as sometimes information being inserted via SELECT/UPDATE, etc., statements can be sensitive/private and it wouldn't be good to keep that data in an unsecured file.

          People

          • Assignee:
            Unassigned
            Reporter:
            Gary Orser
          • Votes:
            2 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development