Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1071

should be possible to get long query results with no prompt

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Cannot Reproduce
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.9.0
    • Component/s: SQL Shell
    • Labels:
      None

      Description

      Hi,

      I need to be able to run queries in batch mode, however, when the query result is long, I get a message to that I need to conform in order to continue:

      continue... ('q' is quit)

      If I run this using my scheduler, the job gets stuck.
      Is it possible to add a way to override this limitation and get the entire query result back?

        Activity

        Hide
        hyunsik Hyunsik Choi added a comment -

        Niv Zvi,

        If you have used 0.8.0, I'm sure you would be amazed to know the stability and improvements of 0.9.0. Enjoy Tajo.

        Thanks,
        Hyunsik

        Show
        hyunsik Hyunsik Choi added a comment - Niv Zvi , If you have used 0.8.0, I'm sure you would be amazed to know the stability and improvements of 0.9.0. Enjoy Tajo. Thanks, Hyunsik
        Hide
        nivo999 Niv Zvi added a comment -

        Thanks!

        Show
        nivo999 Niv Zvi added a comment - Thanks!
        Hide
        hyunsik Hyunsik Choi added a comment -

        According to my survey, the same bug was reported and resolved few months ago.

        Also, tajo.cli.print.pause is used in interactive mode of tsql in order for easy scroll of results. So, you don't need tajo.cli.print.pause option when you use -c or -f option.

        Thanks,
        Hyunsik

        Show
        hyunsik Hyunsik Choi added a comment - According to my survey, the same bug was reported and resolved few months ago. Also, tajo.cli.print.pause is used in interactive mode of tsql in order for easy scroll of results. So, you don't need tajo.cli.print.pause option when you use -c or -f option. Thanks, Hyunsik
        Hide
        nivo999 Niv Zvi added a comment - - edited

        Hi Hyunsik,

        Sure, you can go ahead and close this, I just have one question.
        What is the nature of the fix? Will in 9, after the fix, by default if you use the command line you will not get this message (using -c or -f options)?
        Or by fixed you mean that in 9 I can set the below property and it work (like Jihoon suggested above)?
        <property>
        <name>tajo.cli.print.pause</name>
        <value>false</value>
        </property>

        Thanks,
        Niv

        Show
        nivo999 Niv Zvi added a comment - - edited Hi Hyunsik, Sure, you can go ahead and close this, I just have one question. What is the nature of the fix? Will in 9, after the fix, by default if you use the command line you will not get this message (using -c or -f options)? Or by fixed you mean that in 9 I can set the below property and it work (like Jihoon suggested above)? <property> <name>tajo.cli.print.pause</name> <value>false</value> </property> Thanks, Niv
        Hide
        hyunsik Hyunsik Choi added a comment - - edited

        Hi Niv Zvi,

        I've tested it with the master branch which will be released as 0.9.0 soon. This bug was fixed already. I also fixed other bugs that I found while testing it.

        bin/tsql -c "select * from orders"
        o_orderkey,  o_custkey,  o_orderstatus,  o_totalprice,  o_orderdate,  o_orderpriority,  o_clerk,  o_shippriority,  o_comment
        -------------------------------
        1,  36901,  O,  173665.47,  1996-01-02,  5-LOW,  Clerk#000000951,  0,  nstructions sleep furiously among 
        2,  78002,  O,  46929.18,  1996-12-01,  1-URGENT,  Clerk#000000880,  0,   foxes. pending accounts at the pending, silent asymptot
        ....
        ....
        5999975,  113398,  F,  63216.65,  1993-07-25,  1-URGENT,  Clerk#000000813,  0,  oost! ironic instructions h
        6000000,  110063,  O,  37625.29,  1996-08-31,  2-HIGH,  Clerk#000000411,  0,  ess pinto beans boost slyly regular accounts! furiously even
        (1500000 rows, 0.192 sec, 164.0 MiB selected)
        hyunsik@Hyunsiks-MacBook-Pro:tajo-0.9.0-SNAPSHOT$
        

        If there is no objection, I'll close this issue.

        Thanks,
        Hyunsik

        Show
        hyunsik Hyunsik Choi added a comment - - edited Hi Niv Zvi , I've tested it with the master branch which will be released as 0.9.0 soon. This bug was fixed already. I also fixed other bugs that I found while testing it. bin/tsql -c "select * from orders" o_orderkey, o_custkey, o_orderstatus, o_totalprice, o_orderdate, o_orderpriority, o_clerk, o_shippriority, o_comment ------------------------------- 1, 36901, O, 173665.47, 1996-01-02, 5-LOW, Clerk#000000951, 0, nstructions sleep furiously among 2, 78002, O, 46929.18, 1996-12-01, 1-URGENT, Clerk#000000880, 0, foxes. pending accounts at the pending, silent asymptot .... .... 5999975, 113398, F, 63216.65, 1993-07-25, 1-URGENT, Clerk#000000813, 0, oost! ironic instructions h 6000000, 110063, O, 37625.29, 1996-08-31, 2-HIGH, Clerk#000000411, 0, ess pinto beans boost slyly regular accounts! furiously even (1500000 rows, 0.192 sec, 164.0 MiB selected) hyunsik@Hyunsiks-MacBook-Pro:tajo-0.9.0-SNAPSHOT$ If there is no objection, I'll close this issue. Thanks, Hyunsik
        Hide
        nivo999 Niv Zvi added a comment -

        Thanks Hyunsik,

        Looking forward for your next update.

        Niv

        Show
        nivo999 Niv Zvi added a comment - Thanks Hyunsik, Looking forward for your next update. Niv
        Hide
        hyunsik Hyunsik Choi added a comment -

        Hi Niv,

        I'll dig into this problem. First of all, after I finish the current working issue, I'll check if this bug still occurs in master branch, which will be 0.9.0 release. I'm expecting that this issue does not take long time. Soon, I'll let you know how long it will take to resolve this issue.

        Thanks,
        Hyunsik

        Show
        hyunsik Hyunsik Choi added a comment - Hi Niv, I'll dig into this problem. First of all, after I finish the current working issue, I'll check if this bug still occurs in master branch, which will be 0.9.0 release. I'm expecting that this issue does not take long time. Soon, I'll let you know how long it will take to resolve this issue. Thanks, Hyunsik
        Hide
        nivo999 Niv Zvi added a comment -

        Hi Hyunsik Choi,

        I have deleted my last entry, it was regarding a different issue.
        To your question, the version I'm using is 0.8.0 and it was downloaded from a mirror, http://mirror.nbtelecom.com.br/apache/tajo/tajo-0.8.0/
        I have downloaded the distributable "tajo-0.8.0.tar.gz 30-Apr-2014 06:53 55M".
        Looking forward to hear from you.

        Thanks,
        Niv

        Show
        nivo999 Niv Zvi added a comment - Hi Hyunsik Choi, I have deleted my last entry, it was regarding a different issue. To your question, the version I'm using is 0.8.0 and it was downloaded from a mirror, http://mirror.nbtelecom.com.br/apache/tajo/tajo-0.8.0/ I have downloaded the distributable "tajo-0.8.0.tar.gz 30-Apr-2014 06:53 55M". Looking forward to hear from you. Thanks, Niv
        Hide
        hyunsik Hyunsik Choi added a comment -

        Hi Niv Zvi,

        I need more information. Are you using 0.8.0 release or master branch?

        Show
        hyunsik Hyunsik Choi added a comment - Hi Niv Zvi , I need more information. Are you using 0.8.0 release or master branch?
        Hide
        nivo999 Niv Zvi added a comment - - edited

        Hi,

        I followed the instructions at http://tajo.apache.org/docs/0.8.0/configuration/preliminary.html and created a config file under conf directory.
        It's content now looks as follows:

        <?xml version="1.0"?>
        <!--
          Licensed to the Apache Software Foundation (ASF) under one
          or more contributor license agreements.  See the NOTICE file
          distributed with this work for additional information
          regarding copyright ownership.  The ASF licenses this file
          to you under the Apache License, Version 2.0 (the
          "License"); you may not use this file except in compliance
          with the License.  You may obtain a copy of the License at
        
              http://www.apache.org/licenses/LICENSE-2.0
        
          Unless required by applicable law or agreed to in writing, software
          distributed under the License is distributed on an "AS IS" BASIS,
          WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          See the License for the specific language governing permissions and
          limitations under the License.
          -->
        
        <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
        
        <configuration>
        <property>
          <name>tajo.cli.print.pause</name>
          <value>false</value>
        </property>
        
        </configuration>
        

        I have also stopped and started Tajo but still getting the same results.
        What am I missing?

        $ ls -l
        total 28
        -rw-rw-r--. 1 agmap agmap  909 Apr 23 12:38 catalog-site.xml.template
        -rw-rw-r--. 1 agmap agmap 1329 Apr 23 12:38 configuration.xsl
        -rw-rw-r--. 1 agmap agmap 2521 Apr 23 12:38 log4j.properties
        -rwxrwxr-x. 1 agmap agmap 2594 Sep 23 17:56 tajo-env.sh
        -rwxr-xr-x. 1 agmap agmap  993 Sep 26 13:15 tajo-site.xml
        -rw-rw-r--. 1 agmap agmap  993 Sep 26 13:12 tajo-site.xml.template
        -rw-rw-r--. 1 agmap agmap    9 Apr 23 12:38 workers
        

        Thanks,
        Niv

        Show
        nivo999 Niv Zvi added a comment - - edited Hi, I followed the instructions at http://tajo.apache.org/docs/0.8.0/configuration/preliminary.html and created a config file under conf directory. It's content now looks as follows: <?xml version="1.0"?> <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>tajo.cli.print.pause</name> <value>false</value> </property> </configuration> I have also stopped and started Tajo but still getting the same results. What am I missing? $ ls -l total 28 -rw-rw-r--. 1 agmap agmap 909 Apr 23 12:38 catalog-site.xml.template -rw-rw-r--. 1 agmap agmap 1329 Apr 23 12:38 configuration.xsl -rw-rw-r--. 1 agmap agmap 2521 Apr 23 12:38 log4j.properties -rwxrwxr-x. 1 agmap agmap 2594 Sep 23 17:56 tajo-env.sh -rwxr-xr-x. 1 agmap agmap 993 Sep 26 13:15 tajo-site.xml -rw-rw-r--. 1 agmap agmap 993 Sep 26 13:12 tajo-site.xml.template -rw-rw-r--. 1 agmap agmap 9 Apr 23 12:38 workers Thanks, Niv
        Hide
        jihoonson Jihoon Son added a comment -

        It's strange.. In my machine, it works well.

        Anyway, there is an alternative.
        You can add the following configuration to tajo-site.xml

        <property>
          <name>tajo.cli.print.pause</name>
          <value>false</value>
        </property>
        

        We definitely need to add this configuration to our documents, but have not been yet.
        Sorry for confusing you.

        Sincerely,
        Jihoon

        Show
        jihoonson Jihoon Son added a comment - It's strange.. In my machine, it works well. Anyway, there is an alternative. You can add the following configuration to tajo-site.xml <property> <name>tajo.cli.print.pause</name> <value>false</value> </property> We definitely need to add this configuration to our documents, but have not been yet. Sorry for confusing you. Sincerely, Jihoon
        Hide
        nivo999 Niv Zvi added a comment - - edited

        Hi,

        Thanks for getting back to me!
        I used -c like in your example, however, if you have more than 24 hits on the query results, it blocks:

        $ tsql -c "select * from supplier"
        result: unknown rows (0 B)
        s_suppkey,  s_name,  s_address,  s_nationkey,  s_phone,  s_acctbal,  s_comment
        -------------------------------
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        1,  abc,  def,  1500,  ghi,  100.98,  jkl
        continue... ('q' is quit)
        
        Show
        nivo999 Niv Zvi added a comment - - edited Hi, Thanks for getting back to me! I used -c like in your example, however, if you have more than 24 hits on the query results, it blocks: $ tsql -c "select * from supplier" result: unknown rows (0 B) s_suppkey, s_name, s_address, s_nationkey, s_phone, s_acctbal, s_comment ------------------------------- 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl 1, abc, def, 1500, ghi, 100.98, jkl continue... ('q' is quit)
        Hide
        jihoonson Jihoon Son added a comment -

        Hi Niv,
        thanks for your interest in Tajo.

        If you want to get the entire query results, you can run the tsql with options of '-c' and '-f' as follows.

        $ bin/tsql -c "select * from region"
        r_regionkey,  r_name,  r_comment
        -------------------------------
        0,  AFRICA,  lar deposits. blithely final packages cajole. regular waters are final requests. regular accounts are according to 
        1,  AMERICA,  hs use ironic, even requests. s
        2,  ASIA,  ges. thinly even pinto beans ca
        3,  EUROPE,  ly final courts cajole furiously final excuse
        4,  MIDDLE EAST,  uickly special accounts cajole carefully blithely close requests. carefully final asymptotes haggle furiousl
        (5 rows, 0.095 sec, 384 B selected)
        
        $ cat test.sql
        select * from region
        $ bin/tsql -f test.sql
        r_regionkey,  r_name,  r_comment
        -------------------------------
        0,  AFRICA,  lar deposits. blithely final packages cajole. regular waters are final requests. regular accounts are according to 
        1,  AMERICA,  hs use ironic, even requests. s
        2,  ASIA,  ges. thinly even pinto beans ca
        3,  EUROPE,  ly final courts cajole furiously final excuse
        4,  MIDDLE EAST,  uickly special accounts cajole carefully blithely close requests. carefully final asymptotes haggle furiousl
        (5 rows, 0.095 sec, 384 B selected)
        

        If these options are used, the query results are printed without blocking.
        You can get more detailed information in http://tajo.apache.org/docs/current/cli.html.
        Thanks!

        Show
        jihoonson Jihoon Son added a comment - Hi Niv, thanks for your interest in Tajo. If you want to get the entire query results, you can run the tsql with options of '-c' and '-f' as follows. $ bin/tsql -c "select * from region" r_regionkey, r_name, r_comment ------------------------------- 0, AFRICA, lar deposits. blithely final packages cajole. regular waters are final requests. regular accounts are according to 1, AMERICA, hs use ironic, even requests. s 2, ASIA, ges. thinly even pinto beans ca 3, EUROPE, ly final courts cajole furiously final excuse 4, MIDDLE EAST, uickly special accounts cajole carefully blithely close requests. carefully final asymptotes haggle furiousl (5 rows, 0.095 sec, 384 B selected) $ cat test.sql select * from region $ bin/tsql -f test.sql r_regionkey, r_name, r_comment ------------------------------- 0, AFRICA, lar deposits. blithely final packages cajole. regular waters are final requests. regular accounts are according to 1, AMERICA, hs use ironic, even requests. s 2, ASIA, ges. thinly even pinto beans ca 3, EUROPE, ly final courts cajole furiously final excuse 4, MIDDLE EAST, uickly special accounts cajole carefully blithely close requests. carefully final asymptotes haggle furiousl (5 rows, 0.095 sec, 384 B selected) If these options are used, the query results are printed without blocking. You can get more detailed information in http://tajo.apache.org/docs/current/cli.html . Thanks!

          People

          • Assignee:
            hyunsik Hyunsik Choi
            Reporter:
            nivo999 Niv Zvi
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development