Hive
  1. Hive
  2. HIVE-302

Implement "LINES TERMINATED BY"

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.5.0
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed

      Description

      Specifying "LINES TERMINATED BY" when creating a table currently doesn't do anything when querying that data. It needs to be implemented to support various datasets that ends lines with other characters then just line break.

      1. HIVE-302.1.patch
        1 kB
        Zheng Shao
      2. HIVE-302.2.patch
        4 kB
        Zheng Shao
      3. HIVE-302.3.patch
        20 kB
        Zheng Shao

        Issue Links

          Activity

          Johan Oskarsson created issue -
          Raghotham Murthy made changes -
          Field Original Value New Value
          Issue Type Improvement [ 4 ] Bug [ 1 ]
          Johan Oskarsson made changes -
          Fix Version/s 0.4.0 [ 12313714 ]
          Fix Version/s 0.3.0 [ 12313637 ]
          Johan Oskarsson made changes -
          Fix Version/s 0.5.0 [ 12314156 ]
          Fix Version/s 0.4.0 [ 12313714 ]
          Hide
          Namit Jain added a comment -

          create table nzhang_ctas5 row format delimited fields terminated by ',' lines terminated by '.' stored as textfile as select key, value from src sort by key, value limit 10;

          select * from nzhang_ctas5;

          Modified ctas.q to add the following:

          Got only 1 row as the output.

          Show
          Namit Jain added a comment - create table nzhang_ctas5 row format delimited fields terminated by ',' lines terminated by '.' stored as textfile as select key, value from src sort by key, value limit 10; select * from nzhang_ctas5; Modified ctas.q to add the following: Got only 1 row as the output.
          Namit Jain made changes -
          Assignee Zheng Shao [ zshao ]
          Hide
          Zheng Shao added a comment -

          This is not fixable currently because the line terminator is determined by LineRecordReader.LineReader which is in the Hadoop land.
          However we do support writing to such tables.

          In order to avoid confusion, I will just drop this from the syntax.

          Show
          Zheng Shao added a comment - This is not fixable currently because the line terminator is determined by LineRecordReader.LineReader which is in the Hadoop land. However we do support writing to such tables. In order to avoid confusion, I will just drop this from the syntax.
          Zheng Shao made changes -
          Attachment HIVE-302.1.patch [ 12428673 ]
          Zheng Shao made changes -
          Hadoop Flags [Incompatible change]
          Hide
          Namit Jain added a comment -

          talked with Zheng offline - it might be easier to specify a semantic error instead for compatiblity

          Show
          Namit Jain added a comment - talked with Zheng offline - it might be easier to specify a semantic error instead for compatiblity
          Hide
          He Yongqiang added a comment -

          +1. Looks good. Will commit if tests pass.

          Show
          He Yongqiang added a comment - +1. Looks good. Will commit if tests pass.
          Hide
          He Yongqiang added a comment -

          oh, sorry, ignore my previous comment.

          Show
          He Yongqiang added a comment - oh, sorry, ignore my previous comment.
          Hide
          Zheng Shao added a comment -

          This one keeps the old syntax but will throw out an error in case anything breaks.

          Show
          Zheng Shao added a comment - This one keeps the old syntax but will throw out an error in case anything breaks.
          Zheng Shao made changes -
          Attachment HIVE-302.2.patch [ 12428694 ]
          Zheng Shao made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Namit Jain added a comment -

          ctas.q has a CTAS with line delimiter specified - that also breaks, the tests need to be modified

          Show
          Namit Jain added a comment - ctas.q has a CTAS with line delimiter specified - that also breaks, the tests need to be modified
          Hide
          Zheng Shao added a comment -

          Fixed ctas.q and also tested all test cases.

          Show
          Zheng Shao added a comment - Fixed ctas.q and also tested all test cases.
          Zheng Shao made changes -
          Attachment HIVE-302.3.patch [ 12428757 ]
          Hide
          Namit Jain added a comment -

          +1

          looks good - will commit if the tests pass

          Show
          Namit Jain added a comment - +1 looks good - will commit if the tests pass
          Hide
          Namit Jain added a comment -

          Committed. Thanks Zheng

          Show
          Namit Jain added a comment - Committed. Thanks Zheng
          Namit Jain made changes -
          Resolution Fixed [ 1 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Incompatible change] [Incompatible change, Reviewed]
          Josh Patterson made changes -
          Link This issue is related to HIVE-1898 [ HIVE-1898 ]
          Carl Steinbach made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          300d 12h 34m 1 Zheng Shao 22/Dec/09 05:33
          Patch Available Patch Available Resolved Resolved
          17h 44m 1 Namit Jain 22/Dec/09 23:17
          Resolved Resolved Closed Closed
          724d 48m 1 Carl Steinbach 17/Dec/11 00:05

            People

            • Assignee:
              Zheng Shao
              Reporter:
              Johan Oskarsson
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development