Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-837

virtual column support (filename) in hive

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Query Processor
    • Labels:
      None

      Description

      Copying from some mails:

      I am dumping files into a hive partion on five minute intervals. I am using LOAD DATA into a partition.

      weblogs
      web1.00
      web1.05
      web1.10
      ...
      web2.00
      web2.05
      web1.10
      ....

      Things that would be useful..

      Select files from the folder with a regex or exact name

      select * FROM logs where FILENAME LIKE(WEB1*)

      select * FROM LOGS WHERE FILENAME=web2.00

      Also it would be nice to be able to select offsets in a file, this would make sense with appends

      select * from logs WHERE FILENAME=web2.00 FROMOFFSET=454644 [TOOFFSET=]

      select
      substr(filename, 4, 7) as class_A,
      substr(filename, 8, 10) as class_B
      count( x ) as cnt
      from FOO
      group by
      substr(filename, 4, 7),
      substr(filename, 8, 10) ;

      Hive should support virtual columns

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                he yongqiang He Yongqiang
                Reporter:
                namit Namit Jain
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: