Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4183

select * to console issues in Hive

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.19.0
    • None
    • None

    Description

      the biggest problem is that select * is assuming that the output is a regular text file. it can't read out from compressed text files. (which is what happens when we start using the output compression options).

      the core issue is that select * is special code. and it should not be. it should go through the same 'fileinputformat' and serde code that any map-side task goes through.

      the second issue is that a select * without any data transformations/filters goes through a map-reduce task unnecessarily. we need to fix this. this seems related to the limit N jira that ashish opened. (see hadoop-4086)

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            jsensarma Joydeep Sen Sarma
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment