[MAPREDUCE-1224] Calling "SELECT t.* from <table> AS t" to get meta information is too expensive for big tables

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.20.1

Environment

all platforms, generic jdbc driver

• Hadoop Flags: Reviewed
• Assignee: Spencer Ho
• Reporter: Spencer Ho
• Created: 20/Nov/09 00:32
• Updated: 02/Jul/10 06:31
• Resolved: 15/Dec/09 15:05

Description

The SqlManager uses the query, "SELECT t.* from <table> AS t" to get table spec is too expensive for big tables, and it was called twice to generate column names and types. For tables that are big enough to be map-reduced, this is too expensive to make sqoop useful.