Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-2988

Use of XMLEncoder to serialize MapredWork causes OOM in hive cli

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: CLI
    • Labels:

      Description

      When running queries on tables with 6000 partitions, hive cli if configured with 128M runs into OOM. Heapdump showed 37MB occupied by one XMLEncoder object while the MapredWork was 500K which is highly inefficient. We should switch to using something more efficient like XStream.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              rohini Rohini Palaniswamy
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: