Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4874

Remove schema tuple reference overhead for replicate join hashmap

          Details

          • Type: Improvement
          • Status: Closed
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: None
          • Fix Version/s: 0.16.0
          • Component/s: None
          • Labels:
            None
          • Hadoop Flags:
            Reviewed

            Description

            Currently even if pig.schematuple is set to false which is the default, the usage of TupleToMapKey and TuplesToSchemaTupleList instead of plain HashMap<Object, ArrayList<Tuple>> costs a lot of memory. Also key is currently converted to a tuple which is unnecessary.

              Attachments

              1. PIG-4874-2.patch
                23 kB
                Rohini Palaniswamy
              2. PIG-4874-1.patch
                23 kB
                Rohini Palaniswamy

                Issue Links

                relates to

                Improvement - An improvement or enhancement to an existing feature or task. PIG-4875 Fix SchemaTuple for Tez

                • Major - Major loss of function.
                • Open

                  Activity

                    People

                    • Assignee:
                      rohini Rohini Palaniswamy
                      Reporter:
                      rohini Rohini Palaniswamy
                    • Votes:
                      0 Vote for this issue
                      Watchers:
                      3 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: