Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3446 Umbrella jira for Pig on Tez
  3. PIG-4039

New interface for resetting static variables for jvm reuse

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.14.0
    • tez
    • None
    • Reviewed

    Description

      In Tez, when there is container reuse, static variables, thread locals, etc have to be reinitialized to avoid memory leak or having wrong values. For the short term, we ended up making some of the static variables public or adding destroy method to each class which is hacky. Also it does not help users who will want something similar to be done in their UDFs or LoadFunc. Need to define a interface with a reset/destroy method and find all loaded classes implementing that interface and call destroy on them in PigProcessor.close(). ServiceLoader and annotations are some of the ways to find classes implementing an interface and there are other libraries as well. Need to find the best and fastest way to do that.

      Attachments

        1. PIG-4039-1.patch
          20 kB
          Rohini Palaniswamy
        2. PIG-4039-2.patch
          20 kB
          Rohini Palaniswamy

        Activity

          People

            rohini Rohini Palaniswamy
            rohini Rohini Palaniswamy
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: