Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3482

Mapper only Jobs are not creating intermediate files in /tmp/, instead of creating in user directory.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Invalid
    • 0.11.1
    • 0.14.0
    • impl
    • None
    • RHEL 6.0

    • pig

    Description

      When we run Mapper only jobs, All the intermediate outputs(compressed) are going to the user directory instead of going to tmp. If we run on small datasets, it shouldn't create a problem. But when I run for large datasets like more than 100TB lets say, it taking up so much disk space exceeding the disk space quota(setSpaceQuota) of 100GB also. Problem is happening before clean up.

      Attachments

        Activity

          People

            Unassigned Unassigned
            ravitejach Raviteja Chirala
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: