Pig
  1. Pig
  2. PIG-3441

Allow Pig to use default resources from Configuration objects

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.11.1
    • Fix Version/s: None
    • Component/s: impl
    • Labels:
      None

      Description

      Pig currently ignores parameters from configuration files added statically to Configuration objects as Configuration.addDefaultResource(filename.xml).

      Consider the following scenario -
      In a hadoop FileSystem driver for a non-HDFS filesystem you load properties specific to that FileSystem in a static initializer block in the class that extends org.apache.hadoop.fs.Filesystem for your FileSystem like below -

      
      class MyFileSystem extends FileSystem {
      
              static {
      		Configuration.addDefaultResource("myfs-default.xml");
      		Configuration.addDefaultResource("myfs-site.xml");
      	}
      
      
      }
      

      Interfaces like the Hadoop CLI, Hive, Hadoop M/R can find configuration parameters defined in these configuration files as long as they are on the classpath.

      However, Pig cannot find parameters from these files, because it ignores configuration files added statically.

      Pig should allow users to specify if they would like pig to read parameters from resources loaded statically.

      1. PIG-3441.patch
        4 kB
        Bhooshan Mogal
      2. PIG-3441_1.patch
        4 kB
        Bhooshan Mogal
      3. PIG-3441-2.patch
        17 kB
        Daniel Dai
      4. PIG-3441-3.patch
        18 kB
        Daniel Dai

        Activity

        Daniel Dai made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Daniel Dai made changes -
        Attachment PIG-3441-3.patch [ 12666377 ]
        Daniel Dai made changes -
        Assignee Daniel Dai [ daijy ]
        Daniel Dai made changes -
        Attachment PIG-3441-2.patch [ 12665846 ]
        Bhooshan Mogal made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Bhooshan Mogal made changes -
        Attachment PIG-3441_1.patch [ 12600330 ]
        Bhooshan Mogal made changes -
        Attachment PIG-3441.patch_1 [ 12600329 ]
        Bhooshan Mogal made changes -
        Attachment PIG-3441.patch_1 [ 12600329 ]
        Bhooshan Mogal made changes -
        Field Original Value New Value
        Attachment PIG-3441.patch [ 12600322 ]
        Bhooshan Mogal created issue -

          People

          • Assignee:
            Daniel Dai
            Reporter:
            Bhooshan Mogal
          • Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:

              Development