Pig
  1. Pig
  2. PIG-2046

Properties defined through 'SET' are not passed through to fs commands

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.8.0, 0.9.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      The properties which are set through 'SET' commands are not passed through to FS commands.

      Ex;
      SET dfs.umaskmode '026'
      fs -touchz umasktest/file0

      It looks like the SET commands are processed by GruntParser after the FsShell creation happens with current set of properties. Hence whatever properties defined in SET will not be reflected for fs commands executed in the script.

        Activity

        Hide
        Kevin J. Price added a comment -

        Odd. It definitely works correctly if you set up a "pig-cluster-hadoop-site.xml" file in a conf directory and include it on the class path using -cp. That's the workaround I'm using right now.

        Show
        Kevin J. Price added a comment - Odd. It definitely works correctly if you set up a "pig-cluster-hadoop-site.xml" file in a conf directory and include it on the class path using -cp. That's the workaround I'm using right now.
        Hide
        Daniel Dai added a comment -

        I tried a test code:

        Configuration conf = new Configuration();
        conf.set("dfs.umaskmode", "026");
        FsShell shell = new FsShell(conf);
        shell.run(new String[] {"-touchz", "file0"});
        

        It also does not give the right result. Seems some issue related to FsShell. Will check with hdfs guys.

        Show
        Daniel Dai added a comment - I tried a test code: Configuration conf = new Configuration(); conf.set( "dfs.umaskmode" , "026" ); FsShell shell = new FsShell(conf); shell.run( new String [] { "-touchz" , "file0" }); It also does not give the right result. Seems some issue related to FsShell. Will check with hdfs guys.

          People

          • Assignee:
            Unassigned
            Reporter:
            Vivek Padmanabhan
          • Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development