Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-16837

Spark-SQL test running on Windows: hadoop chgrp warnings

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.6.5
    • Fix Version/s: None
    • Component/s: common, fs
    • Labels:
      None
    • Environment:

      Description

      Running SparkSQL local embedded unit tests on Win10, using winutils.

      Got warnings about 'hadoop chgrp'.

      See environment info.

      -chgrp: 'TEST\Domain users' does not match expected pattern for group
      Usage: hadoop fs [generic options] -chgrp [-R] GROUP PATH...
      -chgrp: 'TEST\Domain users' does not match expected pattern for group
      Usage: hadoop fs [generic options] -chgrp [-R] GROUP PATH...
      -chgrp: 'TEST\Domain users' does not match expected pattern for group
      Usage: hadoop fs [generic options] -chgrp [-R] GROUP PATH...
      

      Related info on SO: https://stackoverflow.com/questions/48605907/error-in-pyspark-when-insert-data-in-hive

      hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsShellPermissions.java:210

      The problem is: backslash character isn't included to allowedChars, see attached HadoopGroupTest.java

      Original issue in Spark: https://issues.apache.org/jira/browse/SPARK-30701

        Attachments

        1. HadoopGroupTest.java
          1 kB
          Guram Savinov

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                gsavinov Guram Savinov
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: