Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11, 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      To reproduce, please do the following:

      1) ensure that no hadoop is installed and therefore no hadoop classes are present in classpath.
      2) ant clean jar -Dhadoopversion=23
      3) ./bin/pig -x local
      4) fail with the following error: (you may not see this if slf4j is available in classpath.)

      cheolsoo@localhost:~/workspace/pig-trunk $cat  /home/cheolsoo/workspace/pig-trunk/pig_1350687456711.log
      Error before Pig is launched
      ----------------------------
      ERROR 2998: Unhandled internal error. org/slf4j/LoggerFactory
      
      java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
      ...
      

      5) cp ./build/ivy/lib/Pig/slf4j-api-1.6.1.jar lib
      6) ./bin/pig -x local
      7) fail with the following error:

      cheolsoo@localhost:~/workspace/pig-trunk $cat /home/cheolsoo/workspace/pig-trunk/pig_1350687052995.log
      Error before Pig is launched
      ----------------------------
      ERROR 2999: Unexpected internal error. Failed to create DataStorage
      
      java.lang.RuntimeException: Failed to create DataStorage
      	at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)
      	at org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58)
      	at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:204)
      	at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:117)
      	at org.apache.pig.impl.PigContext.connect(PigContext.java:229)
      	at org.apache.pig.PigServer.<init>(PigServer.java:213)
      	at org.apache.pig.PigServer.<init>(PigServer.java:198)
      	at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:47)
      	at org.apache.pig.Main.run(Main.java:535)
      	at org.apache.pig.Main.main(Main.java:154)
      Caused by: java.io.IOException: No FileSystem for scheme: file
      	at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2130)
      	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2137)
      	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
      	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2176)
      	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2158)
      	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:302)
      	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:158)
      	at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72)
      	... 9 more
      ================================================================================
      

      8) cp ./build/ivy/lib/Pig/hadoop-common-2.0.0-alpha.jar lib
      9) ./bin/pig -x local
      10) works fine!

      In fact, this is also an issue with running e2e test in local mode:

      ant clean
      ant -Dharness.old.pig=old_pig -Dharness.cluster.conf=hadoop_conf_dir -Dharness.cluster.bin=hadoop_script test-e2e-deploy-local -Dhadoopversion=23
      ant -Dharness.old.pig=old_pig -Dharness.cluster.conf=hadoop_conf_dir -Dharness.cluster.bin=hadoop_script test-e2e-local -Dhadoopversion=23
      

      The ant test-e2e-local fails with the following error:

      java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
              at org.apache.hadoop.security.authentication.util.KerberosName.<clinit>(KerberosName.java:42)
              at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:211)
              at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:274)
              at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:531)
              at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:512)
      

        Attachments

        1. PIG-2979.patch
          1 kB
          Cheolsoo Park

          Issue Links

            Activity

              People

              • Assignee:
                cheolsoo Cheolsoo Park
                Reporter:
                cheolsoo Cheolsoo Park
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: