Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.1
    • Fix Version/s: 0.23.2
    • Component/s: jobhistoryserver, mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      running pig job on oozie hangs due to race condition

      1. MAPREDUCE-3896-20120301.txt
        13 kB
        Vinod Kumar Vavilapalli
      2. MAPREDUCE-3896-20120228.txt
        2 kB
        Vinod Kumar Vavilapalli

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1008 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1008/)
        MAPREDUCE-3896. Add user information to the delegation token issued by the history server. (Contributed by Vinod Kumar Vavilapalli) (Revision 1296401)

        Result = SUCCESS
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296401
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1008 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1008/ ) MAPREDUCE-3896 . Add user information to the delegation token issued by the history server. (Contributed by Vinod Kumar Vavilapalli) (Revision 1296401) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296401 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #214 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/214/)
        merge MAPREDUCE-3896 from trunk (Revision 1296402)

        Result = FAILURE
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296402
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #214 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/214/ ) merge MAPREDUCE-3896 from trunk (Revision 1296402) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296402 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #973 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/973/)
        MAPREDUCE-3896. Add user information to the delegation token issued by the history server. (Contributed by Vinod Kumar Vavilapalli) (Revision 1296401)

        Result = SUCCESS
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296401
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #973 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/973/ ) MAPREDUCE-3896 . Add user information to the delegation token issued by the history server. (Contributed by Vinod Kumar Vavilapalli) (Revision 1296401) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296401 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #186 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/186/)
        merge MAPREDUCE-3896 from trunk (Revision 1296402)

        Result = SUCCESS
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296402
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #186 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/186/ ) merge MAPREDUCE-3896 from trunk (Revision 1296402) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296402 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Commit #629 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/629/)
        merge MAPREDUCE-3896 from trunk (Revision 1296402)

        Result = ABORTED
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296402
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #629 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/629/ ) merge MAPREDUCE-3896 from trunk (Revision 1296402) Result = ABORTED sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296402 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1829 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1829/)
        MAPREDUCE-3896. Add user information to the delegation token issued by the history server. (Contributed by Vinod Kumar Vavilapalli) (Revision 1296401)

        Result = ABORTED
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296401
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1829 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1829/ ) MAPREDUCE-3896 . Add user information to the delegation token issued by the history server. (Contributed by Vinod Kumar Vavilapalli) (Revision 1296401) Result = ABORTED sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296401 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-0.23-Commit #630 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/630/)
        merge MAPREDUCE-3896 from trunk (Revision 1296402)

        Result = SUCCESS
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296402
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Show
        Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #630 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/630/ ) merge MAPREDUCE-3896 from trunk (Revision 1296402) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296402 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1822 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1822/)
        MAPREDUCE-3896. Add user information to the delegation token issued by the history server. (Contributed by Vinod Kumar Vavilapalli) (Revision 1296401)

        Result = SUCCESS
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296401
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1822 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1822/ ) MAPREDUCE-3896 . Add user information to the delegation token issued by the history server. (Contributed by Vinod Kumar Vavilapalli) (Revision 1296401) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296401 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Commit #619 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/619/)
        merge MAPREDUCE-3896 from trunk (Revision 1296402)

        Result = SUCCESS
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296402
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #619 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/619/ ) merge MAPREDUCE-3896 from trunk (Revision 1296402) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296402 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1896 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1896/)
        MAPREDUCE-3896. Add user information to the delegation token issued by the history server. (Contributed by Vinod Kumar Vavilapalli) (Revision 1296401)

        Result = SUCCESS
        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296401
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1896 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1896/ ) MAPREDUCE-3896 . Add user information to the delegation token issued by the history server. (Contributed by Vinod Kumar Vavilapalli) (Revision 1296401) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296401 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/security/client/ClientHSSecurityInfo.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryClientService.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/TestJHSSecurity.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/krb5.conf
        Siddharth Seth made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Resolution Fixed [ 1 ]
        Hide
        Siddharth Seth added a comment -

        Committed to trunk, branch-0.23 and branch-0.23.2
        Thanks Vinod for the fix, and John for verifying it.

        Show
        Siddharth Seth added a comment - Committed to trunk, branch-0.23 and branch-0.23.2 Thanks Vinod for the fix, and John for verifying it.
        Hide
        Siddharth Seth added a comment -

        +1. Patch looks good.

        Show
        Siddharth Seth added a comment - +1. Patch looks good.
        Hide
        Amol Kekre added a comment -

        pls commit asap as this is a blocker for other components of Hadoop stack

        Show
        Amol Kekre added a comment - pls commit asap as this is a blocker for other components of Hadoop stack
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12516783/MAPREDUCE-3896-20120301.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1982//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1982//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12516783/MAPREDUCE-3896-20120301.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1982//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1982//console This message is automatically generated.
        Vinod Kumar Vavilapalli made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 0.24.0 [ 12317654 ]
        Affects Version/s 0.23.2 [ 12319851 ]
        Vinod Kumar Vavilapalli made changes -
        Attachment MAPREDUCE-3896-20120301.txt [ 12516783 ]
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Patch with a test.

        I couldn't exactly reproduce the error in the test case, but without the code fix, the test does fail, albeit with a different exception which still indicates the same underlying problem of missing user-name.

        The test passes after the code-fix.

        Show
        Vinod Kumar Vavilapalli added a comment - Patch with a test. I couldn't exactly reproduce the error in the test case, but without the code fix, the test does fail, albeit with a different exception which still indicates the same underlying problem of missing user-name. The test passes after the code-fix.
        Vinod Kumar Vavilapalli made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12516469/MAPREDUCE-3896-20120228.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1971//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1971//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12516469/MAPREDUCE-3896-20120228.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1971//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1971//console This message is automatically generated.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Thanks for the update John! It's time for the unit tests or atleast the time for investigation of the possibility of writing unit tests..

        Show
        Vinod Kumar Vavilapalli added a comment - Thanks for the update John! It's time for the unit tests or atleast the time for investigation of the possibility of writing unit tests..
        Arun C Murthy made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        John George added a comment -

        Vinod,
        Thanks for the patch.

        +1. Just finished testing this and the issue seems to have gone away with this fix. As you suggested, I commented out the sleep in MRAppMaster and that way was able to reproduce this much more consistently.

        Show
        John George added a comment - Vinod, Thanks for the patch. +1. Just finished testing this and the issue seems to have gone away with this fix. As you suggested, I commented out the sleep in MRAppMaster and that way was able to reproduce this much more consistently.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Still under testing, John is helping me validate this. We will try and resolve it by tomorrow.

        Show
        Vinod Kumar Vavilapalli added a comment - Still under testing, John is helping me validate this. We will try and resolve it by tomorrow.
        Hide
        Amol Kekre added a comment -

        Vinod, We need this asap. Can you complete the patch today . Thks, Amol

        Show
        Amol Kekre added a comment - Vinod, We need this asap. Can you complete the patch today . Thks, Amol
        Vinod Kumar Vavilapalli made changes -
        Fix Version/s 0.23.2 [ 12319851 ]
        Component/s jobhistoryserver [ 12315343 ]
        Vinod Kumar Vavilapalli made changes -
        Attachment MAPREDUCE-3896-20120228.txt [ 12516469 ]
        Hide
        Vinod Kumar Vavilapalli added a comment -

        I think I cornered this one, but have to be sure.

        John, given you already have the setup to test this, can you try running the cluster this patch and let me know?

        Will add tests post that, if possible.

        Show
        Vinod Kumar Vavilapalli added a comment - I think I cornered this one, but have to be sure. John, given you already have the setup to test this, can you try running the cluster this patch and let me know? Will add tests post that, if possible.
        Vinod Kumar Vavilapalli made changes -
        Assignee Mahadev konar [ mahadev ] Vinod Kumar Vavilapalli [ vinodkv ]
        Hide
        Mahadev konar added a comment -

        Tucu,
        This looks more like a Delegation Token issue. I dont think you should be seeing this in a unit test. Are you sure you see the same stack trace?

        Show
        Mahadev konar added a comment - Tucu, This looks more like a Delegation Token issue. I dont think you should be seeing this in a unit test. Are you sure you see the same stack trace?
        Hide
        Alejandro Abdelnur added a comment -

        it looks like i'm hitting a similar issue running Hive from Oozie.

        Show
        Alejandro Abdelnur added a comment - it looks like i'm hitting a similar issue running Hive from Oozie.
        Hide
        Mahadev konar added a comment -

        This is mostly related to DT in History Server. Will update in a bit.

        Show
        Mahadev konar added a comment - This is mostly related to DT in History Server. Will update in a bit.
        John George made changes -
        Field Original Value New Value
        Component/s mrv2 [ 12314301 ]
        Hide
        John George added a comment -

        Pig job through oozie hangs in certain cases due to a race condition. The exact scenario is as follows:
        Oozie launches a job which runs a mapper to launch a pig action. In certain cases, it can be seen that the AM (pig launcher) that started the mapper task is stuck waiting on getCounters(). The pig action itself had completed successfully. The mapper task that was stuck seemed to be talking to the JHS to get counters. JHS itself was throwing an AccessControlException.

        012-02-21 23:59:59,997 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 10020: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Can't retrieve username from tokenIdentifier. from client 98.137.97.247. Count of bytes read: 0
        Socket Reader #5 for port 10020org.apache.hadoop.security.AccessControlException: Can't retrieve username from tokenIdentifier.
        at org.apache.hadoop.ipc.Server$Connection.getAuthorizedUgi(Server.java:997)
        at org.apache.hadoop.ipc.Server$Connection.saslReadAndProcess(Server.java:1090)
        at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1235)
        at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:570)
        at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:369)
        at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)

        It seems that if the mapper task calls getCounter before the pig action AM dies, this oozie job succeeds, otherwise fails..

        Show
        John George added a comment - Pig job through oozie hangs in certain cases due to a race condition. The exact scenario is as follows: Oozie launches a job which runs a mapper to launch a pig action. In certain cases, it can be seen that the AM (pig launcher) that started the mapper task is stuck waiting on getCounters(). The pig action itself had completed successfully. The mapper task that was stuck seemed to be talking to the JHS to get counters. JHS itself was throwing an AccessControlException. 012-02-21 23:59:59,997 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 10020: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Can't retrieve username from tokenIdentifier. from client 98.137.97.247. Count of bytes read: 0 Socket Reader #5 for port 10020 org.apache.hadoop.security.AccessControlException: Can't retrieve username from tokenIdentifier. at org.apache.hadoop.ipc.Server$Connection.getAuthorizedUgi(Server.java:997) at org.apache.hadoop.ipc.Server$Connection.saslReadAndProcess(Server.java:1090) at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1235) at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:570) at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:369) at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344) It seems that if the mapper task calls getCounter before the pig action AM dies, this oozie job succeeds, otherwise fails..
        Hide
        Mahadev konar added a comment -

        @Alejandro,
        Still debugging.

        Show
        Mahadev konar added a comment - @Alejandro, Still debugging.
        Hide
        Alejandro Abdelnur added a comment -

        i've been this on and off on my setup, what is the exact race condition?

        Show
        Alejandro Abdelnur added a comment - i've been this on and off on my setup, what is the exact race condition?
        John George created issue -

          People

          • Assignee:
            Vinod Kumar Vavilapalli
            Reporter:
            John George
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development