Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-1526

Oozie does not work with a secure HA JobTracker or ResourceManager

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: trunk, 3.3.1, 3.3.2, 4.0.0
    • Fix Version/s: 4.1.0
    • Component/s: security
    • Labels:
      None

      Description

      HadoopAccessorService#getMRTokenRenewerInternal handles getting the delegation token for the JT/RM. OOZIE-1159 modified this slightly and it now tries to parse the JT/RM address to get the hostname. However, if you try to use JT HA (which isn't in vanilla Hadoop), it uses a logical name (e.g. "ha-jt-uri") just like we do with HDFS HA. As such, when HadoopAccessorService tries to do

      String addr = NetUtils.createSocketAddr(target).getHostName();
      

      it will get an exception:

      Caused by: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: ha-jt-uri
      	at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:211)
      	at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:163)
      	at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:152)
      	at org.apache.oozie.service.HadoopAccessorService.getMRTokenRenewerInternal(HadoopAccessorService.java:484)
      	at org.apache.oozie.service.HadoopAccessorService.getMRDelegationTokenRenewer(HadoopAccessorService.java:463)
      	at org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:374)
      	at org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:991)
      	at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:743)
      	... 10 more
      

      because there isn't a host or port in the logical name.

      Once RM HA is done (which will be in vanilla Hadoop), it will use a logical name just like JT HA, and will run into the same problem. I think we can fix this by having it fall back to the old behavior when it gets the IllegalArgumentException.

        Attachments

        1. OOZIE-1526.patch
          3 kB
          Robert Kanter
        2. OOZIE-1526.patch
          3 kB
          Robert Kanter
        3. OOZIE-1526.patch
          3 kB
          Robert Kanter

          Issue Links

            Activity

              People

              • Assignee:
                rkanter Robert Kanter
                Reporter:
                rkanter Robert Kanter
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: