Uploaded image for project: 'Apache Twill'
  1. Apache Twill
  2. TWILL-254

Update to use ContainerId.fromString

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.13.0
    • Component/s: yarn
    • Labels:
      None

      Description

      Hello, trying to use 0.12 or master, I get the following on a Hortonworks 2.6.1 (aka roughtly Hadoop 2.7.3) cluster. (One can avoid this by specifically building for Hadoop 2.6+ via mvn package -Dhadoop.version=2.6.0 but I was being a bit slow to realize what was up,) I have a patch to move us away from the deprecated (see YARN-1942) call in the stack trace anyhow and provide Hadoop 2.6 as a supported profile.

      Exception in thread "main" java.lang.reflect.InvocationTargetException
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:73)
      Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
              at com.google.common.base.Throwables.propagate(Throwables.java:160)
              at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61)
              at org.apache.twill.internal.appmaster.ApplicationMasterMain.doMain(ApplicationMasterMain.java:85)
              at org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:71)
              ... 5 more
      Caused by: java.lang.reflect.InvocationTargetException
              at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
              at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
              at org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:58)
              ... 7 more
      Caused by: java.lang.IllegalArgumentException: Invalid ContainerId: container_e164_1516738752188_0024_01_000001
              at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182)
              at org.apache.twill.internal.yarn.AbstractYarnAMClient.<init>(AbstractYarnAMClient.java:83)
              at org.apache.twill.internal.yarn.Hadoop21YarnAMClient.<init>(Hadoop21YarnAMClient.java:65)
              at org.apache.twill.internal.yarn.Hadoop22YarnAMClient.<init>(Hadoop22YarnAMClient.java:34)
              ... 12 more
      Caused by: java.lang.NumberFormatException: For input string: "e164"
              at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
              at java.lang.Long.parseLong(Long.java:589)
              at java.lang.Long.parseLong(Long.java:631)
              at org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137)
              at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177)
              ... 15 more
      

      For reference, it looks like YARN-2182 appears in Hadoop 2.6 to have changed container ID formats and a new API call came out ContainerId.fromString. Meanwhile, the container ID format change was not reflected, of course, in older versions of Hadoop as clarified in YARN-3575.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                clayb Clay B.
                Reporter:
                clayb Clay B.
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: