Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-1634

EC2 launch-hadoop-cluster awk Problem

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.13.0
    • 0.14.0
    • contrib/cloud
    • None
    • ec2-api-tools-1.2-11797

    Description

      With Amazon 'ec2-api-tools-1.2-11797' and Hadoop 0.13.0 the output from ec2-describe-instances is being incorrectly AWK'd on the last few lines of the launch-hadoop-cluster script.

      Specifically, on the
      """
      MASTER_EC2_HOST=`ec2-describe-instances | grep INSTANCE | grep running | awk '

      {if ($7 == 0) print $4}

      '`
      """
      line, it should be comparing column $8 rather than column $7.

      When this command fails, the hostname of the master doesn't get set correctly and the IP lookup finds incorrect results. The last lines of output from the script look something like this:
      """
      Appointing master
      Master is . Please set up DNS so blah.gotdns.org points to D.ROOT-SERVERS.NET.
      I.ROOT-SERVERS.NET.
      F.ROOT-SERVERS.NET.
      M.ROOT-SERVERS.NET.
      J.ROOT-SERVERS.NET.
      C.ROOT-SERVERS.NET.
      H.ROOT-SERVERS.NET.
      E.ROOT-SERVERS.NET.
      A.ROOT-SERVERS.NET.
      K.ROOT-SERVERS.NET.
      G.ROOT-SERVERS.NET.
      B.ROOT-SERVERS.NET.
      L.ROOT-SERVERS.NET..
      Press return to continue.
      """

      Attachments

        1. hadoop-1634.patch
          0.6 kB
          Stu Hood

        Activity

          People

            Unassigned Unassigned
            stuhood Stu Hood
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: