[ACCUMULO-2036] Mappers are not running locally

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Labels: 16_qa_bug
• Assignee: Josh Elser
• Reporter: Keith Turner
• Created: 17/Dec/13 00:00
• Updated: 07/Jan/14 17:17
• Resolved: 03/Jan/14 05:46

Description

I ran listscans in the Accumulo shell while running continuous verify on small cluster and almost not map task were running locally.

I think ACCUMULO-1585 has broken mapper locality in 1.6.0-SNAPSHOT. Before that change Accumulo would always store IP addrs. Code like the following in o.a.a.c.client.mapreduce.AbstractInputFormat.getSplits() would translate IPs to hostnames.

       if (location == null) {
          InetAddress inetAddress = InetAddress.getByName(ip);
          location = inetAddress.getHostName();
          hostNameCache.put(ip, location);
        }

In my case I configured Accumulo to use hostnames, but not fully qualified ones. So I think the above code was just passing the non-quallified hostname through. I suspected hadoop wanted FQDN and changed the code to the following and mappers ran locally. I need to confirm what hadoop is expecting. I think the above code will result in a FQDN if given an IP, so this is not an issue for 1.4 or 1.5.

       if (location == null) {
          InetAddress inetAddress = InetAddress.getByName(ip);
          location = inetAddress.getCanonicalHostName();
          hostNameCache.put(ip, location);
        }

Issue Links

• is related to: ACCUMULO-2147 Create test to verify mappers are running locally