[HADOOP-7160] Configurable initial buffersize for getGroupDetails()

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.22.0
• Component/s: native, security
• Assignee: Unassigned
• Reporter: T Meyarivan
• Created: 03/Mar/11 16:55
• Updated: 04/Mar/11 21:34

Description

trunk/src/native/src/org/apache/hadoop/security/getGroup.c

int getGroupDetails(gid_t group, char **grpBuf) {
  struct group * grp = NULL;
  size_t currBufferSize = sysconf(_SC_GETGR_R_SIZE_MAX);
  if (currBufferSize < 1024) {
    currBufferSize = 1024;
  }
  *grpBuf = NULL; 
  char *buf = (char*)malloc(sizeof(char) * currBufferSize);

  if (!buf) {
    return ENOMEM;
  }
  int error;
  for (;;) {
    error = getgrgid_r(group, (struct group*)buf,
                       buf + sizeof(struct group),
                       currBufferSize - sizeof(struct group), &grp);
    if(error != ERANGE) {
       break;
    }
    free(buf);
    currBufferSize *= 2;
    buf = malloc(sizeof(char) * currBufferSize);
    if(!buf) {
      return ENOMEM;
    }
...

For large groups, this implies at least 2 queries for the group (number of queries = math.ceil(math.log(response_size/1024, 2)))

In the case of a large cluster with central user/group databases (exposed via LDAP etc), this leads to unnecessary load on the central services. This can be alleviated to a large extent by changing the initial buffer size to a configurable parameter

–