[IMPALA-2878] Base64 decode error

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 2.5.0
• Fix Version/s: Impala 2.7.0
• Component/s: Backend
• Labels: impala ramp-up
• Environment: CentOS 64, gcc version 4.4.7
• Target Version: Product Backlog
• Assignee: Yuanhao Luo
• Reporter: Yuanhao Luo
• Created: 22/Jan/16 02:28
• Updated: 04/Jan/17 23:58
• Resolved: 06/Aug/16 04:08

Description

The Base64Decode() method wouldn't return original string in certain circumstances, it would remove trailing '\0' of original string.
When the number of bytes to encode is not divisible by three,(for example, if there are only one byte '\o' of input for the last 24-bit block), it would add extra two bytes with value zero so there are three bytes. This three '\0' bytes would encode into 'AA=='. While decoding, Base64Decode() first replace '=' with 'A', which is 'AAAA', and Base64 encoded 'AAAA' would be decoded as '\0\0\0'. And then the method remove all trailing '\0' besides padding NULL added before. So after decoding, the string is different from original string before encoding.
May be the following code fix this bug:

bool Base64Decode(const string& in, string* out) {
  typedef transform_width<
      binary_from_base64<string::const_iterator> , 8, 6> base64_decode;
  string tmp = in;
  // Replace padding with base64 encoded NULL
  replace(tmp.begin(), tmp.end(), '=', 'A');
  try {
    *out = string(base64_decode(tmp.begin()), base64_decode(tmp.end()));
  } catch(std::exception& e) {
    return false;
  }

  // Remove trailing '\0' that were added as padding.  Since \0 is special,
  // the boost functions get confused so do this manually.
  int num_padded_chars = 0;
  for (int i = in.size() - 1; i >= 0; --i) {
    if (in.at(i) != '=') break;
    ++num_padded_chars;
  }
  out->resize(out->size() - num_padded_chars);
  return true;
}