Uploaded image for project: 'Thrift'
  1. Thrift
  2. THRIFT-3175

fastbinary.c python deserialize can cause huge allocations from garbage

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.3
    • Component/s: Python - Library
    • Labels:
      None

      Description

      In the fastbinary python deserializer, allocating a list is done like so:

          len = readI32(input);
          if (!check_ssize_t_32(len)) {
            return NULL;
          }
      
          ret = PyList_New(len);
      

      The only validation of len is that it's under INT_MAX. I've encountered a situation where upon receiving garbage input, and having len be read as something like 1 billion, the library treated this as a valid input, allocated gigs of RAM, and caused a server to crash.

      The quick fix I made was to limit list sizes to a sane value of a few thousands that more than suits my personal needs.

      But IMO this should be dealt with properly. One way that comes to mind is not pre-allocating the entire list in advance in case it's really big, and resizing it in smaller steps while reading the input.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                dvirsky Dvir Volk
                Reporter:
                dvirsky Dvir Volk
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: