[THRIFT-4134] Fix remaining undefined behavior invalid vptr casts

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.11.0
• Fix Version/s: 0.11.0
• Component/s: Compiler (General), D - Compiler, Haxe - Compiler, Java - Compiler

Environment

Travis CI, Ubuntu 14.04.5

• Assignee: Unassigned
• Reporter: Jim Apple
• Created: 25/Mar/17 15:45
• Updated: 14/Dec/17 13:55
• Resolved: 01/Apr/17 14:54

Description

Removing -fno-sanitize-recover=undefined -fno-sanitize=vptr from ubsan.sh shows vptr errors in the following places:

src/thrift/generate/t_d_generator.cc:384:25: runtime error: 
src/thrift/generate/t_d_generator.cc:385:12: runtime error: 
src/thrift/generate/t_haxe_generator.cc:2576:34: runtime error: 
src/thrift/generate/t_haxe_generator.cc:2576:35: runtime error: 
src/thrift/parse/t_list.h:33:42: runtime error: 
thrift/parse/t_scope.h:88:40: runtime error: 
thrift/parse/t_scope.h:88:41: runtime error: 
/usr/include/boost/lexical_cast.hpp:1620:43: runtime error: 
src/thrift/generate/t_java_generator.cc:2693:27: runtime error: 
src/thrift/generate/t_java_generator.cc:2693:28: runtime error: 
src/thrift/generate/t_java_generator.cc:3756:35: runtime error: 
src/thrift/generate/t_java_generator.cc:3756:36: runtime error: 
src/thrift/generate/t_java_generator.cc:4639:11: runtime error: 
src/thrift/generate/t_java_generator.cc:4639:12: runtime error: 
src/thrift/generate/t_java_generator.cc:4666:10: runtime error: 
src/thrift/generate/t_java_generator.cc:4666:9: runtime error: 
src/thrift/generate/t_json_generator.cc:277:21: runtime error: 
src/thrift/generate/t_json_generator.cc:277:22: runtime error: 
src/thrift/generate/t_xml_generator.cc:395:21: runtime error: 
src/thrift/generate/t_xml_generator.cc:395:22: runtime error: 
src/thrift/parse/t_base_type.h:64:35: runtime error: 
src/thrift/parse/t_list.h:33:42: runtime error: 
src/thrift/parse/t_set.h:33:42: runtime error: 
thrift/parse/t_scope.h:88:40: runtime error: 
thrift/parse/t_scope.h:88:41: runtime error: 
thrift/parse/t_scope.h:88:41: runtime error: 

Here is an example

     [exec] src/thrift/generate/t_json_generator.cc:277:22: runtime error: downcast of address 0x000003f61d40 which does not point to an object of type 't_list'
     [exec] 0x000003f61d40: note: object is of type 't_set'
     [exec]  00 00 00 00  a8 b9 f7 01 00 00 00 00  d8 23 04 8e 7e 2b 00 00  00 00 00 00 00 00 00 00  00 00 00 00
     [exec]               ^~~~~~~~~~~~~~~~~~~~~~~
     [exec]               vptr for 't_set'
     [exec] [WARNING:/thrift/src/test/ThriftTest.thrift:45] No generator named 'noexist' could be found!
     [exec] [WARNING:/thrift/src/test/ThriftTest.thrift:47] cpp generator does not accept 'noexist' as sub-namespace!
     [exec]     #0 0xaa355e in t_json_generator::write_type_spec(t_type*) /thrift/src/compiler/cpp/src/thrift/generate/t_json_generator.cc:277:22
     [exec]     #1 0xaa12e5 in t_json_generator::write_type_spec_object(char const*, t_type*) /thrift/src/compiler/cpp/src/thrift/generate/t_json_generator.cc:252:5
     [exec]     #2 0xabcc0f in t_json_generator::generate_field(t_field*) /thrift/src/compiler/cpp/src/thrift/generate/t_json_generator.cc:663:3
     [exec]     #3 0xabb4cd in t_json_generator::generate_struct(t_struct*) /thrift/src/compiler/cpp/src/thrift/generate/t_json_generator.cc:583:5
     [exec]     #4 0xaad948 in t_json_generator::generate_program() /thrift/src/compiler/cpp/src/thrift/generate/t_json_generator.cc:400:7
     [exec]     #5 0x44d0f6 in generate(t_program*, std::vector<std::string, std::allocator<std::string> > const&) /thrift/src/compiler/cpp/src/thrift/main.cc:1032:9
     [exec]     #6 0x4532ba in main /thrift/src/compiler/cpp/src/thrift/main.cc:1289:5
     [exec]     #7 0x2b7e8e784f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44)
     [exec]     #8 0x417ec3 in _start (/thrift/src/compiler/cpp/thrift+0x417ec3)

The problem here is this:

  } else if (ttype->is_list() || ttype->is_set()) {
    t_type* etype = ((t_list*)ttype)->get_elem_type();

ttype cannot necessarily be cast to t_list*, since sometimes it is a t_set*. When it is a t_set*, the call to get_elem_type() is undefined behavior: section 9.3.1 "Nonstatic member functions [class.mfct.non-static]", paragraph 2 of the C++14 standard states "If a non-static member function of a class X is called for an object that is not of type X, or of a type derived from X, the behavior is undefined."

This will be tough to fix completely, since one of these is an error in Boost that is present all the way through the latest Boost that is a vendor-supplied Ubuntu 14.04 package.

Issue Links

• relates to: IMPALA-5031 UBSAN clean and method for testing UBSAN cleanliness
• links to: GitHub Pull Request #1222

Sub-Tasks

• Align is_binary() method with is_string() to simplify those checks: Closed
• Fix remaining undefined behavior invalid vptr casts in Thrift Compiler: Closed
• Fix remaining undefined behavior invalid vptr casts in C++ library: Closed