[THRIFT-847] Test Framework harmonization across all languages

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.1, 0.2, 0.3
• Fix Version/s: 0.10.0
• Component/s: C# - Library, C++ - Library, Cocoa - Library, Erlang - Library, Haskell - Library, Java - Library, JavaScript - Library, OCaml - Library, Perl - Library, PHP - Library, Python - Library, Ruby - Library, Test Suite
• Labels: gsoc2014 mentor
• Assignee: Roger Meier
• Reporter: Roger Meier
• Created: 11/Aug/10 19:45
• Updated: 06/Jan/16 02:06
• Resolved: 29/Nov/15 15:55

Description

Today each Language supported by Thrift, have its own unit test, all are using the same Thrift IDL's located at the test directory. But the behavior of these tests seems to be different from language to language... this makes it difficult to do tests and bug fixing across different Languages. e.g.

• C++ Test and JavaScript Test Server written in Java have different responses for the same services
• C# and Java Test Server have different responses for testException as C++

I propose the following steps:

• identify the language with the reference implementation (well defined return values for all test cases)
• update the ThriftTest.thrift with details about the required return values that have to be implemented
• update test implementations and move language tests into their appropriate library directory (THRIFT-35)
• a public test server that supports multiple protocols and transports could be another enhancement for testing purposes

I'm ready to help preparing patches, just tell me what you need!

Issue Links

• depends upon: THRIFT-503 "make check"-enabled C++ tests should be in lib/cpp/test
• depends upon: THRIFT-960 add TestServer, TestNonblockingServer and TestClient again
• depends upon: THRIFT-35 Move language tests into their appropriate library directory
• incorporates: THRIFT-1065 Unexpected exceptions not proper handled on JS
• is depended upon by: THRIFT-3102 could not make check for Go Library
• is related to: THRIFT-819 add Enumeration for protocol, transport and server types
• relates to: THRIFT-429 Make binary a full-fledged type of its own
• relates to: THRIFT-953 make check fails in TransportTest.cpp
• relates to: THRIFT-1745 Python JSON protocol
• relates to: THRIFT-1915 Multiplexing Services

Sub-Tasks

• Document the excepted return values and behaviors ThriftTest.thrift so that all languages can implement a test server: Closed
• Test clients should have a return codes that reflect whether they succeeded or not: Closed
• Java test clients should have a return codes that reflect whether it succeeds or not.: Closed
• Java test server should follow the documented behavior as of THRIFT-1590: Closed
• Delphi: Test clients should have a return codes that reflect whether they succeeded or not: Closed
• add ruby to test/test.sh: Closed
• PHP unit test does not work: Closed
• SSL test certificates: Closed
• C++: add compact protocol to cross language test suite: Closed
• Cross language test results should recorded to a status.md or status.html file automatically: Closed
• Moving 'make cross' from test.sh to test.py: Closed
• THRIFT_TEST_PORT variable via to run tests in parallel: Closed
• python Test Server fails when throwing TException: Closed
• TestSuite: detailed result on passed tests by feature: Closed
• Enhance usability of cross test framework: Closed