This patch makes the C++ runtime library buildable in Windows using Visual Studio. No fake Unix environment is needed. It uses Boost and Asio to add work-alike functionality for the existing POSIX-based sockets and thread code. It additionally adds support for Unix Domain Sockets and Windows Named Pipes, so that local client/server communication can be done without the use of localhost. This can simplify the use of Thrift on a single machine because it doesn't run into problems with firewalls, etc. It can also be much faster than using the TCP stack.
Note that all of the new socket and thread code acts exactly like the existing implementations. On a *nix platform, the new socket code can either exist alongside or replace the existing code. The new thread implementation using Boost can only replace the POSIX implementation. The new code does NOT add asynchronous client/server communication.
The first two iterations of this patch were originally attached to JIRA-311. This new version, for JIRA-591, supersedes those, and is applicable to the near-latest Thrift distribution tree. These comments started as a duplicate of my original comment for 311, but they have been modified so if you're interested, please read through them.
The changes from the first patch version are:
- Added the ability to select static or DLL runtime library linkage on Windows
- Added the ability to specify the Boost and Zlib name patterns on a per-config basis for Windows
- Added the ability to select stdint.h implementation on Windows
- Fixed some really stupid bugs in formatted output that I had written. Since Windows doesn't define the PRId64 and PRIu64 formatting strings, I tried using the %j sprintf format strings, casting all the values to be output to either intmax_t or uintmax_t. I was sure that I tested that successfully in Windows before I did my first patch, but I must have been hallucinating, because when I ran the test this time all I saw printed was"ju", not a number. So I undid all that and just defined macros that work for Windows. These were mainly in the CppClient and CppServer test programs, but there was also one in TFileTransport.cpp.
The patch file was generated against the SVN repository revision 818530. It patches the entire distribution tree, but only touches things that are C++ related.
The patch addresses the following unresolved Jira issues:
It also makes changes that are not issues in Jira. For instance:
1. Autoconf support has been extended to all of the C++ tests, as well as the tutorial.
2. The stress-test and stress-test-nb have been extensively reworked to make them more versatile and reliable. I relied on these a lot during my development to test the new socket and server code.
3. The concurrency test has been extensively reworked to make it be reliable and more versatile. I work on a 8 core machine and the test as distributed just doesn't work reliably in that environment. I used this test to verify all of my Boost threading changes.
4. All of the sources have been touched to add Windows-specific support, but I suppose that could be seen as part of this Jira issue.
I had never done anything with autoconf before this. What I have done may not be correct. If changes are required, please let me know.
The configure script displays information at the end that is related to the new configuration stuff. It was useful for me while developing and I thought it might be a good idea to leave it in for a while.
I am really not a Windows guy, I just do what needs doing. If any of you are Windows experts, and you have suggestions or corrections, please let me know.
See the "Known Issues" section in the msvc/README file.
Obviously, I checked the box that says I grant the license to Apache for this code. I also added the Apache license header to all my new files.
However. there are two files that I brought into the source tree that were already open-sourced, with license headers of their own.
The first file is available at http://msinttypes.googlecode.com/svn/trunk/stdint.h, and I have brought it in as the file lib/cpp/src/VisualStudioStdint.h. It has the following notice:
// ISO C9x compliant stdint.h for Miscrosoft Visual Studio
// Based on ISO/IEC 9899:TC2 Committee draft (May 6, 2005) WG14/N1124
// Copyright (c) 2006 Alexander Chemeris
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
// 1. Redistributions of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
// 2. Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// 3. The name of the author may be used to endorse or promote products
// derived from this software without specific prior written permission.
// THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED
// WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
// EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
// OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
// WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
// OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
// ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
The second file is available at http://www.azillionmonkeys.com/qed/pstdint.h and I have brought it in as the file lib/cpp/src/pstdint.h. It has the following notice:
- BSD License:
- Copyright (c) 2005-2007 Paul Hsieh
- All rights reserved.
- Redistribution and use in source and binary forms, with or without
- modification, are permitted provided that the following conditions
- are met:
- 1. Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
- 2. Redistributions in binary form must reproduce the above copyright
- notice, this list of conditions and the following disclaimer in the
- documentation and/or other materials provided with the distribution.
- 3. The name of the author may not be used to endorse or promote products
- derived from this software without specific prior written permission.
- THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
- IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
- OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
- IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
- INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
- NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
- THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
I did not add an Apache license header to either file. I do not know what should be done. They can obviously be used, but there must be some requirement for dual licensing.
To apply the patch, do the following:
1. Checkout from SVN:
svn co --revision 818530 http://svn.apache.org/repos/asf/incubator/thrift/trunk thrift
Note the --revision argument. The patch might work against other revisions, or it might not.
2. Make a copy of your distribution directory, because patch will patch the directory it works on in place.
3. Get the patch file from Jira by following this link:
Use the links there to download the patch zip file, which is named ThriftMsvcPatchForSvnRev818530.txt.zip.
4. Unzip the patch file
5. cd to the renamed distribution directory
6. Apply the patch by execution:
patch -p 8 <patchFile
where patchFile is the path to your downloaded patch file.
After applying the patch, the top level README file will have information about the new "msvc" subdirectory. The new file CPP_LIB_CFG has information about the configurations that are possible on *nix systems. Configurations can keep things the way they are currently, or can add-to and/or replace existing functionality with new functionality provided by Asio/Boost. (I had to do it this way so that I could use the existing test code with my new classes.)
The "msvc" subdirectory has its own README file that contains information particular to the msvc port. There are other msvc subdirectories spread through the tree. They follow a pattern, and they are all documented in the new README.
Distribution with Patch Pre-Applied:
It's pretty hard to apply the patch if all you have is a Windows system. (Get a Mac and use Boot Camp!) To help those poor souls, I have also attached a zip file that contains the 818530 svn checkout with the patch applied. Just download it to your Windows box and unzip it. It is named thrift-818530-patched.zip.
Mac OS X 10.5.7, 10.5.8, and 10.6.1 (Snow Leopard) (me)
Gentoo Linux (Bruce Simpson)
FreeBSD (Bruce Simpson)
Windows XP (me)
Tested Boost and Asio versions:
Standalone Asio 1.2.0 and 1.4.1
On the Mac: Boost 1.36.0, 1.37.0, 1.38.0, and 1.39.0
On Windows XP: Boost 1.36.0
Along with the patch, I will attach a zip file called "MsvcPatchSupportScripts.zip" that contains a number of scripts that are useful for testing all of the variations (there are 10) on a *nix system. These scripts assume a very specific directory structure. It is documented in only one of them. Sorry. If you run "./buildAll.sh --help" at the command line, it will show you what is expected. It also documents two environment variables you can use with the scripts. They are called THRIFT_DEV_ROOT and THRIFT_DISTRO_DIR. Note that the scripts are designed to be run from the directory ABOVE your patched distribution directory.
The various scripts are:
buildAll.sh - Builds all the possible configuration variants by copying the distribution directory, then configuring and building. If you are on a Mac, be sure to copy your pkg.m4 file into the aclocal subdirectory of the distribution tree before you run this script. The build results are written into files called Log_* in the same directory as the script.
buildTutorialAll.sh - After buildAll.sh has run, this script iterates over all of the directories that were created and verifies that the C++ tutorial can be built both using the auto-generated Makefile and the Tutorial.mk makefile. It cleans up after itself.
clean.sh - Finds and removes all of the subdirectory trees and log files that are written by buildAll.sh.
concurrencyTest.sh - After buildAll.sh has run, this script iterates over all the directories that were created, runs the concurrency test in each of them and verifies that the test runs successfully.
cpCfgAndBuild.sh - This script knows how to configure and build Thrift in a single directory, but all it cares about is the C++ stuff. It also runs "make check". This script is used by the buildAll.sh script. This script currently defines the macro THRIFT_ENABLE_CONFIG_WARNINGS via CPPFLAGS. That generates warning messages at library compile time that tell you about the configuration. Remove the macro definition from the script if you don't want the warnings. (Just search for the macro name in the script file. What to do is pretty obvious once you find it.)
doTest.sh - This script runs the standard "make check" test suite on a single directory. It is used by the testAll.sh script.
findThriftDistro.sh - This script attempts to locate the Thrift distribution directory. It looks at the directory passed as the script argument if there is one, or it examines the environment variables THRIFT_DEV_ROOT and THRIFT_DISTRO_DIR if they are set. This script is used by many of the others, and you shouldn't need to mess with it.
testAll.sh - After buildAll.sh has run, this script iterates over all of the directories that were created and runs the standard test suite in each of them.
I hope others find this to be useful.