[STDCXX-215] stdcxx not async-cancel safe

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 4.1.2, 4.1.3
• Component/s: Thread Safety

Environment

all

• Severity: Resource Leak
• Assignee: Unassigned
• Reporter: Martin Sebor
• Created: 29/Jun/06 00:55
• Updated: 26/Nov/07 03:25

Description

Moved from the Rogue Wave bug tracking database:

***Created By: sebor @ Apr 06, 2004 09:02:04 AM***
-------- Original Message --------
Subject: Re: async safe IOstreams
Date: Mon, 5 Apr 2004 19:21:01 -0700 (PDT)
From: Dennis Handly <[e-mail]>
To: [e-mail]
CC: [e-mail], [e-mail]

Do you claim the RW Standard C++ Library (iostreams in particular)
is async cancel safe? With pthread_cancel?

We have a customer where it hangs after one thread is canceled.
(It was holding a mutex.)

#7 0x20000000797990d0:0 in _HPMutexWrapper::lock+0x70 ()
from /usr/lib/hpux32/libstd_v2.so.1
#8 0x4014a50:0 in _rw::_rw_mutex_base::_C_acquire (this=0x795b56c4)
at /opt/aCC/include_std/rw/stdmutex.h:254
#9 0x401b500:0 in _rw::rw_guard::_rw_guard (this=0x7fffe200,
__mutex=0x795b56c4) at /opt/aCC/include_std/rw/stdmutex.h:476
#10 0x40198b0:0 in std::ostream::sentry::sentry (this=0x7fffe200,
__strm=@0x795b592000000000) at /opt/aCC/include_std/ostream:97
#11 0x4018170:0 in std::basic_ostream<char,std::char_traits<char> >& _rw::rw
insert<char,std::char_traits<char>,char const> (__strm=@0x795b592000000000,
__s=0x40132f8 "****** Creation Thread ", __len=23, __width=0)
at /opt/aCC/include_std/ostream.cc:165
#12 0x40174e0:0 in std::basic_ostream<char,std::char_traits<char> >& std::operat
or<<<std::char_traits<char> > (__strm=@0x795b59206861723e,
__s=0x40132f8 "****** Creation Thread ")
at /opt/aCC/include_std/ostream:510
#13 0x401e780:0 in main () at TestThread.cpp:49

Does it work on Tru64 because they have more atomic operations and don't
need mutexes as much as we do?
Since __rw_guard is used above, I don't see this is specific to aC++.
(Though I'm not sure why we need to serialize a call to good():
_C_ok = _C_strm._C_opfx ().good ();
)

Or do you actually call pthread_setcancelstate to block and unblock?

============================================================
>From: "Hofherr, Birgit" <[e-mail]>
When compiling with -AA a multithreaded C++ program, and executing it,
it hangs on a pthread_join when the thread contains cout << ..<<endl.

When the same program is compiled -AP no problem occurs.

It looks like the iostream package from libstd_v2.2 does not implement
the lazy I/O scheme, which means the output ends up on a endl or a cin
input. This brings up for the enclosed multithreaded program and for
the customer who would like to have the cout << .. << endl; performed
in one C statement write like for the -AP option.

============================================================
>From: "Hofherr, Birgit" <[e-mail]>
I just got this additional info from the customer:

Please add a sleep(1) before the call to pthread_cancel. I afterwards
tested the program on Tru64 and it fails without it.

The problem is that, with -AA, cout<<..<<endl; are not monolithic (i.e.:
expressed with a write(1,string,strlen(string)) contrary to the -AP
option. This is what the customer would like with -AA.

>From: "Hofherr, Birgit" <[e-mail]>
Some more analysis from the customer - regarding the lazy io

----Original Message----
>From: Vouters, Philippe
After thorough analysis, /usr/lib/libstd_v2 (option -AA) must implement
a lazy I/O for cin, cout and cerr. This means that the operator << must
buffer data which is added a "\n" at endl time and writes out the buffer
to fd=1. If a operator >> comes up in the code stream, it must write out
cout and cerr buffers and then do input.

I believe this is the way /usr/lib/libstd works (option -AP). Anyway
this the way the customer would like and looks to work like on Tru64
with -std strict_ansi C++ compilation.

Philipppe Vouters (HP Services, Les Ulis, France)

============================================================
>From: Dennis Handly <[e-mail]>
>From: "Hofherr, Birgit" <[e-mail]>
>When compiling with -AA a multithreaded C++ program, and executing it,
>it hangs on a pthread_join when the thread contains cout << ..<<endl.

It doesn't hang on the pthread_join, it hangs after the pthread_cancel.
pthread_cancel is not supported with aC++'s runtime.

The user must not use PTHREAD_CANCEL_ENABLE unless no iostream, etc.
code is being used.

pthread_setcancelstate(3T) says:
NOTES
Only functions that are async-cancel safe should be called from a
thread that is asynchronously cancelable.

>When the same program is compiled -Aa no problem occurs.

(They mean with -AP.) There are no mutexes used. A pthread_cancel
on a string operation may also hang things.

>This brings up for the enclosed multithreaded program and for
>the customer who would like to have the cout << .. << endl; performed
>in one C statement write like for the -AP option.
Birgit

This can't be done with -AA. Each insertion is done with a mutex lock.
for -AP, there is a separate buffer with no mutexes.

>From: "Hofherr, Birgit" <[e-mail]>
>Please add a sleep(1) before the call to pthread_cancel. I afterwards
>tested the program on Tru64 and it fails without it.

The application needs to remove pthread_cancel or make sure
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE) is only used when not
in iostreams.

I've added code with -DWORK to disable and enable correctly.

>The problem is that, with -AA, cout<<..<<endl; are not monolithic (i.e.:
>expressed with a write(1,string,strlen(string)) contrary to the -AP
>option. This is what the customer would like with -AA.

This change can't be done. It isn't part of RW's implementation.

>From: Vouters, Philippe
>After thorough analysis, /usr/lib/libstd_v2 (option -AA) must implement
>a lazy I/O for cin, cout and cerr. This means that the operator << must
>buffer data which is added a "\n" at endl time and writes out the buffer
>to fd=1.

The user could do this by using strstream and fprintf.

>I believe this is the way /usr/lib/libstd works (option -AP).

-AP uses separate buffers.

>Anyway this the way the customer would like and looks to work like on Tru64
>with -std strict_ansi C++ compilation.
Philipppe Vouters (HP Services, Les Ulis, France)

I'm not sure how Tru64 can do this since they also use RW.

============================================================
>From: "Hofherr, Birgit" <[e-mail]>
This is the response from XXX - Is this something RW or we might consider?

----Original Message----
>From: Vouters, Philippe
I do not know much about the internals of the Tru64 C++ library, but I
think this pthread_setcancelstate(PTHREAD_CANCEL_DISABLE,&o)/
pthread_setcancelstate(o,NULL) (to restore in the case of the customer's
program the PTHREAD_CANCEL_ENABLE state) is performed inside the
library. Couldn't it be implemented inside libstd_v2 to make the
iostream async_safe ? Or by the compiler when -mt ? Kind regards from
home. Philippe Vouters (HP Services, Les Ulis, France);

============================================================

>From: Birgit Hofherr (Languages Expert Center) <[e-mail]>
A heads up - XXX have submitted a serious enhancement request to have
the aC++ iostreams behave the same way as Tru64's (compiled with -pthread
and -std strict_ansi). JAGaf18057 requests a change in either the compiler
when compiling with -mt or in the RW library so that iostreams are async-safe -
like the Tru64 implementation.

They are ready to escalate this.
Please let me know what we can do.
Birgit

============================================================
>From: Dennis Handly <[e-mail]>
XXX is claiming that this application works fine on Tru64.
Since you guys use RW, how does it work if the user uses pthread_cancel?

Is it because we use mutexes and Tru64 uses atomic operations?

============================================================
>From: Dennis Handly <[e-mail]>

>From: "Hofherr, Birgit" <[e-mail]>
>This is the response from XXX - Is this something RW or we might consider?

Not likely. It will make performance worse for customers that don't care
about it.

>They are ready to escalate this.
>Please let me know what we can do.

Nothing, we will have to reject it.
Implementing it will kill performance for everyone using -mt.
The most we can do is document it in big bold letters.
============================================================

TestThread.cpp:

#include <pthread.h>
#include <unistd.h>
#include <string>
#include <iostream>
using std::string;
using namespace std;

#define THREADS_NUMBER 1000

void *faire(void *p) {
int o;
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, &o);
pthread_setcanceltype(PTHREAD_CANCEL_DEFERRED, &o);
int i;
for (i = 0; i <= 100000; i++)

{ // PROBLEM WHEN cout IS USED HERE #ifdef WORK pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &o); #endif cout << "C++faire ..." << (int)(long)p << endl; #ifdef WORK pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, &o); #endif // OK WITH printf printf("C faire ...%d\n", (int)(long)p); pthread_testcancel(); sleep(1); }

return NULL;
}
int main() {
int error;
cout << "Test: Thread" << endl;
pthread_t _thread[THREADS_NUMBER];
pthread_attr_t _attributs;
error = pthread_attr_init(&_attributs);
if (error != 0)

{ cout << "Error attr_init" << endl; fprintf(stderr, "Error attr_init\n"); return 1; }

error = pthread_attr_setdetachstate(&_attributs, PTHREAD_CREATE_JOINABLE);
if (error != 0)

{ fprintf(stderr, "Error setdetachstate\n"); return 1; }

int k, i;
for(k=0; k < THREADS_NUMBER; k++) {
cout << "****** Creation Thread " << k << " ******" << endl;
cout << "* Thread " << k << " - create" << endl;
error = pthread_create(&_thread[k], &_attributs, faire, (void*)(long)k);
if (error != 0)

{ fprintf(stderr, "Error create\n"); return 1; }

sleep(1);
fprintf(stderr, "* Thread %d - cancel\n", k);
error = pthread_cancel(_thread[k]);
if (error != 0)

{ fprintf(stderr, "Error cancel\n"); return 1; }

fprintf(stderr, "* Thread %d - join\n", k);
error = pthread_join(_thread[k], NULL);
if (error != 0)

{ fprintf(stderr, "Error join\n"); return 1; }

//cout << "Attente 2s ..." << endl;
//sleep(2);
}
cout << "Attente 4s" << endl;
sleep(4);
cout << "Test: FIN" << endl;
}

***Modified By: sebor @ Apr 06, 2004 09:04:02 AM***
-------- Original Message --------
Subject: Re: async safe IOstreams
Date: Tue, 06 Apr 2004 09:01:56 -0600
From: Martin Sebor <[e-mail]>
To: Dennis Handly <[e-mail]>
CC: [e-mail]
References: <[e-mail]>

Dennis Handly wrote:
> Do you claim the RW Standard C++ Library (iostreams in particular)
> is async cancel safe? With pthread_cancel?

No, we're certainly not that. There are mutexes all over the
place in iostreams. FWIW, I don't know of any implementation
that is.

>
> We have a customer where it hangs after one thread is canceled.
> (It was holding a mutex.)

Right.

>
...
> Does it work on Tru64 because they have more atomic operations and don't
> need mutexes as much as we do?

No, it's not safe on any platform.

> Since __rw_guard is used above, I don't see this is specific to aC++.
> (Though I'm not sure why we need to serialize a call to good():
> _C_ok = _C_strm._C_opfx ().good ();
> )
>
> Or do you actually call pthread_setcancelstate to block and unblock?

No, but it's on my to-do list.

Martin