Bug 56727 - HTTP request failed after listening on multiple ports or enabling SSL Module on my Linux Board
Summary: HTTP request failed after listening on multiple ports or enabling SSL Module ...
Status: REOPENED
Alias: None
Product: APR
Classification: Unclassified
Component: APR (show other bugs)
Version: HEAD
Hardware: Other All
: P5 critical (vote)
Target Milestone: ---
Assignee: Apache Portable Runtime bugs mailinglist
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-07-16 11:00 UTC by Ritesh Prajapati
Modified: 2021-03-30 13:06 UTC (History)
1 user (show)



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ritesh Prajapati 2014-07-16 11:00:59 UTC
I have one Custom Linux board on which I want to run apache web server (httpd) to test HTML and other web based pages.

I have configured, cross compiled and installed httpd (2.2.24, 2.4.1, 2.4.4 and 2.4.9 packages) on my Linux PC (Ubuntu 12.04 LTS) as well as
on my own custom Linux board. Then I have added support of SSL Module (mod_ssl) to test HTTP as well as HTTPS request.

Both HTTP and HTTPS request works fine without any issue on my Linux PC (Ubuntu 12.04 LTS). But when I tried to execute same HTTP request
on my Linux Board using httpd (2.4.4 and 2.4.9 with SSL Module Enabled) at that time browser page goes into loading state and can not be came out from that situation.
Also I have seen that HTTPS request works fine at that time.

I have also did some debugging task through wire-shark tool and found that connection is established successfully after sending request through HTTP
but can not get response of that request. I have also found that response of that HTTP request received on wire-shark after closing that HTTP
requested page from browser.

Also, I can run HTTP and HTTPS requests successfully using httpd (2.2.24 and 2.2.27 with SSL Module enabled) on my Linux Board as well but failed to execute same request
using httpd (2.4.X with SSL Module enabled) package.

I have also changed some configurations by creating different virtual host for HTTP (Port 80) and HTTPS (Port 443) but still failed to
execute that HTTP request.

I have also tried to listen on different ports like (Listen 80 and Listen 8000) without SSL module (using httpd 2.4.4. and 2.4.9 ) at that
time HTTP request goes into loading state.

Does anyone has idea about this issue or help me to solve this type of issue?
Comment 1 Ritesh Prajapati 2014-07-19 05:49:45 UTC
I have also added following flags in configuration script of httpd to solve that issue but still failed to execute HTTP request.

ac_cv_file__dev_zero=yes ac_cv_func_setpgrp_void=yes
apr_cv_process_shared_works=yes apr_cv_mutex_robust_shared=no
apr_cv_tcp_nodelay_with_cork=yes apr_cv_mutex_recursive=yes
ap_cv_void_ptr_lt_long=no ac_cv_sizeof_struct_iovec=8 ac_cv_struct_rlimit=yes \
ac_cv_o_nonblock_inherited=no



I have also debug httpd process using starce and found that read system call goe into blocking state which never returns.

$ ./strace -p 1811
Process 1811 attached
restart_syscall(<... resuming interrupted call ...>) = 0
poll([{fd=6, events=POLLIN}, {fd=4, events=POLLIN}], 2, 10000) = 0 (Timeout)
poll([{fd=6, events=POLLIN}, {fd=4, events=POLLIN}], 2, 10000) = 0 (Timeout)
poll([{fd=6, events=POLLIN}, {fd=4, events=POLLIN}], 2, 10000) = 1 ([{fd=4, revents=POLLIN}])
accept(4, {sa_family=AF_INET6, sin6_port=htons(40827), inet_pton(AF_INET6, "::ffff:192.168.0.45", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 11
fcntl64(11, F_GETFD)                    = 0
fcntl64(11, F_SETFD, FD_CLOEXEC)        = 0
semop(950296, {{0, 1, SEM_UNDO}}, 1)    = 0
gettimeofday({1405747420, 372062}, NULL) = 0
getsockname(11, {sa_family=AF_INET6, sin6_port=htons(80), inet_pton(AF_INET6, "::ffff:192.168.0.183", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
gettimeofday({1405747420, 374162}, NULL) = 0
gettimeofday({1405747420, 374976}, NULL) = 0
read(11, "GET / HTTP/1.1\r\nHost: 192.168.0."..., 8000) = 289
gettimeofday({1405747420, 377850}, NULL) = 0
gettimeofday({1405747420, 380080}, NULL) = 0
gettimeofday({1405747420, 382529}, NULL) = 0
gettimeofday({1405747420, 384638}, NULL) = 0
gettimeofday({1405747420, 386798}, NULL) = 0
gettimeofday({1405747420, 388903}, NULL) = 0
gettimeofday({1405747420, 390971}, NULL) = 0
gettimeofday({1405747420, 393006}, NULL) = 0
gettimeofday({1405747420, 395390}, NULL) = 0
gettimeofday({1405747420, 397474}, NULL) = 0
stat64("/usr/local/apache2/htdocs/", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
stat64("/usr/local/apache2/htdocs/index.html", {st_mode=S_IFREG|0777, st_size=45, ...}) = 0
open("/usr/local/apache2/htdocs/index.html", O_RDONLY|O_CLOEXEC) = 12
open("/etc/localtime", O_RDONLY)        = 13
fstat64(13, {st_mode=S_IFREG|0644, st_size=127, ...}) = 0
fstat64(13, {st_mode=S_IFREG|0644, st_size=127, ...}) = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2a01d000
read(13, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., 4096) = 127
_llseek(13, -11, [116], SEEK_CUR)       = 0
read(13, "\n<GMT-8>-8\n", 4096)         = 11
close(13)                               = 0
munmap(0x2a01d000, 4096)                = 0
gettimeofday({1405747420, 441452}, NULL) = 0
read(11, 

Please let me know if anyone has idea about this issue or any work arounds for that.
Comment 2 Mike Rumph 2014-07-20 15:23:28 UTC
This bug appears to be related to bug 51689.
Comment 3 Ritesh Prajapati 2014-07-21 04:54:36 UTC
I have already tried with all possible compilation flags as well as some other configurations in httpd compilation process to send HTTP request on multiple ports listen directive but still failed to send HTTP request.

Then I have started debugging task from httpd source code and found that request is stuck in check_pipeline() function of http_request.c file.

There is one function call 
"ap_get_brigade(c->input_filters, bb, AP_MODE_SPECULATIVE,APR_NONBLOCK_READ, 1)" 
in check_pipeline() function which never returns after sending HTTP request with multiple ports Listen directive.

ap_get_brigade() function defined in server/util_filter.c file

AP_DECLARE(apr_status_t) ap_get_brigade(ap_filter_t *next,
                                        apr_bucket_brigade *bb,
                                        ap_input_mode_t mode,
                                        apr_read_type_e block,
                                        apr_off_t readbytes)
{
    if (next) {
        return next->frec->filter_func.in_func(next, bb, mode, block,
                                               readbytes);
    }
    return AP_NOBODY_READ;
}

So, This above function call never returns from check_pipeline() function call after sending HTTP request on multiple port Listen.

Please let me know if any one has idea about this problem.
Comment 4 Ritesh Prajapati 2014-07-21 07:24:52 UTC
I have debugged ap_get_brigade() function in details from httpd source code and found that it executes ap_core_input_filter() function from core_filters.c file.

Then, I have looked into ap_core_input_filter() function definition and found that HTTP request execution with multiple listen ports stuck after calling apr_input_read() function which is called from ap_core_input_filter() API.

After that, I have looked for apr_read_bucket() function definition which is defined as read() system call in apr_buckets.h file of Apr-Util Package.

#define apr_bucket_read(e,str,len,block) (e)->type->read(e, str, len, block)

I have also downgraded and tested Apr and Apr-Util packages from 1.5.X to 1.4.X but still failed to execute HTTP request on multiple listen ports.

Please let me know if any one knows about this issue.
Comment 5 Ritesh Prajapati 2014-07-21 13:49:15 UTC
I have also found while debugging apr_bucket_read() function that read() system call executed from apr_socket_recv() function (network_io/unix/sendrecv.c file of Apr Package) blocks HTTP request after listening on multiple ports. 


apr_status_t apr_socket_recv(apr_socket_t *sock, char *buf, apr_size_t *len)
{
    apr_ssize_t rv;
    apr_status_t arv;

    if (sock->options & APR_INCOMPLETE_READ) {
        sock->options &= ~APR_INCOMPLETE_READ;
        goto do_select;
    }

    do {
        rv = read(sock->socketdes, buf, (*len));  ---> Never return from Here
    } while (rv == -1 && errno == EINTR);


That apr_socket_recv() function is called from socket_bucket_read() function of Apr-Util source package (buckets/apr_buckets_socket.c file).

And that socket_bucket_read() function is called from ap_get_brigade() API (util_filters.c file) of http source package to read data from bucket.

So, this issue is caused while reading data from bucket from httpd->Apr-Util->Apr when sending HTTP request on multiple Listen Ports.

We have also checked NON_Block Mode using apr_is_option_set() function and it gives that APR mode is set as NON_Block Mode but still HTTP request never returns from that read() system call of apr_socket_recv() function.

Please let me know if any one has idea about this.
Comment 6 Ritesh Prajapati 2014-07-22 12:26:12 UTC
We have debugged code of APR and found that there is one macro defined 
#define APR_O_NONBLOCK_INHERITED 1
which creates problem to create inherited socket as non block from listener socket while requesting HTTP page in multiple Listen Ports.

So, We have set ac_cv_o_nonblock_inherited=no in configuration options of APR Package and tested HTTP request on multiple Listen ports as well as enabling SSL Modules which works fine without any issue.
Comment 7 Jeff Trawick 2014-07-23 11:57:26 UTC
APR should have some hints for cross-compiling.  Let's leave this open for now and assign to APR.