Fixed crashes

I accidentally initiated the threads and called their functions before initiating the queues.

For some reason, my debug statements weren't writing to the error log. It was actually because I was testing from the browser and I hit the go button instead of refresh so it didn't make a new request.

Here are the results:

~/apache2/bin$ ./apachectl start
~/apache2/bin$ !./ab
./ab -n 100000 -c 1000 localhost:8080/index.html
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Finished 100000 requests


Server Software:        Apache/2.2.4
Server Hostname:        localhost
Server Port:            8080

Document Path:          /index.html
Document Length:        44 bytes

Concurrency Level:      1000
Time taken for tests:   21.20827 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      29301465 bytes
HTML transferred:       4400220 bytes
Requests per second:    4757.19 [#/sec] (mean)
Time per request:       210.208 [ms] (mean)
Time per request:       0.210 [ms] (mean, across all concurrent requests)
Transfer rate:          1361.22 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   31 483.8      1   20999
Processing:     0   65 507.9     21   13177
Waiting:        0   55 507.4     17   13137
Total:         10   97 738.2     23   21003

Percentage of the requests served within a certain time (ms)
  50%     23
  66%     39
  75%     51
  80%     56
  90%     66
  95%     77
  98%     89
  99%   1078
 100%  21003 (longest request)

apr_pollset_poll

Helpful link: text

apr_pollset_poll

APR_DECLARE(apr_status_t) apr_pollset_poll(apr_pollset_t *pollset,
                                           apr_interval_time_t timeout,
                                           apr_int32_t *num,
                                           const apr_pollfd_t **descriptors);

struct apr_pollset_t
{
    apr_pool_t *pool;
    apr_uint32_t nelts;
    apr_uint32_t nalloc;
    struct pollfd *pollset;
    apr_pollfd_t *query_set;
    apr_pollfd_t *result_set;
};

/** Used in apr_pollfd_t to determine what the apr_descriptor is */
typedef enum { 
    APR_NO_DESC,                /**< nothing here */
    APR_POLL_SOCKET,            /**< descriptor refers to a socket */
    APR_POLL_FILE,              /**< descriptor refers to a file */
    APR_POLL_LASTDESC           /**< descriptor is the last one in the list */
} apr_datatype_e ;

/** Union of either an APR file or socket. */
typedef union {
    apr_file_t *f;              /**< file */
    apr_socket_t *s;            /**< socket */
} apr_descriptor;

/** @see apr_pollfd_t */
typedef struct apr_pollfd_t apr_pollfd_t;

/** Poll descriptor set. */
struct apr_pollfd_t {
    apr_pool_t *p;              /**< associated pool */
    apr_datatype_e desc_type;   /**< descriptor type */
    apr_int16_t reqevents;      /**< requested events */
    apr_int16_t rtnevents;      /**< returned events */
    apr_descriptor desc;        /**< @see apr_descriptor */
    void *client_data;          /**< allows app to associate context */
};

Children crashing

I tried threading the security phase but when I checked the log file, it shows that some children are seg faulting. I suspect that since my threads and queues are global variables in the parent, the children cannot access them. It might have worked before because the request parsing and security phases may have not been necessary. I am now looking through the prefork MPM code to see if there is anywhere else I can put the threads. Another option would be to have each child contain a list of threads and then pass the threads to thread function so that it knows which thread to go on to.

Threaded request_parsing_phase

./ab -n 100000 -c 1000 localhost:8080/index.html
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Finished 100000 requests


Server Software:        Apache/2.2.4
Server Hostname:        localhost
Server Port:            8080

Document Path:          /index.html
Document Length:        44 bytes

Concurrency Level:      1000
Time taken for tests:   16.873245 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      29300293 bytes
HTML transferred:       4400044 bytes
Requests per second:    5926.54 [#/sec] (mean)
Time per request:       168.732 [ms] (mean)
Time per request:       0.169 [ms] (mean, across all concurrent requests)
Transfer rate:          1695.76 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   32 439.6      1    9030
Processing:     0   57 330.8     23   13862
Waiting:        0   45 330.0     18   13861
Total:          8   90 638.5     24   16866

Percentage of the requests served within a certain time (ms)
  50%     24
  66%     40
  75%     56
  80%     60
  90%     71
  95%     79
  98%     89
  99%   1190
 100%  16866 (longest request)

Separated phases into functions

Ran after separating into functions:

~/apache2/bin$ ./apachectl start
~/apache2/bin$ ./ab -n 100000 -c 1000 localhost:8080/index.html
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Finished 100000 requests


Server Software:        Apache/2.2.4
Server Hostname:        localhost
Server Port:            8080

Document Path:          /index.html
Document Length:        44 bytes

Concurrency Level:      1000
Time taken for tests:   18.781482 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      29300000 bytes
HTML transferred:       4400000 bytes
Requests per second:    5324.39 [#/sec] (mean)
Time per request:       187.815 [ms] (mean)
Time per request:       0.188 [ms] (mean, across all concurrent requests)
Transfer rate:          1523.47 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   36 480.8      1    9030
Processing:     0   62 502.0     27   12897
Waiting:        0   51 501.7     16   12868
Total:          1   98 805.2     30   18333

Percentage of the requests served within a certain time (ms)
  50%     30
  66%     40
  75%     45
  80%     48
  90%     57
  95%     64
  98%     72
  99%    964
 100%  18333 (longest request)

Request Processing in Apache 2.0

• Request Parsing
• Security
• Preparation
• Handler
  

Link: Request Processing in Apache 2.0

log to log/error_log

#include "http_log.h"

ap_log_error(APLOG_MARK, int debug_level, 0, ap_server_conf, "%d pushing to queue_return", r);

#define APLOG_EMERG 0 /* system is unusable */
#define APLOG_ALERT 1 /* action must be taken immediately */
#define APLOG_CRIT 2 /* critical conditions */
#define APLOG_ERR 3 /* error conditions */
#define APLOG_WARNING 4 /* warning conditions */
#define APLOG_NOTICE 5 /* normal but significant condition */
#define APLOG_INFO 6 /* informational */
#define APLOG_DEBUG 7 /* debug-level messages */

Prefork single process benchmark

~/apache2/bin$ ./httpd -X

In other window:

~/apache2/bin$ ./ab -n 100000 -c 1000 localhost:8080/index.html
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
apr_socket_recv: Connection reset by peer (104)
Total of 10000 requests completed

Same results with

~/apache2/bin$ sudo ./httpd -X

Prefork benchmark

~/apache2/bin$ ./apachectl start

~/apache2/bin$ ./ab -n 100000 -c 1000 localhost:8080/index.html
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Finished 100000 requests


Server Software:        Apache/2.2.4
Server Hostname:        localhost
Server Port:            8080

Document Path:          /index.html
Document Length:        44 bytes

Concurrency Level:      1000
Time taken for tests:   19.266706 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      29300000 bytes
HTML transferred:       4400000 bytes
Requests per second:    5190.30 [#/sec] (mean)
Time per request:       192.667 [ms] (mean)
Time per request:       0.193 [ms] (mean, across all concurrent requests)
Transfer rate:          1485.10 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   27 400.9      1    9015
Processing:     0   58 424.8     20   13145
Waiting:        0   49 424.4     16   13118
Total:          9   86 660.4     20   14302

Percentage of the requests served within a certain time (ms)
  50%     20
  66%     38
  75%     45
  80%     48
  90%     56
  95%     63
  98%     72
  99%   1069
 100%  14302 (longest request)

ApacheBench

~/apache2/bin$ ./ab -n 100000 -c 1000 localhost/index.html

Request Processing in Apache

Link: Request Processing in Apache at ApacheTutor

Web Bench

I found Web Bench.

make
./webbench http://localhost/index.html -c 8

-c|--clients Run HTTP clients at once. Default one.

MPM - Prefork

Prefork

Pros

• Avoid threadings for compatibility with non-thread-safe libraries
  
• Isolates each request, so that a problem with a single request will not affect any other.
  

Apache HTTPD 2.2.4

I downloaded a new version of Apache HTTPD (version 2.2.4).

Then I imported it into SVN:
svn import httpd-2.2.4 file:///home/kle/svn/httpd-2.2.4 -m "Initial import"

Deleted it:
rm -rf httpd-2.2.4

Then checked it out:
svn co file:///home/kle/svn/httpd-2.2.4

Flood

Download flood

I got the following error running configure on an AMD64 system:

checking build system type... Invalid configuration`x86_64-unknown-linux-gnu': machine `x86_64-unknown' not recognized
configure: error: /bin/sh ./config.sub x86_64-unknown-linux-gnu failed

This seemed to fix it:
libtoolize --force
aclocal --force
automake --add-missing
autoconf
autoheader

Then I ran:
$ ./buildconf
$ ./configure --disable-sharing

I get the following error trying to compile it:
$ make all
/usr/share/apr-1.0/build/libtool --silent --mode=compile gcc -g -O2 -pipe -Wall -g -O2 -pthread -DLINUX=2 -D_REENTRANT -D_GNU_SOURCE -I/usr/include/apr-1.0 -I/usr/include/apr-1.0 -I/usr/include/postgresql -I. -c flood.c && touch flood.lo
libtool: compile: unable to infer tagged configuration
libtool: compile: specify a tag with `--tag'
make: *** [flood.lo] Error 1