Wiki Navigation

mod_ssl vs mod_gnutls

This page will contain all information needed to set up an Apache2 environment for benchmarking mod_ssl vs mod_gnutls.

Environment

All testing is performed on a Debian machine, running the following software which should be the latest available when this was written (2008-03-05). Note that we don't build anything ourselves, we use the Debian packages for everything. The benchmarking tool we will use is Siege. The reasons for chosing it was that it was packaged for Debian and was easy to use. We should definitely consider other benchmarking tools as well, suggestions are welcome.

ii  libssl0.9.8                          0.9.8g-4                     SSL shared libraries
ii  openssl                              0.9.8g-4                     Secure Socket Layer (SSL) binary and related cryptographic tools
ii  ssl-cert                             1.0.16                       simple debconf wrapper for OpenSSL
ii  apache2                              2.2.8-1                      Next generation, scalable, extendable web server
ii  apache2-mpm-worker                   2.2.8-1                      High speed threaded model for Apache HTTPD
ii  apache2-utils                        2.2.8-1                      utility programs for webservers
ii  apache2.2-common                     2.2.8-1                      Next generation, scalable, extendable web server
ii  libapache2-mod-gnutls                0.5.0-alpha-1                Apache2 module for SSL and TLS encryption using GnuTLS
ii  siege                                2.65-4                       Http regression testing and benchmarking utility

Apache configuration

It doesn't seem possible to have the mod_ssl and the mod_gnutls modules loaded at the same time. I didn't look into why. Instead, we will change configurations and re-start Apache with mod_ssl or mod_gnutls as we want to test each. This may help to reduce the chance that we mix up results for mod_ssl with mod_gnutls.

GnuTLS configuration

Copy the following into /etc/apache2/sites-available/gnutls.

Listen 443
NameVirtualHost *:443
<VirtualHost *:443>
        GnuTLSEnable on
	GnuTLSCertificateFile /etc/apache2/ssl/apache.pem
	GnuTLSKeyFile /etc/apache2/ssl/apache.pem
	GnuTLSClientCAFile /etc/apache2/ssl/apache.pem
	GnuTLSDHFile /etc/apache2/ssl/dh.pem
	GnuTLSPriorities NORMAL

	ServerAdmin simon@josefsson.org
	DocumentRoot /var/www/
	<Directory />
		Options FollowSymLinks
		AllowOverride None
	</Directory>
	<Directory /var/www/>
		DirectoryIndex index.html
		Options Indexes FollowSymLinks MultiViews +ExecCGI
		AllowOverride None
		Order allow,deny
		allow from all
	</Directory>

	ErrorLog /var/log/apache2/mod_gnutls-error.log
	LogLevel warn

	CustomLog /var/log/apache2/mod_gnutls-access.log combined
	ServerSignature On

    Alias /doc/ "/usr/share/doc/"
    <Directory "/usr/share/doc/">
        Options Indexes MultiViews FollowSymLinks
        AllowOverride None
        Order deny,allow
        Deny from all
        Allow from 127.0.0.0/255.0.0.0 ::1/128
    </Directory>

</VirtualHost>

OpenSSL configuration

Copy the following into /etc/apache2/sites-available/openssl.

NameVirtualHost *:443
<VirtualHost *:443>
	SSLEngine on
	SSLCertificateFile /etc/apache2/ssl/apache.pem
	SSLCertificateKeyFile /etc/apache2/ssl/apache.pem
	SSLCACertificateFile /etc/apache2/ssl/apache.pem

	ServerAdmin simon@josefsson.org
	DocumentRoot /var/www/
	<Directory />
		Options FollowSymLinks
		AllowOverride None
	</Directory>
	<Directory /var/www/>
		DirectoryIndex index.html
		Options Indexes FollowSymLinks MultiViews +ExecCGI
		AllowOverride None
		Order allow,deny
		allow from all
	</Directory>

	ErrorLog /var/log/apache2/mod_ssl-error.log
	LogLevel warn

	CustomLog /var/log/apache2/mod_ssl-access.log combined
	ServerSignature On

    Alias /doc/ "/usr/share/doc/"
    <Directory "/usr/share/doc/">
        Options Indexes MultiViews FollowSymLinks
        AllowOverride None
        Order deny,allow
        Deny from all
        Allow from 127.0.0.0/255.0.0.0 ::1/128
    </Directory>

</VirtualHost>

The reason why the OpenSSL configuration does not need a Listen predicate is because Debian's /etc/apache2/ports.conf contains:

<IfModule mod_ssl.c>
    Listen 443
</IfModule>

Credentials

Put the following into /etc/apache2/ssl/apache.pem.

The key/certificate was generated using OpenSSL by invoking:

mkdir /etc/apache2/ssl
/usr/sbin/make-ssl-cert /usr/share/ssl-cert/ssleay.cnf /etc/apache2/ssl/apache.pem

Put the following in /etc/apache2/ssl/dh.pem:

-----BEGIN DH PARAMETERS-----
MIGHAoGBANZ95EDLu9wZNtaT00r9CtUMhNI5pF9SC7iBdMuYvOlRhJ+RLmOccvsT
tLTXF34W1VrBebpCCyop/jJKRnpjXoH/WQE3e+3c/TMWikYarTty2uiGAHgEWwen
28p4dAh9FRDqn8yd3TMFB91i24iuqnR94PTW4r1osOc5Pg8kIY6zAgEC
-----END DH PARAMETERS-----

This is the Diffie-Hellman parameters extracted from mod_ssl. The file is only used by mod_gnutls. See ModSSLDHParams for more information.

Put the following into /etc/apache2/ssl/apache-dsa.pem:

This was generated using GnuTLS, and is in fact part of our set of test credential.

Factors influencing performance

A number of non-implementation related factors that influence benchmarking have been identified:

The TLS cipher suite used

Generally, all DHE ciphers are slower than their corresponding non-DHE cipher suite. Make sure you confirm the chosen cipher by using tools like Wireshark.

The size of the Diffie-Hellman parameters used

Unfortunately, mod_ssl does not allow users to configure the DH parameters to use. To make sure the comparison is fair, we configure GnuTLS to use the same hard-coded DH parameter as used by mod_ssl. See ModSSLDHParams for more information.

The size of the downloaded file

This shouldn't be a surprise. When we begin comparing compression support, the content of the data will matter too.

Bugs in mod_gnutls

Mod_gnutls is experimental code, so you should expect bugs. One thing I've noticed are related to TLS session caching with DBM. The logs will contain:

[Wed Mar 05 17:04:02 2008] [notice] (89025)Error string not specified yet: [gnutls_cache] error storing in cache '/var/cache/apache2/gnutls_cache'
PANIC: Invalid argument
PANIC: fatal region error detected; run recovery
/var/cache/apache2/gnutls_cache page 25 is on free list with type 13

UPDATE: This was because the session DB grew huge and due to limitations of the default apache dbm database new insertions were prevented with this error. This error is suppressed in newer versions of mod_gnutls.

Since we won't do any TLS resumption in these benchmarks, you could disable such caching for mod_gnutls by making sure your /etc/apache2/mods-available/gnutls.conf looks like:

# mod_gnutls can optionaly use a memcached server to store SSL sessions.  This                                                         
# is useful in a cluster environment, where you want all your servers to share                                                         
# a single SSL session cache.                                                                                                          
#GnuTLSCache memcache "127.0.0.1 server2.example.com server3.example.com"                                                              

# The default method is to use a DBM backed cache.  It is not super fast, but                                                          
# it is portable and does not require another server to be running like                                                                
# memcached.                                                                                                                           
#GnuTLSCache dbm /var/cache/apache2/gnutls_cache                                                                                       

GnuTLSCache none none

Helper scripts

For switching between GnuTLS and OpenSSL configurations, I have written two small scripts. The first is apache-gnutls and restarts apache configured with mod_gnutls:

#!/bin/sh
sudo a2dissite openssl
sudo a2dismod ssl
sudo a2ensite gnutls
sudo a2enmod gnutls
sudo /etc/init.d/apache2 stop
sleep 2
sudo /etc/init.d/apache2 stop
sudo rm -f /var/cache/apache2/gnutls_cache
ps auxww|grep -i -e htt -e apa|grep -v tail|grep -v grep
sudo /etc/init.d/apache2 start
sleep 3
curl --insecure https://localhost/doc/ 2>&1 |tail -2|head -1

The second is apache-openssl which restarts apache configured for mod_ssl:

#!/bin/sh
sudo a2dissite gnutls
sudo a2dismod gnutls
sudo a2ensite openssl
sudo a2enmod ssl
sudo /etc/init.d/apache2 stop
sleep 2
sudo /etc/init.d/apache2 stop
ps auxww|grep -i -e htt -e apa|grep -v tail|grep -v grep
sudo /etc/init.d/apache2 start
curl --insecure https://localhost/doc/ 2>&1 |tail -2|head -1

Chosing another cipher suite

To use the TLS_RSA_WITH_AES_128_CBC_SHA (0x002f) add the following to the mod_ssl configuration:

	SSLCipherSuite AES128-SHA

And to the mod_gnutls configuration:

	GnuTLSPriorities NONE:+VERS-TLS1.0:+AES-128-CBC:+RSA:+SHA1:+COMP-NULL

To use the TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x000a) ciphersuite, use the following configuration for mod_ssl:

	SSLCipherSuite DES-CBC3-SHA

And the following for mod_gnutls:

	GnuTLSPriorities NONE:+VERS-TLS1.0:+3DES-CBC:+RSA:+SHA1:+COMP-NULL

To use TLS_DHE_DSS_WITH_RSA_128_CBC_SHA (0x0032) you need to load the DSA certificate instead, and then use for mod_ssl:

	SSLCipherSuite DHE-DSS-AES128-SHA

And for gnutls:

	GnuTLSPriorities NONE:+VERS-TLS1.0:+AES-128-CBC:+DHE-DSS:+SHA1:+COMP-NULL

Note that you have to replace the key/certificate with the DSA version, so the configuration should look like, for mod_ssl:

	SSLCertificateFile /etc/apache2/ssl/apache-dsa.pem
	SSLCertificateKeyFile /etc/apache2/ssl/apache-dsa.pem
	SSLCACertificateFile /etc/apache2/ssl/apache-dsa.pem

And for mod_gnutls:

	GnuTLSCertificateFile /etc/apache2/ssl/apache-dsa.pem
	GnuTLSKeyFile /etc/apache2/ssl/apache-dsa.pem
	GnuTLSClientCAFile /etc/apache2/ssl/apache-dsa.pem

Results

Benchmark results are collected on a separate page, see BenchmarkingModGnuTLSResults.

Download in other formats:

Plain Text