King23 Benchmark – playing a numbers game

Several People have been asking me why i actually started writing King23,
with that many OpenSource Frameworks arround.

Well, first of all, because i enjoy doing that. Second is my dislike for some of the
things other frameworks do. When i started with ECOTRA/Northern Crusade
(the first website running on this framework), i looked at several PHP Frameworks
and decided to do King23, basically because i did not need most functionality of
other Frameworks, because i wanted to use SithTemplates as a templating system,
and because of the resource/performance waste that comes with the large frameworks.

Today i had a bit of free time and decided, for the fun of it, to do a bit of
speed testing on a few Frameworks.
The Setup for this is quite simple, the machine i tested on is a cloud level 1
jiffybox
running a basic 64bit Debian 5.0 with Apache + PHP 5.2 + suhosin.
The test itself is a simple apache bench with 500 requests
and a concurrency of 20. (please note that i run ab on the same machine as the webserver).

Please note that i’ve chosen to use PHP 5.2 here, since php 5.3 is still a bit
of an underdog at most hosters, and since none of the frameworks that i tested required
5.3 functionality.

For each framework that i tested i set up a virtual host, and run pretty
much with the default install, with the exception of King23, i used the http://king23.net
websites source for this (since the default starting page barely uses the template system).

Now the values that where of interest from me where Requests per second
and time per request.

cakephp:

Requests per second:    23.02 [#/sec] (mean)
Time per request:       868.960 [ms] (mean)

Cakephp was the slowest of all that i tested, however i want to add that
cake did some checks on the filesystem if the caches are readable and such,
so it did a bit more then just showing a welcome page.

symfony:

Requests per second:    29.75 [#/sec] (mean)
Time per request:       672.324 [ms] (mean)

Symfony didnt really have many suprises for me, we are using it at work,
so i was aware of its performance.

zend:

Requests per second:    54.09 [#/sec] (mean)
Time per request:       369.784 [ms] (mean)

Zend actually did suprise me, i expected it to be at arround
the same speed as symfony if not even slower, but it seems
that Zend doesnt load as much unneeded stuff as cake and symfony
do (symfony included several doctrine related classes that where not needed for the example, cake included some form-handling classes that where not needed either)

king23:

Requests per second:    209.76 [#/sec] (mean)
Time per request:       95.345 [ms] (mean)

Not really a suprise here either, king23’s smaller callstacks, less includes, and more simple structure make a difference. Even with using a more complex template system than symfony for example.

Now before people start crucifying me, let me admit it first: Yes this Benchmark is unfair!!!
As mentioned before, King23 does do alot less, also you can get alot of performance out of all frameworks
if you specifically tune the system for that. Also this benchmarks doesnt say much about the quality,
or about the performance under real conditions. If you add Database stuff to it, if you add a bit more logic
the results will be much closer to eachother then they are in this simple case.

However, before people claim that i tuned the setup for King23,
or tuned the King23 website for the benchmark – i didn’t. Its not using any additional caches,
besides the template cache, meaning for example that it is building the maps for the Autoloader
each time etc. Just to show how a small change in the setup can influence the outcome:
king23+m

Requests per second:    2222.91 [#/sec] (mean)
Time per request:       8.997 [ms] (mean)

This is the same sourcecode, the same application configuration etc. than used before, all i did was
kick apache2 out of the equation, in favour of a php extension called “mistral”, more on mistral in another post soon..

Advertisements

One thought on “King23 Benchmark – playing a numbers game

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s