sfToolkit::stripComments 1.2..2.5x optimization

Performance test

Patch

The main technics are:

1. cache constants because they are slow.
2. use a hashtable (assoc array) for defining list of tokens to exclude.
3. is_string($token) is replaced by isset($token[1]) which is a few times faster and completely valid in this case.

PS: my nickname is Wicked and not Wicket :)

Awesome work Wicked! (sorry bout the typo)

i think witspace could be removed as well, but perhaps fabien wants to keep them because a) whitespace removal is more dangerous, because it contains syntax information. and b) because you might still want to read the code.

comment removal is also much more efficient, because it is mainly intended for phpdoc which should ideally be same amount as payload code.

ill doublecheck your patch and apply it, looks fine for me

ps i am getting 10 times better performance on my windows box :-)

(In [13869]) [1.2] applied performance patch for sfToolkit::stripComments from Wicked. Thanks for providing those well done patches. fixes #5237

Hm... it seems that the 2.5x gain is only with xdebug enabled which I accidentally forgot to disable. The code on my ubuntu box produces the following output:

warmup: 
total: 1.71136522293
min:   0.0408270359039
original: 
total: 0.69099521637
min:   0.0159559249878 <-
my2: 
total: 0.522416591644
min:   0.0122780799866
my3: 
total: 0.503157138824
min:   0.0119969844818 <-

so the gain is about 0.016 -> 0.012 which is about only 25%, practically the same I see on windows. This is still some positive result, and the 2.5x is still applicable to those who use xdebug on their development servers. So hopefully you won't revert the [13869].

PS: How did you get the 10x performance? :)

warmup:
total: 5.7169368267059
min:   0.13894510269165
original:
total: 4.8123345375061
min:   0.11645984649658
my2:
total: 0.42525219917297
min:   0.01030707359314
my3:
total: 0.50104832649231
min:   0.011872053146362
 
PHP 5.2.8 (cli) (built: Dec  8 2008 19:31:23) - Windows XP SP 2

see? :-)

unfortunately your case seems too rare :) Do you have any php extensions like xdebug or eaccelerator?

I've tested on 2 other battle servers with different OSes: the gain is still about 22%..23%.

[root@*** ~]# cat /etc/issue
CentOS release 5.2 (Final)
Kernel \r on an \m
[root@*** ~]# php -v
PHP 5.2.6 (cli) (built: May  5 2008 14:41:03) 
Copyright (c) 1997-2008 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies
[root@*** ~]# php stripComments.php 
warmup: 
total: 0.69787049293518
min:   0.017158985137939
original: 
total: 0.26690006256104
min:   0.0065479278564453
my2: 
total: 0.21836137771606
min:   0.0053780078887939
my3: 
total: 0.20673632621765
min:   0.0051088333129883

[17][18:14:06][wicked@***]# cat /etc/issue
Fedora Core release 6 (Zod)
Kernel \r on an \m
[18][18:17:50][wicked@***]# php -v
PHP 5.2.2 (cli) (built: Oct  9 2008 19:10:11) 
Copyright (c) 1997-2007 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies
[19][18:18:00][wicked@***]# php stripComments.php 
warmup: 
total: 1.2217893600464
min:   0.029760837554932
original: 
total: 0.47615599632263
min:   0.01169490814209
my2: 
total: 0.39156889915466
min:   0.0095779895782471
my3: 
total: 0.36756992340088
min:   0.0089600086212158

(In [13892]) [1.1] backported performance patch for sfToolkit::stripComments. refs #5237

(In [13893]) [1.0] backported performance patch for sfToolkit::stripComments. refs #5237