This page should serve as a place to discuss, document and improve automated testing of plugins.
Here are some ideas:
- Document a standardized way for plugin tests (That's what this document aims to become ;-))
- Define a default place for plugin tests (probably 'plugins/myPlugin/test' )
- Create a task to run a plugin's tests
- Enhance the symfony test:all task to run a plugins test, preferable with an option to inlude/exclude plugin tests
Everyone is kindly invited to add his/her experiences!
RFC: include plugin tests in the symfony test:all/unit/functional tasks
We (Bernhard Schussek and I, Klemens Ullmann) thought it would be useful to have a simpler way executing plugin tests. The most obvious solution is to enhance the existing 'symfony test:all/unit/functional' tasks to include this functionality. Transparently testing the plugins is especially handy, if you develop a lot of selfmade plugins for the sake of structure and reuseabilty. Just run 'test:all' and roll...
You can find the patch here: #4873
And a related google-groups thread here: http://groups.google.com/group/symfony-devs/browse_thread/thread/e5b37d99aa41dd4c?hl=en Futhermore a detailed post by Bernhard: http://groups.google.com/group/symfony-devs/browse_thread/thread/5c38ea3cc619c6fd?hl=en
Enable plugin tests
First of all you want to define which plugins to include. It doesn't make much sense to constantly test 3rd party plugins.
Enable the desired plugins in the setup() method of your project's ProjectConfiguration? class ('config/ProjectConfiguration.class.php'):
Enable a single plugin with:
or give multiple plugins as array:
Executing symfony test:all will now include the tests of your desired plugins.
Don't execute any plugin tests at all with the --no-plugins option:
symfony test:all --no-plugins
Execute only the tests of one plugin:
symfony test:all --plugin=myFirstPlugin
Note: this works for ALL plugins, even those not defined in the ProjectConfiguration? class:
symfony test:all --plugin=my3rdPartyPlugin
It works similar for symfony test:unit - all unit tests of your desired plugins are executed.
You can again disable the plugin tests with the --no-plugins option.
You can execute the unit tasks for a single plugin as well:
symfony test:unit --plugin=myFirstPlugin
For symfony test:functional it works a bit differently, since you always launch functional tests for an application, or now for a plugin:
symfony test:functional myFirstPlugin
A way to include symfony core
I would love to see a switch to allow for the testing of symfony included
symfony test:all --symfony-core
Bernhard: Could you elaborate? Where would you specify the symfony directory where the tests are stored? What is the specific use case of this feature?
Stephen: It would use the actual symfony tests. A use case for this is when you take a product from a dev server to the stage and/or live server you could run all test to make sure you didn't forget about something or some unforeseen limitation of the production server. The tests specifically are part of symfony. I believe there are about 7000 tests as part of the core of symfony now.
Bernhard: I see, sounds very reasonable. One problem I see with this option though is that it depends on the way symfony is installed. If you install symfony via PEAR, you don't have the tests installed. Same for projects that integrated symfony using "symfony freeze". These are the officially proposed ways of installing symfony though.
I think people who want to run the symfony tests in production should just integrate symfony into their projects using svn:externals. Then you can simply run the prove.php script manually.
Why not use sfTaskExtraPlugin?
There is a CLI task for testing plugins in the sfTaskExtra plugin. Is there a reason why it would not be sufficient? Just wondering as I'm about to start writing some tests for a plugin of mine. -ezzatron