Not porting anything . I started out trying to create a simple RTTI example for the book, but the next thing I knew, I had a pretty good base for a no-kidding "next-gen" DUnit framework.
Stefan Glienke Indeed, also unless I'm mistaken, the RTTI part mostly comes into play as an alternative to a parser for generating empty test templates. And you still need something like Delphi Code Coverage to ensure the tests are meaningful.
Stefan Glienke I'm not a fan of that approach, as it makes debugging a particular test that fails a pita (need more clicks and filtering in the tests UI, while with the classic code approach, you just place a breakpoint).
Eric Grange Executing only failed tests is one click in the DUnit UI. Ime it makes reading test cases much easier and you avoid much repeating the same code over and over (DRY also applies to test code)
Stefan Glienke You need to click in the UI & place the breakpoint vs just placing the breakpoint, also it means you're using the UI runner (usually with DUnit I'm not using the UI runner because the UI updates slow everything down when you've got many tests, I'm using a text runner). Also color me not convince about making the code more readable, when you have a simple function with 2 simple parameters and a couple cases, maybe, but when you have a dozen test cases, more parameters or record/array parameters? Also with attibutes you've got to enclose those attributes with "ifdef" or they'll end up in your executable, thus adding to the ugly.
Eric Grange I guess you are not very familar with NUnit. The attributes are on the testcase methods and thus don't need any ifdefs. And of course they only work with data types that can be passed to an attribute constructor.
Stefan Glienke I still fail to see why they're simpler than test cases? You can't break-point on attributes, so you need to jump between two UIs to debug anything (the test UI + the IDE).
Stefan Glienke I find DUnit UI very slow once you have a large number of tests, so yes, I just run everything from a text runner, as that's a lot faster than starting the UI and navigating the tests, or doing one run from the UI to be able to click the "select failed tests" button. Unit tests are simple, once you take out the DUnit UI, even a very large number of them can run in a negligible amount of time. I'm talking thousandths of tests. Even on not-really-unit-tests like the DWScript test suite, where most tests involve compiling one or many scripts, the UI is by far taking the lion's share of execution time.
Eric Grange If you don't enable runtime themes the DUnit UI is as fast as the console (just tested with DWS tests, ~5-6secs both) - but if you enable it takes double the time.
Stefan Glienke AFAICT no significant difference here: the bulk of the time when loading is still spent filling up the treeview, and then setting up the check boxes. And when running, most of the times goes to repainting the treeview as it scrolls down the tests. How many tests do you have? More than 10k tests here.
Eric Grange DWS has 269 as you should know. And at work when we are writing unit tests we don't put them all into one project but create several small DUnit projects and let them run by the build server (which of course runs them with the xmltestrunner). So I barely get into the situation of having hundreds or more cases in one project.
Stefan Glienke If individual checks in DWS suite were materialized as different tests (as with the Attributes approach), the DWS suite would bump into the ten thousand as well. The first test class does hundreds of checks per DUnit-tree entry, initially at some point they were enumerated as individual tests, but then I had to backtrack and regroup them. Having to split into multiple project is a workaround in my book, I've seen multiple .Net, Java & JS test suites running in the thousandths, and some C ones in the millions (happen as soon as you want to reach high test coverage, there is just no way around having lots and lots of test cases).
Stefan Glienke What would you say is the base function of your DUnit extensions? My attempt is head towards defining tests with [TestFixture] and [Test], etc. Any class at all can be a TestFixture, and any method of a TestFixture can be a Test.
Nick Hodges My extensions make it possible to write published test case methods with parameters (which is not possible with vanilla DUnit). The argument values are then passed by the attributes you specify.
failure?! :evilsmile:
ReplyDeletemaybe DRTTIUnit?
DMetaUnitTest
ReplyDeleteRTUT
ReplyDeleteRun-time Unit Testing
DUnitXE
ReplyDeleteDolus.
ReplyDeleteTRUST : Trust Requires Unit and System Tests. Want code you can TRUST? Use TRUST.
ReplyDeleteWarren Postma Nice!
ReplyDeleteAT-AT - All Tests Always Tested ;)
ReplyDeleteUse the source, Luke!
A port of JUnit 4?
ReplyDeleteNot porting anything . I started out trying to create a simple RTTI example for the book, but the next thing I knew, I had a pretty good base for a no-kidding "next-gen" DUnit framework.
ReplyDelete"Lucky Tests" because Delphi RTTI is still quite a bit about luck (missing small bits here or there for some parameters, some types, etc.).
ReplyDeleteDRUNITTI
ReplyDeleteWhy create something entirely new when you can build ontop of the current DUnit (like DSharp DUnit extensions)?
ReplyDeleteStefan Glienke Indeed, also unless I'm mistaken, the RTTI part mostly comes into play as an alternative to a parser for generating empty test templates. And you still need something like Delphi Code Coverage to ensure the tests are meaningful.
ReplyDeleteIn my DUnit extensions RTTI/Attributes are used to provide the test data, like NUnit does it (http://www.nunit.org/index.php?p=testCase&r=2.6.1)
ReplyDeleteI think Nicks intention is to go into the BDD direction.
Stefan Glienke I'm not a fan of that approach, as it makes debugging a particular test that fails a pita (need more clicks and filtering in the tests UI, while with the classic code approach, you just place a breakpoint).
ReplyDeleteAutoDUnit
ReplyDeleteDVnit.
ReplyDeleteEric Grange Executing only failed tests is one click in the DUnit UI. Ime it makes reading test cases much easier and you avoid much repeating the same code over and over (DRY also applies to test code)
ReplyDeleteStefan Glienke You need to click in the UI & place the breakpoint vs just placing the breakpoint, also it means you're using the UI runner (usually with DUnit I'm not using the UI runner because the UI updates slow everything down when you've got many tests, I'm using a text runner). Also color me not convince about making the code more readable, when you have a simple function with 2 simple parameters and a couple cases, maybe, but when you have a dozen test cases, more parameters or record/array parameters?
ReplyDeleteAlso with attibutes you've got to enclose those attributes with "ifdef" or they'll end up in your executable, thus adding to the ugly.
Eric Grange I guess you are not very familar with NUnit. The attributes are on the testcase methods and thus don't need any ifdefs. And of course they only work with data types that can be passed to an attribute constructor.
ReplyDeleteStefan Glienke I still fail to see why they're simpler than test cases? You can't break-point on attributes, so you need to jump between two UIs to debug anything (the test UI + the IDE).
ReplyDeleteDo you run all unit tests to debug a failing one?
ReplyDeleteI don't. I just run the failed one and debug that.
Maybe you need to get an idea how my DUnit extensions work: http://stackoverflow.com/questions/8999945/can-i-write-parameterized-tests-in-dunit/9006662#9006662
Stefan Glienke I find DUnit UI very slow once you have a large number of tests, so yes, I just run everything from a text runner, as that's a lot faster than starting the UI and navigating the tests, or doing one run from the UI to be able to click the "select failed tests" button.
ReplyDeleteUnit tests are simple, once you take out the DUnit UI, even a very large number of them can run in a negligible amount of time. I'm talking thousandths of tests.
Even on not-really-unit-tests like the DWScript test suite, where most tests involve compiling one or many scripts, the UI is by far taking the lion's share of execution time.
I'd like a cmd line version too, so that the results can be checked in script files.
ReplyDeleteMichael Thuma don't think TA-TA would be a good idea. Here in the U.S. ta-tas is slang for a female's breasts. Some people might get the wrong idea.
ReplyDeleteEric Grange If you don't enable runtime themes the DUnit UI is as fast as the console (just tested with DWS tests, ~5-6secs both) - but if you enable it takes double the time.
ReplyDeleteStefan Glienke AFAICT no significant difference here: the bulk of the time when loading is still spent filling up the treeview, and then setting up the check boxes. And when running, most of the times goes to repainting the treeview as it scrolls down the tests. How many tests do you have? More than 10k tests here.
ReplyDeleteEric Grange DWS has 269 as you should know. And at work when we are writing unit tests we don't put them all into one project but create several small DUnit projects and let them run by the build server (which of course runs them with the xmltestrunner). So I barely get into the situation of having hundreds or more cases in one project.
ReplyDeleteMichael Thuma - We could always opt for ...
ReplyDeleteRAD-TA-TA's !
TestSmile
ReplyDeleteI propose Test Instant Quality Leveling => Test IQL .. :)
ReplyDeleteIQ Test (Interim Quality Test)
ReplyDeleteStefan Glienke If individual checks in DWS suite were materialized as different tests (as with the Attributes approach), the DWS suite would bump into the ten thousand as well. The first test class does hundreds of checks per DUnit-tree entry, initially at some point they were enumerated as individual tests, but then I had to backtrack and regroup them.
ReplyDeleteHaving to split into multiple project is a workaround in my book, I've seen multiple .Net, Java & JS test suites running in the thousandths, and some C ones in the millions (happen as soon as you want to reach high test coverage, there is just no way around having lots and lots of test cases).
Stefan Glienke What would you say is the base function of your DUnit extensions? My attempt is head towards defining tests with [TestFixture] and [Test], etc. Any class at all can be a TestFixture, and any method of a TestFixture can be a Test.
ReplyDeleteNick if you're heading towards Behaviour Driven Development, have you looked at Kiwi, it's a really cool objective-C bdd + unittest library.
ReplyDeleteWarren Postma Not sure where the notion that I'm heading towards BDD came from. I'm trying to head more towards NUnit. ;-)
ReplyDeleteNick Hodges My extensions make it possible to write published test case methods with parameters (which is not possible with vanilla DUnit). The argument values are then passed by the attributes you specify.
ReplyDelete