Don’t let the title fool you… JUnit, NUnit, FlexUnit, and I have gone round and round for weeks since my last post. Every time I thought I had adequately addressed the issue, a new crop of discoveries would start the battle over again. Despite the rough beginnings, however, I couldn’t be happier with the outcome. Furthermore, the functional distinctions between FUnit and FlexUnit have gone beyond how tests are created and deeper into how they are written. Here is what I’ve discovered.
Discovery 1: All xUnit Frameworks are not created equal.
You may have noticed the addendum in my last post concerning JUnit vs. NUnit array equality. This was my first clue that striking the right balance between authoring language and developer intent was more art than science. The xUnit pattern is a means of providing ubiquity across languages, while it is the responsibility of the framework instance to tailor to the needs of the language. With this in mind, controlled deviations among testing frameworks is not only common but preferable.
Note the JUnit source code for ‘assertEquals’ here:
static public void assertEquals(String message, Object expected, Object actual) {
if (expected == null && actual == null)
return;
if (expected != null && expected.equals(actual))
return;
failNotEquals(message, expected, actual);
}As you see in the above JUnit example, ‘assertEquals’ relies on a call to the ‘expected’ data types Object.equals() implementation. I confess, I was under the false impression that this method served as a member-wise equality comparer. On the contrary, the default behavior for Object.equals() is actually reference equality. I also discovered that this method is rarely overridden outside of primitive types such as String, Boolean, Double, Single, etc. This means that in terms of JUnit, ‘assertEquals’ behaves more like ‘assertSame’ in most cases.
NUnit takes a different approach to object equality. In addition to performing value comparison of primitive types, NUnit adds a few others… Array, ICollection (equatable to IList in AS3), Stream and Date. In such cases, NUnit ‘areEqual’ ensures that the value of the comparing types are equal without relying solely on reference equality. Object.equals() is still used by NUnit but only after all other value comparisons have failed.
The following array equality assertion in NUnit will pass.
String[] array1 = { "one", "two", "three" };
String[] array2 = { "one", "two", "three" };
Assert.assertEquals(array1, array2);There is a similar method available in JUnit called Assert.assertArrayEquals(). I’d like to point out here that there are actually two Assert classes in JUnit, ‘junit.frameworks::Assert’ and ‘org.junit::Assert’. This is a good indicator that JUnit has been rethinking its testing approach as well. Legacy support is a primary factor of “why things are they way they are”. Once an established framework like JUnit has put its stakes in the ground, it’s hard to move them.
Discovery 2: All languages are not created equal… duh.
The JUnit ‘assertEquals’ code sample is a bit misleading. What you don’t see is that there are actually 19 other methods just like it. These method overloads serve as a form of type equality check. In order for two types to be compared by value, they must be of equivalent types. In fact, in some languages comparison of incompatible types won’t even compile.
Compilation of the following condition in C# will fail.
Boolean value = (123456 == "123456") ? true : false;
// The following error will be thrown
Operator '==' cannot be applied to operands
of type 'int' and 'string'Unfortunately, method overloading is not even supported in ActionScript 3. Equally unfortunate, FlexUnit makes no attempt to account for such type mismatches. Type equality is a key component of value equality. The inclusion of FlexUnit ‘assertStrictEquals’ is a step in the right direction but still relies heavily on developers to do the leg work.
The following equality assertion in FlexUnit will pass.
var xml1:XML = <value>Hello World</value>;
var xml2:XML = <value>Hello World</value>;
Assert.assertEquals(xml1, xml2);
The following equality assertions in FlexUnit will fail.
var xml1:XML = <value>Hello World</value>;
var xml2:XML = <value>Hello World</value>;
Assert.assertStrictEquals(xml1, xml2);
In this example, standard equality (==) on XML is a value comparison while strict equality (===) is a reference comparison. It becomes a requirement, therefore, that a developer be intimately familiar with the nuances of each equality type and its variances across data types. I would argue that maintaining separation between Assert.areEqual() and Assert.areSame() is much clearer to the developer (but much harder on me).
Discovery 3: FlexUnit is to JUnit as FUnit is to NUnit.
The more I study the mechanics of the JUnit and NUnit testing frameworks, the clearer the distinction is between the two. Given ActionScript’s strong Java influence, it makes sense that Adobe would choose to pattern FlexUnit after JUnit. I will also say that the developers of FlexUnit did an oustanding job mirroring the testing patterns of JUnit. If you can unit-test in Java with JUnit then you can unit-test in Flash with FlexUnit and vice versa. That’s a very powerful thing.
There’s just one problem… I don’t like JUnit… at all.
Due to patterns established by JUnit, I believe it has made FlexUnit unnecessarily complex for both framing and writing tests. In fact, it was my initial frustrations with FlexUnit (and the JUnit pattern) that led me to establish the FUnit framework. More specifically, I didn’t have the same frustrating experience when using NUnit. Here’s just a couple of ways that NUnit spoiled me.
- No need to extend the TestCase base class
- Tests are easily flagged with the [Test] metadata tag
- No “test” prefix is required for TestCase methods to be reflected
- Expected errors are marked with an [ExpectedError] tag (no wrapper code)
- Supports SetUp, TearDown, FixtureSetUp, and FixtureTearDown
It should come as no surprise that I’ve sided with NUnit on this one. A simple side-by-side comparison of FlexUnit vs. FUnit markup makes it pretty obvious why I like it.
There’s far more to this topic than I was able to post here but I hope I’ve provided sufficient background to describe the issues at hand. For those of you pulling regular svn updates, the core changes have been implemented and are available for use. Since I’d rather not unnecessarily muddle the problem and solution, a detailed explanation of these changes merits an article of it’s own. Look for this article shortly along with an official update release.
