This post represents an exhaustive list of supported value equatable object types. Relevant implementation details and code samples are available for each.

• Primitive types: int, uint, Number, Boolean, and String
• Coerced types: Date, XML, XMLList, Namespace, and QName
• Complex types: Array, ByteArray, and IList

Primitive types:

Assert.areEqual( 123.456, 123.456 );
Assert.areNotEqual( 123, 123.456 );
 
// Note: The following will pass in FUnit but fail in FlexUnit.
//       The condition (NaN == NaN) will actually fail in
//       ActionScript so special handling is required.
Assert.areEqual( NaN, NaN );

Assert.areEqual( true, true );
Assert.areNotEqual( true, false );
 
// Although 'isTrue' or 'isFalse' is more appropriate here.
Assert.isTrue( true );

Assert.areEqual( "hello world!", "hello world!" );
Assert.areNotEqual( "hello world!", "goodbye world!" );
 
// The StringAssert class is better suited for more
// advanced string assertions.
StringAssert.areEqualIgnoringCase( "hello!", "HELLO!" );
StringAssert.isEmpty( "" );
StringAssert.contains( "world", "hello world!" );
StringAssert.startsWith( "hello", "hello world!" );
StringAssert.endsWith( "!", "hello world!" );
// and so on ...

Coerced types:

Important Note: In order to maintain simplicity in expected behavior, value coercion is limited to global top-level classes only. A ‘quick-reference’ should not be necessary while writing equality tests. The only top-level classes not currently handled are Error and RegExp. This is largely due to the fact that they are immutable objects (cannot be changed after they are created). Pending further discussions and/or feedback, value coercion for these types will likely be added as well.

Assert.areEqual( new Date(123456), new Date(123456) );
Assert.areNotEqual( new Date(123456), new Date(654321) );
 
// Note: Unlike the .NET environment ActionScript does not
//       treat dates as a value type. Therefore, FUnit will
//       coerce the Date class to evaluate equality.
//       For reference equality use 'Assert.areSame()'.
Assert.areNotSame( new Date(123456), new Date(123456) );

var xml1:XML = <value>Hello World</value>;
var xml2:XML = <value>Hello World</value>;
Assert.areEqual( xml1, xml2 );
 
// Similar to Date class value coercion, FUnit can distinguish
// value equality from reference equality.
Assert.areNotSame( xml1, xml2 );

var ns1:Namespace =
     new Namespace( "funit", "http://www.funit.org/2009/" );
var ns2:Namespace =
     new Namespace( "funit", "http://www.funit.org/2009/" );
 
Assert.areEqual( ns1, ns2 );
Assert.areNotSame( ns1, ns2 );
 
var qname1:QName =
     new QName ( "http://www.funit.org/2009/", "funit" );
var qname2:QName =
     new QName ( "http://www.funit.org/2009/", "funit" );
 
Assert.areEqual( qname1, qname2 );
Assert.areNotSame( qname1, qname2 );

Complex types:

Assert.areEqual( ["x", "y", "z"], ["x", "y", "z"] );
Assert.areNotEqual( ["x", "y", "z"], ["x", "y", "a"] );
 
// Associative Arrays are also handled.
var array1:Array = new Array();
array1["firstName"] = "John";
array1["middleName"] = "Adam";
array1["lastName"] = "Doe";
 
var array2:Array = new Array();
array2["firstName"] = "John";
array2["middleName"] = "Adam";
array2["lastName"] = "Doe";
 
Assert.areEqual( array1, array2 );
 
// Complex recursive arrays are also handled by internally
// checking for re-entrant behaviors and preventing stack
// overflow.
var recursive1:Array = new Array( null, ["x", "y", "z"], [] );
recursive1[0] = recursive1;
 
var recursive2:Array = new Array( null, ["x", "y", "z"], [] );
recursive2[0] = recursive2;
 
// Note: No stack overflow will occur here, but more variations
//       and complex samples are needed to ensure all potential
//       loopholes are handled.
Assert.areEqual( recursive1, recursive2 );

var array1:ByteArray = new ByteArray();
array1.writeDouble( Math.PI );
array1.writeUTF( "Hello World" );
array1.writeObject( ["x", "y", "z"] );
 
var array2:ByteArray = new ByteArray();
array2.writeDouble( Math.PI );
array2.writeUTF( "Hello World" );
array2.writeObject( ["x", "y", "z"] );
 
Assert.areEqual( array1, array2 );
Assert.areNotSame( array1, array2 );

var set1:ArrayCollection =
     new ArrayCollection( ["x", "y", "z"] );
var set2:ArrayCollection =
     new ArrayCollection( ["x", "y", "z"] );
 
Assert.areEqual( set1, set2 );
Assert.areNotSame( set1, set2 );

The next major revision to the framework core is now available for download. Although there are few changes to the core runtime engine, this release represents significant research surrounding semantics of the Assert ‘areEqual’ method. More in depth coverage of these issues are found here and here.

A detailed article outlining the effects of this change will be available within the next few days.

Don’t let the title fool you… JUnit, NUnit, FlexUnit, and I have gone round and round for weeks since my last post. Every time I thought I had adequately addressed the issue, a new crop of discoveries would start the battle over again. Despite the rough beginnings, however, I couldn’t be happier with the outcome. Furthermore, the functional distinctions between FUnit and FlexUnit have gone beyond how tests are created and deeper into how they are written. Here is what I’ve discovered.

Discovery 1: All xUnit Frameworks are not created equal.

You may have noticed the addendum in my last post concerning JUnit vs. NUnit array equality. This was my first clue that striking the right balance between authoring language and developer intent was more art than science. The xUnit pattern is a means of providing ubiquity across languages, while it is the responsibility of the framework instance to tailor to the needs of the language. With this in mind, controlled deviations among testing frameworks is not only common but preferable.

static public void assertEquals(String message, Object expected, Object actual) {
	if (expected == null && actual == null)
		return;
	if (expected != null && expected.equals(actual))
		return;
	failNotEquals(message, expected, actual);
}

As you see in the above JUnit example, ‘assertEquals’ relies on a call to the ‘expected’ data types Object.equals() implementation. I confess, I was under the false impression that this method served as a member-wise equality comparer. On the contrary, the default behavior for Object.equals() is actually reference equality. I also discovered that this method is rarely overridden outside of primitive types such as String, Boolean, Double, Single, etc. This means that in terms of JUnit, ‘assertEquals’ behaves more like ‘assertSame’ in most cases.

NUnit takes a different approach to object equality. In addition to performing value comparison of primitive types, NUnit adds a few others… Array, ICollection (equatable to IList in AS3), Stream and Date. In such cases, NUnit ‘areEqual’ ensures that the value of the comparing types are equal without relying solely on reference equality. Object.equals() is still used by NUnit but only after all other value comparisons have failed.

String[] array1 = { "one", "two", "three" };
String[] array2 = { "one", "two", "three" };
Assert.assertEquals(array1, array2);

There is a similar method available in JUnit called Assert.assertArrayEquals(). I’d like to point out here that there are actually two Assert classes in JUnit, ‘junit.frameworks::Assert’ and ‘org.junit::Assert’. This is a good indicator that JUnit has been rethinking its testing approach as well. Legacy support is a primary factor of “why things are they way they are”. Once an established framework like JUnit has put its stakes in the ground, it’s hard to move them.

Discovery 2: All languages are not created equal… duh.

The JUnit ‘assertEquals’ code sample is a bit misleading. What you don’t see is that there are actually 19 other methods just like it. These method overloads serve as a form of type equality check. In order for two types to be compared by value, they must be of equivalent types. In fact, in some languages comparison of incompatible types won’t even compile.

Boolean value = (123456 == "123456") ? true : false;
 
// The following error will be thrown
Operator '==' cannot be applied to operands
     of type 'int' and 'string'

Unfortunately, method overloading is not even supported in ActionScript 3. Equally unfortunate, FlexUnit makes no attempt to account for such type mismatches. Type equality is a key component of value equality. The inclusion of FlexUnit ‘assertStrictEquals’ is a step in the right direction but still relies heavily on developers to do the leg work.

var xml1:XML = <value>Hello World</value>;
var xml2:XML = <value>Hello World</value>;
Assert.assertEquals(xml1, xml2);

var xml1:XML = <value>Hello World</value>;
var xml2:XML = <value>Hello World</value>;
Assert.assertStrictEquals(xml1, xml2);

In this example, standard equality (==) on XML is a value comparison while strict equality (===) is a reference comparison. It becomes a requirement, therefore, that a developer be intimately familiar with the nuances of each equality type and its variances across data types. I would argue that maintaining separation between Assert.areEqual() and Assert.areSame() is much clearer to the developer (but much harder on me).

Discovery 3: FlexUnit is to JUnit as FUnit is to NUnit.

The more I study the mechanics of the JUnit and NUnit testing frameworks, the clearer the distinction is between the two. Given ActionScript’s strong Java influence, it makes sense that Adobe would choose to pattern FlexUnit after JUnit. I will also say that the developers of FlexUnit did an oustanding job mirroring the testing patterns of JUnit. If you can unit-test in Java with JUnit then you can unit-test in Flash with FlexUnit and vice versa. That’s a very powerful thing.

There’s just one problem… I don’t like JUnit… at all.

Due to patterns established by JUnit, I believe it has made FlexUnit unnecessarily complex for both framing and writing tests. In fact, it was my initial frustrations with FlexUnit (and the JUnit pattern) that led me to establish the FUnit framework. More specifically, I didn’t have the same frustrating experience when using NUnit. Here’s just a couple of ways that NUnit spoiled me.

• No need to extend the TestCase base class
• Tests are easily flagged with the [Test] metadata tag
• No “test” prefix is required for TestCase methods to be reflected
• Expected errors are marked with an [ExpectedError] tag (no wrapper code)
• Supports SetUp, TearDown, FixtureSetUp, and FixtureTearDown

It should come as no surprise that I’ve sided with NUnit on this one. A simple side-by-side comparison of FlexUnit vs. FUnit markup makes it pretty obvious why I like it.

There’s far more to this topic than I was able to post here but I hope I’ve provided sufficient background to describe the issues at hand. For those of you pulling regular svn updates, the core changes have been implemented and are available for use. Since I’d rather not unnecessarily muddle the problem and solution, a detailed explanation of these changes merits an article of it’s own. Look for this article shortly along with an official update release.

It is not my intention to discredit FlexUnit’s decision to diverge it’s implementation from other testing frameworks, but rather to ensure that FUnit is firmly planted in what the testing community considers the de facto standard.

I will cite JUnit and NUnit as references for xUnit best practice. These frameworks are long established and widely adopted tools for unit-testing in the Java and .NET languages. The discrepancy between JUnit/NUnit and FlexUnit can be summarized as follows:

JUnit ‘assertEquals‘ / NUnit ‘areEqual‘ assert value equality.
FlexUnit ‘assertEquals‘ asserts reference equality.

When evaluating equality of value types such as Boolean, String, Number, etc. the behavior will be similar in most cases (we’ll outline these differences later). Test outcome for Object and Array assertions, however, may be drastically different between FlexUnit and other xUnit frameworks.

Important Correction: Since posting this article I’ve discovered that the JUnit method overload Assert.assertEquals(object[], object[]) has been deprecated and replaced by Assert.assertArrayEquals(object[], object[]). Variance between JUnit and NUnit may indicate that minor inconsistencies exist between testing frameworks as a whole and is not limited to FlexUnit alone. Further discussions on this topic and how it pertains to FUnit will continue here.

String[] array1 = { "one", "two", "three" };
String[] array2 = { "one", "two", "three" };
Assert.assertArrayEquals(array1, array2);

String[] array1 = { "one", "two", "three" };
String[] array2 = { "one", "two", "three" };
Assert.assertEquals(array1, array2);

var array1:Array = [ "one", "two", "three" ];
var array2:Array = [ "one", "two", "three" ];
Assert.assertEquals(array1, array2);

The intent of ‘assertEquals’ according to xUnit convention is to ensure that the value of two objects are identical, not that they reference the same object. A second assertion type ‘assertSame’ exists in JUnit (’areSame’ for NUnit) to ensure that two objects share the same memory reference.

There are 3 possible reasons that may have lead FlexUnit developers to change the assertEquals implementation.

• Inconsistencies between the ActionScript Object class and other languages.
• Existence of two Flash equality types; equals (==) and strict equals (===).
• Flash developer expectations and implied meaning of ‘assertEquals’.

Relevant Language Differences Between ActionScript and Java and .NET

The primary difference between the ActionScript Object definition and it’s Java/.NET counterparts is the absence of an Object.equals() method. All Java/.NET objects inherit from or override this method. Unfortunately, there is no ActionScript equivalent for member-wise equality comparison.

Object.equals() plays a crucial role in the ‘assertEquals’ implementation.

static public void assertEquals(String message, Object expected, Object actual) {
	if (expected == null && actual == null)
		return;
	if (expected != null && expected.equals(actual))
		return;
	failNotEquals(message, expected, actual);
}

Standard Equality Vs. Strict Equality

It is important to note here that ’standard equality’ (==) is a looser form of equality that does not exist in Java/.NET. These languages use what Flash considers ’strict equality’ (===). Behavioral differences between these types become apparent when evaluating equality of value types (Boolean, String, Number, etc.).

int value1 = 5;
String value2 = "5";
Assert.assertEquals(value1, value2);

var value1:int = 5;
var value2:String= "5";
Assert.assertEquals(value1, value2);

According to the Flash Language Reference:

The strict equality (===) operator differs from the equality (==) operator in only two ways:

• The strict equality operator performs automatic data conversion only for the number types (Number, int, and uint), whereas the equality operator performs automatic data conversion for all primitive data types.
• When comparing null and undefined, the strict equality operator returns false.

In other words, ’standard equality’ does not enforce type equality of value types. This is inconsistent with other strong-typed languages.

User Expectation

Admittedly, my initial impressions of ‘assertEquals’ is that i would perform reference comparison. If you say “Assert that value1 and value2 are equal.”, it would be easy to visualize “value1 == value2″. Since this is not what will be executed, this could be viewed as misleading.

Proposed Solution

The FUnit framework was designed to be intuitive. It was also designed to be ubiquitous with other frameworks. I don’t want developers of JUnit, NUnit, or FUnit tests to have to change their logic just because they switch languages. The less you have to think about the tools, the more focused you can be on using them.

I would opt for a stronger more consistent xUnit form of equality assertion. This however might require minor changes when migrating tests from FlexUnit implementations. I would also create a secondary LegacyAssert class that would perform identically to it’s FlexUnit counterpart. This would allow the developer to choose between xUnit convention or a more familiar FlexUnit behavior (or both in combination). This will ultimately empower the developer to be more explicit in their unit-testing.

function areEqual( expected:Object, actual:Object ) : void
function areSame( expected:Object, actual:Object ) : void

function areEqual( expected:Object, actual:Object ) : void
function areStrictEqual( expected:Object, actual:Object ) : void

I realize there are countless web-based issue tracking systems available. These systems vary widely from open source to high dollar enterprise installations. Unfortunately, I had a number restricting factors that virtually eliminated all of them.

• Guru Not Included: My server-side experience is… less than ideal.
• Hosting Restrictions: I don’t have the resources to support X or Y technology.
• Complex Setup: Near impossible in absence of said “server-side experience”
• Expensive: Open source projects aren’t always funded, including this one.
• Option Nightmare: It crams so many “features”, it’s practically unusable.
• Wrong Target User: I don’t need much, but give me what I need.
• Ugly Ugly Ugly: People DO judge a book by it’s cover…

I finally found “just what I was looking for”. There is room to grow but Lighthouse sailed past my limitation gauntlet with flying colors. I was able to create my account and begin roughing out community-based milestones and tickets almost immediately. The tools are simple and intuitive, and a good looking interface never hurt anybody. Better still, this hosted solution is free for open-source projects. Although I found this option available on other commercial products, I still preferred this one.

Lighthouse will help provide better visibility into FUnit development planning and changes as well as a sounding board for desired features and bug fixes. The new Issue Tracking Dashboard is now active and I encourage community involvment. Please do not hesitate to create new tickets for any desirables or bugs you may encounter. You may also comment on existing tickets to raise their priority and I will do my best to expedite those areas.

I discovered that within the past few days, Joseph Berkovitz and Alex Uhlmann have released Flexcover 0.50. I believe this project will be a tremendous asset for FUnit and I will be closely following it’s progress.

There is one feature in particular I’m especially impressed with. FlexCover now supports Branch Coverage. This is the first major step in not only providing test coverage, but test coverage quality. Just because a test executes your class method, doesn’t mean that all the code in that method is called. This is where Branch Coverage steps in. In a nutshell, FlexCover tracks every condition (if, else if) and whether it passes or fails. Obviously, if the condition fails, it’s code will never be executed…or tested.

One major caveat to FlexCover, however, is that it requires a “modified” Flex compiler. This is how the coverage “hooks” are injected into your application code. In addition, this step is needed emit detailed metadata of your code, files, and application structure. I anticipate that given Alex’s ties with Adobe that there will be better compiler extensibility available with the introduction of Flex 4 next year. Perhaps then this will no longer be necessary.

I have to say, the documentation for such an early project is excellent and very well outlined. I was able to get good coverage analysis for FUnit itself. I was able to create a small sub-set of FlexCover’s AIR based CoverageViewer for the web. Sadly, the source view that demonstrates the branch count won’t be available online (for now). The sample coverage data is viewable here.

Copyright © 2008 FUnit.org. All rights reserved.

The first major update of the framework core has now been completed. This is the first of several phases to create a more sophisticated asynchronous testing platform.

There are a few key differences between Flash and more sophisticated runtime environments like .NET or Java. One such disparity is the single-threaded nature of the Flash Player. Flash does not have the ability to “stop” the main thread or spawn new threads while execution completes.

An obvious example of this threading limitation surrounds the rendering model. Because rendering is executed on the same thread as all other scripts, the UI cannot be invalidated and redrawn until the entire stack has been executed. Therefore, in order for the upcoming FUnit GUI to display real-time testing feedback, this requires a more creative deviation from the conventional “for” loop.

This update uses an internal testing scheduler that will delegate testing execution across multiple execution stacks (i.e. frames). At this stage, the scheduler merely executes one test per frame but because the class has been externalized, it can be easily enhanced. It is possible, for example, for the scheduler to execute as many tests as possible while still maintaining the same frame rate.

So what’s next for FUnit? Although there’s much more work to be done surrounding asynchronous testing, it’s finally time for a GUI.

FUnit is the first generation of xUnit based unit-testing tool to leverage the custom metadata capabilities of the Flex mxmlc compiler. FUnit is comparable to patterns observed in NUnit, and carries with it a widely adopted framework API with tightly integrated strong-typed metadata and other reflection related capabilities.

The tag-based attribute model of FUnit makes it the most intuitive unit-testing framework ever created for the Flex platform.

FlexUnit Implementation

public class TestExecute extends TestCase
{
   public static function getSuite() : TestSuite
   {
      var suite : TestSuite = new TestSuite();
      suite.addTest(new TestExecute("testGetString"));
      suite.addTest(new TestExecute("testExecuteNull"));
 
      return suite;
   }
 
   public override function setup() : void
   {
      CustomClass.setString("TestString");
   }
 
   public function testGetString() : void
   {
      // sample code to test
      Assert.assertEquals( CustomClass.getString(), "TestString");
   }
 
   public function testExecuteNull() : void
   {
      try
      {
         // sample code to test
         CustomClass.execute( null );
      }
      catch ( e:ArgumentError )
      {
         // exception thrown correctly
         return;
      }
      catch ( e:Error )
      {
         // exception thrown but of wrong type
         throw new AssertionFailedError("Invalid exception.");
      }
 
      // no exception was thrown
      throw new AssertionFailedError("Uncaught exception");
   }
}

FUnit Implementation

[TestFixture]
public class TestExecute
{
   public static function getSuite() : TestSuite
   {
      var suite : TestSuite = new TestSuite();
      suite.add(TestExecute);
 
      return suite;
   }
 
   [SetUp]
   public function setup() : void
   {
      CustomClass.setString("TestString");
   }
 
   [Test]
   public function getString() : void
   {
      // sample code to test
      Assert.areEqual( CustomClass.getString(), "TestString");
   }
 
   [Test]
   [ExpectedError("ArgumentError")]
   public function executeNull() : void
   {
      // sample code to test
      CustomClass.execute( null );
   }
}

Key Differences:

• No need to extend the TestCase base class
• Tests are marked with the [Test] metadata tag
• No “test” prefix is required for TestCase methods to be reflected
• Expected failures are easily marked with an [ExpectedError] tag
• Supports SetUp, TearDown, FixtureSetUp, and FixtureTearDown
• SetUp is marked with the [SetUp] tag - No override needed
• Advanced Collection and String assertion support
• and much more…