I have received several inquiries regarding the SimpleTestRunner and a means of visualizing test failures. It is important to note that the current SimpleTestRunner GUI is really intended for automation purposes. A full GUI similar to what you’d find in FlexUnit or Fluint, is on the roadmap. It’s just not ready yet.
In the meantime, there are several options for recording test results.
Option 1:
The most longstanding option is the DebugTestListener. Just run the following…
var runner:TestRunner = new TestRunner();
runner.test = LibraryTestSuite.getInstance();
runner.run( new DebugTestListener() );
Note: This must be run in debug mode.
You will see the following in the debug console.
(I’ve intentionally introduced some errors so you see the output)
------------------------------------------------------------------
FUnit Framework - Flex Unit Testing Environment
Framework Version: 0.72.0452 (Build 1106)
Adobe Windows (StandAlone)
Player Version: WIN 9,0,124,0
------------------------------------------------------------------
################################ UNIT TESTS ################################
Running tests in ''...
***** funit.framework::ArrayAssertTests.isNotSubsetOf
***** funit.framework::ArrayAssertTests.doesNotContain
***** funit.framework::ArrayAssertTests.containsFailureOnEmpty
***** funit.framework::ArrayAssertTests.areNotEqualHandlesNull
***** funit.framework::ArrayAssertTests.isNotEmpty
[continued ...]
############################################################################
############## F A I L U R E S #################
1) funit.framework::ArrayAssertTests.itemsOfTypeFailure :
Expected: all items instance of <String>
But was: [ "x", "y", <Object> ]
at funit.framework::CollectionAssert$/allItemsAreInstancesOf()[..\Framework\main\source\funit\framework\CollectionAssert.as:82]
at funit.framework::ArrayAssertTests/itemsOfTypeFailure()[..\Framework\tests\source\funit\framework\ArrayAssertTests.as:59]
at Function/http://adobe.com/AS3/2006/builtin::apply()
at InternalMethodInfo/invoke()[..\SwirlyVision\Corelib\main\source\sv\reflection\InternalMethodInfo.as:96]
at funit.core::TestMethod/invokeTestCase()[..\Framework\main\source\funit\core\TestMethod.as:127]
at funit.core::TestMethod/runTestCaseImpl()[..\Framework\main\source\funit\core\TestMethod.as:105]
at funit.core::TestMethod/runTestCase()[..\Framework\main\source\funit\core\TestMethod.as:82]
at funit.core::TestCase/run()[..\Framework\main\source\funit\core\TestCase.as:56]
at funit.core::TestSuite/runTest()[..\Framework\main\source\funit\core\TestSuite.as:243]
at <anonymous>()[..\Framework\main\source\funit\core\TestSuite.as:180]
at flash.events::EventDispatcher/dispatchEventFunction()
at flash.events::EventDispatcher/dispatchEvent()
at funit.core::TestScheduler/scheduleTest()[..\Framework\main\source\funit\core\TestScheduler.as:128]
at <anonymous>()[..\Framework\main\source\funit\core\TestScheduler.as:105]
at flash.events::EventDispatcher/dispatchEventFunction()
at flash.events::EventDispatcher/dispatchEvent()
at flash.utils::Timer/tick()
2) funit.framework::ArrayAssertTests.itemsUniqueFailure :
Expected: all items unique
But was: [ "x", "y", "x" ]
at funit.framework::CollectionAssert$/allItemsAreUnique()[..\Framework\main\source\funit\framework\CollectionAssert.as:124]
at funit.framework::ArrayAssertTests/itemsUniqueFailure()[..\Framework\tests\source\funit\framework\ArrayAssertTests.as:115]
at Function/http://adobe.com/AS3/2006/builtin::apply()
at InternalMethodInfo/invoke()[..\SwirlyVision\Corelib\main\source\sv\reflection\InternalMethodInfo.as:96]
at funit.core::TestMethod/invokeTestCase()[..\Framework\main\source\funit\core\TestMethod.as:127]
at funit.core::TestMethod/runTestCaseImpl()[..\Framework\main\source\funit\core\TestMethod.as:105]
at funit.core::TestMethod/runTestCase()[..\Framework\main\source\funit\core\TestMethod.as:82]
at funit.core::TestCase/run()[..\Framework\main\source\funit\core\TestCase.as:56]
at funit.core::TestSuite/runTest()[..\Framework\main\source\funit\core\TestSuite.as:243]
at <anonymous>()[..\Framework\main\source\funit\core\TestSuite.as:180]
at flash.events::EventDispatcher/dispatchEventFunction()
at flash.events::EventDispatcher/dispatchEvent()
at funit.core::TestScheduler/scheduleTest()[..\Framework\main\source\funit\core\TestScheduler.as:128]
at <anonymous>()[..\Framework\main\source\funit\core\TestScheduler.as:105]
at flash.events::EventDispatcher/dispatchEventFunction()
at flash.events::EventDispatcher/dispatchEvent()
at flash.utils::Timer/tick()
############################################################################
Executed tests : 160
Ignored tests : 1
Failed tests : 2
Unhandled exceptions : 0
Total time : 1.231 seconds
############################################################################Option 2:
The XmlResultWriter will take an TestResult object and translate it into an NUnit-style xml schema. If you choose, you can simply trace the result to the debug console or write it to a flat file with Adobe AIR.
If you want to get a little more fancy can use this in combination with ANT and write this out to an xml file. Unlike other approaches, this does not require use of either the AIR runtime or XMLSocket connections. Instead, you need only target the standalone debug player distributed with the Flex SDK. This approach is ideal for with automated build integration, especially those with test report generation.
Here’s how you might set up your test runner…
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml"
xmlns:funit="http://www.funit.org/2007/mxml">
<mx:Script>
<![CDATA[
import mx.utils.StringUtil;
import funit.LibraryTestSuite;
import funit.listeners.events.RunFinishEvent;
import funit.utils.XmlResultWriter;
private function runTests() : void
{
runner.addEventListener(
RunFinishEvent.RUN_FINISH,
runFinishedHandler
);
runner.run();
}
private function runFinishedHandler( event:RunFinishEvent ) : void
{
var writer:XmlResultWriter =
new XmlResultWriter(event.result);
var result:String =
"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n" +
writer.result.toXMLString();
trace( StringUtil.substitute("##funit\[TestResult {0}]##", result) );
}
]]>
</mx:Script>
<mx:creationComplete>runTests();</mx:creationComplete>
<funit:AutomationTestRunner id="runner">
<funit:test>{LibraryTestSuite.getInstance()}</funit:test>
</funit:AutomationTestRunner>
</mx:Application>Believe it or not, the ANT side is just as easy.
(Properties are used for clarity and reuse)
<!--
Execute Tests
-->
<target name="test" description="Test All">
<exec executable="${flashplayer.debug.exe}" errorproperty="trace.output">
<arg line="${bin.debug.dir}/FUnitTestRunner.swf" />
<redirector>
<errorfilterchain>
<ignoreblank />
</errorfilterchain>
</redirector>
</exec>
<!-- Extract results and write TestResult.xml -->
<echo file="${bin.debug.dir}/TestResult.xml">${trace.output}</echo>
<replaceregexp file="${bin.debug.dir}/TestResult.xml" match="(?s).*##funit\[TestResult (.+)\]##" replace="\1" />
<!-- Display original trace statements -->
<echo>${trace.output}</echo>
</target>The in the redirector is necessary to prevent double line spacing. I’m not sure why this happens but it helps keep our final output pretty.
The regular expression helps ensure that only the result xml is written to the final file. Without it, any other trace calls would be written as well, causing invalid xml markup.
trace(StringUtil.substitute("##funit\[TestResult {0}]##", result));To avoid this, I’ve wrapped the result in a unique closure. Simply put, only the value of result would make it into TestResult.xml.
