Webcast Review: Testing the Most Critical Part of Your Application

In this webcast, Microsoft MVP and frequent MSDN Webcast presenter Jim Wilson provides an overview and a demonstration of the Local Server Framework. The Local Server Framework, also known as FakeServer, is a new tool included with the Windows Mobile 6 SDK. The Local Server Framework provides both a native dll and a managed wrapper for setting up a testing server which runs directly on a Windows Mobile device.

Running a test server on a Windows Mobile device provides greater control over an application's test environment by removing network setup and maintenance from the equation. When building mobile applications, often times the server component is being developed in parallel with the client. Using the Local Server Framework for testing, therefore, is a great way to cut down on the impact of client and server modifications. Jim mentions how blame for miscommunication when versions are out of sync is usually placed on the client because it is the application that exposes the error. For this and other reasons, he believes that a mobile client should not trust server data unconditionally and that the application should validate transmitted data just like user-entered data. By using the Local Server Framework, developers can modify the FakeServer to see how their mobile applications respond to these and other spurious inputs.

From an information perspective, this is a great resource on the Local Server Framework, a tool which has not seen nearly any publicity since the release of Windows Mobile 6. This webcast shares the necessary information needed to get started using the Local Server Framework making it very effective.

Windows Mobile Native Unit Testing

When applied with discipline and care, unit testing can be a powerful weapon in a software developer's arsenal. It allows a developer to garner feedback on functional correctness as well as refactor an existing codebase with security and confidence. Additional benefits of unit testing can be greater modularity, looser coupling, and an easier ability to measure whether software is meeting functional requirements.

There is a wealth of information on the Internet regarding unit testing and unit testing frameworks, but very few items are Windows Mobile specific. With the .NET Compact Framework, unit testing can currently be done with the Mobile Client Software Factory, released in mid-2006. And with the upcoming release of Visual Studio 2008 and the .NET Compact Framework version 3.5, unit tests are integrated into Visual Studio for managed mobile development.

The managed side is covered pretty well at this point (or will be when Visual Studio 2008 comes out), but what about the native side? What is available for C++ developers on Windows Mobile to use? One answer is CppUnitLite.

CppUnitLite was written by Michael C. Feathers, author of Working Effectively with Legacy Code. Feathers was also the original author of CppUnit and developed CppUnitLite when he realized that CppUnit could have been "smaller, easier to use, and far more portable if it used some C idioms and only a bare subset of the C++ language." [Feathers p48]

Here are steps to setup CppUnitLite for use with Windows Mobile.

1. Download CppUnitLite from the Object Mentor website.

2. Extract the following files from the om/CppUnitLite section of the zip file:

• Failure.h
• SimpleString.h
• Test.h
• TestHarness.h
• TestRegistry.h
• TestResult.h
• Failure.cpp
• SimpleString.cpp
• Test.cpp
• TestRegistry.cpp
• TestResult.cpp

(In this demo, these files have been extracted to C:\CppUnitLite)

3. Open the Solution which requires unit tests.(In this demo, the solution is named StackExample. This solution contains one project, Stack, which contains two files, Stack.h and Stack.cpp. Image 1 contains a screenshot of the solution loaded in the Solution Explorer. Stack.h is displayed in Code Listing 1. Stack.cpp is displayed in Code Listing 2.)

Image 1 - StackExample solution in Solution Explorer

 

Code Listing 1 - Stack.h

#ifndef Stack_h
#define Stack_h

#if (_MSC_VER > 1000)
#pragma once
#endif

/*!
 * \brief
 * This class is a stack which holds integer
 * values.
 *
 * The size of the stack is determined at time
 * of construction. A stack uses the last-in
 * first-out approach.
 */
class Stack
{
public:
    Stack(int maxSize);
    virtual ~Stack();

    bool IsEmpty() const;
    void Push(int item);
    int Pop();
private:
    int* m_items;
    int  m_currentSize;
    int  m_maxSize;
};

#endif

Code Listing 2 - Stack.cpp

#include "Stack.h"
#include <stdexcept>

using namespace std;

/*!
 * \brief
 * This method is the constructor for the Stack class.
 *
 * \param maxSize
 * This parameter is the maximum number of integers
 * the stack can hold.
 */
Stack::Stack(int maxSize)
{
    m_maxSize = maxSize;
    m_items = new int[maxSize];
    m_currentSize = 0;
}

/*!
 * \brief
 * This method is the destructor for the Stack class.
 */
Stack::~Stack()
{
    delete[] m_items;
}

/*!
 * \brief
 * This function calculates whether or not the
 * stack is currently empty.
 *
 * \returns
 * This function returns true if the stack is
 * empty, or false otherwise.
 */
bool Stack::IsEmpty() const
{
    return (m_currentSize == 0);
}

/*!
 * \brief
 * This function adds an integer value to the stack.
 *
 * \param item
 * This parameter is the integer value to add to the stack.
 *
 * \throws exception
 * This function throws an exception if a value is pushed
 * when the stack is already full.
 */
void Stack::Push(int item)
{
    if (m_currentSize >= m_maxSize)
    {
        throw exception("Stack is full.");
    }
    else
    {
        m_items[m_currentSize++] = item;
    }
}

/*!
 * \brief
 * This function removes an integer value from the stack.
 *
 * \returns
 * This function returns the last integer value added to
 * the stack and removes it.
 *
 * \throws exception
 * This function throws an exception if this function is
 * called when the stack is empty.
 */
int Stack::Pop()
{
    if (IsEmpty())
    {
        throw exception("Stack is empty.");
    }
    else
    {
        return m_items[--m_currentSize];
    }
}

4. Add a new Win32 Smart Device Project to the solution named CppUnitLite and place it in the directory where the CppUnitLite source code was extracted in Step 2.

Image 2 - Add New Project Wizard

5. Add all Windows Mobile platform SDKs to the new project.

Image 3 - Win32 Smart Device Project Wizard (Platform)

6. Set the Application type to Static Library and turn off Precompiled header in the Additional options section.

Image 4 - Win32 Smart Device Project Wizard (Application Settings)

7. Move the project file and the readme.txt file into the main directory where the CppUnitLite files are stored. The project needs to be removed and readded to the solution within Visual Studio.

Image 5 - Directory listing of CppUnitLite files.

8. Using the Add->Existing Item... mechanism, add the header files (*.h) to the Header files section and add the source files (*.cpp) to the Source files section.

Image 6 - CppUnitLite project with all header and source files added

This walkthrough assumes that the code to be tested is in a static library (*.lib). In the example, Stack.h and Stack.cpp are in a static library project named Stack.

9. Add a new project to the solution and name it LibraryName.Tests. In the example, the project is named Stack.Tests.

10. Right-click on the project and choose properties. In the Common Properties->References section add the other two projects as references by using the Add New Reference button.

Image 7 - References added to the project properties.

11. In the Configuration Properties->C/C++->General section is an option named Additional Include Directories. Add the directories which contain the header files of the classes to be tested as well as the directory which contains the CppUnitLite header files. In the example, these two directories are ..\Stack and C:\CppUnitLite.

Image 8 - Additional Include Directories added to the project properties.

12. Add a .cpp file with a command line main method for Windows Mobile in it. This file should include TestHarness.h from CppUnitLite. In the main method, create a TestResult variable and pass it into TestRegistry::runAllTests as done in the StackMain.cpp example in Code Listing 3.

Code Listing 3 - StackMain.cpp

#include "TestHarness.h"
#include <windows.h>

int _tmain(int argc, _TCHAR* argv[])
{
    TestResult tr;
    TestRegistry::runAllTests(tr);

    return 0;
}

13. Add a file named corresponding to the class to be tested. In the example, the Stack class is being tested, so the file is named StackTest.cpp. The code for this file is listed in Code Listing 4.

Code Listing 4 - StackTest.cpp

#include "TestHarness.h"
#include "Stack.h"
#include <string>

TEST( MultiPushPop, Stack )
{
    Stack s(10);
    s.Push(14); 
    LONGS_EQUAL(14, s.Pop());
    s.Push(12);
    s.Push(18);
    s.Push(27); 
    LONGS_EQUAL(27, s.Pop());
    s.Push(38);
    s.Push(2);
    s.Push(12);
    LONGS_EQUAL(12, s.Pop());
    LONGS_EQUAL( 2, s.Pop());
    LONGS_EQUAL(38, s.Pop());
    LONGS_EQUAL(18, s.Pop());
    LONGS_EQUAL(12, s.Pop());
    CHECK(s.IsEmpty());
}

TEST( PushExceedsMax, Stack )
{
    Stack s(5);
    s.Push(1);
    s.Push(2);
    s.Push(3);
    s.Push(4);
    s.Push(5);
    try
    {
        s.Push(6);
        FAIL("Push should have exceeded max stack size");
    }
    catch (std::exception ex)
    {
        // Expected behavior
    }
}

TEST( PushPop, Stack )
{
    Stack s(10);
    s.Push(27000);
    CHECK(!s.IsEmpty());
    int value = s.Pop();
    LONGS_EQUAL(27000, value);
    CHECK(s.IsEmpty());
}

TEST( PopEmpty, Stack )
{
    Stack s(10);
    try
    {
        // This test should throw an exception
        int value = s.Pop();
        FAIL("Popping an empty stack should throw an exception");
    }
    catch (std::exception ex)
    {
        // Expected behavior
    }
}

TEST( IsEmpty, Stack )
{
    Stack s(10);
    CHECK(s.IsEmpty());
}

TEST( Creation, Stack )
{
    Stack s(10);
}

13. Set the .Tests project (Stack.Tests) as the startup project by right-clicking on the project and choosing Set as Startup Project.

14. Build all the projects.

15. Make sure to choose the same emulator or device for all three projects otherwise multiple emulator windows may pop up.

16. Run the project. Hopefully, a message stating "There were no test failures" is then displayed in the Output window as shown in Image 9.

Image 9 - Successful Test Run

Aside from the examples that come with CppUnitLite, there are additional examples within Working Effectively with Legacy Code.

Webcast Review: Building High-Performance Applications Using the .NET Compact Framework

On August 8th, 2007, the Microsoft .NET Compact Framework team's lead software developer, Steven Pratschner, conducted an extremely informative MSDN webcast entitled "Building High-Performance Applications Using the .NET Compact Framework". For people unfamiliar with Steven's blog, performance is a topic which he has covered quite extensively there. This webcast included discussion of a wide array of topics ranging from performance comparisons with native code to the implementation of the .NET CF garbage collector.
There are many notable highlights in this 71 minute webcast including the following subjects:

• A comparison of method calls on performance:
  
• Managed Method calls are 2.3x the cost of a native call
• Virtual Method calls are 1.5x the cost of a method instance call
• P/Invoke Method calls are 5x the cost of a managed instance call
• (Unfortunately though it did not discuss the performance of static methods versus instance methods nor the use of interfaces)
  
• The effect of boxing on performance demonstrated through the use of a simple demo game containing a grid of blocks being rotated.
• How to use the .NET CF Remote Performance Monitor to look at metrics including the total number of garbage collections, memory usage, and objects allocated.
• How to use the new CLR Profiler, which will be included in version 3.5 of the .NET Compact Framework. This provided detailed information to the level of showing which methods within the demo application were causing a high percentage of object allocations.
  
• Why it is better to use an indexer or custom iterator over a foreach statement

Early in the session, Steven mentions how code is just-in-time compiled or jitted, on a per method basis. How code is jitted is important when testing the performance of methods. Steven reveals an extremely useful approach. He says to run the method once so that it gets jitted. Then, call the method again this time measuring for performance. Following these steps gives the true runtime of the method as opposed to the jit time and runtime combined.

Other fascinating topics are discussed including collections, reducing application startup times and reflection. This is definitely a webcast that developers working on enterprise level line of business applications using the .NET CF should watch.