2013年10月27日 星期日

Windows Mobile Test Framework Overview

Windows Mobile Test Framework Overview

Framework Layers

This document will give a high level overview of the pieces that make up the Windows Mobile Test Framework. This framework is made primarily of .Net class libraries which are used to automate interaction with the device UI.
These class libraries are broken out into separate layers with differing purposes. The layers are shown in the diagram below, which illustrates how the different pieces are built on top of one another. Tux.Net, not shown here, simply executes code from the test suites, but is built primarily on Logging and Utils.Net components.



Logging

The Logging library contains classes associated with logging test data and test result management. The Logging library is generic enough that it can be used by any tool, not just .Net test automation, and it can even be used by desktop tools. The main interfaces into the Logging library for .Net Automation are GlobalLogger and GlobalLogResultManager.
GlobalLogger is used for logging text and comments to log files during testing. It is used by Mtk, UIAL, Area Library, and Test components. It frees you from worrying about where and how to log data - it will make sure your data gets in the right file, where the "right" file is defined by whatever file the GlobalLogger has been configured to deliver to. GlobalLogger also has the notion of logging levels so that you can assign varying importance to different comments. For instance, UIAL comments may be logged at debug level and only turned on during debugging, whereas comments at the test level would always be logged. GlobalLogResultManager is used for keeping track of the result of your test - Pass or Fail - as you perform multiple steps in each test case. It is used almost exclusively in the Test and Area Library layers.

Utils.Net

Utils.Net is a toolbox of common device oriented code that can be reused in any automation component including MTK, UIAL, Area Libraries, and Tests. The .Net Framework provides a very rich library of code for desktop tools to reuse, but the .Net Compact Framework contains only a subset of this library. Utils.Net includes many things missing from the .Net Compact Framework that we need on a regular basis to work with and test our devices. As we have need of these missing library components we crate and add them to Utils.Net.

Utils.Net holds code that isn't just useful to .Net Automation, but potentially for anyone who is working with the .Net Compact Framework. In addition, Utils.Net builds for the desktop to facilitate sharing code between device and desktop.

Device Automation Toolkit

The Device Automation ToolKit (DATK) is primarily a C# library of classes that mirror controls on our devices. Most classes in DATK map to a control found in our devices, such as edit boxes, list views, and buttons. In addition to these control classes there is the WindowFinder class, which is the key to working with DATK. It allows you to specify the characteristics (such as type, text, and owning process) of the control you are looking for on the device, and bind that control to a DATK control object. Using the DATK control object you can get information about the control on the device such as the text, selected/unselected status, focused/unfocused status, and more. Even more importantly you can use the DATK control object to interact with the control on the device. In this way we can use DATK to automate the use of UI on the device.

The DATK does not currently depend on the Logging or Utils.Net components.

Mobility Toolkit

The Mobility ToolKit (MTK) is simply an Windows Mobile specific extension to the DATK class library. Because DATK is dedicated to all graphical Windows CE platforms, it has been kept it free of classes representing controls that are not provided as part of the Windows CE operating system. For instance, Softkeys are a very important component in Mobile devices but do not exist in Windows CE itself. We have placed Windows Mobile specific DATK controls into their own library, the MTK. In addition, MTK holds some other classes which are useful to automation. ApplicationManager is the most obvious example of this, which we use to launch applications on Windows Mobile devices.

The MTK has many classes that derive from DATK classes, and it also depends heavily on Logging and Utils.Net.

Abstraction Layer

DATK provides classes for most of the UI elements found on our devices. However, in order to initialize one of these objects to an actual UI element at the Datk layer, you need to use the Datk WindowFinder class. A WndowFinder is provided search criteria such as class name, title, label, control ID, etc and then asked to find the control that matches the criteria it was given. The search criteria are especially sensitive to design and implementation change of the UI, and so this warrants a mode of abstraction to help protect test automation from these changes.
The UI Abstraction Layer, or UIAL, is meant to encapsulate all controls on every form of an application so that the automation developer does not have to determine the properties of every control he uses or use WindowFinder to get that control. It is a C# library that provides a 1:1 mapping between particular controls on the device and DATK control objects. There should be a UIAL for every application, and there should be a property in each UIAL that maps to every control found in that application. This centralizes the work of characterizing each control so that when elements of the UI change only the UIAL needs to change, not all of the code that automates that element.
An application developer must create a UIAL component in order to make their application testable. Anyone can do this for most applications, actually, but only the application developer can reliably ensure that the UIAL adheres to a contract between the automation and the application as the application changes.

The UIAL depends heavily on DATK, MTK, Utils.Net, and Logging. It is used almost exclusively by the Area Libraries. The UIAL is largely tool-generated code, not hand-written code.

Area Libraries

The UIAL is used to hide the complexity of finding controls in DATK, but it does not combine the use of these controls into common usage scenarios. This is what the Area Libraries are for - providing a library of routines that act out common UI scenarios on the device.
The goal of the libraries is to make test script writing as straightforward as typing in descriptions of manual test case steps. The code is very generally comparable to the aggregate steps taken in manual testing. For instance, the first step in a test case for Microsoft Word would be open Word.exe, which corresponds to a series of UI based commands: click Start, click Programs File, Scroll Down, Click Word. "Open Word" might be implemented in the Area Library, so that the test case doesn't need to deal with the specifics of what it takes to open Microsoft Word. The Area Library also abstracts away the notions of a Windows Mobile SKU (Professional, Classic, Standard) to provide functions that act appropriately depending on which version is being used.


Test Cases

In our .Net Automation Framework test suites are simply C# classes that inherit from a class called TestSuite. Tests are simply C# methods in these classes decorated with an attribute that designates them as test cases. Tests are generally organized into test suites by feature. Tests rely almost solely on the area library for functionality, and consist of not much more than a series of calls into the area library, to string together common UI scenarios into a single test scenario. Tests do not call into the UIAL unless the test is specific to a particular kind of Windows Mobile device, which should be fairly rare, depending on the application being tested.
A Test developer should create a test assembly to host test cases for their application.

The Tests depend heavily on Area Libraries, for scenario and object creational patterns, and Logging. They occasionally (rarely in theory) need to use UIAL, MTK, or DATK.

Tux.Net

Tux.Net is a C# program, specifically a test harness, used to run tests on devices. When run from a device with command line or a configuration file it will locate the specified test suites and tests inside managed assemblies and execute them. It has many options for specifying what tests or suites will be run, and how they should be run (for instance, should they be shuffled or repeated). Tux.Net looks for classes inheriting the TestSuite class and methods with the TestCase attribute to know what it can run, and it can run any assembly meeting these requirements.

Tux.Net executes Tests, but it is built primarily with the Logging library and Utils.Net.

Source: WMTF Document

沒有留言:

張貼留言