This document describes the API for the Wacom Feel™ Multi-Touch support in the Wacom WebPlugin component. The WebPlugin module is a subset of the Wacom Feel™ Multi-Touch API [Loftus2011], which allows developers to access finger touch data from supported Wacom Feel™ Multi-Touch enabled tablets. Note that not all Wacom branded tablets support Feel™ Multi-Touch capabilities. A Feel™ Multi-Touch API supported tablet will be recognized by the WebPlugin API, enabling the developer to query for that tablet's multi-touch properties and acquire finger touch data.

The Feel™ Multi-Touch API can be used alongside the Wintab pen interface API, which is supported in the same plugin. If a tablet supports simultaneous pen and touch data, then the web application will continue to get touch data even while the pen is in proximity. If a tablet does not support this, then the pen will trump the touch. This means that no touch data is sent while the pen is in proximity. The WebPlugin Feel™ Multi-Touch API does not process or inform about the pen proximity. If the application needs to provide special processing for a touch stroke interpreted by a pen stroke, then the application should monitor both APIs.

1.1 Supported Versions

The WebPlugin Feel™ Multi-Touch API is supported in the following components:

· WebPlugin: Version 2.0 and greater

· Tablet Driver: Version 5.2.5 and greater

· The Webplugin is supported on these browsers and operating systems with the given minimum versions:

	Windows 7, Vista, XP	Mac OSX 10.5.8, 10.6
Firefox v4	yes	yes
Safari v5	yes	yes
Chrome v10	Yes	n/a
Internet Explorer v8	Yes	n/a

1.2 WebPlugin usage

In a nutshell, the WebPlugin is used in a javascript program to access Feel™ Multi-Touch API properties and events. The HTML page containing the javascript program can then be used with any supported browser components (such as Canvas), other plugins (such as Adobe Flash), or frameworks (such as Silverlight). By using the WebPlugin, much of the Feel™ Multi-Touch API initialization has been performed automatically so the developer can focus on the application. See [Cohn2011] for more details on using the WebPlugin.

This document contains snippets of javascript code demonstrating various Feel™ Multi-Touch API properties, data and event handling.

1.3 Example Demos and Related Resources

See http://www.wacomeng.com/ for WebPlugin Feel™ Multi-Touch API example code and related documentation.

2 WebPlugin Feel™ Multi-Touch API Properties

2.1 TouchRawFingerData

Returns latest raw finger touch data to the user from the first touch device found.

2.1.1 Params:

deviceID Int

For now, the only deviceID accepted will be -1 or a deviceID returned from TouchDeviceCapabilities, which will mean to return data from the first touch device found, or the data for that specified device respectively. If a deviceID other than a valid ID is used, then the Status flag in the returned FB::VariantMap will indicate an error.

2.1.2 Returns:

FB::VariantMap consisting of the following members:

NumFingers Int // #fingers in this data packet

DeviceID Int // device from which data is sent

Status Int // indicates if data is valid (see below)

FingerList FB::VariantList // list of finger data

FingerList[0]

FB::VariantMap // Data for Finger1

FingerID Int // as specified in data packet

PosX float // X coord passed back from API (see below)

PosY float // Y coord passed back from API (see below)

Width float // logical width of finger contact area

Height float // logical height of finger contact area

Orientation float // angle of finger contact ellipse from vertical

Confidence Int // true if driver believe this is valid touch

Sensitivity Short // strength of the contact

FingerState Int // latest activity for this finger (see below)

FingerList[1]

FB::VariantMap // Data for Finger2

FingerID Int

PosX float

PosY float

Width float

Height float

Orientation float

Confidence Int

Sensitivity Short

FingerState Int

...

The size of the returned FB::VariantMap will depend on the number of fingers in the most recent touch data packet. If dynamic FB::VariantMap is not supported, then the plugin will pre-allocate a map for 10 fingers, and fill in the number of fingers of data from the latest touch API data packet.

NumFingers – number of detected fingers in the latest data packet. The WebPlugin will fill in touch data for each finger.

DeviceID – the unique identifier of the tablet from which the touch data comes

Status – indicates whether the FB::Variant returned contains valid data. If data is valid then, Status will be zero. If data is not valid, then Status will be the value -1.

PosX and PosY –

will be logical units in the range of 0 to 1.0 for opaque tablets.

will be pixel screen coordinates in the virtual desktop range for integrated tablets. *

Width and Height – represents logical width and height of a finger contact. *

Sensitivity – strength of the contact. This is not pressure. This is a device/user specific indication of the strength of the contact point. Only valid in relation to other fingers within the same frame/gesture. *

Orientation – angle of finger contact ellipse from vertical*

FingerState = 0 (unknown), 1 (Down), 2 (Hold), 3 (Up)

Note that there is no Move state. The Hold state can be interpreted as a move if the finger position changes.

2.1.3 Sample Javascript Usage:

var fingerdata = plugin.TouchAPI.TouchRawFingerData(0);

var numFingers = fingerdata.NumFingers;

var status = fingerdata.Status;

if ( status == 0 )

{

var finger1= FingerData.FingerList[0];

var finger1ID = finger1.FingerID;

if (finger1.Confidence == 1 && finger1.FingerState != 0)

{

var finger1PosX = finger1.PosX;

var finger1PosY = finger1.PosY;

...

}

var finger2 = FingerData.FingerList[1];

etc...

}

2.1.4 Design notes:

FB::VariantMap and FB::VariantList are supported Firebreath variant types.

Upon plugin initialization, the plugin will do the following:

· Initialize its touch API connection with WacomMTInitialize()

· Register for touch devices being attached with: WacomMTRegisterAttachCallback(). Upon attach, initialize all plugin touch API structures.

· Register for touch devices being detached with: WacomMTRegisterDetachCallback() . Upon detach, delete/clear all plugin touch API structures.

· The webplugin client can poll for data or use the TouchDataEvent (see below).

Upon plugin destruction, the plugin will do the following:

· Call WacomMTQuit() to close connection with the touch driver service and delete/clear all plugin touch API structures.

2.2 TouchDeviceCapabilities

Returns the capabilities for the specified touch device.

2.2.1 Params:

deviceID Int

For now, the only device ID accepted will be -1. See TouchRawFingerData for details.

2.2.2 Returns:

FB::VariantMap consisting of the following members:

Version Int // data structure version

DeviceID Int // identifier of this device

DeviceType Int // a WacomMTDeviceType value

MaxFingers Int // max number of fingers supported by device

ReportedSizeX Int // Number of touch points in X direction

ReportedSizeY Int // Number of touch points in Y direction

PhysicalSizeX float // Width of the device active touch surface in mm

PhysicalSizeY float // Height of the device active touch surface in mm

LogicalOriginX float // Min X of device. Can be negative if device larger than display

LogicalOriginY float // Min Y of device. Can be negative if device larger than display

LogicalWidth float // Logical width of device. MaxX = LogicalOriginX + LogicalWidth

LogicalHeight float // Logical height of device. MaxY = LogicalOriginY + LogicalHeight

If the call fails, then the returned FB::VariantMap will have Version == -1.

2.2.3 Sample Javascript Usage:

var touchDeviceCaps = plugin.TouchAPI.TouchDeviceCapabilities(0);

var maxNumFingers = touchDeviceCaps. MaxFingers;

var maxXTouchPoints = touchDeviceCaps.ReportedSizeX;

var maxYTouchPoints = touchDeviceCaps.ReportedSizeY;

etc...

2.2.4 Design notes:

As indicated above, the initial release of this API will only recognize -1 as a legal deviceID. So the capabilities returned will be for the first touch device found when deviceID = -1 is used. The DeviceID value returned in this structure can be used in the other touch API properties that required a deviceID (TouchRawFingerData, Open, Close).

2.3 TouchDeviceAttached

Returns status of whether any touch devices are physically attached to the system.

2.3.1 Params:

None

2.3.2 Returns:

bool True -> at least one touch device was found

2.3.3 Sample Javascript Usage:

var isTDAttached = plugin.IsTouchDeviceAttached();

2.3.4 Design notes:

Get a count of the attached touch devices using WacomMTGetAttachedDeviceIDs(). If this returns a non-zero list, then the IsTouchDeviceAttached property (below) can return true.

2.4 TouchDeviceIDList

Returns list of attached touch device IDs.

2.4.1 Params:

None

2.4.2 Returns:

FB::VariantList of attached touch device IDs. If no touch device IDs are attached, the array is of zero size.

2.4.3 Sample Javascript Usage:

_touchDeviceIDList = plugin().touchAPI.TouchDeviceIDList;

2.4.4 Design notes:

Returned list is built from a list of active touch devices. The list is maintained by the plugin by adding newly attached devices, when the plugin receives an Attach message from the touch API, or by deleting touch devices from the list when the plugin receives a Detach message from the touch API.

2.5 Open

This is a method that will attach to the specified device and initialize the touch API for the client so that data from that device will be sent to the client. The client can elect to connect so that touch data is NOT passed to the system, or can connect so that touch data is passed to him (the client) as well as being sent to the system.

2.5.1 Params:

deviceID Int // -1 -> open first touch device found; else open specified deviceID

passed from TouchDeviceCapabilities

passThru Int // 1: data is passed to system; 0: data is not passed to system

2.5.2 Returns:

0 on success; non-zero if error.

2.5.3 Sample Javascript Usage:

var caps = plugin.TouchAPI.TouchDeviceCapabilities(-1); // get caps of first touch device

Either of these will work:

var status = plugin.TouchAPI.Open(-1, 0); // open 1st device; do NOT pass data to system

var status = plugin.TouchAPI.Open(caps.DeviceID, 0); // open specified device

2.5.4 Design notes:

When this method is called, the plugin will do the following:

· Register for touch data using WacomMTRegisterFingerReadCallback(), with the following args:

hitRect = NULL - uses the full tablet surface

mode = passThru = 0 -> ProcessingModeAPIOnly; passThru = 1 -> ProcessModeOS

ProcessingModeAPIOnly = only the client gets touch data

ProcessModeOS = both client and system will get touch data

fingerCallback = plugin handler to cache the touch data for use by the webplugin API

userData = NULL (can't think of a use for this yet)

· For now, we'll pass any data back to the user (since -1 is used); in the future, we'll only forward data that matches the specified deviceID

2.6 Close

This method detaches client from the specified device, so that no further data from that device nor all touch devices will be sent to the client.

2.6.1 Params:

deviceID Int // -1 -> close all touch devices; else close deviceID

passed from TouchDeviceCapabilities

2.6.2 Returns:

0 on success; non-zero if error.

2.6.3 Sample Javascript Usage:

Either of these will work:

var status = plugin.TouchAPI.Close (-1); // close all devices

var status = plugin.TouchAPI.Close (caps.DeviceID); // close specified device

2.6.4 Design notes:

· Unregister for touch data using WacomMTRegisterFingerReadCallback(), by setting the fingerCallback arg to NULL.

2.7 Modes

This method allows the plugin to be set for different conditions. The conditions supported at this time are:

1. CHILD_BIT: Telling the API that the plugin will be run in a browser child process or in the browser main thread. Using this setting, the API is told that the plugin is configured to work in a browser that launches plugins either in a child process, or launches plugins in the main thread. Currently, most browsers (Internet Explorer, Firefox, Chrome, etc.) launch plugins in a separate process. This guards against a failing plugin crashing the whole browser. Although the plugin API assumes that it will be run in a child process by default, there is a way to switch off that behavior and have the plugin API assume the webplugin will be run in the main thread.

2. OBSERVER_BIT: Setting the plugin to run as a touch consumer or touch observer. Using this setting, the plugin is configured to run in observer or consumer mode. In observer mode, the plugin processes touch data, and then passes the touch data on to the system, where it can be used by other applications or by the system to execute gestures or cursor navigation. In consumer mode, the plugin processes touch data, but does not let it be passed to the system.

2.7.1 Params:

mode Int // A bitwise-OR of conditions:

// CHILD_BIT: 0x00000001 -> plugin assumed to be in a child browser process

// OBSERVER_BIT: 0x00000010 -> plugin passes data to the system (Observer)

// By default, the plugin executes in a child browser process as a Consumer.

2.7.2 Returns:

0 on success; non-zero if error.

2.7.3 Sample Javascript Usage:

var status = plugin.TouchAPI.Modes (0); // tells API that plugin runs in browser main thread, and is a consumer

var status = plugin.TouchAPI.Modes (1); // tells API that plugin runs in a child process, and is a consumer

var status = plugin.TouchAPI.Modes (3); // tells API that plugin runs in a child process, and is an observer

2.7.4 Design notes:

For most browsers (Opera is the only exception we know of), you don't need to change mode to run the plugin in the main browser thread.
Set the observer bit if you don't want any other application, or the system, to process the touch data while the browser window with your application has the focus.

2 WebPlugin Feel™ Multi-Touch API Events

2.1 TouchDataEvent

Fires when touch data is received when any touch data is received. The client's handler, upon receiving this event, can query for data (see below).

2.1.1 Plugin Event Name

"onTouchDataEvent"

2.1.2 Example Javascript Usage

Registering for the event:

if (window.addEventListener)

{

obj.addEventListener("TouchDataEvent", touchDataHandler, false);

}

else

{

obj.attachEvent("onTouchDataEvent", touchDataHandler);

}

Function handler for the event:

function touchDataHandler()

{

// Use TouchRawFingerData to query for cached data

// process/render finger data in client display area

}

2.1.3 Design Notes

The plugin will register this event using:

registerEvent("onTouchDataEvent");

When the plugin receives notification from the touch API of an incoming FingerRead event, it will fire notification to the client using the following:

this->FireEvent("onTouchDataEvent ");

2.2 TouchDeviceAttachEvent

2.2.1 Plugin Event Name

"onTouchDeviceAttachEvent"

2.2.2 Example Javascript Usage

Registering for the event:

if (window.addEventListener)

{

obj.addEventListener("TouchDeviceAttachEvent", touchDeviceAttachHandler, false);

}

else

{

obj.attachEvent("onTouchDeviceAttachEvent ", touchDeviceAttachHandler);

}

Function handler for the event:

function touchDeviceAttachHandler ()

{

// Get current list of touch devices and open first device.

_touchDeviceIDList = plugin().touchAPI.TouchDeviceIDList;

_touchDeviceID = _touchDeviceIDList[0];

plugin().touchAPI.Open(_touchDeviceID, 0);

}

2.2.3 Design Notes

The plugin will register this event using:

registerEvent("onTouchDeviceAttachEvent ");

When the plugin receives notification from the touch API of an Attach event , it will fire notification to the client using the following:

this->FireEvent("onTouchDeviceAttachEvent");

2.3 TouchDeviceDetachEvent

2.3.1 Plugin Event Name

"onTouchDeviceDetachEvent"

2.3.2 Example Javascript Usage

Registering for the event:

if (window.addEventListener)

{

obj.addEventListener("TouchDeviceDetachEvent", touchDeviceDetachHandler, false);

}

else

{

obj.attachEvent("onTouchDeviceDetachEvent", touchDeviceDetachHandler);

}

Function handler for the event:

function touchDeviceDetachHandler ()

{

// close connection to the current touch device

plugin().touchAPI.Close(_touchDeviceID);

}

2.3.3 Design Notes

The plugin will register this event using:

registerEvent("onTouchDeviceDetachEvent ");

When the plugin receives notification from the touch API of an Detach event , it will fire notification to the client using the following:

this->FireEvent("onTouchDeviceDetachEvent");

3 References

[Cohn2011] Robert Cohn, EN-0265: WebPlugin User Notes, Version 1.0, 5/29/2011.

[Loftus2011] Ken Loftus, EN-0269: Branded Multi-Touch API Document, Version 1.0, 9/26/2011.

* This property may not be supported in all Feel™ Multi-Touch devices and is reserved for future use.