Since 2006, there has been quite a lot of research into multitouch user interfaces, largely thanks to Jeff Han.
Many people have started building their own hardware, and commercial devices are also starting to become
available.
To enable easy development of applications for such multitouch- and tangible-based GUIs, we present
libTISCH,
a cross-platform, cross-device multitouch development framework. Some of the more prominent features include:
- ready-to-use multitouch widgets based on OpenGL
- reconfigurable, hardware-independent gesture recognition engine
- support for widely used (move, scale, rotate..) as well as custom gestures
- hardware drivers for Kinect, FTIR, DI, Wiimote, DiamondTouch..
- TUIO converters: source and sink
- cross-platform: Linux, MacOS X, Windows (32 and 64 bit)
- cross-language: C++ with bindings for C#, Java, Python

Tracking daemon in shadow-tracking mode.

Demo of various multitouch widgets.

The ubiquitous picture browser.
Of course, an abstract architecture is much more valuable with a reference implementation. Our cross-platform (Linux, Windows, Mac)
reference implementation is
libTISCH. The following releases are available:
Additional releases may be available on our
project site.
In order to reach a large user base, we decided to release
libTISCH under
LGPL.
If you are interested in using
libTISCH, please subscribe to the
mailing list where I'll post updates.
Documentation (sort of ;-) is available in the
Wiki.
For bleeding-edge development versions, please see the
Sourceforge SVN.
Update: this patch has been merged into the FreeGLUT codebase and will be
part of the upcoming release 2.8.0. Consequently, the patch is no longer needed,
but will remain here for archival purposes.
Here's a patch to
FreeGLUT which
enables the Windows 7 multitouch and the XInput2 multi-pointer callbacks.
The patch was tested against the current FreeGLUT 2.6.0 SVN version.
Note that the Windows part is still not very well tested.
freeglut-2.6.0-multi.patch
As stated above, we believe that the logical next step is the design of a software architecture that allows developers to
focus on writing applications instead of focusing on low-level stuff like hardware issues, gesture recognition
and so on. Especially the gesture recognition is of importance here; a common interface which provides applications
with semi-standardized events instead of raw input data should be a great step towards easier development. In our paper
"A Multitouch Software Architecture",
we provide a draft of such an architecture. For an instant overview, see the image on the right.
The ultra-short summary of the four layers, from bottom to top:
- hardware abstraction layer: takes raw data from the input hardware and generates data packets containing the positions of hands, fingers and objects (if available)
- transformation layer: converts the data into screen coordinates and outputs transformed data packets
- interpretation layer: reads screen-aligned input positions and converts them to gesture events
- widget layer: registers for gesture events and generates visual output
All data exchange between the layers is done through clear-text messages over UDP as transport protocol. Although this
might turn out to be a source of latency, we selected UDP because it allows for the most flexibility in choosing platforms
and languages for alternative implementations of any layer (which we want to encourage!). Moreover, our first tests show a
surprisingly small additional latency resulting from the use of UDP.
No library is ever complete, which is why we still need..
- improved developer documentation in the Wiki
- more gesture/feature specifications
- more widgets, e.g.
..and above all:
discussion!
If you notice any sort of bug or glitch, please take a second to open a
tracker item, post to the
mailing list or simply send me an email ( echtler ät in.tum.de ).
My name is
Florian Echtler,
and I designed and wrote large parts of this library during my
PhD thesis
work with
Prof. Gudrun Klinker
at the
computer science department of
TUM in Munich, Germany. Other contributors include Andreas Dippon, Nikolas Dörfler, Daniel Pustka, Martin Schanzenbach, Eva Artinger, Martin Kleinhans, Lode Vandevenne, Johannes Woehler, Darren Leigh and Norbert Wiedermann. Thanks!
Validated

-
Hosting provided by: