Text Entry Metrics for Android

Text Entry Metrics for Android (TEMA) is an application to calculate text entry performance and accuracy measurements on portable devices running Android OS 1.5 or later. TEMA logs individual input events, as well as words-per-minute and error rate values.

This application functions independently of the input method (IME) used. Therefore, multiple IMEs can be evaluated without any modification or reinstallation of TEMA. TEMA facilitates analyses of existing IMEs as well as new prototypes.

For additional information regarding TEMA, please view this video and the following academic papers:

Castellucci, S. J., and MacKenzie, I. S. (2013). Gathering Text Entry Metrics on Android Devices. Proceedings of the International Conference on Multimedia and Human Computer Interaction � MHCI 2013. Ottawa: International ASET Inc. [PDF]

Castellucci, S. J., and MacKenzie, I. S. (2012). Encouraging consistency in mobile text entry evaluations. CHI 2012 Workshop on Designing and Evaluating Text Entry Methods. New York: ACM. [PDF]

Castellucci, S. J., and MacKenzie, I. S. (2011). Gathering text entry metrics on Android devices. Extended Abstracts of the ACM Conference on Human Factors in Computing Systems � CHI 2011. New York: ACM. [PDF]

Download

TEMA v1.0, 122 kB

• Initial release
• The folder to contain log files must be created manually (the default location is /sdcard/TEMA_logs)

TEMA v1.1, 122 kB

• Accidental or premature presses of Enter are ignored. A trial only terminates if Enter is pressed and the length of the transcribed phrase is at least as long as the presented phrase, minus five characters.
• The log directory (/sdcard/TEMA_logs) is static and cannot be changed. However, the directory is created automatically.
• The stats logs include the number of deleted characters (delChars). Previously, TEMA assumed each press of Bksp deleted one character. This change allows for correct error rate calculations with IMEs that delete multiple characters at a time. The calculation of CER and TER now use delChars, rather than numBksp. Note: The order of columns in the stats logs is affected.

TEMA v1.2, 125 kB

• Trial data and some devices specifications (eg, OS version, display size, IME in use) can be uploaded to our server. This data will be used for academic research on mobile text entry. No personal information is sent. To disable the data upload, uncheck the "Upload trial data" option in the parameters dialog.

TEMA v1.3, 127 kB

• To encourage participant diligence, text input must have a UER less than a user-specified threshold. Otherwise, the trial will repeat.
• TEMA now ignores the first token (not just the first character) when calculating entry speed. This change improves timing accuracy for word-based input techniques.

TEMA v2.0, 243 kB

• added option to convert all phrases to lowercase
• added option to remove non-letters from phrases
• added 3 new corpora (1500 phrases) that better represent mobile text entry
• included screen dpi in the uploaded log data to calculate actual screen size
• added IME selection to parameters dialog
• changed the size of the transcription field to match the presented one (i.e., the size of the transcription field does not change during input)
• changed the log format from comma-separated vector (.csv) to tab-delimited (to avoid collisions with commas in phrases); also changed the file extension to ".tema"
• changed the comment delimiter in the logs from "#" to "[#]" to avoid collisions with hashtags in the Twitter and Mobile corpora

TEMA v2.1, 245 kB

• added inter-process communication between IME and TEMA to allow TEMA to log IME-specific data; to enable this, IME developers should use the TemaImeLogger class in their IME package to log data