Saturday, December 19, 2009

MatchTalk



                           

 

 

“MATCHTALK”

VOICE ENABLED WINDOW MANAGER

International Conference on Computers for People with Special Needs,

CPSN’ 2005

 

Vishal Saberwal

Computer Science Department,

California State University, Fresno

CA 93740 USA

vishalcyberwall@csufresno.edu,

+1-559-681-6745

Tarek Alameldin

Computer Science Department,

California State University, Fresno

CA 93740 USA

tareka@csufresno.edu,

+1-559-278-3965

Director Russ Perry

California State University, Fresno

2320 E. San Ramon Ave, Fresno,

CA 93740 USA

russ@csufresno.edu,

+1-559-681-6745



 



Abstract

This paper tries to find solutions to these questions by enabling a Window Manager with Voice Output.

Sit in front of you computer and close your eyes. How would you know what is being shown on the computer screen? Even if someone tells you what you have on the screen, would you be able to imagine or understand it? Would you know what colors you have on the screen or what graphics are being displayed? Would you be able to use the mouse: Would you know where you are moving your cursor? What if someone takes the monitor away: would it cause any difference to you having your eyes closed? How would you read your online news, your papers, and your emails?

 

Keywords

MatchBox: A suite of tools for the user management of X11 Applications.

Festival: A general framework for building speech synthesis systems.

Window Manager: A program that manages a graphical user interface, determining the appearance of windows and determining the response to operations such as clicking on the desktop.

 

Introduction

This paper conceptualizes the idea of a Window Manager for the Visually Impaired. If you cannot see a Window on the screen, how do you operate a modern computer?

The paper explores the capability of Window Manager with Voice output. The goal is to be able to navigate already created applications using only the keyboard and no screen (Voice Output only). The goal of the paper is not to create a different interface, but to make it possible for visually impaired to use the existing ones.

The developed system’s interface has proved to be useful for visually impaired people. The system was named Matchtalk.

 

Overview

The paper is divided into five sections. The first section explains the system configuration used to develop the system. This section also gives a brief overview of the packages that the system depends upon. Section two provides with the information on the steps taken and goals achieved to achieve the system’s functionality. This section also describes the underlying API (Application Programming Interface) that has been developed on and used. Section three gives details on the developed system’s code and explains the constructs. The system was tested for visually impaired and their usability tests are described in section four. There are a lot of advanced features that if implemented, will create great applications to assist visually impaired. Some of these future advancements to our system are discussed in section five.

 

1. System Configuration

The developed system is Linux based computer with Festival text-to-speech package and the MatchBox Window Manager. The developed system is portable to the handheld computers. Our developed system is configured for voice output.

 

1.1. Matchbox

MatchBox has been developed over the past 3 years. This Window Manager is designed for pocket type computers. It is a suite of tools for X11 user management on PDAs.

 

Matchbox’s Window Manager maximizes application windows to maximum display size, thus viewing only one at a time. The user navigates through these windows by clicking on a dropdown list on the window title bar.

 

Since it was designed to be used on PDAs, it is small in size, fast, easy to understand and flexible. It is written in C and is very well commented.  It facilitates an environment for arranging and manipulating applications in a graphical environment. All these advantages for an open source window manager made it suitable for our system.

 

1.2. Festival

Festival is part of the work at Carnegie Mellon University for advanced speech synthesis. It provides the capability to build new voice using its internal constructs.

 

Festival offers a text to speech synthesizer to create speech. It supports a number of APIs: for shell, Scheme, a C and C++ API library and an Emacs interface. It also provides multi-language support.

 

Festival is designed as a speech synthesis system for at least three levels of users: those who want high quality speech from arbitrary text with minimum effort, those who are developing applications and wish to include synthesis output, those who are developing new synthesis methods. The developed system falls in both the first and the second categories, for providing high quality speech with least effort and enabling window manager and its applications for voice output. These advantages provided by an open source text to speech synthesizer made Festival suitable for our developed system.

 

2. Project Planning

The project was easy to achieve and the results could not only be seen but also heard. The following were the steps I used to achieve the project goal:

·       Prototype computer was installed and configured with our system requirements.

·       An event list was created and the system was designed and analyzed. The test cases were written.

·       An API library was designed and implemented as an interface between the Window Manager and the Text to speech synthesizer.

·       The portable system was implemented and packaged with desired code and installation scripts.

·       The complexity of voice tags were increased and incorporated in the optimized code.

·       The developed system was tested on the prototype computers.

 

The developed system did not need a mouse or a monitor. Currently our system needs keyboard for keyboard events for corresponding voice output.  The test prototype would consist of a Computer and a keyboard. This in future will be replaced by voice recognition system requiring no keyboard, mouse or a monitor.



We had various options available for a window manager for the development. Microsoft Window manager and XWindows [1] Window manager were the two prominent options. Microsoft has all their code hidden, thereby provided the least flexibility to change or customize the window manager to our system’s needs. XWindows has open source code and hence can provide an insight to how the windows and events are being managed. XWindows has a big array of applications written to run on it. Though it had all the features needed, the code size and architecture was complexand may require a lot of processing resources on handhelds for our system applications.

 

The looks of Window manager did not matter. We needed a window manager because applications are written using windows. This provides easy management for the windows and easy control of the events in object-oriented architecture. The system needed a Window Manager, which is fairly small, has an open source and supports many applications.

 

MatchBox Window Manager being made for Palmtops/Handhelds, which has space as a limiting factor, was small, easy to understand, and supported all applications running on XWindows. Since, this windows manager was made for Linux, we chose Linux as the base operating system for our development project.

 

The Text-to-Speech synthesizer for our system needed to be open source, for configuration and modification flexibility; reasonable in cost, and one that is under progressive development.

 

Festival, [2] a text-to-speech synthesizer was being developed in Carnegie Mellon University. It was under development to incorporate speech recognition, which would satisfy MatchTalk’s proposed future needs. Further it had a C Application Programming Interface available to extend its use to suit our needs. It satisfied all our requirements and hence was suitable for the MatchTalk system.

 

The window manager had to be modified to suit our needs. The voice tags were enabled for various components of the window manager. Then designed keyboard events were mapped for shortcuts to tasks like opening main menu.

 

The directory structure was created and install and build scripts were created. MatchTalk finally started talking, rendering voice outputs corresponding to the events generated on the system.

 

2.1. Window Manager Understanding

To make the window manager talk, first we need to understand the flow of events and actions.

Matchbox is an Event based Window Manager. The Window Manager starting point is main.c. Initially a new window manager (wm) object is created by calling wm_new(). This structure object stores the environment and information for the system it is running on. This loads up any user configuration data using wm_load_config() from command line parameters. Various wm X11 parameters are then initialized and a new wm object is returned which is used throughout the program to manipulate and control the window system.

 

After taking care of the windowing system for the matchbox window manager, the program enters its main loop, wm_event_loop(). Here it captures system and application events for configuring or performing requested/required actions.

 

When a new client window appears, MatchBox receives a notify event for the new window and a new ‘Client’ object is made for the new window by wm_make_new_client(). This decides the type of client object to be initialized (Application/Dialog/Panel window).

 

All created client objects are stored in doubly linked list referenced by pointers in Wm structure.

 

Once a client object is created, ewmh_set_allowed_actions() sets client variables to Window Manager properties. Suitable client geometry is calculated by calling client’s configure (). The client window is then resized and positioned to fit MatchBox style. The client’s show () is called which then displays the client.

 

While the client is runing, it will generate events, which will be sent to client’s methods and will be managed.

 

Key presses are monitored by wm_handle_keypress(), which searches for the relevant action defined in keys.c.

 

Button press event are be handled by wm_handle_button_event(), which will get the location of the button press and may call client’s button_press() if required.

 

The file client_common.c takes care of button management. The files above were modified to suit the project needs.



Festival incorporation into the program is done using the file fest.cc. This file uses C API to program voice output for the text. These functions defined in the file have been extensively used wherever voice enabled output was desired.

X messages for MatchBox are handled by the function wm_handle_client_message() before being sent to ewmh_handle_root_message() for the root window.

 

3. The Application

Now after having an understanding of how Matchbox works, an Interface File is created to communicate with Window Manager and Festival.

The ‘C’ API for festival is used to create this file.

 

/* fest.cc : Made by: Vishal Saberwal : CSUFresno : MatchTalk*/

#include <festival/festival.h>

extern "C" {

 

int fest_init() {

//Default scheme heap size

                  int heap_size = 210000; 

//We want the festival init

// files loaded

int load_init_files = 1;  festival_initialize

  (load_init_files,heap_size);

}

int fest_say(char *s) {

              festival_say_text(s);

}

 

}

 

We need to call fest_init() only once to initialize festival configuration for speech synthesis. From then on, every call to fest_say() will render voice output to the text sent in.

 

When Matchtalk (modified Matchbox for visually impaired) boots up, it starts with a welcome speech. Remember that, we do not have any monitor, hence all the input is provided using keyboard and the output is speech only. The arrow keys on the keyboard are used to navigate the menu. The <Alt>+<Tab> keys are used to toggle between two clients (application windows).

 

To implement the speech for the events happening on the system, we need to provide fest_say() function calls for keyboard handling functions (mainly in wm.h for testing).

 

Client are identified by the name defined in the window manager and whenever they get focus fest_say()  call is used in wm_activate_client() to notify the user of the application receiving focus.

 

Makefile was created to compile the above festival functions into MatchTalk. The relevant contents of the Makefile are as follows:

 

FEST_LIBS = -lFestival -lestools -lestbase -leststring -ltermcap

 

matchbox_LDADD =  -L/usr/X11R6/lib -lX11 -lXext $(XFT_LIBS) $(PNG_LIBS) $(XSET_LIBS) $(EXPAT_LIBS) $(SN_LIBS) $(standalone_libs) $(FEST_LIBS)

 

matchbox_SOURCES = main.c structs.h

wm.c wm.h 

base_client.c base_client.h 

main_client.c main_client.h

toolbar_client.c toolbar_client.h              

dockbar_client.c dockbar_client.h  

dialog_client.c dialog_client.h              

select_client.c select_client.h              

desktop_client.c desktop_client.h  

ewmh.c ewmh.h     

misc.c misc.h              

client_common.c client_common.h

keys.c keys.h   

list.c list.h  

fest.cc $(standalone_src)

 

matchbox_OBJECTS =  main.$(OBJEXT) wm.$(OBJEXT) fest.$(OBJEXT) \

base_client.$(OBJEXT) main_client.$(OBJEXT) \

toolbar_client.$(OBJEXT) \

dockbar_client.$(OBJEXT) dialog_client.$(OBJEXT) \

select_client.$(OBJEXT) desktop_client.$(OBJEXT) \

ewmh.$(OBJEXT) misc.$(OBJEXT) \

client_common.$(OBJEXT) keys.$(OBJEXT) \

list.$(OBJEXT) mbtheme.$(OBJEXT) xml.$(OBJEXT)

#matchbox_OBJECTS =  main.$(OBJEXT) wm.$(OBJEXT) \

main_client.$(OBJEXT) \

dockbar_client.$(OBJEXT) \

select_client.$(OBJEXT) \

ewmh.$(OBJEXT) \

 

fest.o: fest.cc

              g++ -c fest.cc

 

To install the system run at the shell prompt the configure script ($./configure), make the source ($make) and ($make install) install the MatchTalk.

 

MatchTalk can be modified to voice enable most applications that can be supported by the MatchBox or XWindows Window Manager. The more applications you make voice enabled, the more technology you provide to the visually impaired.

 

4. Testing

The system was tested for editor and a few other applications. The users were students (including those who are visually impaired) at the university from different majors.  The visually impaired students needed to be provided with training to use the system.

The system was marked as 4 on a scale of 1 through 5. In terms of usability, the system was marked 3 on a scale of 1 through 5.

One phase of the development is working on supporting the applications requested by the students. Another phase of the development is providing the system with voice recognition so as to reduce the training effort and provide more benefits to the visually impaired.

 

5. Future Work

The advancement of this developed system may lead to providing voice-enabled applications for future Operating Systems. What merely seems like an aid for Visually Impaired can be used as talking computers as in science fictions and is portable to handhelds for providing voice output. When festival comes up with stable version of Voice Recognition System; this can provide a new version of MatchTalk System, making keyboard, mouse and monitor redundant, thereby providing least cost systems. This will significantly reduce the resource requirements of the handhelds and computers.

 

Future scope of the project includes:

·       Voice recognition and voice output computers (Talking Computers),

·       Teachers/Teaching aids,

·       News/Notes/Book readers

 

In addition, MatchTalk system will provide voice authentication systems for the security industry. It can also be used to provide voice commands to the remote systems, thereby raising preconfigured events.

 

MatchTalk, if blended with Artificial Intelligence, can benefit teachers by providing preformatted lectures and answering/explaining a set of questions. Our system then can provide answers based on the reactions. Future extensions to the developed system and the scope of its use are versatile and immense.

 

The system will bring is intended to provide equal opportunity for people with disabilities to learn modern computers and applications.

 

Acknowledgements

This work was partially funded with the grant from Edwards Air Force Base. The authors are thankful to their support.

 

References

[1]  XWindows CVS and Documentation: http://www.xfree.org/

[2]  Festival & Speech synthesis CVS and Documentation: http://www.speech.cs.cmu.edu/festival/ 

[3]  MatchBox CVS and Documentation: http://matchbox.handhelds.org/

[4]  The Speakup Project:

http://www.linux-speakup.org/

[5]              File Specification for the Digital Talking Book ANSI/NSO:

              http://www.linux-speakup.org/niso-dtb.html

[6]              Department Of Justice Section 508:

              http://www.section508.gov/

[7]              W3C Auxiliary Benefits of Accessible Web Design & Policies:

              http://www.w3.org/WAI/Policy/

[8]              Accessibility at IBM:

              http://www-306.ibm.com/able/access_ibm/index.html

[9]              Linux Accessibility Solutions:

http://www.linorg.cirp.usp.br/RedHat8.0/README-Accessibility

 


No comments:

Post a Comment