A MULTIMEDIA EDITOR FOR MATHEMATICAL DOCUMENTS

Gerhard Weber

Harz University of Applied Studies and Research
Friedrichstr. 57, 38855 Wernigerode, Germany  
gweber@hs-harz.de

Abstract

1. Introduction

Mathematics is a scientific discipline students experience as both a source of knowledge and a challenge for acquiring mathematical skills. As the World Wide Web, has mathematics a universal approach, it‘s notation can be read in almost any language and culture. But unlike browsing the Web is "doing mathematics" with computers still only possible for a small group of professionals and academics.

Techniques for editing mathematical documents on computers are as diverse as the users who create single equations, solve problems by transforming equations, or document a calculation. In this paper we concentrate on students solving mathematical assignments and teachers preparing those assignments. By explicitly addressing users who are blind, we will demonstrate the use of multimedia techniques for doing mathematics. For use by both blind and sighted users, each group requires a user interface to create, modify, or simply to read equations and text . Combinations of acoustic media with GUIs can possibly increase the effectiveness of computer based mathematics for all users.  

An approach for universal design of an interactive system for mathematical equations is needed as documents are prepared both for non-visual and visual use by others. Exchange of comments on text as well as on equations and terms facilitates the process of learning mathematics considerably and is the standard didactic principle in teaching mathematics. As a pre-requisite, only through the use of appropriate data formats it can be attempted to allow coherence between visual and non-visual presentation of complete documents or their elements.

2. Mathematical documents

Mathematical documents stored in computers have developed in several generations which can be described as

·    first generation documents: proprietary data formats for desktop publishing which do not allow electronic exchange,

·    second generation documents: non-standardised mark-up languages such as LaTeX and a variety of SGML-based document type definitions (DTD, see Sydow, 1994) which allow submission to publishers, and

·    third generation documents: standardized mark-up languages such as XML and the MathsML DTD which have been defined to allow electronic exchange with all readers .

Editing using second generation mark-up languages for mathematical documents has allowed to become independend from particular printers. Recent developments allow to translate from LaTeX documents with sophisticated mathematical contents to the portable document format (PDF) directly (Goossens, Rahtz, 1999).

Unlike any other approach is LaTeX open for extensions. Authors are free to create new commands in order to extend the mark-up language to avoid repetitive use of formatting commands. For example can a new binary operator be based on combining several glyphs.

This advantage turns into an disadvantage since exchange of mathematical contents using non-standard mark-up information is difficult. For example, semantic information about each operator within an equations is needed to transform equations within symbolic processors such as Mathematica or Maple. Up to now, several incompatible sets of LaTeX macros have been developed for their use within  symbolic processors.

Third generation documents can carry information such as the number of operands an operator takes and hence can be exchanged between typesetter and symbolic processor. MathsML has received the status of a recommendation by the W3C and was designed to be a third generation mark-up language. MathsML assists no method for extending the mark-up language but provides both a set of formatting tags as well as a set of mark-up tags for semantic interpretation of operands and operators. The following table 1 compares different MathsML representations of the term  \[ x^{3} + 5x - 8 = 0 \]

Table 1: Different versions of MathsML markup for  \[ x^{3} + 5x - 8 = 0 \]

Neither LaTeX nor MathsML were designed to be read by students and teachers. We have therefor extended a commercially available converter and typesetter accepting both MathsML and LaTeX (Techexplorer, see Goossens and Rahtz, 1999). Our extension is based on file transfer and a converter program. The converter  uses a Java-based XSLT processor. It is driven by an XSL file covering a subset of MathsML tags (see Figure 1).  

Figure 1: Conversion of LateX/MathsML to EuroMaths

The result of this conversion is a document which consists of simplified form of HTML with embedded equations. Equations are in this file are described using the EuroMath mark-up language. Within the MATHS project structure described by this mark-up method have shown to be suitable for students (Maths, 1997).

Various limitations in automatic transcription have shown that a subset of the EuroMath DTD can be used at the moment. For example is the end of an integral not clearly marked in MathsML output generated by TechExplorer (version 2.0). But polynoms, fractions, roots and nested terms put into parentheses produce sufficient mark-up.

Functional testing has to be based on manipulations of equations. In the following we will discuss therefor a editor accepting the generated file and which is able to generate input for our converter. A round-trip therefor consists of creating a web page with LaTeX mathematics, converting it to an input file for an editor, editing text and equations and saving modifications. The saved file can be exported again into LaTeX.

3. Developing mathematical documents

Development of proofs and other transformations by use of interactive systems has developed in a several steps:

·    first generation editing: symbolic representation programming languages require programming-like skills to use the computer for interactive development of equations and mathematical terms.

·    second generation editing: repetitive use of cut and paste operations for text-only linear character strings.

·    third generation editing: WYSIWYG Systems allow to develop equations by means of interaction objects such as menus, toolbars, and even handwriting (see Suzuki, 1999).

·    forth generation editing: tutoring systems which develop a user model and accompany the user (student, teacher, lecturer, etc.) by means of an intelligent expert in an adaptive manner.

Consistency among interactive systems has not yet been developed. There is no industry standard interactive equation editor as commercial and non-commercial systems (e.g. W3C’s Amaya) implement various of above aspects differently.

Non-visual presentation of mathematics is required if users cannot see equations either due to lack of vision or due to constraints such as lack of display space. Both auditory (Stevens and Edwards 1994; Raman, 1994) and braille based presentation (Schweikhardt, 2000) are suitable alternatives. In general, multimedia presentation of mathematical equations combined with multimodal input can assist all users in learning mathematics in a better way.

An interactive WYSIWYG editor for interaction via auditory output as well as braille based input and output has been developed within the MATHS project. One of the strengths of this system is the semantically oriented EuroMath DTD which allows synchronisation between different output techniques for each entity carrying mark-up information (MATHS, 1997). As no standard approach for access to interactive systems through assistive devices was available the MATHS workstation is a hardware dependent implementation.

4. M3 Editor

We have developed the M3 (multimedia mathematics) protoypical editor as a new interactive system on the basis of rendering techniques used within the MATHS workstation. Requirements for M3 are:

·    printing documents for desktop publishing using a wide range of printers and electronic file formats (PDF),  

·    multimedia capabilities of the MATHS workstations,

·    accessible through industry standard assistive devices and screen readers,

·    synchronisation of different media for collaborative use between sighted and blind people,

·    integration with MathsML-based documents, and

·    open for transcription into different mathematical braille notations.

In order to print documents, we rely on typesetting capability of TechExplorer. Since this is a plug-in for Netscape printing, file save and file open are standard operations provided by the browser.

To ensure multimedia capabilities the software architecture of M3 forsees a layered approach (see figure 2):

·         a web page written in HTML and Euromath DTD mark-up is parsed,

·         menus control document appearance and browsing of equations and text.

·         a graphical user interface both visualizes text only

·         a renderer transcribes mathematical terms into braille or spoken output , and

·         a screen reader and assistive devices access the graphical user interface.

As a novel approach to the design of GUIs for accessible editors we have visualized the concept of synchronising a focussing interaction object with spoken output  through a textfield. This field is writeable and readable as long as it contains text. The editor recognizes any equations and makes the textfield read-only. If the user moves on to text the textfield will accept input in a normal way again.

 
Figure 2: Layers in the M3 editor

By separation of text from equations within the graphical user interface it becomes possible to pass implicitely control of the user interface between a screen reader and a specific renderer for mathematical terms. Figure 3 shows a snapshot of the M3 editor prototype.

Figure 3: The M3 Editor

Only if there is a caret or selection being displayed then the screen reader becomes active. Making the textfield read-only requires the screen reader to be practically silent and the editor can generate speech output while the user is browsing the structure of an equation.

Input of this editor is menu-driven. A verb such as "next" combined with an object such as "fraction" forms the command "Alt-NF" consisting of two keys and a modifier key. The number of terms (and implicitly tags) is limited and motivates semantic structures. The phrases for verbs and objects  have been selected to fit naturally with unique characters of the alphabet. A configuration file allows to localize the menu to different languages.

At the point of this writing only browsing through an equation is implemented. Input of new terms and  their manipulation will be based on the same menu technique. As has been shown in the Maths workstation braille-based input is a possible extension as well.

Conclusions

MathsML, a recommendation for mathematical document mark-up by the World Wide Web Consortium, provides the basis for "doing mathematics" with an editing program, but it provides not the basic structure for a multimedia user interface. The graphical user interface of an multimedia editor should separate equations from text to allow also access through standard assistive devices and screen readers.

Acknowledgements

This work has been partially funded by the Human Language Technologies unit under the IST Programme by the Commission of the European Union (Project IST-2000-27513). We want to thank also Sebastian Breit, Ine Langer, Michael Menz, Robert Nitschke, Tino Reichardt, and Kurt Weimann for their help in implementing the M3 editor.

References

Goossens, M. and Rahtz, S. (1999). The LaTeX Web Companion, Reading, Mass.: Addison-Wesley.

Maths (1997). Final Report, available from F.H.Papenmeier, Talweg 2, 58840 Schwerte, Germany.

Raman, T.V. (1994). Audio Systems for Technical Reading, Ph.D. thesis, Department of Computer Science, Cornell University, NY, USA.

Schweikhardt, W. (2000). Requirements on a mathematical notation for the blind, in Vollmar, R. und Wagner, R. (eds.) Computers Helping People with Special Needs ICCHP 2000, Wien: Österreichische Computer Gesellschaft, 661-670.

Stevens, R. D. and Edwards, A. D. N (1994). Mathtalk: The design of an interface for reading algebra using speech, in W. L. Zagler, G. Busby and R. R. Wagner (eds), Computers for Handicapped Persons: Proceedings of  ICCHP'94, Lecture Notes in Computer Science 860, Berlin: Springer, 313-320.

Suzuki, M.; Fukuda, R.; Ohtake, N. (2000). Optical Recognition and Braille Transcription of Mathematical Documents, in Vollmar, R. und Wagner, R. (eds.) Computers Helping People with Special Needs ICCHP 2000, Wien: Österreichische Computer Gesellschaft, 711-718.

von Sydow, B. (1994). Editing mathematics in the Euromath System, Euromath Bulletin, 1 (2), 17-23.

Weber, G. Stevens, R.D. (1996). Integration of Speech and Braille in the MATHS Workstation, in Klaus, J.; Auff, E.; Kremser, W.; Zagler, W. (eds.) Interdiscplinary Aspects on Computers Helping People with Special Needs, ICCHP’96, (July, 17-19, 1996, Linz), Wien: Oldenbourg, 617-626.