Project Reading 6

MathPaper: Mathematical Sketching with Fluid Support for Interactive Computation

Citation:

Zeleznik, R., Miller, T., Li, C., and Laviola Jr, J. J. Mathpaper: Mathematical sketching with fluid support for interactive computation. In Smart Graphics, Springer (2008), 20–32.

Summary:

This paper presents a framework called MathPaper. It represents a system for fluid pen-based entry and editing of mathematics which also has the support for interactive computation. 

Writing on MathPaper interface is just like writing on a paper, in which multiple mathematical expressions and even algorithms can be entered anywhere on the page. It also talks about modification of Mathematical expressions using simple deletion and dragging gestures along with real-time recognition and computation feedback. 

MathPaper also supports extended notations and gestures for controlling computational assistance, simplifying input, and entering algorithms, making MathPaper a user-friendly system for mathematical sketching and computation.

Discussion:

This paper also presents the idea of writing mathematical equations on paper and not on the computer, which is the idea we are trying to work on in our term project. It also talks about modification of mathematical expression modification which we are also performing in our project.

Project Reading 5

Designing a Sketch Recognition Front-End: User Perception of Interface Elements

Citation:

Wais, P., Wolin, A., and Alvarado, C. Designing a sketch recognition front-end: User perception of interface elements. In Proceedings of the 4th Eurographics Workshop on Sketch-based Interfaces and Modeling, SBIM ’07, ACM (New York, NY, USA, 2007), 99–106.

Summary:

This paper focusses on presenting a user study which investigates critical sketch recognition user Interface issues in systems that recognize hand-drawn diagrams. There are four critical sketch recognition user interface issues:

1) how users integrate the process of triggering recognition into their work.

2) when users prefer to indicate which portions of the diagram should be recognized.

3) how users prefer to receive recognition feedback.

4) and how users perceive recognition errors.

The paper believes that programs that can recognize students’ hand-drawn diagrams can revolutionize education. This is because they break down the barriers between diagram creation and simulation. This paper thus focusses on understanding how to support this new interaction paradigm from a user’s perspective.

Discussion:

The paper presents a very well conducted user study to understand the drawbacks and limitations of the exisiting systems. They have shown the data for Electrical circuits for the user study. It would have been really helpful to perform the user study in any other domain also.

Project Reading 4

Citation:

LogicPad: A Pen-Based Application for Visualization and Verification of Boolean Algebra. Kang, B., and LaViola, J. Logicpad: A pen-based application for visualization and verification of boolean algebra. In Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces, IUI ’12, ACM (New York, NY, USA, 2012), 265–268. 

Summary:

The paper presents a framework called LogicPad, which a pen-based application for boolean algebra

visualization. It lets users manipulate boolean function representations through handwritten symbol and gesture recognition. It also has a drag-and-drop interface which makes interaction with the interface much easier. LogicPad’s user interface and the general algorithm are discussed in the paper which are used for verifying the equivalence of three different boolean function representations: boolean expressions, truth tables, and logic gate diagrams. A user study is also conducted by the authors evaluating LogicPad’s user interface, visualization techniques, and overall performance.

Discussion:

As per the results, visualizations were generally well-liked and verification results matched

user expectations, which is pretty good. The sketching area on which the users write as they normally would using pen and paper, is very intuitive and which we are trying to follow in our project too.

Project Reading 3

Citation:

A Handwriting-Based Equation Editor. Written by: Steve Smithies, Kevin Novins, James Arvo.

Summary:

Equation editing systems relying on either textbased equation description languages or on interactive

construction by means of structure templates and menus are tedious to use, even for experts, because the user is forced to “parse” the expressions mentally before they are entered. The system described below is a prototype equation editor that is based on handwriting recognition and automatic equation

parsing. 

The recognition system performs Character Recognition, Stroke grouping and Equation parsing. Parsing uses LATEX like notation.The main drawbacks of a graph grammar approach are

the number and complexity of subgraph searches. The system also provides a basic User interface for ease of use. It also provides correcting of stroke grouping errors and correcting of character recognition errors. The next step is providing correction of Equation parsing errors. This system provides a natural and familiar interaction method or recognizing equations.

Discussion:

The time taken for parsing simple formulæ is less than two seconds, though more complicated formulæ may take twenty seconds or more. This time taken for parsing is very less and shows the efficiency this system has.

Project Reading 2

Recognition and retrieval of mathematical expressions

Citation:

Recognition and retrieval of mathematical expressions. Written by: Richard Zanibbi , Dorothea Blostein.

Summary:
This paper surveys recognition and retrieval of mathematical expressions, organized around four key problems in math retrieval (query construction, normalization, indexing and relevance feedback), and four key problems in math recognition (detecting expressions, detecting and classifying symbols, analyzing symbol layout, and constructing a representation of meaning).
The problems arising in Math Notation recognition are:
1) Expression Detection
2) Symbol Extraction or Symbol Recognition
3) Layout Analysis
4) Mathematical content Interpretation

It creates two kinds of trees for equation recognition: Symbol layout tree and Operator tree. The following problems arise in retrieval of Math notation:
1) Query languages and query formulation
2) Normalization
3) Indexing and matching
4) Relevance feedback

In solving the ambiguity of mathematical expressions, we first need to understand the meaning of what do each of the symbols represent.

Discussion:
This paper has given many future work ideas. Also, while doing our project, we felt on the same lines as the author (that solving the ambiguity in Math expressions, we need to understand meaning of each symbol). The use of various tools like MathML and Latex is very intuitive.

Project Reading 1

Mathpad: A Fuzzy Logic-Based Recognition System for Handwritten Mathematics

Citation:

Mathpad: A Fuzzy Logic-Based Recognition System for Handwritten Mathematics

Fitzgerald, J.A. ; Univ. Coll. Dublin, Dublin ; Geiselbrechtinger, F. ; Kechadi, T.

Summary:

A recognition system which enables the users to enter mathematical equations in customized fashion is presented. The approach uses Fuzzy logics and fuzzy rules to extract features, classify symbols and assess spatial relationships. Using all this, the resultant outcome is a tool for Mathematical problem-solving and visualization. 

Two types of fuzzy rules are used for symbol recognition: High-level and low-level fuzzy rules. Mathpad performs syntax checking through Shift Reduce parsing. Some changes made to SRP are in Input attributes, Fuzzy constraints, Semantic rules and Fuzzy Parse trees. This paper then established the use of Fuzzy Shift Reduce parsing. 

Discussion:

The system does not change its computation method based on complexity of equation written. Also, the results of ME recognition rate are better, but I think there still is a scope for improvement. I really liked the idea of using Fuzzy logics in understanding and parsing the equation.

Reading 27

Reading 27: Visual Symbolic Recognizer

Citation

Tom Y. Ouyang and Randall Davis. 2009. A visual approach to sketched symbol recognition. In Proceedings of the 21st international jont conference on Artifical intelligence (IJCAI'09), Hiroaki Kitano (Ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1463-1468.

Direct Link: http://dspace.mit.edu/openaccess-disseminate/1721.1/71572

Summary

The paper presents a sketched symbol recognition that focuses on the visual appearance of the symbols. It uses a new symbol classifier that is computationally efficient and invariant to rotation and local deformations. It evaluates performance on following three domains: handwritten digits, PowerPoint shapes, and electrical circuit symbols.

The paper represents symbols as feature images rather than as geometric primitives. It also proposes a set of visual features which captures on-line stroke properties (eg. orientation, endpoint location). The paper introduces a computationally efficient classification technique which is robust to rotation and local deformations. The paper then talks about it exceeding performance on all of the three datasets evaluated. 

Four reference angles are used as orientation features correspond. The endpoint feature identifies stroke endpoints. The stroke is then sent for smoothening and Downsampling. 

IDM Distance between two symbols I1 and I2 is represented as:

D2 = X x,y min dx,dy ||I1(x + dx, y + dy) − I2(x, y)||2 

Discussion

Getting good accuracy in three different domains is appreciable. There still is some scope of improvement as SVM-linear seems to perform better. Probably it can be combined with SVM-linear and used as a collaborative system.

Reading 26

Reading 26: Shape Context

Citation

Oltmans, Michael. Envisioning sketch recognition: a local feature based approach to recognizing informal sketches. Diss. Massachusetts Institute of Technology, 2007.

Direct Link: http://rationale.csail.mit.edu/publications/Oltmans2007Envisioning.pdf

Summary

Discussion

Reading 25

Reading 25: Who Dotted That i?

Citation

Eoff, Brian David, and Tracy Hammond. "Who dotted that'i'?: context free user differentiation through pressure and tilt pen data." Proceedings of Graphics Interface 2009. Canadian Information Processing Society, 2009.

Publication Link: http://dl.acm.org/citation.cfm?id=1555916

Summary

The paper presents  a system that shows that the creator of an individual pen strokes can be determined with high accuracy. 
The paper presents following as the benefits of unobtrusive user differentiation:
1) Security and Verification
2) Improving sketch recognition through user modelled recognition
3) Collaboration
Multiple experiments which have been done for developing this system have been described in the paper. In a collaborative sketch environment, the authors are able to identify the creator of individual strokes without forcing the user to use a specific pen, or to select a specific mode to indicate who is drawing, which is very impressive.

Regarding the features, it says that even poor features hold some value when determining user differentiation. Also, there is no single feature which could be used as a discriminator for different users. 

Discussion

This paper presents a good point about some features being better than the others in differentiating one user from the other. The idea of giving some score to even the poor features is intuitive and useful.