Introduction to Sketch Recognition

-Tracy Hammond and Kenneth Mock

COMMENTS ON OTHER PEOPLE'S BLOGS

1. Comment on Nabeel's blog

SUMMARY

The paper presents an overview of the existing technology and ongoing research in the field of sketch recognition (SR). It also enumerates the ways in which SR could make certain tasks simpler and more efficient. It begins with a brief introduction of Ivan Sutherland's sketch pad and a probable reason why it couldn't take off. Raster graphic displays despite their inability to produce smooth lines overshadowed vector graphics (used in sketch pad) due to the flicker free display and lower cost of the former.
Next, types of digitizers (technology used to determine the location of a pen while writing or navigating) are discussed. Passive digitizers use only touch data and don't require a special pen to navigate. However they suffer from various disadvantages like Vectoring (unintended click), jumpy mouse cursor, difficult secondary inputs like right click, and lower resolution. Active digitizers on the other hand use electromagnetic signals reflected off a special pen to get position data, but are free of other disadvantages associated with passive digitizers.
After that, various hardware and software technologies that are used in sketch recognition systems are described. Convertible tablet PCs, slates, Wacom pen tablets are some of the hardware technologies to enable pen based input. Microsoft Vista and XP to some extent have handwriting recognition capabilities. Camtasia screen capture allows users to record their pen interactions.
The next part is related to applications in education. Instructors can deliver their lectures with the help of tablet PCs and large displays. In addition to previously prepared content of their slides, they could show on the fly data by simply sketching on the slides. User studies show that students have shown an increase in performance when such methods were used by instructors. There are few disadvantages associated with this method; one of which is the initial learning curve.
The paper then presents several pointers as to how lectures could be prepared and delivered using the above mentioned technologies. These technologies have found some nice application in describing molecular structure to students, in the field of high school physics and mathematics.
After that the FLUID framework is described. FLUID framework enables end-users to describe their own shapes and domains. So this framework is sustainable and self learning in that way. Users could either describe shapes by entering text data or just by drawing an example shape.
In the end two user studies illustrate how sketch based systems have actually shown positive results in a classroom setting. Statistics show an increase in tablet-PC usage especially in the field of education. So it might not be long before we see tablet PCs installed with sketch recognition systems as a ubiquitous piece of technology in classroom and elsewhere.

DISCUSSION

The paper gives a cursory overview of sketch recognition technology to the uninitiated. I liked the idea of the FLUID framework, where the end-user does not have to wait for a new version every time he wants to work with a new domain. The system can be taught to recognize new shapes. In essence, the intelligence of the system will evolve with usage, very much in line with what humans go through. I was very impressed with the idea of teaching the system a new shape by drawing an example. The text input method, however, would seem a little intimidating to the user.
One more thing that impressed me was the pressure sensitive capabilities of active digitizers. This technology could go a long way in giving digital pens a natural feel of writing pen. I can see it being used by professional painters and artists in future.

Sketch Pad

-Ivan E. Sutherland

COMMENTS ON OTHER PEOPLE'S BLOGS

1. Comment on Nabeel's blog

2. Comment on Daniel's Blog

3. The Comment on my own blog!!


SUMMARY

The paper contains the description of sketch pad, a tool that could be used for simplifying creation, manipulation and storage of drawings. It begins with an example of drawing a regular hexagon and associated patterns to give readers an idea of what Sketch pad can accomplish. The author uses this example to illustrate underlying concepts like sub picture, constraints and definition copying. The sub picture is an instance of a master figure which can be used repeatedly. This saves a lot of design time in creation of highly repetitive drawings. Enforcing a constraint implies establishing a relationship between different parts of a picture. Thus if one component among all the components involved in a constraint relationship is changed (translated, rotated, deleted etc), other components will automatically change in order to satisfy the constraints. All such elements are stored as the n-component ring structure (which actuates the propagation of constraints). One of the remarkable things about Sketch pad is the hierarchical storage of drawing elements. This ensures that changes made to a basic element will automatically propagate to higher levels. Moreover, this allows usage of general functions which act on higher level drawing objects by acting recursively on the lower levels. The concept of these general functions is akin to the modern day OOPS concept of abstraction. The author then describes the light pen tracking, pointing and display generation. For pointing to a picture element the system only selects the spots which are within a threshold distance from the center of the position of the light pen. For display generation, the coordinates of the spots are stored in a file with 36 bits allotted to each display spots, out of which 20 bits contain coordinate information and remaining 16 bits contain the address of the n-component element to which the spot belongs. This 16 bit tag allows the system to know which picture element is being aimed at. Another very useful feature of sketch pad is that it enables user to draw intricate details with high accuracy using the magnification feature. In the end, several practical applications of sketch pad are suggested like analysis of force distribution in bridge structures; in artistic drawings and animation; in electrical circuit diagram. The author suggests that future efforts could be directed towards 3D drawing and capability of defining transformation functions on drawing objects.

DISCUSSION

Sutherland’s paper must have been a groundbreaking idea when it was published. The fact that despite the vast difference in technology - both hardware and software, between now and back in 1960s, there is not much difference between present day design tools and Sutherlands sketchpad is what is remarkable. Having said that, as far as the recognition part of Sketch recognition (SR) is concerned, the sketch pad doesn’t do much, nothing actually. My interpretation of recognition is inputting noisy data say a set of pixels (with a high cardinality), and getting in return pure information, a set which consists only of classifying features. The Sketch pad doesn’t do that. When we move the light pen and turn the knobs, in effect we are providing it with pure information (feeding it with coordinates, the critical defining ones and telling it what shape we are trying to draw) and all it does is manipulate that information in various ways (which I must admit it does brilliantly). Now I am not claiming this to be a fault with the system. I am just suggesting that sketch pad might not be relevant to recognition. Therefore, in the context of SR, I would not think of improving this system. Sketch pad indeed could be made better by using SR algorithms in it. But sketchpad itself does not provide any knowledge base for the development of SR per se.

Something about me..

E-mail address : akb2810 at tamu dot edu

Graduate standing : 1st year Masters

Why am I taking this class? I tried to search for a book related to Sketch Recognition. I could not find even one. Less explored field -> huge probability that research will lead to new findings.

What experience do I bring to this class? I built a simple interface in the final year of my bachelors where I could navigate the mouse cursor and perform other functions by waving my hand from a distance. Thats all I have.

What do I expect to be doing in 10 years? Working in a research laboratory in some field of Computer Science, or may be sociology and economics. ( I have no idea what these two latter fields are all about. But I keep developing theories in my head, rejecting or accepting them. I am my own audience)

What do I think will be the next biggest technological advancement in computer science? It could be virtual reality, like virtual office spaces etc.

What was my favorite course in undergrad (CS or otherwise)? Design and analysis of algorithms.

If I could be another animal, what would it be and why? I could never be any other animal. I love all the confusion in human head. It would feel too tied up being driven by instinct alone.

What is my favorite motto or slogan? Imagination is better than knowledge. -- Albert Einstein

What is my favorite movie? Right now ... Hotel Rawanda, Ghandhi, Top Gun.

Some interesting fact about myself? I used to paint a lot and I was good at it. But midway I lost interest because I could not find a defining factor which makes one piece of art better than the other. I find this reason funny and interesting.... and sometimes sad(!). I like photorealistic paintings though because these can be evaluated by a deterministic method of how close to reality they are. So there is a defining factor....