notes/docs/lectures/mdp/14_touch_tech.md

Touch Technology

Resistive Touch Screen

• Two sheets of transparent layers with metallic (resistive) coating facing each other with a thin gap in between

• Can be used with a finger or any pointing device (passive)
• Requires a certain amount of pressure (to get the resistance to register)
• Can be modified for multi-touch (only two touches)
• Has high tolerance for liquids and contaminants
  • They're still used in hospitals and industry where fluids have a high spill rate

Surface Capacitive Touch Screen

• Screen is covered in a capacitive material
  • This is 90% transparent
• Capacitance - ability to store electrical charge (this works through glass)
• We apply a small voltage to generate an electrostatic field
• Humans naturally act as small capacitors; touching the screen with a finger creates a dynamic capacitor.
• Measure the effective capacitance at each corner of the screen
  • The larger the change, the closer to the corner the touch was
  • Combine these measurements from all corners to find the exact location of the touch

Using the distance from all 4 corners to determine the location. This only actually needs 3 corners; the 4th is used for accuracy.

Projected (Mutual) Capacitance

For multi-touch, there is a grid of sensors.

• Conductive material is etched with rows & columns
• An electric field is projected through the top layer of the glass
• A human acts as a conductor; a decrease in capacitance between electrodes is detected as a touch

The 1st-generation iPhone had a 10 x 15 (150) grid of sensors.

To get a higher resolution:

Similarly, we can do this with multiple touches

3D touch

• Pressure-sensitive touch
• This involves flexible glass
  • Pressing forces the finger closer to the rear capacitor
    • We can use this to measure depth, not location

Gestures

Gesture - A series of touch events that occur over a period of time

TouchBegin(), TouchMoved(), TouchEnded()

Two ways to register these:

1. Implement onTouchEvent() in an activity
2. Register a new OnTouchListener with a view - setOnTouchListener(...)

One or more fingers on the screen will trigger the callbacks of onTouchEvent() on the view.

Either way, the detailed interactions are delivered using a MotionEvents object

MotionEvent

This object encapsulates information about:

• Touch events
  • A touch begins (ACTION_DOWN)
  • The finger moves (ACTION_MOVE)
  • The touch ends (ACTION_UP)
• (x,y) coordinates of the touch, information about pressure, size, orientation, etc.
• Additional information for multi-touch, e.g. pointer ID, action index, etc.

ACTION_DOWN

• A gesture starts when a finger is pressed
• A MotionEvent is generated for this
• Find the action by calling getAction()
• This can also have the identifier of the pointer, so use getActionMasked() for multi-touch

ACTION_MOVE

• As the finger moves, a series of ACTION_MOVE events will be sent
• Find the new position using getX() and getY() (returns floats)
• Note that Android bundles these into a series of touch events, so we can get historic touches

ACTION_UP

• A gesture ends in three ways
  1. ACTION_UP - last finger has been taken off the display
  2. ACTION_CANCEL event - another event happens, cancelling the gesture, e.g. the phone rings
  3. ACTION_OUTSIDE event - if the finger moves outside the relevant view

Single Touch Gesture

Formed of:

1. A single ACTION_DOWN
2. Zero or more ACTION_MOVE
3. An ACTION_UP to finish

Single-touch interactions

• Positions are tracked to move an object
• Use movement velocity for a swipe / fling

Dragging & Scrolling

• Store the original (x,y) touch location from ACTION_DOWN
• Calculate changes from stored value and value returned from ACTION_MOVE or ACTION_UP
• Adding the coordinate changes to the original object location

Swipe / Fling

• Rather than moving the object, we can calculate the velocity and direction
• On ACTION_UP, continue to move the object with that velocity
• Gives the user obvious visual feedback of flinging UI elements across the screen

Customised Gestures

Several different ways of tracking the movement of a gesture

• Use the starting and ending point of a pointer
• Use the direction the pointer is travelling
• Use velocity of the pointer
• Use getHistorical<Value> to get historical movements

Multi-touch Gestures

• Very similar to single touch
• Same sequence of events with a few more events
  • ACTION_POINTER_DOWN & ACTION_POINTER_UP
  • Support for 256 touches at the same time

Determining Finger

• Index: position within the array in a MotionEvent
• ID: Unique for each pointer (finger) to allow tracking an individual pointer across the entire gesture
• The number of pointers can change as fingers are lifted or placed; so do the indices of the pointers
• Each pointer is given an ID that won’t change
  • Need to track both the ID and past locations of pointers to move things about

onTouchEvent and motionEvent

• onTouchEvent is a callback method
• motionEvent is an object that contains gesture information
• When a user places fingers on the screen, it triggers the callback onTouchEvent() on the view
• After some touch actions, the motionEvent delivered to onTouchEvent() provides the details of every interaction

public boolean onTouchEvent(MotionEvent event) {
	...
    int maskedAction = event.getActionMasked();
    switch (maskedAction) {
        case MotionEvent.ACTION_DOWN:
        case MotionEvent.ACTION_POINTER_DOWN:
        case MotionEvent.ACTION_MOVE:
        case MotionEvent.ACTION_UP:
        case MotionEvent.ACTION_POINTER_UP:
        case MotionEvent.ACTION_CANCEL:
}

Pinch to zoom

• Obtain the IDs of the two pointers
  • Use this to get the index so we can get locations
• Calculate and store the distance between the two pointers
• The ratio of the new distance to the old one gives the zoom / scale ratio

Two-finger rotation

• Obtain the IDs of the two pointers
  • Use this to get the index so we can get locations
• Use the vertical and horizontal location difference to calculate the initial angle
• Obtain the new locations of the two pointers to derive the new angle
• Object can be rotated using the difference in angles