rev. 0.1 1/15 copyright ? 2015 by silicon laboratories AN580 AN580 i nfrared g esture s ensing 1. introduction touchless user interfaces are an emerging trend in embedded electronics as product designers seek out innovative control methods and more intu itive ways for users to interact with electronics. active infrared proximity motion sensing can solve this challeng e. silicon labs si114x proximity and ambient light sensor products are ideally suited to touchless gesturing ap plications such as page turning on an e-reader, scrolling on a tablet pc, or gui navigation. the si114x features up to three led drivers and ha s the ability to sense gestures within a 7 to 15 cm product interaction region, assuming a hand as th e detectable object. this document will discuss in detail how silicon labs implements motion se nsing using infrared technology. ther e are two primary methods used for gesture sensing ? position-based and phase-based. position-based gesture sensing involves finding gestures based on the calculated location of an object while phas e-based gesture sensing is based on the timing of the changes in signal to determine th e direction of an object?s motion. 2. hardware considerations this application note focuses on detecting gestures made by a user?s hand. it is important to recognize that the concepts introduced in this application note can be applie d to targets other than the hand, as long as the hardware is designed appropriately. the end application and i ndividual system constraints will each dictate the range requirements for ir gesture sensing. since object refl ectance is the main measurable component for touchless gesturing, a hand is presumed to be the detectable object for the examples in this document. whereas a hand can achieve gesture sensing up to 15 cm away from the si114x sensor, a finger, with dramatically lower reflectance, can achieve gesture sensing at a range of < 1 cm for thumb-scroll type applications. the general guideline for designing a gesture sensing system with multiple leds is to make sure that there is no ?dead spot? in the middle of the detectable area. when a ta rget is placed above the system and is not detected, the target is in a reflectivity dead spot . to avoid dead spots in the system, the leds must be placed such that the emitted infrared light can reflect off the target and onto the sensor from the desired detection range. figure 1 shows systems designed to detect a hand or a finger. the most susceptible area for a dead spot is directly above the sensor in between the two leds. the two leds are plac ed as close to the edge of the target as possible to provide feedback in the middle while also keeping enough distance between the leds so that it can be detected when the finger or hand moves to the left or right.
AN580 2 rev. 0.1 figure 1. ideal hardware formations?hand versus finger detection other than the placement of the leds in relation to the size of the hand, the location and reflectance of the target in relation to the system is also very important. notice that the si11xx device is placed under the palm of the hand and middle of the finger. for the hand detecting system, the fingers are not a good focal point because light can slip between the fingers, and the shape/curve of the outline of the fingers also makes for unpredictable measurements. for the finger detecting system, the tip of the finger is cu rved and reflects less light than the middle of the finger. for more information about other hardware considerations for infrared proximity sensing in general, refer to ?an498: si114x designer?s guide?. for more information about leds, refer to ?an521: irled selection guide for si114x proximity applications?. 2.1. method 1?position- based gesture sensing the position-based motion sensing algorithm involves three primary steps. the first step is the conversion of the raw data inputs to usable distance data. the second step uses the distance data to estimate the position of the target object, and the third step checks the timing of t he movement of the position data to see if any gestures occurred. ? step 1 ? converting counts to distance : the infrared (ir) proximity sensor outputs a value for the amount of infrared light fed back to the device fr om the ir leds. these values increase as more light is reflected by an object or hand moving closer to th e system. this example presumes that a hand is the defined target for detection. the system can estimate how far one?s hand is based on characterization of the proximity sensing (ps) feedback for a hand at certain distances. for example, if a hand is placed in front of the system about 10 cm away and the ps measurement yielded 5000 counts, then subsequent ps measurements around 5000 counts also mean that there is a similarly reflective hand approximately 10 cm away from the system. taking multiple data points at vary ing distances from the system helps to interpolate between these points and create a piece-wise equation for the conversion from ra w ps counts to distance estimation. note that for systems with multiple leds, each led will have a different ps feedba ck for each hand distance so each led will need its own counts to distance equation. si11xx d2 d1 si11xx d2 d1
AN580 rev. 0.1 3 in figure 2, the characterization for a two-led system is displayed. each led must be characterized with a target suspended over the midpoint between the led and the sensing device. when target 1 is suspended over the sensing device and ir led 1, the measured feedback will correlate to a distance d1 above the system. the same is true for target 2, ir led 2, and d2. figure 2. distance approximation ir led 2 si11xx d 2 ir led 1 d 1 p 1 p 2 target 2 target 1
AN580 4 rev. 0.1 ? step 2 ? estimating position : the next step in the process of detecting gestures is to take the distance data from step 1 and estimate the target?s position. the positi on estimate comes from the formula for the intersection of two circles. an led?s light output is conical in nature, but for this approximation, the led?s output is considered to be spherical. with the led?s on the same axis, the intersection of these two spheres can be considered using equations for the intersection of two circles. consider the characterization performed in figure 2 in the previous section. figure 3 shows what the estimations d1 and d2 mean when a single target is su spended over the middle area of the system. d1 and d2 are estimations of the distance from the points p1 and p2 to the target above the system. if d1 and d2 are thought of as radii of two circles, the intersection of these two circles is the point where the target is located. figure 3. single target distance approximation figure 4 is an expanded version of figure 3. the measurements ?a? and ?b? have been added to label the location of the target along the axis between the two points p1 and p2. also note that the distance measurements d1 and d2 have been renamed to r1 and r2 to represent that they are being considered as radii now. ir led 2 si11xx d 2 ir led 1 d 1 p 1 p 2 target
AN580 rev. 0.1 5 figure 4. intersection of two circles implementation the value of ?a? will be the location of the object along the axis between p1 and p2. a negative value is possible and will mean that the ta rget is on the left side of p1. ? step 3 ? timing gestures : with a positioning algorithm in place, keepi ng track of timing a llows the system to search for and acknowledge gestures. for hand-swiping gestures, the entry and exit positions are the most important to consider. if the hand enters the right side wit h a very high ?a? value and exits the left side with a very low ?a? value, then a left swip e occurred, as long as the timing betwe en the entry and exit happened within a chosen time window. if the position stays steadily in the middle area for a certain period of time, then this gesture can be considered a pause gesture. the system must keep track of the timestamps for the en try, exit, and current positions of the target in the detectable area. with timing informati on and position informatio n, most gestures can be recognized. timing will need to be custom tuned to each application and each system designer?s preference. silicon labs reference designs use a quarter of a second for the gesture time requirement. silicon labs? irslider2ek is a great example and starting poin t for comprehending the fundamentals of position- based gesturing. r 2 r 1 p 1 p 2 target a b using d a b a r1 2 r2 2 d 2 +? ?? 2d ? ?? ? = ? +=
AN580 6 rev. 0.1 2.2. method 2?phase-b ased gesture sensing for the second method of gest ure sensing, the location of the target is never calculated. this means that steps 1 and 2 from method 1 above are not used in this process. this method involves looking solely at the raw data from the proximity measurements and looking for the timing of the changes in feedback for each led. the maximum feedback point for each led will occur wh en a hand is right above that led. if a hand is swiped across two leds, the direction of the swipe can be determined by lookin g at which led?s feedback raised first. figure 5 shows a three-led system with a hand about to swipe over the system. figure 5. swiping left over a 3-led system when the hand in figure 5 is swiped left across the th ree leds and proximity sensing device, the hand will first cross over d2, then d3, then d1. the sensing algorithm will recognize the rise in feedback fo r d2 and record the timestamp for this raise. then the sensing algorithm will detect the same raises for d3 and d1 with a higher timestamp than the one before it. the sensing algorithm ca n also detect the return back to no-detect state of each led?s measurement and record a timestam p for this as well. first d2 will retu rn back to normal, then d3, then d1. for up and down gestures in the system in figure 5, d1 and d2 will rise and fall simultan eously with d3 coming either before or after d1/d2 for the up or down gesture. if all three channels rise and fall at the same time, then a hand approached from directly above (z-axis) a nd was retracted to indicate a ?select? gesture. in figure 6, the four gestures right, left, down, and up are shown on graphs of adc counts versus time. the green line represents ps measurements using d1, the pink line represents ps data from d2, and the yellow line shows data from d3. observe for the right swip e that d1 spikes first, then d3, and fi nally d2. this is expected for a swipe from left to right. fo r the up and down swipes, d1 and d2 spike at the same time since yo ur hand will cross these leds at the same time when swiping up or down. si11xx d2 d3 d1
AN580 rev. 0.1 7 figure 6. gesture data right swipe left swipe down swipe up swipe si11xx d2 d3 d1 d2 d3 d1 d2 d3 d1 d2 d3 d1 d2 d3 d1
AN580 8 rev. 0.1 3. position-based meth od advantages and drawbacks the advantage of the position-based method over phas e-based is that the position-based method can offer information on the location of the target. this will allow for ratiometric co ntrol of systems. for example, to scroll through a book several pages you could suspend your hand over the right side of the detectable area instead of having to make several right swipe gestures. the main drawback of the position-based algorithm is the accuracy of the position calculations. the positioning algorithm assumes a spherical output from the leds, but in practice led output is more conical than spherical. the algorithm also assumes uniform light intensity across the entire output of the led, but the light intensity decays away from the normal. another issue is that this algorithm does not account for the shape of the target. a target that is uniquely shaped will cause inconsistencies with the po sitioning output. for example, the system cannot tell the difference between the hand and the wrist, so any gestures involving move ment that puts the wrist in the area of detection will be less accurately located. the result is that the positioning information provided in this algorithm is good enough for low resolution systems that only need a 3x3 grid of detection, but the curr ent positioning algorithm is not well-suited for a pointing application. this algorithm?s output is not an ideal touchscreen replacement. 4. phase-based method advantages and drawbacks for applications not requiring position information, th e phase-based method provides a very robust way of detecting gestures. each gesture can be detected on either the entry or exit from the detectable area, and the entry and exit can be double-checked with each other to prov ide much higher certainty for each gesture observed. the drawback of this method when compared with the posi tion-based method is that no positioning information is provided. this means that the number of gestures that can be implemented are more limited than the position- based method. the phase-based method can only tell the di rection of entry and exit from the detectable area so any movement in the middle of the detectable area is not detected. 5. improved gestures through combination of the two methods these two methods can be implemented alongside one an other to help mask the other?s deficiencies. the position-based algorithm can provide some positional information for ratiometric control, and the phase-based algorithm can be used for detection of most gestures. these two algorithms working together can provide a strong solution for gesture sensing, but the drawback here is the code space requirements of implementing two separate algorithms as well as the cpu cycles to process both algorithms at all times.
AN580 rev. 0.1 9 n otes :
disclaimer silicon laboratories intends to provide customers with the latest, accurate, and in-depth documentation of all peripherals and modules available for system and software implementers using or intending to use the silicon laboratories products. characterization data, available modules and peripherals, memory sizes and memory addresses refer to each specific device, and "typical" parameters provided can and do vary in different applications. application examples described herein are for illustrative purposes only. silicon laboratories reserves the right to make changes without further notice and limitation to product information, specifications, and descriptions herein, and does not give warranties as to the accuracy or completeness of the included information. silicon laboratories shall have no liability for the consequences of use of the information supplied herein. this document does not imply or express copyright licenses granted hereunder to design or fabricate any integrated circuits. the products must not be used within any life support system without the specific written consent of silicon laboratories. a "life support system" is any product or system intended to support or sustain life and/or health, which, if it fails, can be reasonably expected to result in significant personal injury or death. silicon laboratories products are generally not intended for military applications. silicon laboratories products shall under no circumstances be used in weapons of mass destruction including (but not limited to) nuclear, biological or chemical weapons, or missiles capable of delivering such weapons. trademark information silicon laboratories inc., silicon laboratories, silicon labs, silabs and the silicon labs logo, cmems?, efm, efm32, efr, energy micro, energy micro logo and combinations thereof, "the world?s most energy friendly microcontrollers", ember?, ezlink?, ezmac?, ezradio?, ezradiopro?, dspll?, isomodem ?, precision32?, proslic?, siphy?, usbxpress? and others are trademarks or registered trademarks of silicon laboratories inc. arm, cortex, cortex-m3 and thumb are trademarks or registered trademarks of arm holdings. keil is a registered trademark of arm limited. all other products or brand names mentioned herein are trademarks of their respective holders. http://www.silabs.com silicon laboratories inc. 400 west cesar chavez austin, tx 78701 usa smart. connected. energy-friendly products www.silabs.com/products quality www.silabs.com/quality support and community community.silabs.com
|
