Mobile visual interaction: enhancing local communication and collaboration with visual interactions
MUM '06 Proceedings of the 5th international conference on Mobile and ubiquitous multimedia
PPR: partial packet recovery for wireless networks
Proceedings of the 2007 conference on Applications, technologies, architectures, and protocols for computer communications
Unsynchronized 4D barcodes: coding and decoding time-multiplexed 2D colorcodes
ISVC'07 Proceedings of the 3rd international conference on Advances in visual computing - Volume Part I
Challenge: mobile optical networks through visual MIMO
Proceedings of the sixteenth annual international conference on Mobile computing and networking
Real-world interaction with camera phones
UCS'04 Proceedings of the Second international conference on Ubiquitous Computing Systems
COBRA: color barcode streaming for smartphone systems
Proceedings of the 10th international conference on Mobile systems, applications, and services
VRCodes: Unobtrusive and active visual codes for interaction by exploiting rolling shutter
ISMAR '12 Proceedings of the 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)
| Hi-index | 0.00 |
A key challenge for smartphone based visual communication over screen-camera links is imperfect frame synchronization. The difficulty arises from frame rate diversity and variability due to camera capability, lighting conditions, and system factors. On the 4 smartphone cameras we tested, the frame rate varies between 8 and 30 frames per second. If the transmit frame rate is too high, the receiver might lose original frames or capture mixed frames, which are normally not decodable. Previous systems simply reduce the effective screen frame rate to be half the camera frame capture rate, to guarantee receiving a decodable frame every other frame. This under-utilizes the transmitter side capacity and is inefficient. We achieve frame synchronization with LightSync, which features in-frame color tracking to decode imperfect frames and a linear erasure code across frames to recover lost frames. LightSync allows smooth communication between the screen and the camera at any combination of the transmit and receive frame rates, as long as the receive rate is at least half the transmit rate. This means that each receiver can scale the decoding performance with its own camera capability. Across several phones, our system can more than double the average throughput compared to previous approaches.