An Optimal Randomised Cell Probe Lower Bound for Approximate Nearest Neighbour Searching

• Authors: Amit Chakrabarti;Oded Regev
• Venue: FOCS '04 Proceedings of the 45th Annual IEEE Symposium on Foundations of Computer Science
• Year: 2004

We consider the approximate nearest neighbour search problem on the Hamming Cube {0, 1}^d. We show that a randomised cell probe algorithm that uses polynomial storage and word size d^{0(1)} requires a worst case query time \Omega ({{\log \log d} \mathord{\left/ {\vphantom {{\log \log d} {\log \log \log d}}} \right. \kern-\nulldelimiterspace} {\log \log \log d}}). The approximation factor may be as loose as 2^{\log ^{1 - n} d} for any fixed. n 0. This generalises an earlier result [5] on the deterministic complexity of the same problem and, more importantly, fills a major gap the study of this problem since all earlier lower bounds either did not allow randomisation [5, 18] or did not allow approximation [4, 2, 15]. We also give a cell probe algorithm which proves that our lower bound is optimal. Our proof uses a lower bound on the round complexity of the related communication problem. We show, additionally, that considerations of bit complexity alone cannot prove any nontrivial cell probe lower bound for the problem. This shows that the Richness Technique [20] used in lot of recent research around this problem would not have helped here. Our proof is based on information theoretic techniques for communication complexity, a theme that has been prominent in recent research [6, 1, 23, 14]. In particular, we make heavy use of the round elimination and message compression ideas in the recent work of Sen [23] and Jain, Radhakrishnan, and Sen [14], and also introduce a new technique which we call message switching.