Return-oriented rootkit without returns (on the x86)

  • Authors:
  • Ping Chen;Xiao Xing;Bing Mao;Li Xie

  • Affiliations:
  • State Key Laboratory for Novel Software Technology, Nanjing University and Department of Computer Science and Technology, Nanjing University, Nanjing;State Key Laboratory for Novel Software Technology, Nanjing University and Department of Computer Science and Technology, Nanjing University, Nanjing;State Key Laboratory for Novel Software Technology, Nanjing University and Department of Computer Science and Technology, Nanjing University, Nanjing;State Key Laboratory for Novel Software Technology, Nanjing University and Department of Computer Science and Technology, Nanjing University, Nanjing

  • Venue:
  • ICICS'10 Proceedings of the 12th international conference on Information and communications security
  • Year:
  • 2010

Quantified Score

Hi-index 0.01

Visualization

Abstract

Return Oriented Programming(ROP) is a new technique which can be leveraged to construct a rootkit by reusing the existing code within the kernel. Such ROP rootkit can be designed to evade existing kernel integrity protection mechanism. In this paper, we show that, it is also possible to mount a new type of return-oriented programming rootkit without using any return instructions on x86 platform. Our new attack makes use of certain instruction sequences ending in jmp instead of ret; we show that these sequences occur with sufficient frequency in OS kernel, thereby enabling to construct arbitrary x86 behaviors. Since it does not make use of return instructions, our new attack has negative implications for existing defense methods against traditional ROP attack. Further, we present a design of memory layout arrangement technique for this type of ROP rootkit, whose size is not limited by the kernel stack. Finally, we propose the implementation of this practical attack to demonstrate the feasibility and effectiveness of our approach.