The Farsite project: a retrospective
ACM SIGOPS Operating Systems Review - Systems work at Microsoft Research
Distributed directory service in the Farsite file system
OSDI '06 Proceedings of the 7th symposium on Operating systems design and implementation
| Hi-index | 0.00 |
From the Book:This book shows how to use the Win32 Application Programming Interface (API) and the emerging Win64 API, concentrating on the core system services, including the file system, process and thread management, interprocess communication, network programming, and synchronization. User interfaces, internals, and I/O drivers, although important and interesting topics, are beyond the scope of this book. The examples concentrate on scenarios that are likely to arise in practice, and in many cases the examples can be used as bases for real applications. The Win32 API is supported by Microsoft's family of 32-bit operating systems: Windows 2000, Windows NT, Windows 98, Windows 95, and Windows CE. Win64, to be supported as a 64-bit interface on Windows 2000, is very similar, and migration issues are discussed as required. There is no doubt that Win32, along with Win64, is an important factor for application developers, in many cases replacing UNIX as the preferred API for application programs targeted at desktop and server systems. Many observers predict that Win32 will become the dominant programming interface, although it appears that UNIX and Win32 will continue to coexist and that each will find its own niche. Regardless of the outcome of the operating system wars, many experienced programmers will want to learn the Win32 API quickly, and this book is designed to help them do so. The first objectives are to explain what Win32 is, show how to use it in realistic situations, and do so as quickly as possible without burdening the reader with unnecessary detail. This book is, therefore, not comprehensive, but it explains the central features of the most important functions and shows how to use them in realistic programming situations. Equipped with this knowledge, the reader will be able to use the comprehensive Microsoft reference documentation to explore details, advanced options, and the more obscure functions as requirements or interests dictate. I have found the Win32 API easy to learn using this approach, and I have greatly enjoyed developing Win32 programs, despite occasional frustration. This enthusiasm will show through at times, as it should. This does not mean that I feel that Win32 is necessarily better than other operating system APIs, but it certainly has many attractive features. Many Win32 books assume that the user is familiar only with 16-bit Windows 3.1 programming. These books spend a great deal of time explaining how processes, virtual memory, interprocess communication, and preemptive scheduling work without showing how to use them in realistic situations. A programmer experienced in UNIX, VAX VMS, IBM MVS, or another high-end operating system--that is, nearly anything other than the long-obsolete Windows 3.1 Win16 API--will be familiar with these concepts and will be impatient to find out how they are implemented in Win32. Most Win32 books also spend a great deal of space on user interface programming. This book avoids the user interface, beyond discussing simple character-based console I/O, in the interest of concentrating on the important core features. This book takes the point of view that Win32 is just an operating system (OS) API, providing a well understood set of features. Many programmers, regardless of experience level, need to learn Win32 and the Windows operating systems quickly, and an understanding of Win32 is invaluable in discussing subjects such as Microsoft's Component Object Model (COM). The Windows systems, when compared with other systems, have good, bad, and average features and quality. The purpose of this book is to show how to use those features efficiently and in realistic situations to develop useful, high-quality, and high-performance applications. Audience Anyone who has experience programming in UNIX or another operating system, even Windows 3.1, and who wants to learn about Win32 quickly. Programmers and software engineers who must port existing applications, often in UNIX, to Win32 for operation under Windows 2000 or any of the other Windows platforms. This book contains many comparisons among Win32, UNIX, and standard C library functions and programming models. All common UNIX functionality, including process management, synchronization, file systems, and interprocess communication, is covered in Win32 terms. Programmers who are developing servers or other systems or components in which management of resources such as processes and threads is of primary importance and in which the user interface is of secondary importance. Programmers using COM or DCOM will find much of the information here helpful in understanding COM's DLL usage, thread models, interfaces, and synchronization. Readers starting new projects who are not constrained by the need to port existing code. Many aspects of program design and implementation are covered, and Win32 functions are used to create useful applications and to solve common programming problems. Computer science students at the upperclass undergraduate or beginning graduate level in courses covering systems programming or application development. This book will also be useful to those who are learning multithreaded programming or need to build networked applications. This book would be a useful complementary text to a book such as W. Richard Stevens' Advanced Programming in the UNIX Environment (see the Bibliography) so that students could compare Win32 and UNIX. Students in operating systems courses will find this book to be a useful supplement as it illustrates how a commercially important operating system provides essential OS functionality. The only other assumption, implicit in all the others, is a knowledge of C programming. Organization Chapters are organized topically so that the features required in even a single-threaded application are covered first, followed by process and thread management features, and finally network programming in a multithreaded environment. This organization allows the reader to advance logically from file systems to security, memory management, and file mapping, and then to processes, threads, and synchronization, followed by interprocess and network communication. This organization also allows the examples to evolve in a natural way, much as a developer might create a simple prototype and then add additional capability. The advanced features, such as asynchronous I/O, appear last. Within each chapter, after introducing the functionality area, such as process management or memory-mapped files, we discuss important Win32 functions and their relationships in detail. Illustrative examples follow. Within the text, only essential parts of programs are listed; complete programs and the required include files, utility functions, and the like are in an appendix or on the disc provided with the book. Throughout, we identify those features supported only by Windows 2000 and NT, because Windows 98, 95, and CE do not implement many advanced features. Each chapter suggests related additional reading and gives some exercises. Many exercises address interesting and important issues that did not fit within the normal text, and others allow the reader to explore advanced or specialized topics. Chapter 1 is a high-level introduction to the Windows OS family and Win32. A simple example program shows the basic elements of Win32 programming style and lays the foundation for more advanced Win32 features. Win64 and migration issues are introduced in Chapter 1, described extensively in Chapter 16, and included throughout the book as required. Chapters 2 and 3 deal with file systems, console I/O, file locking, and directory management. Unicode, the extended character set used by Windows 2000/NT, is also introduced in Chapter 2. Examples include sequential and direct file processing and a directory traversal program. Chapter 3 ends with a discussion of registry management programming, which is similar in many ways to file and directory management. Chapter 4 introduces Win32's Structured Exception Handling (SEH) capability, which will be used extensively throughout the book. Many books defer SEH to later chapters, but by introducing it early we will be able to use SEH throughout and thus simplify some programming tasks and improve quality. Chapter 5 explains Windows NT object security, showing, in an example, how to emulate UNIX-style file permissions. Security upgrades can then be applied to the examples as appropriate. Although security is used in the later chapters, feel free to skip this chapter if you are not interested in this topic. Chapter 6 treats Win32 memory management and shows how to use memory-mapped files both to simplify programming and for performance. This chapter also covers dynamic link libraries (DLLs). Chapter 7 introduces Win32 processes, process management, and simple process synchronization. Chapter 8 then describes thread management in similar terms. Examples in each chapter show the many benefits, including program simplicity and performance, of threads and processes. Chapters 9 and 10 provide an extended, in-depth treatment of Win32 thread synchronization, one of Win32's strong features. Synchronization is a complex topic, and these two chapters use extended examples and well understood models to help you obtain the programming and performance benefits of threads while avoiding many of the pitfalls. Chapters 11 and 12 are concerned with interprocess and interthread communication and networking. Chapter 11 concentrates on the features that are properly part of Win32--namely, pipes, named pipes, and mailslots. Chapter 12 treats Windows Sockets, which allow interoperability with non-Windows systems using industry-standard protocols, primarily TCP/IP. Windows Sockets, while not strictly part of Win32, provide for network and Internet communication and interoperability, and the subject matter is consistent with the rest of the book. A multithreaded client/server system illustrates how to use interprocess communication along with threads. Chapter 13 describes how Windows 2000 and Windows NT provide "NT services" that allow you to manage background servers, such as the ones created in Chapters 11 and 12. Some small programming changes will turn the servers into NT services. Chapter 14 shows how to use overlapped I/O with events and completion routines. For file systems, this feature applies only to Windows 2000 and NT, and you can achieve much the same thing with threads. The closely related I/O completion ports are, however, necessary for scalable multithreaded servers, so this feature is illustrated with the servers created in Chapters 11, 12, and 13. Waitable timers are described, because they require concepts first introduced in Chapter 14. Chapter 15 concludes with a survey of two specialized topics: Remote Procedure Calls (RPCs) and Microsoft's COM object model, which integrates many of the concepts in the book. Remote procedures and COM objects frequently use the features described throughout this book. Simple examples are included, and this chapter illustrates that readers now have the necessary information to learn other Win32 topics beyond the core system services. Chapter 16 describes the Win64 programming issues and how to plan for application migration and portability. There are three appendices. Appendix A describes the programs on the disc and how to use them. Appendix B contains several tables that compare Win32 functions with their counterparts in UNIX and the Standard C library. Appendix C compares the performance of alternative implementations of some of the examples in the text so that you can gauge the trade-offs between Win32 features, both basic and advanced, and the C library. Notes on the Second Edition The Second Edition includes extensive new material along with significant updating and reorganization. Objectives of the Second Edition include: Cover Windows 2000 and Win64 migration. Provide increased coverage of threads and synchronization. Chapter 10 is new, as are the examples in Chapter 9. Add coverage of sockets and remote procedure calls (Chapters 12 and 15) and NT services (Chapter 13), thereby dealing with issues that are important when developing services and servers, particularly for Windows 2000. Describe the performance implications of threads and synchronization in more depth (Chapters 9 and 10). Incorporate reader and student feedback to fix defects, improve explanations, improve the organization, and address numerous small details. UNIX and C Library Notes and Tables Within the text at appropriate points, we contrast Win32 style and functionality with the comparable UNIX (and LINUX) and ANSI Standard C library features. Tables listing the comparable functions are presented in Appendix B. This information is included because many readers are familiar with UNIX and are interested in the comparisons between the two systems. Readers without a UNIX background should feel free to skip these paragraphs. Such discussions are indented, in a smaller font. Examples The examples are designed to do the following: Illustrate common, representative, and useful applications of the Win32 functions. Correspond to real programming situations encountered in program development, consulting, and teaching. Some of my clients and course participants have used the code examples as the bases for their own systems. During my consulting activities, I frequently encounter code that is similar to that used in the examples, and on several occasions I have seen code taken directly from the First Edition. (Feel free to do so yourself, and an acknowledgment in your documentation would be greatly appreciated.) Frequently, this code occurs as part of COM or C++ objects. The examples are "real-world" examples and solve "real-world" problems. Emphasize how the functions actually behave and interact, which is not always as you might first expect after reading the documentation. Throughout this book, the text and the examples concentrate on interactions between functions rather than the functions themselves. Grow and expand, adding new capability to a previous solution in an easy and natural manner and exploring alternative implementation techniques. In the earlier chapters, many examples implement UNIX commands, such as ls, touch, chmod, and sort, showing the Win32 functions in a familiar context while creating a useful set of utilities. Different implementations of the same command will also give us an easy way to compare performance benefits available with advanced Win32 features. Appendix C contains the results of these performance tests. Examples in the early chapters are usually short, but the later chapters present longer examples when appropriate. Exercises at the end of each chapter suggest alternative designs, subjects for investigation, and additional functionality that is important but beyond the scope of this book. Some exercises are easy, and a few are very challenging. Frequently, clearly labeled defective solutions are provided, because fixing the bugs is an excellent way to sharpen skills. All examples have been debugged and tested under Windows 2000, Windows NT, and, where appropriate, Windows 98 and 95. For Windows NT testing we used Version 4.0, and although the bulk of the development was performed on single-processor, Intel-based systems, many programs were also tested on multiprocessor systems. The client/server applications have been tested using multiple clients simultaneously interacting with a server. Nonetheless, there is no guarantee or assurance of program correctness, completeness, or fitness for any purpose. Undoubtedly, even the simplest examples contain defects or will fail under some conditions; such is the fate of nearly all software. I will, however, gratefully appreciate any messages regarding program defects--and, better still, fixes. Book errata, along with additional examples, reader contributions, additional explanations, and much more, will be maintained at my home page: http://world.std.com/~jmhart. The code will be updated as required when defects are found and fixed and as reader input is received. If you encounter any difficulties with the programs or any material in the book, check this location first, because there may already be a fix or explanation. If that does not answer your question, feel free to send e-mail to jmhart@world.std.com.