Exploring the Linux Kernel's System Call Execution Model

Published On Tue Sep 10 2024
Exploring the Linux Kernel's System Call Execution Model

The Linux System Call Execution Model: An Insight - Open Source Perspective

This two-part series of articles delves into the intricate workings of the system call execution model in Linux-based operating systems. In this first part, we explore the fundamentals of system calls, their necessity, and the pivotal role played by the glibc wrapper in facilitating system call execution. Furthermore, we delve into the system call execution model from the vantage point of the Linux kernel.

A system call denotes a solicitation made by a user space program or application to communicate with an operating system functioning in the kernel space. Essentially, a user application triggers a system call when it necessitates access to services that are exclusively available through a higher privilege mode. Examples of such services include creating a new task, performing network I/O or file I/O operations, and accessing hardware resources. These operations cannot be directly carried out by user space applications. Hence, operating systems like Linux furnish a repertoire of routines known as system calls, which essentially are C functions executing in the kernel space.

When a user space program initiates a system call, a software interrupt occurs (modern x86-64 architecture provides the syscall instruction for expedited system call execution), causing a transition from user space to kernel space. Subsequently, the system call handler in the kernel space executes the requisite operation on behalf of the user space application and relays the response back to it.

Task Trace Internals — NuttX latest documentation

The Role of the Standard C Library in System Call Execution

The standard C library, commonly known as glibc or GNU C library, is extensively distributed within Linux-based operating systems. This C library aids in the implementation of standard C functions and APIs such as print(), scanf(), malloc(), fopen(), strcpy(), among others. These standard functions may internally invoke system calls. For instance, printf() internally triggers the write(2) system call. Nevertheless, these internal invocations of system calls remain concealed from the user space application.

When a user space application invokes a system call, such as the open(2) system call, the invocation occurs through the wrapper function implemented in the C library (glibc). This wrapper function internally triggers the actual system call implemented in the kernel. Almost all system calls in Linux have corresponding wrappers provided by the C library. Hence, from the perspective of an application program, invoking a system call mirrors calling a C function.

Execution of System Calls: The Role of the Wrapper Function

The wrapper function undertakes validation, initialization (in certain scenarios), and error checking of the arguments supplied by the application program. In case of an error, it promptly returns to the user program with the relevant error number. Subsequent to successful validation, the wrapper functions load the system call number and arguments into designated CPU registers as stipulated by the Linux kernel.

Direct Syscalls: A journey from high to low - RedOps - English

System calls or system call handlers are intrinsic components of kernel code, executed in the kernel mode (high privilege mode 0). As a result, user space applications or glibc wrappers cannot directly execute kernel code. To transition from user to kernel space, the glibc wrapper function must signal the execution of the system call handler to the kernel.

Evolution in System Call Execution: From Software Interrupts to Syscall Instruction

In earlier versions of x86 processors, Linux kernel, and glibc, system call wrappers triggered a software interrupt (exception) to switch to kernel mode and execute the system call handler. This software interrupt, denoted as interrupt number 128 in x86, was invoked through the int $0x80 instruction. However, the int assembly language instruction was inherently slow due to multiple checks. Contemporary x86 processors provide a faster alternative for system calls through the syscall instruction in the x86-64 architecture.

System Calls in Operating System Explained | phoenixNAP KB

The updated glibc system call wrapper employs the syscall instruction to transition from user space to kernel space and execute the system call handler. By concealing the architectural intricacies related to argument preparation and syscall instruction execution, the system call wrapper simplifies the process for user space application programs.

In scenarios where the glibc wrapper is absent for certain system calls, such as futex(2), glibc furnishes a generic library function termed syscall(2). This function, implemented in assembly within glibc, caters to invoking such system calls from user space applications, managing architecture-specific assembly instructions.

Sensors | Free Full-Text | DDoS Flood and Destination Service ...

As elucidated, the glibc system call wrapper streamlines system call invocation complexities, shielding user space applications from the low-level assembly intricacies. By invoking the wrapper function akin to a conventional C function or API, user space applications seamlessly interact with system calls, sans concerns regarding underlying architecture intricacies.