At the kernel boundary

System calls, scheduling, memory, and privilege.

9 min readChapter 30 of 50Intermediate → advanced
In this chapter

Before you start#

Required earlier lessons: Memory and system calls, C/C++ for source readers, and Concurrency and object lifetime. You need the ideas of process, thread, virtual address, file descriptor, C pointer, and callback; the source reading below explains the kernel-specific terms it adds. No kernel checkout, privileged command, or device modification is required.

Learning goals#

You will distinguish Android user space from the Linux kernel; follow a file-read request across its major layers; explain why a syscall is not the same as a context switch, interrupt, or ordinary function call; and read a small pinned kernel excerpt without mistaking it for a complete execution trace.

Practice#

A boundary, not a wall with one door

The kernel is privileged code that manages CPU time, memory, devices, and isolation for the whole machine. Apps normally execute in user mode, where processor rules prevent direct access to kernel memory or arbitrary hardware. Kernel code executes with greater privilege. Android's framework, native libraries, runtime, and apps are above this boundary. A Java call, a Binder transaction, and a Linux filesystem syscall are different interfaces; Binder uses kernel support but is not an ordinary filesystem syscall. Changing the kernel does not directly add a Java API, and a framework change does not automatically require a kernel change. AOSP kernel overview

A process is a running program plus resources such as an address space and open file descriptors. A thread is a schedulable execution stream. Linux represents schedulable entities with task_struct; in Linux usage a thread is often called a task. Threads in one process commonly share an address space (mm_struct) and file descriptors, while separate processes usually have different address spaces. clone can choose which resources to share, so these are useful models rather than absolute rules. Each thread has its own register state and kernel stack. The scheduler chooses runnable threads/tasks, not one indivisible “app slot.”

Five-box read path (and what is omitted)

A read across the kernel boundary
  1. User space: library read wrapper

    Prepare descriptor, buffer, and requested count.

  2. CPU entry and syscall dispatch

    Architecture-specific mechanism selects the kernel implementation.

  3. ksys_read → vfs_read

    Resolve the descriptor, validate access/range, and choose the file operation.

  4. file_operations callback

    Perform object-specific work with fault-aware user-buffer handling.

  5. Return through wrapper

    Short byte count or error; a syscall need not switch to a different task.

Source map. Conceptual entry plus the pinned fs/read_write.c path. Architecture entry/dispatch are not shown by the source excerpt. A callback can serve a filesystem or device, not necessarily storage hardware.
text
1. App/library calls read(fd, buffer, count)       user space
                 ↓ wrapper prepares request
2. CPU executes architecture-specific entry       boundary
   (for example AArch64 svc; x86-64 syscall)
                 ↓ kernel entry/dispatch            NOT shown in source excerpt
3. syscall dispatch selects read implementation    kernel
                 ↓ fd lookup and generic VFS checks
4. VFS chooses file/device operation              kernel
                 ↓ callback copies/produces bytes
5. result returns: byte count or error             user space

This diagram is conceptual: Android libraries, framework code, Binder, buffering, caching, and asynchronous APIs may add work. The architecture-specific entry and syscall-table dispatch are deliberately not shown in the implementation excerpt below. VFS means Virtual Filesystem: common kernel file operations that dispatch to the particular opened object. A callback is a function supplied to another component to call later; a file's operation table contains callbacks such as .read.

On AArch64 the entry instruction is commonly svc; on x86-64 it is syscall (syscall(2) architecture table). A syscall is a controlled request from user space. A hardware interrupt arrives asynchronously (for example, a device completion); an exception is caused synchronously by the current instruction (for example, a page fault). Neither is a userspace syscall. A syscall can enter and return to the same task. A context switch saves enough state to pause one task and resume another; a timer interrupt can lead to a context switch, but entering kernel mode does not itself imply one.

Virtual addresses and scheduling

Each process sees a virtual address space translated through page tables by the processor's MMU. Two processes can use numeric address 0x1000 and still refer to different memory. A page fault means a requested mapping/permission was not immediately available; the kernel may satisfy it (normal demand paging) or signal the process (often SIGSEGV). It is not automatically a crash. Page size depends on architecture/build; Android devices can use 16 KiB pages as well as other sizes (AOSP 16 KB page size).

The scheduler chooses a runnable task to run. A task waiting for I/O or a lock may sleep without consuming a CPU, then become runnable when its condition changes. Android uses Linux scheduling mechanisms and device-specific tuning; do not infer the scheduling policy from the Android release alone.

A harmless host observation

In a Linux host terminal (not adb shell), run:

sh
strace -e trace=read,write,openat -o /tmp/syscalls.log cat /etc/hostname
cat /tmp/syscalls.log

This reads the host's hostname file and creates/replaces only /tmp/syscalls.log. Expected: cat prints a hostname and the log shows file-open/read/write activity; exact calls vary. If strace or /etc/hostname is absent, record “not available” rather than installing tools or changing permissions. strace shows observed system calls, not every internal kernel function. Android production images commonly restrict tracing; do not try to bypass policy. The visible hostname does not prove which physical storage driver was used.

Read one pinned source path

The examples below are from Android Common Kernel (ACK) android14-6.1, commit 2f0d864181b9acf801e9b51c345e11b6156b767a. The commit—not the moving branch name—is the reproducible snapshot. Open the pinned fs/read_write.c, use the browser's find command for vfs_read, and compare the shown lines. In C, struct file *file is a pointer to a structure; file->f_mode accesses one of its fields; ssize_t is a signed byte-count/error return type; __user marks a pointer that refers to userspace memory. FMODE_READ, -EBADF, and -EFAULT are kernel constants for access modes/errors.

The following is an abbreviated excerpt from ACK fs/read_write.c; comments and trailing error-accounting are omitted, indentation normalized. It shows generic VFS handling, not CPU entry assembly, and is a reading excerpt—not a complete replacement function:

c
ssize_t vfs_read(struct file *file, char __user *buf, size_t count, loff_t *pos)
{
        ssize_t ret;

        if (!(file->f_mode & FMODE_READ))
                return -EBADF;
        if (!(file->f_mode & FMODE_CAN_READ))
                return -EINVAL;
        if (unlikely(!access_ok(buf, count)))
                return -EFAULT;
        ret = rw_verify_area(READ, file, pos, count);
        if (ret)
                return ret;
        if (count > MAX_RW_COUNT)
                count = MAX_RW_COUNT;
        if (file->f_op->read)
                ret = file->f_op->read(file, buf, count, pos);
        else if (file->f_op->read_iter)
                ret = new_sync_read(file, buf, count, pos);
        /* Remaining branches, accounting, and return omitted. */
}

access_ok checks an address range but does not pin every page or promise that later access cannot fault. The selected operation still has to use fault-aware user-copy helpers. f_op is a pointer to the opened file's operation table; the if chooses a supplied callback. A callback can be supplied by a filesystem or character device, not necessarily a physical storage driver. A successful read can return fewer bytes than requested; negative kernel error values are translated by user-space wrappers into their documented convention (often -1 plus errno).

The syscall wrapper is short and makes the handoff explicit:

c
SYSCALL_DEFINE3(read, unsigned int, fd, char __user *, buf, size_t, count)
{
        return ksys_read(fd, buf, count);
}

SYSCALL_DEFINE3 is a kernel macro that declares a syscall with three arguments; it is not the architecture's svc instruction. ksys_read obtains a referenced struct file from the descriptor, calls vfs_read, updates the shared file position after a nonnegative result, then drops the descriptor reference. The path separates architecture entry (not shown), syscall wrapper, descriptor lookup, generic VFS validation, and file-specific operation. An invalid descriptor can fail before VFS; a valid descriptor can refer to an unreadable object; user-memory access can fail; a callback can return an error or short count. Those symptoms do not identify the same cause. Shared open-file descriptions can also make file offsets observable across threads.

Source-browser worksheet

Use the pinned URLs, not a current branch search. If the source browser is unavailable, save this worksheet and mark the source step “blocked”; do not substitute an unpinned version.

StepFixed path/symbolRecord as evidence
1fs/read_write.c, vfs_readWhich checks happen before the file operation callback? Cite line or code anchor.
2Same file, SYSCALL_DEFINE3(readWhat function does the wrapper call? What layer is still omitted?
3Same file, ksys_readWhere does the descriptor become a referenced struct file? What happens to position on success?
4include/linux/fs.h, struct file_operationsFind the declaration of .read and explain why the VFS can call an implementation without knowing its device-specific details.

Artifact: a four-row table containing source path + symbol, one quoted identifier/line per row, “this establishes…”, and “this does not establish…”. Expected: the path distinguishes wrapper, fd lookup, generic VFS, and callback. A screenshot of output or an unpinned search result is not equivalent evidence. Do not claim to have traced architecture entry assembly; it is intentionally outside this exercise.

Knowledge check#

Answer before opening the key.

  1. A read syscall returns to the same thread. Did a context switch necessarily occur? Explain.
  2. A team sees a short read and blames the storage driver. Give two other places in the five-box path that could explain it.
  3. An app and another app both use virtual address 0x1000. Why is that not direct shared access?
  4. What exactly does the vfs_read excerpt prove, and which earlier layer does it omit?

Answers#

  1. No. A user/kernel privilege transition can return to the same task; a context switch is a scheduler change to another task.
  2. The wrapper/application may request a limited count or handle partial reads; VFS/file operations may return a short count. The observation alone does not identify physical storage.
  3. Each process has its own address-space mappings/page tables; the same number can map to different memory and permissions.
  4. It shows generic VFS validation and dispatch to a file operation in this pinned source. It omits architecture-specific CPU entry/syscall dispatch and is not a full call trace.

Checkpoint#

  • Ready: you can place syscall wrapper, VFS, and callback in order, and distinguish context switch from privilege transition.
  • Revise: you treat vfs_read as the CPU entry or a syscall as a guaranteed disk operation; redraw the five boxes and label the omitted entry.
  • Not yet evidenced: your worksheet lacks pinned path + specific source evidence + limitation. Repeat the four-row artifact before continuing.

Sources#

A little further beneath the surface.

Check your understanding with the checkpoint. Marking a chapter as read saves a bookmark, not a proficiency assessment.