try/catch in Linux Kernel

In higher level languages like Java and C#, one can recover from unexpected bahaviour using try/catch like mechanism. Things are different inside Linux kernel. The code is considered trusted and reliable. Faults and exceptions have severe penalities. For example, division by zero will likely cause a kernel oops and the whole system to hang. So how would you run some code which you know can fault? Enter extable, a kernel mechanism by which one can have try/catch like facility. It is not a high-level extension to C programming language. Instead it works at assembly level.

We will take following piece of blatant division by zero and make it safe to run inside kernel using this exception handling mechanism. We will be assuming x86_64 platform in this article.

void blatant_div_by_zero(void)
{
        /* quotient and divisor */
        int q, d;

        d = 0;
        asm ("movl $20, %%eax;"
        "movl $0, %%edx;"
        "div %1;"
        "movl %%eax, %0;"
        : "=r"(q)
        : "b"(d)
        :"%eax"
        );

        pr_debug("quotient is %d\n", q);
}

extable

extable is basically an ELF section inside Linux kernel binary image which contains mappings between potentially faulting instructions and their respective handlers. Users interface with extable through _ASM_EXTABLE* family of macros. All macros delegate to one macro:

_ASM_EXTABLE_HANDLE(from, to, handler)

`from`: address of faulting instruction, i.e. the ‘try’ part
`to`: address to which control will be transferred when fault occurs, i.e. the ‘catch’ part
`handler`: is the function which transfers execution to catch part

`handler` function is important here. It has following signature:

bool handler(const struct exception_table_entry *fixup,
        struct pt_regs *regs, int trapnr)

`fixup` contains address of catch part, i.e. ‘to’ argument of _ASM_EXTABLE_HANDLE. `regs` contains registers, as they were when the fault happened and `trapnr` is the fault number. handler transfers control to catch part by simply setting instruction pointer register in regs argument to fixup address. However, before doing that, it has an opportunity to set up environment for catch part. That’s where the different wrappers around _ASM_EXTABLE_HANDLE come in. Each wrapper uses a differnt handler function. Let’s take a couple of examples from arch/x86/include/asm/asm.h.

# define _ASM_EXTABLE(from, to) \
_ASM_EXTABLE_HANDLE(from, to, ex_handler_default)

# define _ASM_EXTABLE_FAULT(from, to) \
_ASM_EXTABLE_HANDLE(from, to, ex_handler_fault)

ex_handler_default() is a handler function which simply sets instruction pointer to catch part. ex_handler_fault() moves fault number to rax register in addition to setting instruction pointer to catch part, so catch part can access fault number.

Context of handler()

At this point, some might be wondering which context does handler() function execute in. Short answer is interrupt handler context. If that works for you, you may jump to the next section. Here we will do a very quick detour of how this interrupt context comes about.

After executing an instruction and before going into next instruction, the control unit checks if the just-executed instruction resulted in an interrupt or exception. If an exception did occur, then it determines vector associated with it. In case of our divide-by-zero fault, it will be vector 0. The control unit will then read the corresponding entry – zeroth entry for divide-by-zero – of Interrupt Descriptor Table (IDT). It is an array of entries inside processor’s RAM. Address of that table is stored inside the ‘idtr’ register. That entry in IDT in turn points to an entry in another table of entries called Global Descriptor Table (GDT). The entry inside GDT then info – such as base address of segment – related to handler of the exception whose vector we started out with. Processor then performs some checks and stores context when the exception occurred (e.g. register contents) on stack. When the handler() function discussed above sets instruction pointer, it actually sets instruction pointer in this saved context. After that process jumps to interrupt handler. It is this interrupt handler which then looks up extable section to find the handler() function for the faulting instruction. If such a handler function is found, the processor jumps to it. Therefore our handler function is executed inside interrupt context. Ultimately, when execution returns from interrupt context, the context that was saved on the stack before interrupt handler was executed, is restored. Since our handler function above sets instruction pointer and potentially other registers which are part of that pre-interrupt context which will be restored, execution will jump to ‘catch’ part after interrupt handler returns.

Solution and How it Works

Using extable macro’s we can re-write our blatant_div_by_zero() function in a safe way. Let’s first see what the final solution looks like. Then we will break it down into easily understandable parts.

void blatant_div_by_zero(void)
{
        /* quotient and divisor */
        int q, d;

        d = 0;
        asm volatile ("movl $20, %%eax;"
        "movl $0, %%edx;"
        "1: div %1;"
        "movl %%eax, %0;"
        "2:\t\n"
        "\t.section .fixup,\"ax\"\n"
        "3:\tmov\t$-1, %0\n"
        "\tjmp\t2b\n"
        "\t.previous\n"
        _ASM_EXTABLE(1b, 3b)
        : "=r"(q)
        : "b"(d)
        :"%eax"
        );

        pr_debug("quotient is %d\n", q);
}

There are two key differences here. First, we are adding some code to a section called .fixup. Second, _ASM_EXTABLE() macro. Code in .fixup section is the ‘catch’ part. _ASM_EXTABLE(), as we saw above, uses a default handler which merely sets instruction pointer to ‘catch’ part:

__visible bool ex_handler_default(const struct exception_table_entry *fixup,
        struct pt_regs *regs, int trapnr)
{
        regs->ip = ex_fixup_addr(fixup);
        return true;
}

First argument of _ASM_EXTABLE is label 1b for faulting instruction “1: div %1;”. Letter ‘b’ in 1b means backwards and doesn’t have functional significance in our context. Second argument, 3b, is the catch part which is inside .fixup section. Now let’s look at the execution flow with this set up. Division by zero instruction is executed, it faults, goes into exception handler at vector 0, which finds the correct handler – in our case ex_handler_default() – which in turn, sets instruction pointer to address at label 3, our catch part. As noted, label 3 is inside .fixup section. Inside label 3, we set quotient q to -1 and jump to label 2 which is known safe point after faulting instruction. From there, execution continues to next instruction like normal. Note that the next instruction is neither “\t.section .fixup,\”ax\”\n” (which is a directive to linker) nor “3:\tmov\t$-1, %0\n” (which lives in a different section from where instruction at label 2 is).

Now let’s see what a real object file looks like when compiled with above extable code. Here is a hello-world kernel module, modified to contain blatant_div_by_zero() function. After compiling it, we can inspect its sections using readelf*:

$ readelf -S hello.ko
There are 34 section headers, starting at offset 0x10b8:

Section Headers:
[Nr] Name   Type Address Offset
        Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
       0000000000000000 0000000000000000 0 0 0
[ 1] .note.gnu.build-i NOTE 0000000000000000 00000040
       0000000000000024 0000000000000000 A 0 0 4
[ 2] .text PROGBITS 0000000000000000 00000064
       0000000000000000 0000000000000000 AX 0 0 1
[ 3] .init.text PROGBITS 0000000000000000 00000064
       0000000000000034 0000000000000000 AX 0 0 1
[ 4] .rela.init.text RELA 0000000000000000 00000cd8
       0000000000000060 0000000000000018 I 31 3 8
[ 5] .fixup PROGBITS 0000000000000000 00000098
       000000000000000a 0000000000000000 AX 0 0 1
[ 6] .rela.fixup RELA 0000000000000000 00000d38
       0000000000000018 0000000000000018 I 31 5 8
[ 7] .exit.text PROGBITS 0000000000000000 000000a2
       000000000000000c 0000000000000000 AX 0 0 1
[ 8] .rela.exit.text RELA 0000000000000000 00000d50
       0000000000000030 0000000000000018 I 31 7 8
[ 9] .rodata.str1.1 PROGBITS 0000000000000000 000000ae
       000000000000001c 0000000000000001 AMS 0 0 1
[10] __ex_table PROGBITS 0000000000000000 000000cc
       000000000000000c 0000000000000000 A 0 0 4
[11] .rela__ex_table RELA 0000000000000000 00000d80
       0000000000000048 0000000000000018 I 31 10 8
[...]

Above is partial output, containing the sections which are relevant to our discussion. First, there is .fixup section at number 5. Its type of PROGBITS means it will be loaded in memory and flag X means it will be executable, which is what we want since this will execute the catch part. Further down at number 10 is __ex_table section. This is loaded into memory (type PROGBITS) but not marked as executable. We will only use it to read address of handler function but won’t need to execute the section itself. There is a function in kernel/extable.c which searches extables for correct handler for a given faulting address. It is reproduced below:

/* Given an address, look for it in the exception tables. */
const struct exception_table_entry *search_exception_tables(unsigned long addr)
{
        const struct exception_table_entry *e;

        e = search_extable(__start___ex_table,
        __stop___ex_table - __start___ex_table, addr);
        if (!e)
                e = search_module_extables(addr);
        return e;
}

As you can see, if the search in kernel image fails, the function looks in kernel modules’ extables for a corresponding entry through the call to search_module_extables(addr). This is the function which will search our hello.ko module for entry corresponding to division-by-zero instruction.


* Apologies for the formatting. Please bear until I find a way to fix it in the wordpress editor.

Title Image: Catch taken when playing cricket in street – a popular pastime in South Asia. Taken from https://www.flickr.com/photos/flickcoolpix/

How Does an Intel Processor Boot?

When we switch on a computer, it goes through a series of steps before it is able to load the operating system. In this post we will see how a typical x86 processor boots. This is a very complex and involved process. We will only present a basic overall structure. Also what path is actually taken by the processor to reach a state where it can load an OS, is dependent on boot firmware. We will follow example of coreboot, an open source boot firmware.

Before Power is Applied

Let us start with BIOS chip, also known as boot ROM. BIOS chip is a piece of silicon on the motherboard of a computer and it can store bytes. It has two characteristics which are of interest to us. First, it (or a part of it) is memory mapped into the CPU’s address space, which means that the CPU can access it in the same way it would access RAM. In particular, the CPU can point its instruction pointer to executed code inside BIOS chip. Second, the bytes that BIOS chip stores, represent the very first instructions that are executed by the CPU. BIOS chop also contains other pieces of code and data. A typical BIOS contains flash descriptor (a contents table for BIOS chip), BIOS region (the first instructions to be executed), Intel ME (Intel Management Engine) and GbE (gigabit ethernet). As you can see, BIOS chip is shared between serveral components of the system and not exclusive to CPU.

When power is applied

Modern Intel chips come with what is called Intel Management Engine. As soon as power is available – through battery or from mains – Intel ME comes on. It does its own initialisations which requires it to read BIOS’s flash descriptor to find where Intel ME region is and then from Intel ME region of BIOS, read in code and config data. Next when we press power button on the computer, the CPU comes on. On a multiprocessor system, there is always a designated processor, called Bootstrap Processor (BSP), which comes on. In either case, the processor always comes on in what is called 16-bit Real Mode with insruction pointer pointing to address 0xffff.fff0, the reset vector.

EDIT: (thanks to burfog for indicating that this needs explaination)

You might be wondering how could a 16-bit system address 0xffff.fff0 which is clearly beyond 0xffff, the max 16-bit value? In 16-bit mode,  physical address is calculated by left shifting code segment (CS) selector register by 4 bits and then adding instruction pointer (IP) address. On reset, IP cotains value 0xfff0 and CS has value 0xf000 [1]. By the above formula the physical address should be:

CS << 4 + IP = 0x000f.0000 + 0xfff0 = 0x000f.fff0

which is still not what we expected. This is because on reset, the system is in a “special” Real Mode, where the first 12 address lines are asserted. So all addresses look like 0xfffx.xxxx. This means in our case, we need to set the most significant 12 bits in the address we derived, which results in our expected address 0xffff.fff0. These 12 address lines remain asserted until a long JMP is executed, after which they are de-asserted and normal Real Mode addressing calculations resume.

The BIOS chip is also set up in such a way that first instruction to be executed from the BIOS is at physical address 0xffff.fff0 of the processor. Hence processor is able to execute the first instruction from BIOS region of the BIOS chip. This region contains what is called boot firmware. Examples of boot firmware are UEFI implementations, coreboot and the classic BIOS.

One of the first things that the boot firmware does is switch to 32-bit mode. It is also “protected mode”, i.e. segmentation is turned on and various segments of processor’s address space can be managed with different access permissions. Boot firmware however would have just one segment, effectively turning off segmentation. This is called flat mode.

Early Initialisations

It is worth noting that at this point in boot process, DRAM is not available. DRAM Initialisation is one of the main objectives of boot firmware. But before it can initialise DRAM, it needs to do some preparation.

Microcode patches are like patches for CPU to function correctly. Intel keeps publishing microcode patches for different CPUs. The boot firmware applies those patches very early on in boot process. Part of the processor is what is called south bridge or I/O controller hub (ICH) or peripheral controller hub (PCH). There are some initialisations to be performed for ICH also. For example, ICH may contain a watchdog timer which can go off which DRAM is being initialised. That watchdog timer must be turned off first.

Of course all of this is being done by firmware which is code written by someone. Now most of the code we know utilises stack. But we have mentioned that DRAM hasn’t been initialised yet so there is no memory. So how is this code written and run? Answer is that this is stackless code. Either it is hand written x86 assembly or, as in case of coreboot, it is written in C and compiled using special compiler called ROMCC which translates C to stackless assembly instructions. This of course comes with some restrictions so ROMCC compiled code is not how we want to execute everything. We need stack as soon as possible.

So, the next step is setting up what is called cache-as-RAM (CAR). Boot firmware basically sets up CPU caches so that they can be temporarily used as RAM. This way the firmware can run code which is not stackless, but still restricted in terms of stack size and general amount of memory available.

Memory Initialisation and Intel FSP

On Intel systems, memory initialisation is performed using a blob called Intel Firmware Support Package (FSP). This is supplied by Intel in binary form. Intel FSP does a lot of heavy lifting when it comes to bootstrapping Intel processors and is not just limited to memory init. It is basically a three stage API. The way boot firmware interacts with FSP is set up some parameters and a return address, and jump into an FSP stage. The FSP stage would execute taking into account the parameters and then use the return address to jump back into boot firmware. This continues across these three FSP stages and in that order:

  • TempRamInit(): This performs some init for RAM and hand control back to boot firmware. Boot firmware can kick off some actions and then go on to next stage. This is because the next step performs chipset and memory initialisation which may take some time. For example memory training is a time consuming operation. So this is an opportunity for boot firmware to kick off other initialisations, like spinning up hard drive, which can take time to stabilise.
  • FspInitEntry(): This is where actual DRAM is achieved. This also performs other silicon init, like PCH and CPU itself. After this finishes, it passes control back to boot firmware. However, since this time, the memory has been initialised, the passing back of control and data is different from TempRamInit stage. After this stage, firmware does most of the rest of initialisations – described in the next section ‘After Memory Init’ – before passing control to the next stage of FSP.
  • NotifyPhase(): This is where boot firmware would pass control back to FSP and set params which would tell FSP what sort of actions it needs to take before winding down. The types of things that FSP can do here are platform dependent but they include things like post PCI enumeration.

After Memory Init

Once DRAM is ready, it breathes a new life into boot process. First that the firmware does is copy itself into DRAM. This is done with help of “memory aliasing”, which means that reads and writes to addresses below 1MB are routed to and from DRAM. Then, firmware sets up the stack and transfer control to DRAM.

Next, some platform specific inits are done, such as GPIO configuration and re-enabling the watchdog timer in ICH which was disabled before memory init, paving the way for interrupts enabling. Local Advanced Programmable Interrupt Controller (LAPIC) sites inside each processor, i.e. it is local to each CPU in a multiprocessor system. LAPIC determines how each interrupt is delivered to that particular CPU. I/O APIC (IOxAPIC) lives inside ICH and there is one IOxAPIC for all processors. There can also be a Programmable Interrupt Controller (PIC) which is for use in Real Mode as is Interrupt Vector Table which contains 256 interrupt vectors – pointers to handlers for corresponding interrupts. Interrupt Descriptor Table on the other hand, is used to hold interrupt vectors when in Protected Mode.

Firmware then sets up various timers depending upon platform and the firmware. Programmable Interrupt Timer (PIT) is the system timer and sits on IRQ0. It lives inside ICH. High Precision Event Time (HPET) also sits inside ICH but boot firmware may not initialise it, letting the OS to set it up if needed. There is also a clock, the Real Time Clock (RTC) which too resides in ICH. There are other timers too, particularly LAPIC timer which is inside each CPU. Next, the firmware sets up memory caching. This basically means setting up different cache characteristics – write-back, uncached etc – for different ranges of memory.

Other Processors, I/O Devices and PCI

Finally, it is time to bring up other processors as all the work so far was being handled by the bootstrap processor. To find out about other application processors (AP) on the same package, BSP runs CPUID instruction. Then using its LAPIC, BSP sends an interrupt called SIPI, to each AP. Each SIPI points to the physical address at which the receiving AP should start executing. It is worth noting that each AP comes up in Real Mode, therefore the SIPI address must be less than 1MB, the maximum addressable in Real Mode. Usually soon after initialisation, each AP executes HLT instruction and gets into halt state, waiting for further instructions from BSP. However, just before OS gains control, APs are supposed to be in “waiting-for-SIPI” state. BSP achieves this by sending a couple of inter-processor interrupts to each AP.

Next come I/O devices like Embedded Controller (EC) and Super I/O, and after that PCI init. PCI init basically boils down to:

  1. enumerating all PCI devices
  2. allocating resources to each PCI device

This discussion here applies to PCIe also. PCI is a hierarchical bus system where for each bus, leaf is either a PCI device or a PCI bridge leading to another PCI bus. CPU communicates with PCI by reading and writing PCI registers. The resources needed by PCI devices are range inside memory address space, range inside I/O address space and IRQ assignment. CPU finds out about address ranges and their types (memory-mapped or I/O) by writing to and reading from Base Address Registers (BARs) of PCI devices. IRQs are usually set up based how the board is designed.

During PCI enumeration, firmware also reads Option ROM register. If that register is not empty then it contains address of Option ROM. This is ROM chip that is physically situated on the PCI device. For example the network card may contain Option ROM which holds iPXE firmware. When an Option ROM is encountered then it is read into DRAM and executed.

Handing Control to OS loader

Before handing over control to next stage loader which is usually an OS loader like GRUB2 or LILO, the firmware sets up some information inside memory which is later to be consumed by the OS. This information is things like Advanced Configuration and Power Interface (ACPI) tables and memory map itself. Memory map tells the OS what address ranges have been set up for what purposes. The regions can be gerenal memory for OS use, ACPI related address ranges, reserved (i.e. not to be used by OS), IOAPIC (to be used by IOAPIC), LAPIC (to be used by LAPICs). Boot firmware also sets up handlers for System Management Mode (SMM) interrupts. SMM is an operating mode of Intel CPUs, just like Real, Protected and Long (64-bit) modes. A CPU enters SMM mode upon receipt of an SMM interrupt which can be triggered by a number of things like chip’s temperature reaching a certain level. Before handing control to OS loader, the firmware also locks down some registers and CPU capability, so that it can’t be changed afterwards by the OS.

Actual transfer of control to the OS loader usually takes form of a JMP to that part of memory. An OS loader like GRUB2 will perform actions based on its config and ultimately pass controle to an operating system like Linux. For Linux, this will usually be a bzImage (big zImage, not bz compression). It is worth noting that the OS, like Linux would enumerate PCI devices again and may have other overlap with some of the final initialisations done by boot firmware. Linux usually picks up the system in 32-bit mode with paging turned off and performs its own initialisations which include setting up page tables, enabling paging and switching to long mode, i.e. 64-bit.


[1] userbinator on Hacker News pointed out that IP hasn’t always held the value 0xfff0 on a reset. On 8086/8088 it was 0x0. Here’s what he found from Intel’s documentation:

8086/88:   CS:IP = FFFF:0000 first instruction at FFFF0
80186/188: CS:IP = FFFF:0000 first instruction at FFFF0
80286:     CS:IP = F000:FFF0 first instruction at FFFF0
80386:     CS:IP = 0000:0000FFF0 or F000:0000FFF0[1], first instruction at FFFFFFF0
80486+:    CS:IP = F000:0000FFF0(?) first instruction at FFFFFFF0

 

Paging in Linux on x86

In our last post we covered how x86 logical address is translated into linear address. In this one we will look at translation from linear to physical. We will use the terms ‘virtual address’ and ‘linear address’ interchangeably.

A piece of hardware called paging unit is responsible for converting virtual addresses to physical. However, the operating system needs to set it up with correct data structures – page tables. On x86, paging is enabled by setting a flag inside a special register. When that flag is zero, paging is not enabled and linear addresses are treated as physical addresses. Linux first sets up page tables and then enables paging.

Pages and page tables

For ease of management of memory, e.g. access rights, physical memory is divided into `page frames`. These are contiguous cells of RAM, usually 4KB in size. Corresponding to each physical page frame there is a `page` of virtual addresses. For instance virtual addresses 0x20300000 – 0x20301000 represent a page which corresponds to 4096 physical addresses each of which points to a cell (one byte) in RAM. A page as well as a page frame represent contiguous addresses, so inside a page, the virtual-to-physical mapping is one-to-one. Page is basic unit of memory management in Linux. A key function of paging unit is to check type of access to a virtual address (read or write) against access rights of the page to which that virtual address belongs. When access right is violated, paging unit generates a Page Fault.

Page table is an array in RAM which maps virtual address to physical address. Each user process has its own page table and when a context switch happens, the page tables are changed as part of it. Each entry inside page table points to a page frame inside RAM. So a 32-bit virtual address has two parts: page table index (20 most significant bits) and page offset (12 bits because page size is 4096). Using page tabele index, we will get page frame. Inside page frame we use page offset to get the exact memory cell, the byte that the virtual address points to.

A naive way of organising page table would be to have one page table whose indices are 20 most significant bits of virtual address and whose values contain (among other things) physical address of page frame. That would be wasteful. If each entry is 4 bytes, a page table would require (2^20 * 4) bytes = 4MB of RAM. That is for each process. x86 instead breaks single page table into two: Page Directory and Page Table. Virtual address is also divided into three parts: index inside Page Directory, to get Page Table entry, index inside Page Table entry to get page frame address, and then the same 12-bit page offset to find the cell inside page frame. This way, each process will have to have a Page Directory but there is no need to allocate all Page Tables upfront. Instead Page Tables can be set up when they are needed.

Management of Pages

Physical address of Page Directory is stored in a special register and that registered is updated when there is a context switch. Entries in Page Directory and Page Table have same format. Along with address of corresponding page frames (or Page Table in case of Page Directory’s entry), it stores privilege level needed to access that page. The privilege level is a single byte so has two possible values. It depends upon CPU Privilege Level (CPL) – a two byte value on x86 which represents four levels. In page table entry, it only checks whether a page requires supervisor mode (CPL = 0) or not (CPL = 1, 2 or 3).

Page table entry also contains access type allowed: read and write. In contrast, access rights for segments are three: read, write and execute. So a page which is read only cannot be written to.

What about Linux?

As you might have noticed, this post hasn’t really lived up to its title and only talks about paging in x86. Time and other conditions permitting, we will discuss paging in Linux in a follow-up article.