DM510: Adding a System Call

This document describes how to add a system call to User-mode Linux to the 5.6-rc1 kernel. Make sure you have a running UML kernel (see here). Understanding this will be a prerequisite to solve the first mandatory assignment.

Define the entry point

The first thing you need to do is modify the file unistd_64.h in the directory arch/x86/include/generated/uapi/asm/unistd_64.h. In this file, you need to add a line providing an id for your system call. Locate the bunch of lines of the form #define __NR_somename (in this example __NR_hellokernel), and add a new entry for your system call by adding the following (last) line:

#define __NR_openat2 437
#define __NR_pidfd_getfd 438
#define __NR_hellokernel 439

Next, you must add and entry refering to your call in the system calls table. To do this, modify syscall_64.tbl file in ./arch/x86/entry/syscalls/ and add the following (last) line:

   common  openat2                 __x64_sys_openat2
   common  pidfd_getfd             __x64_sys_pidfd_getfd
   common  hellokernel             sys_hellokernel

Now we will implement a system call.

Implementation code

First of all, create a header file hellokernel.h for your system call and place it in arch/um/include/asm as shown here:

Download hellokernel.h

#ifndef __UML_HELLOKERNEL_H__
#define __UML_HELLOKERNEL_H__

extern int sys_hellokernel( int );

#endif

Then, write out the implementation hellokernel.c of your system call in arch/um/kernel/ as shown here. (Note the use of the asmlinkage modifier. This macro tells the compiler to pass all function arguments on the stack.):

Download hellokernel.c

#include "linux/kernel.h"
#include "linux/unistd.h"

asmlinkage
int sys_hellokernel( int flag ) {
  printk("Your kernel greets you %d times! \n", flag);
  return 0;
}

Finally, modify the respective Makefile in arch/um/kernel and add somename.o (in this example: hellokernel.o) to the list of build targets.

extra-y := vmlinux.lds

obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
        physmem.o process.o ptrace.o reboot.o sigio.o \
        signal.o syscall.o sysrq.o time.o tlb.o trap.o \
        um_arch.o umid.o maccess.o kmsg_dump.o hellokernel.o skas/

Now you have to recompile the kernel. Change to the kernel source root directory and recompile :

cd ~/dm510/linux-5-6-rc1
make ARCH=um linux

The next step is to use the new system call from a user-space application.

Creating a stub for your system call

You call the new functions as identified by their index through the syscall function. With the syscall function, you can call, what a surprise, a system call. For example, the short application shown below calls your sys_hellokernel using the index of the system call and 42 as argument. The program can be compiled with the command gcc -o testsystemcall testsystemcall.c.

Download testsystemcall.c

#include <stdio.h>
#include <errno.h>
#include <unistd.h>
#include "arch/x86/include/generated/uapi/asm/unistd_64.h"

int main(int argc, char ** argv) {
	printf("Calling ... \n");
	syscall(__NR_hellokernel,42);
}

Run user mode linux

./linux

mount none /mnt/tmp -t hostfs -o /home/your_imada_login/dm510
cd /mnt/tmp/linux-5.6-rc1

Execute the program. The result should be similar to the following:

dm510:/mnt/tmp/linux-5.6-rc1# ./testsyscall 
Calling ... 
Your kernel greets you 42 times!

Note that the kernel messages are maybe printed in another terminal than the messages that are printed by printf commands of the program. Besides of that, you will find the messages from the kernel in /var/log/kern.log.

Further (partially very old) information can e.g. be found here:

Kernel command using Linux system calls link.
Adding a System Call to UML link (partially outdated).
Linux System Call Howto link. Note: this is for adding a system call to Linux in general, i.e., slightly different than what you have to do in UML.
How System Calls Are Implemented on i386 Architecture: link.