Adding work to kthread worker

#

Version

Ubuntu

Ubuntu 22.10

Kernel

6.7.9

  • In this program, you are going to learn

  • How to create the kthread work ?

  • How to add kthread work to kthread worker?

  • How to destroy the kthread worker?

#include <linux/kthread.h>

struct kthread_worker *kthread_create_worker(unsigned int flags, const char namefmt[], ...);
  • where

    • kthread_create_worker: creates a kthread worker.

    • return type:

      • returns a pointer (i.e struct kthread_worker*) to the allocated worker on success

      • ERR_PTR(-ENOMEM) when the needed structures could not get allocated

      • ERR_PTR(-EINTR) when the caller was killed by a fatal signal.

    • flags: flags modifying the default behaviour of the kthread worker.

    • namefmt: printf-style name for the kthread worker.

  • Here is an example of how to use the API,

kthread_create_worker(0,"sample kthread worker");
  • Here is the function prototype of the API: kthread_run

#include <linux/kthread.h>

#define kthread_run(threadfn, data, namefmt, ...)
  • where

    • kthread_run : is used to create and wake a kthread

    • return type: struct task_struct* (i.e) address of the kthread

    • threadfn: function to executed by kthread

    • data: data pointer for threadfn which can be used to send any possible arguments required for threadfn.

    • namefmt: printf-style format for the thread name which will be displayed on ps output when the thread is in execution.

  • Here is an example of how to use the API,

kthread_run(mythread,NULL,"sample kthread");
#include <linux/kthread.h>

#define kthread_init_work(work, fn)                    \
    do {                                               \
        memset((work), 0, sizeof(struct kthread_work));\
        INIT_LIST_HEAD(&(work)->node);                 \
        (work)->func = (fn);                           \
    } while (0)
  • where

    • kthread_init_work : used to initialize a work.

    • work: kthread work which needs to be initialized

    • fn: function which is needs to be executed once kthread work is created.

  • Here is an example of how to use the API,

kthread_init_work(mywork,myworkfn);
#include <linux/kthread.h>

bool kthread_queue_work(struct kthread_worker *worker, struct kthread_work *work);
  • where

    • kthread_queue_work : queue a kthread work

    • worker: target kthread_worker

    • work: kthread_work to queue

    • return type: returns type if it is successfully queue to kthread_worker, false if its already pending

  • Here is an example of how to use the API,

kthread_queue_work(myworker,mywork);
#include <linux/kthread.h>

bool kthread_cancel_work_sync(struct kthread_work *work);
  • where

    • kthread_cancel_work_sync : cancel a kthread_work and wait for it to finish

    • work: the kthread_work to cancel.

    • return type: true if work is pending , false otherwise.

  • Here is an example of how to use the API,

kthread_cancel_work_sync(myworker);
#include <linux/kthread.h>

void kthread_destroy_worker(struct kthread_worker *worker);
  • where

  • Here is an example of how to use the API,

kthread_cancel_work_sync(mywork);
#include <linux/kthread.h>

bool kthread_should_stop(void);
  • where

    • kthread_should_stop: this is used to determine whether the thread should return now Whenever the kthread_stop() is called it will be woken and returns true.

    • return type: returns true when the thread is stopped, false when the thread is still in execution

  • Here is an example of how to use the API,

while (!kthread_should_stop())
{
    //add the instructions to be performed during thread execution.
}
  • Here is the prototype of module paramter APIs

#include <linux/module.h>

#define MODULE_LICENSE(_license) MODULE_FILE MODULE_INFO(license, _license)
#define MODULE_AUTHOR(_author) MODULE_INFO(author, _author)
#define MODULE_DESCRIPTION(_description) MODULE_INFO(description, _description)
  • where

    • MODULE_LICENSE: tells the kernel what license is used by our module.

    • MODULE_AUTHOR: denotes the author of this kernel module.

    • MODULE_DESCRIPTION: gives a basic idea about what the kernel module does.

    • These information can be found when modinfo command is used which lists out all these above mentioned information.

  • Here is the example of how to use the Module parameter APIs,

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Linux Usr");
MODULE_DESCRIPTION("Sample kernel module");
  • Here is the prototype of the API: IS_ERR

#include <linux/err.h>

static inline bool __must_check IS_ERR(__force const void *ptr);
  • where

    • IS_ERR: Detects an error pointer

    • return type: return true if it’s an error pointer else return false.

    • ptr: pointer which needs to detected.

  • Here is an example of how to use the API,

if(IS_ERR(sample_ptr)) {
    //instructions to be executed when error ptr is detected
} else {
    //instructions to be executed when error ptr is not detected
}
#include <linux/kernel.h>

#define container_of(ptr, type, member) ({                  \
    const typeof(((type *)0)->member) * __mptr = (ptr);     \
    (type *)((char *)__mptr - offsetof(type, member)); })
#endif
  • where

    • container_of : cast a member of the structure to the containing structure.

    • ptr: the pointer to the member

    • type: the type of container struct this is embedded in

    • member: the name of the member within the struct

  • Here is an example of how to use the API,

container_of(myptr,struct mysamplestruct,samplemember);
  • Here is the prototype of the API: kmalloc

#include <linux/slab.h>

void *kmalloc(size_t size, gfp_t gfp);
  • where

    • kmalloc: allocate kernel memory

    • size: how many bytes of memory is required

    • flags: describle the allocation context

  • Here is an example of how to use the API,

kmalloc(sizeof(struct mystruct),GFP_KERNEL);
  • Here is the prototype of the API: pr_info

#include <linux/printk.h>

#define pr_info(fmt, ...) \
    printk(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__)
  • where

    • pr_info: Prints an info-level messages

    • fmt: format string

  • Here is an example of how to use the API,

pr_info("//sample print statement");
#include <linux/sched.h>

extern int wake_up_process(struct task_struct *tsk);
  • where

    • wake_up_process: wake up a specific process

    • return type: returns 1 if the process is woken up, 0 if the process is in running state.

    • tsk: process to be woken up.

  • Here is the example of how to use the API,

wake_up_process(mythread);
  • Here is the example of the API: msleep

#include <linux/delay.h>

void msleep(unsigned int msecs);
  • where

    • msleep: will put it in sleep for a certain amount of msecs time.

    • msecs: time in milliseconds to sleep for

  • Here is the example of how to use the API,

msleep(1000);
  • Here is the prototype of the Driver entry point API’s

#include <linux/module.h>

#define module_init(x)      __initcall(x);
#define module_exit(x)      __exitcall(x);
  • where

    • module_init: driver initialization entry point which will be called at module insertion time.

    • module_exit: driver exit entry point which will be called during the removal of module.

    • x:
      • function to be run at module insertion for module_init function.

      • function to be run when driver is removed for module_exit function.

  • Here is an example of how to use the driver entry point API’s

module_init(myinitmodule);
module_exit(myexitmodule);
  • In this example let’s see how to create kthread work using kthread_init_work and execute it.

  • Include the follow header files(.h) to refer the API being used for the execution.

#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/kthread.h>
#include <linux/slab.h>
#include <linux/delay.h>
  • Add the following module macros to display information about the license, author and description about the module.

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Linux Usr");
MODULE_DESCRIPTION("Example of kthread_init_work");
  • Declare the thread variables which we are going to create and use in this example

static struct task_struct *my_thread;
static struct kthread_worker *worker;
bool ret_val;
struct my_work {
    struct kthread_work my_kthread_work;
    int i;
};
  • Add the module init function which will be executed once we load the kernel module using insmod command.

static int __init kthread_init_work_init(void)
{
    pr_info("Inside kthread init function\n");
    thread_start();
    return 0;
}
  • Add the thread start function which creates kthread, on success creation it starts executing the task.

void thread_start(void)
{
    my_thread = kmalloc(sizeof(*my_thread),GFP_KERNEL);
    my_thread = kthread_run(def_thread_fun,NULL,"kthread_init_work thread example");
    if(IS_ERR(my_thread))
        pr_info("Error creating thread\n");
    else
        pr_info("Created kthread\n");
}
  • Add the module exit function which will be executed once we unload the kernel module using rmmod command.

static void __exit kthread_init_work_exit(void)
{
    pr_info("Inside kthread exit function\n");
    thread_stop();
}
  • Add the thread stop function which destroys the thread and stops the execution.

void thread_stop(void)
{
    if(my_thread) {
        kthread_stop(my_thread);
        kthread_destroy_worker(worker);
        pr_info("Destroyed kthread and worker");
    }
}
  • Add the thread function API which will be called as soon as the kthread is created and is in running state.

int def_thread_fun(void *data)
{
    struct my_work my_def_work;
    kthread_init_work(&my_def_work.my_kthread_work,def_work_function);
    pr_info("Initializing kthread work\n");
    worker = kthread_create_worker(0,"kthread_init_work worker example");
    if (IS_ERR(worker))
        pr_info("Error creating worker\n");
    else
        pr_info("Created kthread worker\n");
    ret_val = kthread_queue_work(worker,&my_def_work.my_kthread_work);
    if (ret_val)
        pr_info("Queue the work to worker\n");
    else
        pr_info("Work is still in pending state\n");
    while (!kthread_should_stop()) {
        pr_info("thread execution\n");
        msleep(100);
    }
    return 0;
}
  • Add the work function which is executed once the work is queued to kthread_worker.

void def_work_function(struct kthread_work *work)
{
    pr_info("Inside work function\n");
    struct my_work *my_data = container_of(work,struct my_work,my_kthread_work);
    my_data->i = 0;
    while (my_data->i <= 10) {
        pr_info("my_data->i = %d\n",my_data->i);
        my_data->i++;
    }
}
  • Add the driver entry points which will be executed once the module is inserted or removed from the kernel.

module_init(kthread_init_work_init);
module_exit(kthread_init_work_exit);
  1#include <linux/init.h>
  2#include <linux/kernel.h>
  3#include <linux/module.h>
  4#include <linux/kthread.h>
  5#include <linux/slab.h>
  6#include <linux/delay.h>
  7
  8MODULE_LICENSE("GPL");
  9MODULE_AUTHOR("Linux usr");
 10MODULE_DESCRIPTION("Example of kthread_init_work");
 11
 12static struct task_struct *my_thread;
 13static struct kthread_worker *worker;
 14bool ret_val;
 15struct my_work {
 16    struct kthread_work my_kthread_work;
 17    int i;
 18};
 19
 20/* def_work_function - executes once kthread_work is queued to worker,
 21 * will print a message until my_data->i reaches 10 */
 22
 23void def_work_function(struct kthread_work *work)
 24{
 25    pr_info("Inside work function\n");
 26    struct my_work *my_data = container_of(work,struct my_work,my_kthread_work);
 27    my_data->i = 0;
 28    while (my_data->i <= 10) {
 29        pr_info("my_data->i = %d\n",my_data->i);
 30        my_data->i++;
 31        msleep(100);
 32    }
 33}
 34
 35/* def_thread_fun - will execute on successful creation of thread,
 36 * work and worker is created, work is queued to worker,
 37 * then goes into an infinite loop prints a message and sleep,
 38 * will stop its execution when kthread_stop() is called*/
 39
 40int def_thread_fun(void *data)
 41{
 42    struct my_work my_def_work;
 43    kthread_init_work(&my_def_work.my_kthread_work,def_work_function);
 44    pr_info("Initializing kthread work\n");
 45    worker = kthread_create_worker(0,"kthread_init_work worker example");
 46    if (IS_ERR(worker))
 47        pr_info("Error creating worker\n");
 48    else
 49        pr_info("Created kthread worker\n");
 50    ret_val = kthread_queue_work(worker,&my_def_work.my_kthread_work);
 51    if (ret_val)
 52        pr_info("Queue the work to worker\n");
 53    else
 54        pr_info("Work is still in pending state\n");
 55    while (!kthread_should_stop()) {
 56        pr_info("thread execution\n");
 57        msleep(100);
 58    }
 59    return 0;
 60}
 61
 62/* thread_start - creates kthread and starts its execution*/
 63
 64void thread_start(void)
 65{
 66    my_thread = kmalloc(sizeof(*my_thread),GFP_KERNEL);
 67    my_thread = kthread_run(def_thread_fun,NULL,"kthread_init_work thread example");
 68    if(IS_ERR(my_thread))
 69        pr_info("Error creating thread\n");
 70    else
 71        pr_info("Created kthread\n");
 72}
 73
 74/* kthread_init_work_init - calls thread_start function,
 75 * executes when the module is loaded into linux kernel*/
 76
 77static int __init kthread_init_work_init(void)
 78{
 79    pr_info("Inside kthread_init function\n");	
 80    thread_start();
 81    return 0;
 82}
 83
 84/* thread_stop - destroys thread, worker and stops its execution*/
 85
 86void thread_stop(void)
 87{
 88    if(my_thread) {
 89        kthread_stop(my_thread);
 90        kthread_destroy_worker(worker);
 91        pr_info("Destroyed kthread and worker");
 92    }
 93}
 94
 95/* kthread_init_work_exit - calls thread_stop function,
 96 * executes when the module is unloaded from kernel*/
 97
 98static void __exit kthread_init_work_exit(void)
 99{
100    pr_info("Inside kthread exit function\n");
101    thread_stop();
102}
103
104module_init(kthread_init_work_init);
105module_exit(kthread_init_work_exit);
1obj-m += kthread.o
2
3all:
4	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
5
6clean:
7	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
  • Run make to compile the kernel source and generate the .ko image.

make -C /lib/modules/6.7.9/build M=$HOME/kthread_examples/
make[1]: Entering directory '/usr/src/linux-headers-6.7.9'
warning: the compiler differs from the one used to build the kernel
The kernel was built by: x86_64-linux-gnu-gcc (Ubuntu 12.2.0-3ubuntu1) 12.2.0
You are using:           gcc (Ubuntu 12.2.0-3ubuntu1) 12.2.0
CC [M]  $HOME/kthread_examples/kthread.o
MODPOST $HOME/kthread_examples/Module.symvers
CC [M]  $HOME/kthread_examples/kthread.mod.o
LD [M]  $HOME/kthread_examples/kthread.ko
BTF [M] $HOME/kthread_examples/kthread.ko
Skipping BTF generation for $HOME/kthread_examples/kthread.ko due to unavailability of vmlinux
make[1]: Leaving directory '/usr/src/linux-headers-6.7.9'
  • Check if the .ko is generated or not using ls command.

test@test-V520-15IKL:~$ ls -l
total 360
-rw-rw-r-- 1 test test    713 Mar 18 16:06 kthread.c
-rw-rw-r-- 1 test test 169784 Mar 18 16:08 kthread.ko
-rw-rw-r-- 1 test test     58 Mar 18 16:08 kthread.mod
-rw-rw-r-- 1 test test   1047 Mar 18 16:08 kthread.mod.c
-rw-rw-r-- 1 test test  96512 Mar 18 16:08 kthread.mod.o
-rw-rw-r-- 1 test test  74696 Mar 18 16:08 kthread.o
-rw-rw-r-- 1 test test    161 Mar 18 16:00 Makefile
-rw-rw-r-- 1 test test     58 Mar 18 16:08 modules.order
-rw-rw-r-- 1 test test      0 Mar 18 16:08 Module.symvers
  • Run modinfo command to get the information about the kernel module.

test@test-V520-15IKL:~/.../tc_1$ modinfo kthread.ko
filename:       $HOME/kthread_examples/kthread.ko
description:    Example of kthread_init_work
author:         Linux Usr
license:        GPL
srcversion:     8D2147F67AB01CF0E482DAC
depends:
retpoline:      Y
name:           kthread
vermagic:       6.7.9 SMP preempt mod_unload modversions
  • insert the module using insmod command.

test@test-V520-15IKL:~$ sudo insmod ./kthread.ko
  • check if the module is loaded or not using lsmod command.

test@test-V520-15IKL:~$ sudo lsmod | grep kthread
kthread                16384  0
  • check if the thread is created or not using ps command.

test@test-V520-15IKL:~$ ps -N | grep kthread_init_work
15149 ?        00:00:00 kthread_init_work thread example
15150 ?        00:00:00 kthread_init_work worker example
  • check for the kernel messages from init function and thread function once the module is loaded and thread is created.

test@test-V520-15IKL:~$ sudo dmesg
[ 5989.857961] Inside kthread_init function
[ 5989.857991] Created kthread
[ 5989.857992] Initializing kthread work
[ 5989.858009] Created kthread worker
[ 5989.858010] Queue the work to worker
[ 5989.858011] thread execution
[ 5989.858011] Inside work function
[ 5989.858014] my_data->i = 0
[ 5989.963688] thread execution
[ 5989.963689] my_data->i = 1
[ 5990.071690] thread execution
[ 5990.071689] my_data->i = 2
  • remove the module from kernel using rmmod command.

test@test-V520-15IKL:~$ sudo rmmod kthread
  • check if the module is still loaded after removing the kernel module using lsmod if it is not displayed in lsmod output it is verified that the module is removed successfully.

test@test-V520-15IKL:~$ sudo lsmod | grep kthread
test@test-V520-15IKL:~$
  • check if the thread is destroyed using ps command if it is not displayed in ps output we can confirm that the thread is destroyed successfully.

test@test-V520-15IKL:~$ ps -N | grep kthread_init_work
test@test-V520-15IKL:~$
  • Check for kernel messages from exit function using dmesg command.

test@test-V520-15IKL:~$ sudo dmesg
[ 5993.133210] Inside kthread exit function
[ 5993.207808] Destroyed kthread and worker
  • In this example let’s see how to cancel kthread_work using kthread_cancel_work_sync.

  • Include the follow header files(.h) to refer the API being used for the execution.

#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/kthread.h>
#include <linux/slab.h>
#include <linux/delay.h>
  • Add the following module macros to display information about the license, author and description about the module.

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Linux Usr");
MODULE_DESCRIPTION("Example of kthread_cancel_work_sync");
  • Declare the thread variables which we are going to create and use in this example

bool ret_val;
static struct task_struct *my_thread;
static struct kthread_worker *worker;
struct my_work {
    struct kthread_work my_kthread_work;
    int i;
};
  • Add the module init function which will be executed once we load the kernel module using insmod command.

static int __init kthread_cancel_work_sync_init(void)
{
    pr_info("Inside kthread init function\n");
    thread_start();
    return 0;
}
  • Add the thread start function in which the worker and thread is created and starts its execution.

void thread_start(void)
{
    my_thread = kthread_run(def_thread_fun,NULL,"kthread_cancel_work_sync thread example");
    if(IS_ERR(my_thread))
        pr_info("Error creating kthread\n");
    else
        pr_info("Created kthread\n");
}
  • Add the module exit function which will be executed once we unload the kernel module using rmmod command.

static void __exit kthread_cancel_work_sync_exit(void)
{
    pr_info("Inside kthread exit function\n");
    thread_stop();
}
  • Add the thread stop function in which worker is destroyed and the thread stops its execution.

void thread_stop(void)
{
    if(my_thread) {
        kthread_stop(my_thread);
        kthread_destroy_worker(worker);
        pr_info("Destroyed kthread and worker");
    }
}
  • Add the thread function API which will be called as soon as the kthread is created and is in running state.

int def_thread_fun(void *data)
{
    struct my_work my_def_work;
    kthread_init_work(&my_def_work.my_kthread_work,def_work_function);
    pr_info("Initializing kthread work\n");
    worker = kthread_create_worker(0,"kthread_cancel_work_sync worker example");
    if (IS_ERR(worker))
        pr_info("Error creating worker\n");
    else
        pr_info("Created kthread worker\n");
    ret_val = kthread_queue_work(worker,&my_def_work.my_kthread_work);
    if (ret_val)
        pr_info("Queue the work to worker\n");
    else
        pr_info("Work is in pending state");
    while (!kthread_should_stop()) {
        pr_info("thread execution\n");
        kthread_cancel_work_sync(&my_def_work.my_kthread_work);
        msleep(100);
    }
    pr_info("Cancelled kthread work\n");
    return 0;
}
  • Add the work function which will be executed once kthread_work is queued to kthread_worker.

void def_work_function(struct kthread_work *work)
{
    pr_info("Inside work function\n");
    struct my_work *my_data = container_of(work,struct my_work,my_kthread_work);
    my_data->i = 0;
    while (my_data->i <= 10) {
        pr_info("my_data->i = %d\n",my_data->i);
        my_data->i++;
        msleep(100);
    }
}
  • Add the driver entry points which will be executed once the module is inserted or removed from the kernel.

module_init(kthread_cancel_work_sync_init);
module_exit(kthread_cancel_work_sync_exit);
  1#include <linux/init.h>
  2#include <linux/kernel.h>
  3#include <linux/module.h>
  4#include <linux/kthread.h>
  5#include <linux/slab.h>
  6#include <linux/delay.h>
  7
  8MODULE_LICENSE("GPL");
  9MODULE_AUTHOR("Linux usr");
 10MODULE_DESCRIPTION("Example of kthread_cancel_work_sync");
 11
 12bool ret_val;
 13static struct task_struct *my_thread;
 14static struct kthread_worker *worker;
 15struct my_work {
 16	struct kthread_work my_kthread_work;
 17	int i;
 18};
 19
 20/* def_work_function - executes once the work is queued to worker,
 21 * will print a message until my_data->i reaches 10*/
 22
 23void def_work_function(struct kthread_work *work)
 24{
 25	pr_info("Inside work function\n");
 26	struct my_work *my_data = container_of(work,struct my_work,my_kthread_work);
 27	my_data->i = 0;
 28	while (my_data->i <= 10) {
 29		pr_info("my_data->i = %d\n",my_data->i);
 30		my_data->i++;
 31        msleep(100);
 32	}
 33}
 34
 35/* def_thread_fun - executes when the thread is created,
 36 * creates worker,work and queues work to the worker,
 37 * prints a messages and goes to sleep for 100ms,
 38 * stops its execution once kthread_stop() is called*/
 39
 40int def_thread_fun(void *data)
 41{
 42	struct my_work my_def_work;
 43	kthread_init_work(&my_def_work.my_kthread_work,def_work_function);
 44	pr_info("Initializing kthread work\n");
 45	worker = kthread_create_worker(0,"kthread_cancel_work_sync worker example");
 46	if (IS_ERR(worker))
 47		pr_info("Error creating worker\n");
 48    else
 49        pr_info("Created kthread worker\n");
 50	ret_val = kthread_queue_work(worker,&my_def_work.my_kthread_work);
 51	if (ret_val)
 52        pr_info("Queue the work to worker\n");
 53    else
 54        pr_info("Work is in pending state");
 55    while (!kthread_should_stop()) {
 56        pr_info("thread execution\n");
 57        kthread_cancel_work_sync(&my_def_work.my_kthread_work);
 58        msleep(100);
 59    }
 60	pr_info("Cancelled kthread work\n");
 61	return 0;
 62}
 63
 64/* thread_start - creates thread and starts execution*/
 65
 66void thread_start(void)
 67{
 68    my_thread = kthread_run(def_thread_fun,NULL,"kthread_cancel_work_sync thread example");
 69    if(IS_ERR(my_thread))
 70        pr_info("Error creating thread\n");
 71    else
 72        pr_info("Created kthread\n");
 73}
 74
 75/* kthread_cancel_work_sync_init - calls thread_start function,
 76 * executes when module is loaded */
 77
 78static int __init kthread_cancel_work_sync_init(void)
 79{
 80	pr_info("Inside kthread_init function\n");	
 81	thread_start();
 82    return 0;
 83}
 84
 85/* thread_stop - destroys worker,thread and stops execution*/
 86
 87void thread_stop(void)
 88{
 89    if(my_thread) {
 90        kthread_stop(my_thread);
 91        kthread_destroy_worker(worker);
 92        pr_info("Destroyed kthread and worker");
 93    }
 94}
 95
 96/* kthread_cancel_work_sync_exit - calls thread_stop function,
 97 * executes when module is unloaded*/
 98
 99static void __exit kthread_cancel_work_sync_exit(void)
100{
101	pr_info("Inside kthread exit function\n");
102    thread_stop();
103}
104
105module_init(kthread_cancel_work_sync_init);
106module_exit(kthread_cancel_work_sync_exit);
1obj-m += kthread.o
2
3all:
4	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
5
6clean:
7	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
  • Run make to compile the kernel source and generate the .ko image.

make -C /lib/modules/6.7.9/build M=$HOME/kthread_examples/
make[1]: Entering directory '/usr/src/linux-headers-6.7.9'
warning: the compiler differs from the one used to build the kernel
The kernel was built by: x86_64-linux-gnu-gcc (Ubuntu 12.2.0-3ubuntu1) 12.2.0
You are using:           gcc (Ubuntu 12.2.0-3ubuntu1) 12.2.0
CC [M]  $HOME/kthread_examples/kthread.o
MODPOST $HOME/kthread_examples/Module.symvers
CC [M]  $HOME/kthread_examples/kthread.mod.o
LD [M]  $HOME/kthread_examples/kthread.ko
BTF [M] $HOME/kthread_examples/kthread.ko
Skipping BTF generation for $HOME/kthread_examples/kthread.ko due to unavailability of vmlinux
make[1]: Leaving directory '/usr/src/linux-headers-6.7.9'
  • Check if the .ko is generated or not using ls command.

test@test-V520-15IKL:~$ ls -l
total 360
-rw-rw-r-- 1 test test    713 Mar 18 16:06 kthread.c
-rw-rw-r-- 1 test test 169784 Mar 18 16:08 kthread.ko
-rw-rw-r-- 1 test test     58 Mar 18 16:08 kthread.mod
-rw-rw-r-- 1 test test   1047 Mar 18 16:08 kthread.mod.c
-rw-rw-r-- 1 test test  96512 Mar 18 16:08 kthread.mod.o
-rw-rw-r-- 1 test test  74696 Mar 18 16:08 kthread.o
-rw-rw-r-- 1 test test    161 Mar 18 16:00 Makefile
-rw-rw-r-- 1 test test     58 Mar 18 16:08 modules.order
-rw-rw-r-- 1 test test      0 Mar 18 16:08 Module.symvers
  • Run modinfo command to get the information about the kernel module.

test@test-V520-15IKL:~/.../tc_1$ modinfo kthread.ko
filename:       $HOME/kthread_examples/kthread.ko
description:    Example of kthread_cancel_work_sync
author:         Linux Usr
license:        GPL
srcversion:     8D2147F67AB01CF0E482DAC
depends:
retpoline:      Y
name:           kthread
vermagic:       6.7.9 SMP preempt mod_unload modversions
  • insert the module using insmod command.

test@test-V520-15IKL:~$ sudo insmod ./kthread.ko
  • check if the module is loaded or not using lsmod command.

test@test-V520-15IKL:~$ sudo lsmod | grep kthread
kthread                16384  0
  • check if the thread is created or not using ps command.

test@test-V520-15IKL:~$ ps -N | grep kthread_cancel_work_sync
6234 ?        00:00:00 kthread_cancel_work_sync thread example
6235 ?        00:00:00 kthread_cancel_work_sync worker example
  • check for the kernel messages from init function and thread function once the module is loaded and thread is created.

test@test-V520-15IKL:~$ sudo dmesg
[  835.288192] Inside kthread_init function
[  835.288360] Created kthread
[  835.288365] Initializing kthread work
[  835.288436] Created kthread worker
[  835.288440] Queue the work to worker
[  835.288441] thread execution
[  835.288441] Inside work function
[  835.288446] my_data->i = 0
[  835.394864] my_data->i = 1
[  835.502852] my_data->i = 2
  • remove the module from kernel using rmmod command.

test@test-V520-15IKL:~$ sudo rmmod kthread
  • check if the module is still loaded after removing the kernel module using lsmod if it is not displayed in lsmod output it is verified that the module is removed successfully.

test@test-V520-15IKL:~$ sudo lsmod | grep kthread
test@test-V520-15IKL:~$
  • check if the thread is destroyed using ps command if it is not displayed in ps output we can confirm that the thread is destroyed successfully.

test@test-V520-15IKL:~$ ps -N | grep kthread_cancel_work_sync
test@test-V520-15IKL:~$
  • Check for kernel messages from exit function using dmesg command.

test@test-V520-15IKL:~$ sudo dmesg
[  850.683827] Inside kthread exit function
[  850.775050] Cancelled kthread work
[  850.775410] Destroyed kthread and worker
  • In this example let’s see how to queue multiple works using kthread_queue_work

  • Include the follow header files(.h) to refer the API being used for the execution.

#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/kthread.h>
#include <linux/delay.h>
#include <linux/slab.h>
  • Add the following module macros to display information about the license, author and description about the module.

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Linux Usr");
MODULE_DESCRIPTION("example of kthread_queue_work");
  • Declare the thread variables which we are going to create and use in this example

bool ret_val;
static struct task_struct *my_thread;
static struct kthread_worker *my_worker;
struct my_work {
    struct kthread_work my_kthread_work;
    int i;
};
  • Add the module init function which will be executed once we load the kernel module using insmod command.

static int __init kthread_queue_work_init(void)
{
    pr_info("Inside kthread init function\n");
    thread_start();
    return 0;
}
  • Add the thread start function in which the thread is created and starts its execution.

void thread_start(void)
{
    my_thread = kthread_run(my_thread_fun,NULL,"kthread_queue_work thread example");
    if (IS_ERR(my_thread))
        pr_info("Error in creating thread\n");
    else
        pr_info("Thread created successfully\n");
}
  • Add the module exit function which will be executed once we unload the kernel module using rmmod command.

static void __exit kthread_queue_work_exit(void)
{
    pr_info("Inside kthread exit function\n");
    thread_stop();
}
  • Add the thread stop function which destroys worker,thread and stops the execution.

void thread_stop(void)
{
    if (my_thread) {
        kthread_stop(my_thread);
        kthread_destroy_worker(my_worker);
        pr_info("Destroyed threads and worker\n");
    }
}
  • Add the thread function API which will be called as soon as the kthread is created and is in running state.

int my_thread_fun(void *data)
{
    struct my_work my_def_work_1;
    struct my_work my_def_work_2;
    kthread_init_work(&my_def_work_1.my_kthread_work,def_work_fn_1);
    pr_info("kthread work 1 intialized\n");
    my_worker = kthread_create_worker(0,"kthread_queue_work worker example");
    if (IS_ERR(my_worker))
        pr_info("error in creating kthread_worker\n");
    else
        pr_info("kthread worker created\n");
    ret_val = kthread_queue_work(my_worker,&my_def_work_1.my_kthread_work);
    if (ret_val)
        pr_info("kthread work 1 is queued to worker\n");
    else
        pr_info("work 1 is pending\n");

    msleep(1000);

    kthread_init_work(&my_def_work_2.my_kthread_work,def_work_fn_2);
    pr_info("kthread_work_2_initialized\n");
    ret_val = kthread_queue_work(my_worker,&my_def_work_2.my_kthread_work);
    if(ret_val)
        pr_info("kthread work 2 is queued to worker\n");
    else
        pr_info("work 2 is pending\n");
    while (!kthread_should_stop()) {
        pr_info("thread execution\n");
        msleep(1000);
    }
    return 0;
}
  • Add the work function API which is executed once the work is queued to worker.

void def_work_fn_1(struct kthread_work *work)
{
    struct my_work *my_work_fn_1;
    my_work_fn_1 = kmalloc(sizeof(struct my_work*),GFP_KERNEL);
    pr_info("Inside work function 1\n");
    my_work_fn_1->i = 0;
    while (my_work_fn_1->i <= 3) {
        pr_info("my_work_fn_1 = %d\n",my_work_fn_1->i);
        my_work_fn_1->i++;
        msleep(100);
    }
}

void def_work_fn_2(struct kthread_work *work)
{
    struct my_work *my_work_fn_2;
    my_work_fn_2 = kmalloc(sizeof(struct my_work*),GFP_KERNEL);
    pr_info("Inside work function 2\n");
    my_work_fn_2->i=3;
    while (my_work_fn_2->i >= 0) {
        pr_info("my_work_fn_2 = %d\n",my_work_fn_2->i);
        my_work_fn_2->i--;
        msleep(100);
    }
}
  • Add the driver entry points which will be executed once the module is inserted or removed from the kernel.

module_init(kthread_queue_work_init);
module_exit(kthread_queue_work_exit);
  1#include <linux/init.h>
  2#include <linux/kernel.h>
  3#include <linux/module.h>
  4#include <linux/kthread.h>
  5#include <linux/delay.h>
  6#include <linux/slab.h>
  7
  8MODULE_LICENSE("GPL");
  9MODULE_AUTHOR("Linux Usr");
 10MODULE_DESCRIPTION("Example of kthread_queue_work");
 11
 12bool ret_val;
 13static struct task_struct *my_thread;
 14static struct kthread_worker *my_worker;
 15struct my_work {
 16	struct kthread_work my_kthread_work;
 17	int i;
 18};
 19
 20/* def_work_fn_1 - executed when work is queued to worker,
 21 * will print a message and sleeps for 100ms,
 22 * will stop when my_work_fn_1 is equal to 3*/
 23
 24void def_work_fn_1(struct kthread_work *work)
 25{
 26	struct my_work *my_work_fn_1;
 27    my_work_fn_1 = kmalloc(sizeof(struct my_work*),GFP_KERNEL);
 28	pr_info("Inside work function 1\n");
 29	my_work_fn_1->i = 0;
 30	while (my_work_fn_1->i <= 3) {
 31		pr_info("my_work_fn_1 = %d\n",my_work_fn_1->i);
 32		my_work_fn_1->i++;
 33        msleep(200);
 34	}
 35}
 36
 37/* def_work_fn_2 - executed when work is queued to worker,
 38 * will print a message and sleeps for 100ms,
 39 * will stop when my_work_fn_2 is equal to 0*/
 40
 41void def_work_fn_2(struct kthread_work *work)
 42{
 43	struct my_work *my_work_fn_2;
 44    my_work_fn_2 = kmalloc(sizeof(struct my_work*),GFP_KERNEL);
 45	pr_info("Inside work function 2\n");
 46	my_work_fn_2->i=3;
 47	while (my_work_fn_2->i >= 0) {
 48		pr_info("my_work_fn_2 = %d\n",my_work_fn_2->i);
 49		my_work_fn_2->i--;
 50        msleep(50);
 51	}
 52}
 53
 54/*my_thread_fun - executed when thread is created,
 55 * creates worker and work, queue created work to worker,
 56 * thread prints a messages and sleeps for 1000ms,
 57 * stops when kthread_stop is called*/
 58
 59int my_thread_fun(void *data)
 60{
 61	struct my_work my_def_work_1;
 62	struct my_work my_def_work_2;
 63	kthread_init_work(&my_def_work_1.my_kthread_work,def_work_fn_1);
 64	pr_info("kthread work 1 intialized\n");
 65	my_worker = kthread_create_worker(0,"kthread_queue_work worker example");
 66	if (IS_ERR(my_worker))
 67		pr_info("error in creating kthread_worker\n");
 68    else
 69	    pr_info("kthread worker created\n");
 70	ret_val = kthread_queue_work(my_worker,&my_def_work_1.my_kthread_work);
 71    if (ret_val)
 72	    pr_info("kthread work 1 is queued to worker\n");
 73    else
 74        pr_info("work 1 is pending\n");
 75    kthread_init_work(&my_def_work_2.my_kthread_work,def_work_fn_2);
 76	pr_info("kthread_work_2_initialized\n");
 77	ret_val = kthread_queue_work(my_worker,&my_def_work_2.my_kthread_work);
 78    if(ret_val)
 79	    pr_info("kthread work 2 is queued to worker\n");
 80    else
 81        pr_info("work 2 is pending\n");
 82    while (!kthread_should_stop()) {
 83        pr_info("thread execution\n");
 84	    msleep(100);
 85    }
 86	return 0;
 87}
 88
 89/* thread_start - creates thread and starts its execution */
 90
 91void thread_start(void)
 92{
 93    my_thread = kthread_run(my_thread_fun,NULL,"kthread_queue_work thread example");
 94    if (IS_ERR(my_thread))
 95        pr_info("Error in creating thread\n");
 96    else
 97        pr_info("Thread created successfully\n");
 98}
 99
100/* kthread_queue_work_init - calls thread_start function,
101 * executes when module is loaded*/
102
103static int __init kthread_queue_work_init(void)
104{
105	pr_info("Inside kthread init function\n");
106	thread_start();
107    return 0;
108}
109
110/* thread_stop - destroys thread,worker and stops its execution */
111
112void thread_stop(void)
113{
114    if (my_thread) {
115        kthread_stop(my_thread);
116        kthread_destroy_worker(my_worker);
117        pr_info("Destroyed threads and worker\n");
118    }
119}
120
121/* kthread_queue_work_exit - calls thread_stop function,
122 * executes when the module is unloaded */
123
124static void __exit kthread_queue_work_exit(void)
125{
126	pr_info("Inside kthread exit function\n");
127    thread_stop();
128}
129
130module_init(kthread_queue_work_init);
131module_exit(kthread_queue_work_exit);
1obj-m += kthread.o
2
3all:
4	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
5
6clean:
7	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
  • Run make to compile the kernel source and generate the .ko image.

make -C /lib/modules/6.7.9/build M=$HOME/kthread_examples/
make[1]: Entering directory '/usr/src/linux-headers-6.7.9'
warning: the compiler differs from the one used to build the kernel
The kernel was built by: x86_64-linux-gnu-gcc (Ubuntu 12.2.0-3ubuntu1) 12.2.0
You are using:           gcc (Ubuntu 12.2.0-3ubuntu1) 12.2.0
CC [M]  $HOME/kthread_examples/kthread.o
MODPOST $HOME/kthread_examples/Module.symvers
CC [M]  $HOME/kthread_examples/kthread.mod.o
LD [M]  $HOME/kthread_examples/kthread.ko
BTF [M] $HOME/kthread_examples/kthread.ko
Skipping BTF generation for $HOME/kthread_examples/kthread.ko due to unavailability of vmlinux
make[1]: Leaving directory '/usr/src/linux-headers-6.7.9'
  • Check if the .ko is generated or not using ls command.

test@test-V520-15IKL:~$ ls -l
total 360
-rw-rw-r-- 1 test test    713 Mar 18 16:06 kthread.c
-rw-rw-r-- 1 test test 169784 Mar 18 16:08 kthread.ko
-rw-rw-r-- 1 test test     58 Mar 18 16:08 kthread.mod
-rw-rw-r-- 1 test test   1047 Mar 18 16:08 kthread.mod.c
-rw-rw-r-- 1 test test  96512 Mar 18 16:08 kthread.mod.o
-rw-rw-r-- 1 test test  74696 Mar 18 16:08 kthread.o
-rw-rw-r-- 1 test test    161 Mar 18 16:00 Makefile
-rw-rw-r-- 1 test test     58 Mar 18 16:08 modules.order
-rw-rw-r-- 1 test test      0 Mar 18 16:08 Module.symvers
  • Run modinfo command to get the information about the kernel module.

test@test-V520-15IKL:~/.../tc_1$ modinfo kthread.ko
filename:       $HOME/kthread_examples/kthread.ko
description:    Example of kthread_queue_work
author:         Linux Usr
license:        GPL
srcversion:     8D2147F67AB01CF0E482DAC
depends:
retpoline:      Y
name:           kthread
vermagic:       6.7.9 SMP preempt mod_unload modversions
  • insert the module using insmod command.

test@test-V520-15IKL:~$ sudo insmod ./kthread.ko
  • check if the module is loaded or not using lsmod command.

test@test-V520-15IKL:~$ sudo lsmod | grep kthread
kthread                16384  0
  • check if the thread is created or not using ps command.

test@test-V520-15IKL:~$ ps -N | grep kthread_queue_work
6772 ?        00:00:00 kthread_queue_work thread example
6773 ?        00:00:00 kthread_queue_work worker example
  • check for the kernel messages from init function and thread function once the module is loaded and thread is created.

test@test-V520-15IKL:~$ sudo dmesg
[  422.559136] Inside kthread init function
[  422.559212] Thread created successfully
[  422.559214] kthread work 1 intialized
[  422.559231] kthread worker created
[  422.559232] kthread work 1 is queued to worker
[  422.559234] Inside work function 1
[  422.559235] my_work_fn_1 = 0
[  422.665290] my_work_fn_1 = 1
[  422.773245] my_work_fn_1 = 2
[  422.881178] my_work_fn_1 = 3
[  423.593158] kthread_work_2_initialized
[  423.593169] kthread work 2 is queued to worker
[  423.593171] thread execution
[  423.593267] Inside work function 2
[  423.593271] my_work_fn_2 = 3
[  423.701065] my_work_fn_2 = 2
[  423.809081] my_work_fn_2 = 1
[  423.917107] my_work_fn_2 = 0
  • remove the module from kernel using rmmod command.

test@test-V520-15IKL:~$ sudo rmmod kthread
  • check if the module is still loaded after removing the kernel module using lsmod if it is not displayed in lsmod output it is verified that the module is removed successfully.

test@test-V520-15IKL:~$ sudo lsmod | grep kthread
test@test-V520-15IKL:~$
  • check if the thread is destroyed using ps command if it is not displayed in ps output we can confirm that the thread is destroyed successfully.

test@test-V520-15IKL:~$ ps -N | grep kthread_queue_work
test@test-V520-15IKL:~$
  • Check for kernel messages from exit function using dmesg command.

test@test-V520-15IKL:~$ sudo dmesg
[  431.502097] Inside kthread exit function
[  431.784486] Destroyed threads and worker

kthread API

Learning

kthread_create_worker

Create kthread worker

kthread_init_work

Create kthread work

kthread_queue_work

Queue work to worker

kthread_cancel_work_sync

Cancel the kthread work and wait for it to finish

kthread_run

Create and wake a thread

kthread_destroy_worker

Destroy kthread worker

kthread_should_stop

Should kthread return or not

API

Learning

MODULE_LICENSE

Used to denote the license used in the kernel module

MODULE_AUTHOR

Used to mention the author of the kernel module

MODULE_DESCRIPTION

Used to describe what the module does

IS_ERR

Detects an error pointer

container_of

cast a member of the structure to the containing structure

kmalloc

Allocate kernel memory

wake_up_process

wake up a specific process

msleep

will put in sleep for a certain amount of msecs time.

module_init

Driver initialization entry point

module_exit

Driver exit entry point

pr_info

Print an info-level message