IO wait for completion
# |
Version |
---|---|
Ubuntu |
Ubuntu 22.10 |
Kernel |
6.7.9 |
In this program, you are going to learn
How to create completion variable ?
How to send completion signal for io waiting threads?
How to signal once the task is completed?
How to use Kthread APIs ?
How to use Completion APIs ?
How to use below APIs ?
Here is the function prototype of the API: kthread_should_stop
#include <linux/kthread.h>
bool kthread_should_stop(void);
where
kthread_should_stop: this is used to determine whether the thread should return now. Whenever the kthread_stop() is called it will be woken and returns true.
return type: returns true when the thread is stopped, false when the thread is still in execution
Here is an example of how to use the API,
while (!kthread_should_stop())
{
//add the instructions to be performed during thread execution.
}
Here is the function prototype of the API: kthread_run
#include <linux/kthread.h>
#define kthread_run(threadfn, data, namefmt, ...)
where
kthread_run : is used to create and wake a kthread
return type: struct task_struct* (i.e) address of the kthread
threadfn: function to executed by kthread
data: data pointer for threadfn which can be used to send any possible arguments required for threadfn.
namefmt: printf-style format for the thread name which will be displayed on ps output when the thread is in execution.
Here is an example of how to use the API,
kthread_run(mythread,NULL,"sample kthread");
Here is the function prototype of the API: kthread_stop
#include <linux/kthread.h>
int kthread_stop(struct task_struct *k);
where
kthread_stop: stops a kthread.
return type: returns the return value of threadfn() which is passed as an argument during kthread creation.
k: kthread created by one of the API used to created kthread.
Here is the example of how to use the API,
kthread_stop(mythread);
Here is the prototype of the API: init_completion
#include <linux/completion.h>
static inline void init_completion(struct completion *x);
where
init_completion : initialize a dynamically allocated completion
x : pointer to completion structure that is to be initialized
Here is the example of how to use the API
init_completion(mycompletionvar);
Here is the prototype of the API: wait_for_completion_io
#include <linux/completion.h>
void wait_for_completion_io(struct completion *x);
where
wait_for_completion_io : waits for completion of a task. It is NOT interruptible and there is no timeout. The caller is accounted as waiting for IO
x : holds the state of this particular completion
Here is an example of how to use the API,
wait_for_completion_io(mycompletionvar);
Here is the prototype of the API: wait_for_completion_io_timeout
#include <linux/completion.h>
extern unsigned long wait_for_completion_io_timeout(struct completion *x, unsigned long timeout);
where
wait_for_completion_io_timeout : this waits for either a completion of a specific task to be signaled or for a specified timeout to expire. The timeout is in jiffies. It is not interruptible. The caller is accounted as waiting for IO
x : holds the state of this particular completion
timeout : timeout value in jiffies
Here is an example of how to use the API
wait_for_completion_io_timeout(mycompvar,msecs_to_jiffies(10));
Here is the prototype of module paramter APIs
#include <linux/module.h>
#define MODULE_LICENSE(_license) MODULE_FILE MODULE_INFO(license, _license)
#define MODULE_AUTHOR(_author) MODULE_INFO(author, _author)
#define MODULE_DESCRIPTION(_description) MODULE_INFO(description, _description)
where
MODULE_LICENSE: tells the kernel what license is used by our module.
MODULE_AUTHOR: denotes the author of this kernel module.
MODULE_DESCRIPTION: gives a basic idea about what the kernel module does.
These information can be found when modinfo command is used which lists out all these above mentioned information.
Here is the example of how to use the Module parameter APIs,
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Linux Usr");
MODULE_DESCRIPTION("Sample kernel module");
Here is the prototype of the API: IS_ERR
#include <linux/err.h>
static inline bool __must_check IS_ERR(__force const void *ptr);
where
IS_ERR: Detects an error pointer
return type: return true if it’s an error pointer else return false.
ptr: pointer which needs to detected.
Here is an example of how to use the API,
if(IS_ERR(sample_ptr)) {
//instructions to be executed when error ptr is detected
} else {
//instructions to be executed when error ptr is not detected
}
Here is the example of the API: wake_up_process
#include <linux/sched.h>
extern int wake_up_process(struct task_struct *tsk);
where
wake_up_process: wake up a specific process
return type: returns 1 if the process is woken up, 0 if the process is in running state.
tsk: process to be woken up.
Here is the example of how to use the API,
wake_up_process(mythread);
Here is the prototype of the Driver entry point API’s
#include <linux/module.h>
#define module_init(x) __initcall(x);
#define module_exit(x) __exitcall(x);
where
module_init: driver initialization entry point which will be called at module insertion time.
module_exit: driver exit entry point which will be called during the removal of module.
- x:
function to be run at module insertion for module_init function.
function to be run when driver is removed for module_exit function.
Here is an example of how to use the driver entry point API’s
module_init(myinitmodule);
module_exit(myexitmodule);
In this example let’s see how to wait for completion signal using wait_for_completion_io and use with kthread.
Include the follow header files(.h) to refer the API being used for the execution.
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kthread.h>
#include <linux/completion.h>
#include <linux/delay.h>
Add the following module macros to display information about the license, author and description about the module.
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Linux Usr");
MODULE_DESCRIPTION("Example of wait_for_io_completion");
Declare the thread and completion variables which we are going to create and use in this example
static struct task_struct *my_thread1;
static struct task_struct *my_thread2;
struct completion thread_done;
Add the module init function which will be executed once we load the kernel module using insmod command.
static int __init wait_for_io_completion_init(void)
{
pr_info("Inside thread init function\n");
thread_start();
return 0;
}
Add the thread start function which is called from module init function, creates the thread and starts it’s execution.
void thread_start(void)
{
init_completion(&thread_done);
my_thread1 = kthread_run(my_thread_fn1,NULL,"wait_for_io_completion thread1 example");
if (IS_ERR(my_thread1))
pr_info("error creating in kthread\n");
else
pr_info("successfully created kthread\n");
my_thread2 = kthread_run(my_thread_fn2,NULL,"wait_for_io_completion thread2 example");
if (IS_ERR(my_thread2))
pr_info("error creating in kthread\n");
else
pr_info("successfully created kthread\n");
}
Add the module exit function which will be executed once we unload the kernel module using rmmod command.
static void __exit wait_for_io_completion_exit(void)
{
pr_info("Inside kthread exit function\n");
thread_stop();
}
Add the thread stop function which is called from module exit function, destroys the thread created and stops it’s execution.
void thread_stop(void)
{
kthread_stop(my_thread2);
kthread_stop(my_thread1);
pr_info("destroyed thread and completed task\n");
}
Add the thread function API which will be called as soon as the kthread is created and is in running state.
static int my_thread_fn1(void *data)
{
while (!kthread_should_stop()) {
wait_for_completion_io(&thread_done);
pr_info("my_thread_fn1 execution\n");
msleep(1000);
}
return 0;
}
static int my_thread_fn2(void *data)
{
while(!kthread_should_stop()) {
pr_info("my_thread_fn2 execution\n");
msleep(1000);
}
complete(&thread_done);
return 0;
}
Add the driver entry points which will be executed once the module is inserted or removed from the kernel.
module_init(wait_for_io_completion_init);
module_exit(wait_for_io_completion_exit);
1#include <linux/kernel.h>
2#include <linux/init.h>
3#include <linux/module.h>
4#include <linux/kthread.h>
5#include <linux/completion.h>
6#include <linux/delay.h>
7
8MODULE_LICENSE("GPL");
9MODULE_DESCRIPTION("Example of wait_for_io_completion");
10MODULE_AUTHOR("linux usr");
11
12static struct task_struct *my_thread1;
13static struct task_struct *my_thread2;
14struct completion thread_done;
15
16/* my_thread_fn1 - executes when thread1 created,
17 * wait for completion signal from thread2,
18 * prints a message and sleeps for 1000ms,
19 * stops when kthread_stop is called */
20
21static int my_thread_fn1(void *data)
22{
23 while (!kthread_should_stop()) {
24 wait_for_completion_io(&thread_done);
25 pr_info("my_thread_fn1 execution\n");
26 msleep(1000);
27 }
28
29 return 0;
30}
31
32/* my_thread_fn2 - executes when thread2 created,
33 * sends completion signal to thread1,
34 * prints a message and sleeps for 1000ms,
35 * stops when kthread_stop is called */
36
37static int my_thread_fn2(void *data)
38{
39 while(!kthread_should_stop()) {
40 pr_info("my_thread_fn2 execution\n");
41 msleep(1000);
42 }
43 complete(&thread_done);
44 return 0;
45}
46
47/* thread_start - creates thread and starts execution */
48
49void thread_start(void)
50{
51 init_completion(&thread_done);
52 my_thread1 = kthread_run(my_thread_fn1,NULL,"wait_for_io_completion thread1 example");
53 if (IS_ERR(my_thread1))
54 pr_info("error creating in kthread\n");
55 else
56 pr_info("successfully created kthread\n");
57 my_thread2 = kthread_run(my_thread_fn2,NULL,"wait_for_io_completion thread2 example");
58 if (IS_ERR(my_thread2))
59 pr_info("error creating in kthread\n");
60 else
61 pr_info("successfully created kthread\n");
62}
63
64/* wait_for_io_completion_init - calls thread_start,
65 * executes when module is loaded */
66
67static int __init wait_for_io_completion_init(void)
68{
69 pr_info("inside thread init function\n");
70 thread_start();
71 return 0;
72}
73
74/* thread_stop - destroys thread and stops execution */
75
76void thread_stop(void)
77{
78 kthread_stop(my_thread2);
79 kthread_stop(my_thread1);
80 pr_info("destroyed thread and completed task\n");
81}
82
83/* wait_for_io_completion_exit - calls thread_stop,
84 * executes when module is unloaded */
85
86static void __exit wait_for_io_completion_exit(void)
87{
88 pr_info("inside kthread exit function\n");
89 thread_stop();
90}
91
92module_init(wait_for_io_completion_init);
93module_exit(wait_for_io_completion_exit);
1obj-m += kthread.o
2
3all:
4 make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
5
6clean:
7 make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
Run make to compile the kernel source and generate the .ko image.
make -C /lib/modules/6.7.9/build M=$HOME/kthread_examples/
make[1]: Entering directory '/usr/src/linux-headers-6.7.9'
warning: the compiler differs from the one used to build the kernel
The kernel was built by: x86_64-linux-gnu-gcc (Ubuntu 12.2.0-3ubuntu1) 12.2.0
You are using: gcc (Ubuntu 12.2.0-3ubuntu1) 12.2.0
CC [M] $HOME/kthread_examples/kthread.o
MODPOST $HOME/kthread_examples/Module.symvers
CC [M] $HOME/kthread_examples/kthread.mod.o
LD [M] $HOME/kthread_examples/kthread.ko
BTF [M] $HOME/kthread_examples/kthread.ko
Skipping BTF generation for $HOME/kthread_examples/kthread.ko due to unavailability of vmlinux
make[1]: Leaving directory '/usr/src/linux-headers-6.7.9'
Check if the .ko is generated or not using ls command.
test@test-V520-15IKL:~$ ls -l
total 360
-rw-rw-r-- 1 test test 713 Mar 18 16:06 kthread.c
-rw-rw-r-- 1 test test 169784 Mar 18 16:08 kthread.ko
-rw-rw-r-- 1 test test 58 Mar 18 16:08 kthread.mod
-rw-rw-r-- 1 test test 1047 Mar 18 16:08 kthread.mod.c
-rw-rw-r-- 1 test test 96512 Mar 18 16:08 kthread.mod.o
-rw-rw-r-- 1 test test 74696 Mar 18 16:08 kthread.o
-rw-rw-r-- 1 test test 161 Mar 18 16:00 Makefile
-rw-rw-r-- 1 test test 58 Mar 18 16:08 modules.order
-rw-rw-r-- 1 test test 0 Mar 18 16:08 Module.symvers
Run modinfo command to get the information about the kernel module.
test@test-V520-15IKL:~/.../tc_1$ modinfo kthread.ko
filename: $HOME/kthread_examples/kthread.ko
description: Example of wait_for_io_completion
author: Linux Usr
license: GPL
srcversion: 8D2147F67AB01CF0E482DAC
depends:
retpoline: Y
name: kthread
vermagic: 6.7.9 SMP preempt mod_unload modversions
insert the module using insmod command.
test@test-V520-15IKL:~$ sudo insmod ./kthread.ko
check if the module is loaded or not using lsmod command.
test@test-V520-15IKL:~$ sudo lsmod | grep kthread
kthread 16384 0
check if the thread is created or not using ps command.
test@test-V520-15IKL:~$ ps -N | grep wait_for_io_completion
14979 ? 00:00:00 wait_for_io_completion thread1 example
14980 ? 00:00:00 wait_for_io_completion thread2 example
check for the kernel messages from init function and thread function once the module is loaded and thread is created.
test@test-V520-15IKL:~$ sudo dmesg
[11615.081970] inside thread init function
[11615.081996] successfully created kthread
[11615.082011] successfully created kthread
[11615.082013] my_thread_fn2 execution
[11616.091512] my_thread_fn2 execution
[11617.115541] my_thread_fn2 execution
remove the module from kernel using rmmod command.
test@test-V520-15IKL:~$ sudo rmmod kthread
check if the module is still loaded after removing the kernel module using lsmod if it is not displayed in lsmod output it is verified that the module is removed successfully.
test@test-V520-15IKL:~$ sudo lsmod | grep kthread
test@test-V520-15IKL:~$
check if the thread is destroyed using ps command if it is not displayed in ps output we can confirm that the thread is destroyed successfully.
test@test-V520-15IKL:~$ ps -N | grep wait_for_io_completion
test@test-V520-15IKL:~$
Check for kernel messages from exit function using dmesg command.
test@test-V520-15IKL:~$ sudo dmesg
[11627.908766] inside kthread exit function
[11628.379925] my_thread_fn1 execution
[11629.403996] destroyed thread and completed task
In this example let’s see kthread waiting for completion using wait_for_io_completion_timeout.
Include the follow header files(.h) to refer the API being used for the execution.
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kthread.h>
#include <linux/completion.h>
#include <linux/delay.h>
#include <linux/jiffies.h>
Add the following module macros to display information about the license, author and description about the module.
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Linux Usr");
MODULE_DESCRIPTION("example of wait_for_completion_io_timeout");
Declare the thread variables which we are going to create and use in this example
static struct task_struct *my_thread1;
static struct task_struct *my_thread2;
struct completion thread_done;
Add the module init function which will be executed once we load the kernel module using insmod command.
static int __init wait_for_completion_timeout_init(void)
{
pr_info("Inside kthread init function\n");
thread_start();
return 0;
}
Add the thread start function called from the module init function which is used to create the thread and execute it.
void thread_start(void)
{
init_completion(&thread_done);
my_thread1 = kthread_run(my_thread_fn1,NULL,"wait_for_completion_io_timeout thread1 example");
if (IS_ERR(my_thread1))
pr_info("error creating in kthread\n");
else
pr_info("successfully created kthread\n");
my_thread2 = kthread_run(my_thread_fn2,NULL,"wait_for_completion_io_timeout thread2 example");
if (IS_ERR(my_thread2))
pr_info("error creating in kthread\n");
else
pr_info("successfully created kthread\n");
}
Add the module exit function which will be executed once we unload the kernel module using rmmod command.
static void __exit wait_for_completion_io_timeout_exit(void)
{
pr_info("Inside kthread exit function\n");
thread_stop();
}
Add the thread stop function called from the module exit function which is used to destroy the thread and stop its execution.
void thread_stop(void)
{
kthread_stop(my_thread2);
kthread_stop(my_thread1);
pr_info("destroyed thread and completed task\n");
}
Add the thread function API which will be called as soon as the kthread is created and is in running state.
static int my_thread_fn1(void *data)
{
while (!kthread_should_stop()) {
wait_for_completion_io_timeout(&thread_done,msecs_to_jiffies(100));
pr_info("my_thread_fn1 execution\n");
msleep(1000);
}
return 0;
}
static int my_thread_fn2(void *data)
{
while(!kthread_should_stop()) {
pr_info("my_thread_fn2 execution\n");
msleep(1000);
}
complete(&thread_done);
return 0;
}
Add the driver entry points which will be executed once the module is inserted or removed from the kernel.
module_init(wait_for_completion_io_timeout_init);
module_exit(wait_for_completion_io_timeout_exit);
1#include <linux/kernel.h>
2#include <linux/init.h>
3#include <linux/module.h>
4#include <linux/kthread.h>
5#include <linux/completion.h>
6#include <linux/delay.h>
7#include <linux/jiffies.h>
8
9MODULE_LICENSE("GPL");
10MODULE_DESCRIPTION("Example of wait_for_completion_io_timeout");
11MODULE_AUTHOR("linux usr");
12
13static struct task_struct *my_thread1;
14static struct task_struct *my_thread2;
15struct completion thread_done;
16
17/* my_thread_fn1 - executes when thread1 is created,
18 * waits for completion signal from thread2,
19 * prints a message and sleeps for 1000ms,
20 * stops when kthread_stop is called */
21
22static int my_thread_fn1(void *data)
23{
24 while (!kthread_should_stop()) {
25 wait_for_completion_io_timeout(&thread_done,msecs_to_jiffies(1));
26 pr_info("my_thread_fn1 execution\n");
27 msleep(1000);
28 }
29 return 0;
30}
31
32/* my_thread_2 - executes when thread2 is created,
33 * prints a message and sleeps for 1000ms,
34 * sends completion signal to thread1,
35 * stops when kthread_stop is called */
36
37static int my_thread_fn2(void *data)
38{
39 while(!kthread_should_stop()) {
40 pr_info("my_thread_fn2 execution\n");
41 msleep(1000);
42 }
43 complete(&thread_done);
44 return 0;
45}
46
47/* thread_start - creates thread and starts execution */
48
49void thread_start(void)
50{
51 init_completion(&thread_done);
52 my_thread1 = kthread_run(my_thread_fn1,NULL,"wait_for_completion_io_timeout thread1 example");
53 if (IS_ERR(my_thread1))
54 pr_info("error creating in kthread\n");
55 else
56 pr_info("successfully created kthread\n");
57 my_thread2 = kthread_run(my_thread_fn2,NULL,"wait_for_completion_io_timeout thread2 example");
58 if (IS_ERR(my_thread2))
59 pr_info("error creating in kthread\n");
60 else
61 pr_info("successfully created kthread\n");
62}
63
64/* wait_for_completion_io_timeout_init_init_init_init_init_init_init_init_init - calls thread_start,
65 * executes when module is loaded */
66
67static int __init wait_for_completion_io_timeout_init(void)
68{
69 pr_info("inside thread init function\n");
70 thread_start();
71 return 0;
72}
73
74/* thread_stop - destroys thread and stops execution */
75
76void thread_stop(void)
77{
78 kthread_stop(my_thread2);
79 kthread_stop(my_thread1);
80 pr_info("destroyed thread and completed task\n");
81}
82
83/* wait_for_completion_io_timeout_exit - calls thread_stop,
84 * executes when module is unloaded */
85
86static void __exit wait_for_completion_io_timeout_exit(void)
87{
88 pr_info("inside thread exit function\n");
89 thread_stop();
90}
91
92module_init(wait_for_completion_io_timeout_init);
93module_exit(wait_for_completion_io_timeout_exit);
1obj-m += kthread.o
2
3all:
4 make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
5
6clean:
7 make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
Run make to compile the kernel source and generate the .ko image.
make -C /lib/modules/6.7.9/build M=$HOME/kthread_examples/
make[1]: Entering directory '/usr/src/linux-headers-6.7.9'
warning: the compiler differs from the one used to build the kernel
The kernel was built by: x86_64-linux-gnu-gcc (Ubuntu 12.2.0-3ubuntu1) 12.2.0
You are using: gcc (Ubuntu 12.2.0-3ubuntu1) 12.2.0
CC [M] $HOME/kthread_examples/kthread.o
MODPOST $HOME/kthread_examples/Module.symvers
CC [M] $HOME/kthread_examples/kthread.mod.o
LD [M] $HOME/kthread_examples/kthread.ko
BTF [M] $HOME/kthread_examples/kthread.ko
Skipping BTF generation for $HOME/kthread_examples/kthread.ko due to unavailability of vmlinux
make[1]: Leaving directory '/usr/src/linux-headers-6.7.9'
Check if the .ko is generated or not using ls command.
test@test-V520-15IKL:~$ ls -l
total 360
-rw-rw-r-- 1 test test 713 Mar 18 16:06 kthread.c
-rw-rw-r-- 1 test test 169784 Mar 18 16:08 kthread.ko
-rw-rw-r-- 1 test test 58 Mar 18 16:08 kthread.mod
-rw-rw-r-- 1 test test 1047 Mar 18 16:08 kthread.mod.c
-rw-rw-r-- 1 test test 96512 Mar 18 16:08 kthread.mod.o
-rw-rw-r-- 1 test test 74696 Mar 18 16:08 kthread.o
-rw-rw-r-- 1 test test 161 Mar 18 16:00 Makefile
-rw-rw-r-- 1 test test 58 Mar 18 16:08 modules.order
-rw-rw-r-- 1 test test 0 Mar 18 16:08 Module.symvers
Run modinfo command to get the information about the kernel module.
test@test-V520-15IKL:~/.../tc_1$ modinfo kthread.ko
filename: $HOME/kthread_examples/kthread.ko
description: Example of wait_for_completion_io_timeout
author: Linux Usr
license: GPL
srcversion: 8D2147F67AB01CF0E482DAC
depends:
retpoline: Y
name: kthread
vermagic: 6.7.9 SMP preempt mod_unload modversions
insert the module using insmod command.
test@test-V520-15IKL:~$ sudo insmod ./kthread.ko
check if the module is loaded or not using lsmod command.
test@test-V520-15IKL:~$ sudo lsmod | grep kthread
kthread 16384 0
check if the thread is created or not using ps command.
test@test-V520-15IKL:~$ ps -N | grep wait_for_completion_io_timeout
17390 ? 00:00:00 wait_for_completion_io_timeout thread1 example
17391 ? 00:00:00 wait_for_completion_io_timeout thread2 example
check for the kernel messages from init function and thread function once the module is loaded and thread is created.
test@test-V520-15IKL:~$ sudo dmesg
[13836.567009] inside thread init function
[13836.567065] successfully created kthread
[13836.567080] successfully created kthread
[13836.567081] my_thread_fn2 execution
[13836.572332] my_thread_fn1 execution
[13837.600487] my_thread_fn2 execution
[13837.608377] my_thread_fn1 execution
remove the module from kernel using rmmod command.
test@test-V520-15IKL:~$ sudo rmmod kthread
check if the module is still loaded after removing the kernel module using lsmod if it is not displayed in lsmod output it is verified that the module is removed successfully.
test@test-V520-15IKL:~$ sudo lsmod | grep kthread
test@test-V520-15IKL:~$
check if the thread is destroyed using ps command if it is not displayed in ps output we can confirm that the thread is destroyed successfully.
test@test-V520-15IKL:~$ ps -N | grep wait_for_completion_io_timeout
test@test-V520-15IKL:~$
Check for kernel messages from exit function using dmesg command.
test@test-V520-15IKL:~$ sudo dmesg
[13855.735410] inside thread exit function
[13856.032716] my_thread_fn1 execution
[13857.056817] destroyed thread and completed task
kthread API |
Learning |
---|---|
kthread_run |
Create and wake a thread |
kthread_should_stop |
To determine when thread should exit |
kthread_stop |
Stop a thread created by kthread_create |
Completion API |
Learning |
---|---|
init_completion |
Initialize completion variable |
complete |
signals a signal thread waiting on this completion |
wait_for_io_completion |
waits for completion of task |
wait_for_completion_io_timeout |
waits for completion of task or waits until the timer expires |
API |
Learning |
---|---|
MODULE_LICENSE |
Used to denote the license used in the kernel module |
MODULE_AUTHOR |
Used to mention the author of the kernel module |
MODULE_DESCRIPTION |
Used to describe what the module does |
IS_ERR |
Detects an error pointer |
wake_up_process |
wake up a specific process |
msleep |
will put in sleep for a certain amount of msecs time. |
module_init |
Driver initialization entry point |
module_exit |
Driver exit entry point |
pr_info |
Print an info-level message |