feat: preemption between schedulers by kobayu858 · Pull Request #266 · tier4/awkernel

kobayu858 · 2025-01-08T07:01:41Z

Implementation of inter-scheduler preemption for priority inversion.
Specifically, re-transmission of IPI, limitation of the period during which task status can be obtained, etc. were added.
Details are given below.

Test

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

awkernel_async_lib/src/scheduler.rs

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

awkernel_async_lib/src/task.rs

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

awkernel_async_lib/src/task.rs

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

Koichi98 · 2025-01-09T08:53:28Z

awkernel_async_lib/src/scheduler.rs

+    fn invoke_preemption(&self, wake_task_priority: PriorityInfo) {
+        while let Some((task_id, cpu_id, lowest_priority_info)) = get_lowest_priority_task_info() {
+            if wake_task_priority < lowest_priority_info {
+                if IS_SEND_IPI.load(Ordering::Relaxed) {


You may have to use compare_exchange() here.
Checking and storing the value should be atomic.

Between executing get_lowest_priority_task_info and checking the IS_SEND_IPI value, is there a possibility that preemption could occur on another CPU, changing the lowest_priority task, and even completing the process of setting IS_SEND_IPI back to false?

To help me understand, could you answer the following question for both before and after merging this PR?

"Can a CPU receive and handle an IPI from another CPU while it's in invoke_preemption(), and then resume its original execution?"

I noticed that interrupts during lock would cause a deadlock or if invoke_preemption occurs during the task termination process, priority inversion may occur, so I changed the implementation.
Details are summarized in Doc and Doc

You may have to use compare_exchange() here.
Checking and storing the value should be atomic.

Correction. Thank you very much.

Between executing get_lowest_priority_task_info and checking the IS_SEND_IPI value, is there a possibility that preemption could occur on another CPU, changing the lowest_priority task, and even completing the process of setting IS_SEND_IPI back to false?

Yes. See Doc for how to deal with this after the implementation change.

To help me understand, could you answer the following question for both before and after merging this PR?

"Can a CPU receive and handle an IPI from another CPU while it's in invoke_preemption(), and then resume its original execution?"

As mentioned earlier, interrupts during locking cause deadlocks, so no work properly after the merge.
Before merging, the GEDF data was locked, so no interruptions occurred during invoke_preemption(), and there were no issues.

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

atsushi421 · 2025-01-19T23:10:05Z

awkernel_async_lib/src/task.rs

+            let mut is_send_ipi = IS_SEND_IPI.lock(&mut node);
+            if *is_send_ipi {
+                continue;
+            }
+            *is_send_ipi = true;


You can use AtomicBool::compare_exchange().
https://doc.rust-lang.org/std/sync/atomic/struct.AtomicBool.html#method.compare_exchange

Correction. Thank you very much.

atsushi421 · 2025-01-19T23:37:17Z

@kobayu858
For readability, could you change the NOT_IN_TRANSITION to the IN_TRANSITION (affirmative form)?
If you rename them, you will also need to reverse their bool values.

atsushi421 · 2025-01-20T00:10:39Z

awkernel_async_lib/src/task.rs

-        let mut node = MCSNode::new();
-        let tasks = TASKS.lock(&mut node);
-        running_tasks
+            if true_count >= non_primary_cpus {


Suggested change

if true_count >= non_primary_cpus {

if true_count == non_primary_cpus {

Correction. Thank you very much.

atsushi421 · 2025-01-20T00:18:39Z

awkernel_async_lib/src/task.rs

-            .is_none_or(|(_, _, lowest_priority_info)| priority_info > *lowest_priority_info)
-        {
-            lowest_task = Some((task_id, cpu_id, priority_info));
+        if running_tasks.is_empty() || running_tasks.len() != non_primary_cpus {


Suggested change

if running_tasks.is_empty() || running_tasks.len() != non_primary_cpus {

if running_tasks.len() < non_primary_cpus {

running_tasks.is_empty() is required to pass make test.
Because at make tast, there is only one core, and running_tasks and non_primary_cpus seem to be set to 0.
Therefore, the above fix will cause an infinite loop during make test.

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

kobayu858 added 12 commits December 27, 2024 17:33

feat:sched_preempt

0cfd74b

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

feat:check num ipi

c73e53a

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

feat:check preempt done

d9baee0

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

fix: cargo fmt

e1accb0

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

feat: wait until all task statuses are ready to load

8340bed

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

feat: wait until all task statuses are ready to load

980f87a

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

refactor: variable name

444a983

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

feat: support for Interruptions

9425d6e

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

fix: revert white line

beb0c0e

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

fix: add import

a2a651d

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

fix: remove from

ad0822e

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

fix: remove from

f4907b9

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

Koichi98 requested review from Koichi98, atsushi421 and ytakano January 8, 2025 07:33

refactor: IS_SEND_IPI

1a3af6f

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

Koichi98 reviewed Jan 8, 2025

View reviewed changes

awkernel_async_lib/src/scheduler.rs Outdated Show resolved Hide resolved

docs: fix comment

6ac8ee4

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

kobayu858 requested a review from Koichi98 January 8, 2025 08:04

Koichi98 reviewed Jan 8, 2025

View reviewed changes

awkernel_async_lib/src/task.rs Outdated Show resolved Hide resolved

Koichi98 reviewed Jan 8, 2025

View reviewed changes

awkernel_async_lib/src/task.rs Outdated Show resolved Hide resolved

atsushi421 reviewed Jan 8, 2025

View reviewed changes

awkernel_async_lib/src/task.rs Outdated Show resolved Hide resolved

Koichi98 reviewed Jan 8, 2025

View reviewed changes

awkernel_async_lib/src/task.rs Outdated Show resolved Hide resolved

kobayu858 added 5 commits January 8, 2025 18:00

fix: IS_LOAD_RUNNING

9146636

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

fix: get_lowest_priority_task_info

e3f05e3

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

fix: get_lowest_priority_task_info

70c015e

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

refactor: where to get preempt_irq

52ee165

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

refactor:rename NOT_IN_TRANSITION

25d703b

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

kobayu858 requested a review from atsushi421 January 8, 2025 11:10

atsushi421 reviewed Jan 8, 2025

View reviewed changes

awkernel_async_lib/src/task.rs Show resolved Hide resolved

kobayu858 added 3 commits January 8, 2025 21:00

refactor: check preemption trigger

cefb7fd

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

refactor: where to init lowest_task

3ead65f

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

fix: cargo fmt

85ffac0

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

kobayu858 requested a review from atsushi421 January 8, 2025 12:07

fix: build error

7cf1a43

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

kobayu858 requested a review from Koichi98 January 8, 2025 12:46

kobayu858 added 3 commits January 8, 2025 22:00

fix: make test

793372f

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

fix: cargo fmt

b2bf365

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

fix: clippy

70ac8af

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

Koichi98 reviewed Jan 9, 2025

View reviewed changes

kobayu858 added 2 commits January 14, 2025 19:12

fix: preemption

23fa42e

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

fix: revert

e8e2006

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

kobayu858 requested a review from Koichi98 January 14, 2025 10:28

refactor: deleted unuse variable

4496ac8

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

atsushi421 reviewed Jan 19, 2025

View reviewed changes

atsushi421 reviewed Jan 20, 2025

View reviewed changes

kobayu858 added 2 commits January 20, 2025 10:52

refactor: minor corrections

84a51a0

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

docs: add explain

d5b1cbe

Signed-off-by: kobayu858 <yutaro.kobayashi@tier4.jp>

ytakano closed this Feb 10, 2025

kobayu858 deleted the feat_sched_preempt branch February 3, 2026 08:32

	if true_count >= non_primary_cpus {
	if true_count == non_primary_cpus {

	if running_tasks.is_empty() \|\| running_tasks.len() != non_primary_cpus {
	if running_tasks.len() < non_primary_cpus {

Comments

Conversation

kobayu858 commented Jan 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kobayu858 Jan 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kobayu858 Jan 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kobayu858 Jan 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

atsushi421 commented Jan 19, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kobayu858 commented Jan 8, 2025 •

edited

Loading

kobayu858 Jan 14, 2025 •

edited

Loading

kobayu858 Jan 14, 2025 •

edited

Loading

kobayu858 Jan 14, 2025 •

edited

Loading