The most important part is the embassy_executor::main macro. So we will focus on it.
Embassy_executor::main Macro Summary
When the main func begins to run, the #[embassy_executor::main] will :
create an Executor
initializing
create a Spawner
create a TaskPool for the executor
spawn main task
“poll” the Executor
set up the alarm
wake task in time queue
poll the ready task
set the expiration of alarm
Wake up the task and the Executor
about the waker
it’s code is :
from file—embassy-executor/src/arch/cortex_m.rs:54
and this procedural macro define here :
from file—embassy-executor-macros/src/lib.rs:72
and the most important code is below:
from file—embassy-executor/src/arch/cortex_m.rs:99
This is a func of the Executor.
Create an Executor
create an Executor is nothing special, for the initializing is done in run .
the struct code is below(call new() to create Executor).
from file embassy-executor/src/arch/cortex_m.rs
Initializing
In func run (from code above) ,there is a func init ,which take duty to initialize anything such as the Spawner, the TaskPool, the Task and so on. It is called like this:
Init the Spawner
explain the init(self.inner.spawner());
Actually this is the parameter of init. The Spawner owns to main task to “spawn” new task
the code is:
and go into Spawner::new
Create the TaskPool
This func’s feature is to alloc a TaskPool. The size of the TaskPool is due to a static var-ARENA . In Embassy, it is defined as TASK_ARENA_SIZE , which is 4096 (default).static ARENA: Arena<{ crate::config::TASK_ARENA_SIZE }> = Arena::new();
the type: Arena is:
the TASK_AREAN_SIZE config setting’s description is in :
you can change it to 1024. As an example, our setting in cargo.toml is(note the "task-arena-size-1024") :
The TaskPoll is an array wrapped by a struct:
in file — embassy-executor/src/raw/mod.rs:250
And the TaskStorage is a struct which store the task(future)
Spawn the Main Task
From my perspective, this is the most important part in initializing.
from file—embassy-executor/src/raw/mod.rs:197
to be clear, this func is called by spawn_impl which we will explain in a quote:
from file—embassy-executor/src/raw/mod.rs:265
This is the func of AvailableTask . In this func, the parameter future actually is our main func.
and the TaskStorage::<F>poll is a function pointer which point to the func:
from file — embassy-executor/src/raw/mod.rs:153
💡
AvailableTask is claimed by func AvailableTask::claim ,which just iter the TaskPool and try to claim a TaskStorage ,the inner of AvailableTask . TaskStorage contains a TaskHeader and a future. In my view, it just like the Control Block of main task.
the func AvailableTask::claim is ：
and the iteration process is done in spawn_impl(self.pool.iter().find_map(AvailableTask::claim) ):
To understanding the code above, I recommend you to learn how find_map() works.
and to further explore, this func spawn_impl which call the important function initialize_impl is also called by another func _spawn_async_fn:
self.task is a member variable which has type TaskHeader . The TaskHeader ’s definition looks like:
💡
There are some comments in the code above. it maybe useful to understand the following parts of the article, for the TaskHeader almost contains all information about a task.
Here, we set the poll_fn , a func pointer to TaskStorage::<F>::poll instead of our main task’s poll, which is generated by compiler. When we set the value, we won’t call the func poll .
This func will be mentioned below.
Then we set the future to our main future. And create a TaskRef , a pointer to the TaskStorage .
Finally, we wrap the task ptr to a SpawnTokenso that the task will be scheduled by the executor.
But now, the main task is just in the TaskPool. Our executor is not aware of the main task. Everything we get now is a SpawnToken, which contains the ptr of main task’s TaskStorage. So we need to spawn our main task into the executor.
The following code finishes the task:
this function’s called trace is (I want to give a extensive few of the code):
from file—embassy-executor/src/spawner.rs:102
and the self.executor.spawn(task) will call:
from file—embassy-executor/src/raw/mod.rs:495
and the self.inner.spawn(task) is eventually the final point we call this important spawn,just repeat this important code again:
from file—embassy-executor/src/raw/mod.rs:364
The parameter task actually comes from our SpawnToken , which contains the TaskRef points to main task’s TaskStorage.
the SpawnToken code is:
Spawning the main task is surprisingly simple—just sets the member variable executor in TaskHeader to the executor we create and then add the main task to the run_queue of the SyncExecutor , which is the inner of our executor. The run_queue is a linked table and the new task will be insert into the list header.
💡
There is a line of code is not mentioned which has feature "rtos-trace". It used to trace the state of the rtos. With this, we can trace the rtos by tool like SystemView.
SyncExecutor is defines as:
It owns a RunQueueand a Penderto pend the executor. The enqueue func of the executor is defined as:
from file—embassy-executor/src/raw/mod.rs:342
The return value of run_queue’s enqueue method is was_empty , which is easy to guess it’s feature. So, if the ready_queue is empty before we insert the main task, we will pend the executor.
Why the executor should be pended?
OK, now let’s come to the ready_queue part:
from file—embassy-executor/src/raw/run_queue_atomics.rs:42
When enqueue the main task to the ready queue, the main task is inserted into the first place of the single linked list.
The next part is the pend func. Before we analyse the func, let’s have a look at the Pender’definition:
from file—embassy-executor/src/raw/mod.rs:302
the pender is just a raw ptr. It is set as THREAD_PENDER (usize::MAX)when we new the executor:
The func is defined as(in file cortex_m.rs):
In blinky, we just have one executor, which is running in mode thread. So the part of the func compiled is only:
💡
There are two mode in ARM Cortex-M: Thread and Process. User code is running in thread mode and other code, like ISR is running in Process mode(just consider it as Privileged mode and Unprivileged mode). The biggest difference of the mode in Cortex-M is that the SP register is different physically, which will increase the speed of context switch.
As it is said, in blinky we only have a executor in tread mode. there is another type of executor. It is interrupt executor, whose pend is also shown above. If you want to know more about it, just see the blog of my teammate. I will focus on the flow of embassy, instead of the type of executor.
It is easy to see that the feature of pend is just executing a line of assembly code: sev, which means Set Event. sev should be execute here because if there is no task in the ready_queue before, the MCU must be in low power state.(because if there is no task in ready_queue to poll, the executor will “sleep”, which makes MCU into low power mode by instruction wfe)
💡
The function of sev is to wake up the MCU from low power mode, which is cased by instruction wfe that is used to suspend our executor to wait a future. So pend is used to wake up our MCU form wfe state.
Beside, every time we call enqueue of SyncExecutor , it may wake up the executor.
“Poll” the Executor
Congratulations! We finish the init part. Now let’s come to the async time.
Below is the main loop in run . I paste it here again:
When we come into the main loop, the first thing the executor does is to call poll . For the executor is wrapped in layers, so as the poll func. So I just paste the final poll we call, which is a func of SyncExecutor:
from file—embassy-executor/src/raw/mod.rs:373
This func is so important that I will explain it row by row.
Set up the Alarm
In blinky, we should use the Timer, so we have to set the call back func:
The para’s meaning is:
alarm—type: AlarmHandle
The alarm is set as self.alarm . The AlarmHandle is allocated by the embassy_time_driver::allocate_alarm() when we new the executor(in file time_driver.rs):
The Time Drive has several alarm. The number of it depends on ALARM_COUNT :
The comments above show how the alarm works(even can have a view of the time driver). The time driver doesn’t use the Timer’s output or the update interrupt directly, but use the CC1(capture/compare register, which is usually used to analyze or output PWM) register to generate the interrupt.
There is a comment:
this driver is implemented using CC1 as the halfway rollover interrupt
In the time driver’s init func, it looks like:
The code’s function is to set the CC1 register’s value as half of the ARR(auto-reload register, which depends the value of the Timer’s count number) register. When the count num reaches the value of CC1 register, there will be an interrupt(if we enable it), which makes the time driver work.
The remaining 3 CC channel will be used as the alarm, which will generate mutually independent interrupts. For there is no Timer 12 and Timer 15 in STMF401ReTx and the Timer of STM32F401 has four CC channel, so the ALARM_COUNT should be set as 3.
💡
If you’re interested in the CC’s interrupt, the theory of the interrupt generated by CC is(CC1 as example):
callback(type:fn(*mut ()))
The para needs a ptr to a func. We provide it with Self::alarm_callback , which is defined as:
We have introduced pend func. It is used to wake up our executor. Judging from the para’s name, the func we provide will be called if the alarm “bells”, and wakes up the executor by func pend.
💡
If the alarm bells, there will be an interrupt. The callback func will be called in the ISR.
ctx(type:*mut ())
We set it as our executor’s ptr so that the alarm can wake up the executor bound with the alarm.
💡
There is only one RtcDriver. But there can be many alarm.
In conclusion, we set the alarm and define the func will be called when the alarm “bells”. For the procedure is outside the loop, so it will be execute once.
Wake Task in Time Queue
The code is:
And the dequeue_expired and some needed func are:
Actually, dequeue_expired calls func retain . Let’s introduce is recursively. In func retain , we traverse the time queue and call f(p) , which is the closure in dequeue_expired . If the task in time queue expires, we will call wake_task_no_pend to change the task’s state, add the task to ready queue and return false to func retain , which will take the task out of the time queue.
💡
Maybe the func is annoying because there are so many func closures. But actually it does a simple task: traverse the time queue, find tasks expiring, remove it from time queue and add it to ready queue.
Poll the Ready Task
The code is:
Here we pass a func closure to func dequeue_all . Let’s name the closure as on_task , which is the same to the name in dequeue_all
dequeue_all looks like:
Let’s understand dequeue_all first. Firstly, we get the ready queue’s head ptr and clean the ready queue. Then we traverse the simple linked table whose head ptr points the ready queue before and call func on_task on every member in the linked table.
💡
The ready queue is clear here because if a task is polled, it can’t be ready. Besides, when we enqueue a task, we will insert it at the head position and when we dequeue, we will traverse from head to tail. So if a task is added to the ready queue later, it will be polled former, just like a stack.
Now let’s see what we do on the task in the ready queue. Firstly, we will set all of our task’s expiring time as MAX.
Why set MAX here?
Then we will check the task’s state. The state of the task is set as STATE_SPAWNED | STATE_RUN_QUEUED when we claim a TaskStorage so run_dequeue will return true and we won’t go into the if:
It should be noted that in run_dequeue , the run_queued in state will be set as false, which means the task is not in the ready queue.
After the state check, the poll of the TaskStorage will be called, which needs a parameter: the task’s TaskRef.
💡
When we init the task, the poll_fn is set as the ptr to TaskStorage::<F>::poll.
The poll looks like:
The main task in the poll of TaskStorage is to create a waker bound with a task, which will be used to wake up the executor. This is also the main reason that Embassy use the poll of TaskStorage instead of the poll of the future. The wake func will be introduced later.
Then the poll of our main task will be executed. In our main function, there is a Timer we create to delay. The Timer is created as:
The Timer’s expires_at records the delay finishing time.
Actually, Timer implement Future trait, so when we call the poll of our main function, the poll of Timer will be called:
When poll func is called for the first time, the yielded_once is false for the default,which means current future is not Ready. So the flow will go to the else part and call the schedule_wake func, passing the waker to the executor.
When the poll call schedule_wake (a function of embassy_time_queue_driver),actually it call _embassy_time_schedule_wake , which is written by macro. So I have to learn the macro too. My teammate finds the article below, just learn macro by it.
Now back to Embassy, we can take a look at the code
This is a macro, which is used in file embassy-executor-0.5.0/src/raw/mod.rs only.The macro match the pattern:
The use of the macro is:
from file embassy-executor/src/raw/mod.rs
When the macro is used, _embassy_time_schedule_wake will be define. So the function is defined in the embassy-executor/src/raw/mod.rs. According to the pattern, the name is TIMER_QUEUE ,which is a static parameter. t is TimerQueue (is a structure which implement the embassy_time_queue_driver::TimerQueue trait). The val is TimerQueue(this is a Zero-Sized Type ,ZST or specifically the Unit-like Struct) ,which is defined in the same file.
in the macro/(it expand to the complete _embassy_time_schedule_wake function), it convert the t :TimerQueue structure to TimerQueue trait in order to use the schedule_wake func and offer the function we need—_embassy_time_schedule_wake
The function here just do the uncoupled part, the embassy offers the macro timer_queue_impl to make it easy to implement the required inner function _embassy_time_schedule_wake
it’s in the file embassy_time-queue-driver/src/lib.rs
The next func is schedule_wake of TimerQueue structure.
In this function, task_from_waker will be called first. As it is introduced above, the task is bound with the waker, so the effect of the function is just as its name.
After get the task, we need to change the task’s expires_at , whose meaning is: the time point the task should be wake. We set here as expires_at.min(at) . for we set the task’s expires_at as MAX when we run the poll of SyncExecutor and if there is a smaller latency set by other Timer, it will be set as the closest time point.
After change the expires_at of task, we also need to set the yielded_once as true because we just need to delay once. Finally, return Poll::Pending , which is also the return value of the poll of our main task. So far, we finish the first poll of our main task.
Now let’s come back to the on_task . The last part is:
The function is just get the task that is delayed and insert it at the first position of the time queue.
💡
If a task is not delayed, the expires_at will be u64::MAX and actually it will pop out of the run_queue and also won’t be added to the timer_queue, so it just poll once and quit(will never run again).
Set the Expiration of Alarm
Let’s come back to the poll of SyncExecutor . This is the last part of the poll of SyncExecutor :
OK, here is a our old friend: retain . It just traverse the time queue and if the closure returns false, it will remove current task from the time queue.
💡
So the false means: the task should not be in the time queue.
So the part is simple: just traverse the time queue and find the closest expiring time. And set the alarm so that it will bell when the closest expiring time arrives.
Wait, here is a func set_alarm:
Because we didn’t enable the alarm when we allocate it(Maybe a lazy policy?), there are so many register’s settings to enable it, which may annoyed you if you don’t have knowledge in STM32F401. OK, there is no need to understand the code. Just know that the code is setting the closest expiring time. When it arrives, there will be an interrupt generated by the alarm.
the implementation of now in stm32:
in file—embassy-stm32/src/timer_driver.rs:518
After we set up the alarm, there is no task in the ready queue. So the loop in poll of SyncExecutor will be broken and our executor will sleep and the MCU will be turned into low power mode by the instruction of wfe .
Wake up the Task and Executor
As stated above, if the time point we expect arrives, there will be an interrupt generated by the alarm. The ISR of the interrupt looks like:
We have no need to care the update interrupt and the driver’s interrupt(CC1 interrupt). Let’s focus on the alarm’s interrupt:
Here, we check the interrupt’s flag, which shows whether the interrupt occurs and the interrupt’s enable bit.
Here we call the func trigger_alarm :
If the alarm bells, there must be some futures in the main task ready. So we need to wake up our executor and the MCU. Do you remember the callback function we set in the poll of SyncExecutor ? I paste it here again:
OK, the code is familiar to us. Here we just call pend , it will wake up our MCU and the executor by instruction of sev. After wake up, the run will loop again. I paste run here again.(The beginning and the end echo…)
In the second poll, the main task will be removed from the time queue, and be moved into the run_queue. Then the main task will be polled again, return Poll::Ready, destroy the Timer_delay’s future, and continue our main task.
💡
pend will be called in ISR or when a task enqueues.
About the Waker
Maybe you notice that there is no waker in the flow. OK, let me tell you. The waker is used to wake the executor. The same to the Timer’s interrupt? The wake looks like:
It just enqueues the task. As stated above, the pend will be called when a task enqueues.
For the embassy supports the time_delay well, so the enqueue part is written in the poll of SyncExecutor directly and so there is no need to call the wake of the timer. So in the ISR of time’interrupt, we just need to wake up the MCU and the executor.
But for other futures, we need to enqueue the task manually in ISR.
💡
In other words, the wake of Timer is called in the poll of the SyncExecutor
Q
by Noah, all solved.
when I debug by stepping, there is something wrong with my gdb:
💡
After I cargo clean, it works well.
This happens when I debug the code to create a new executor, the first step of main macro
My rust-analyzer is stuck on Build Crate-Graph
💡
Maybe there is something wrong with v0.3.1992. When I change to v0.3.1983, RA works will.
I am confused with some code:
Why the inner of the executor is called SyncExecutor ?
I don’t know why my GDB closed unexpectedly when I called self.timer_queue.update(p); in poll of SyncExecutor .
💡
After I cargo clean, it works well.
I can run into the alarm init part.
💡
Actually, when I debug the code, the Timer is running too because it is a peripheral. So if I debug the code is too slow, now maybe large than my timestamp. Just need to set the delay time large enough.
