PebbleOS v4.47.0-512-g5285d4236
Loading...
Searching...
No Matches
Task watchdog

Software watchdog that catches stuck threads and turns them into core dumps. More...

Data Structures

struct  pbl_task_wdt_stall
 Expired channels, as reported with pbl_task_wdt_stalled(). More...
 

Typedefs

typedef void *(* pbl_task_wdt_callback_t) (int channel_id, void *user_data)
 Callback of an expired channel.
 

Functions

void pbl_task_wdt_stalled (const struct pbl_task_wdt_stall *stall)
 Report expired channels.
 
void pbl_task_wdt_recovered (void)
 Report that the channels found expired have all been fed again.
 
void pbl_task_wdt_expired (void)
 Reset the system: channels stayed expired for the grace period.
 
void pbl_task_wdt_init (void)
 Start the watchdog thread.
 
int pbl_task_wdt_add (struct pbl_thread *thread, uint32_t timeout_ms, pbl_task_wdt_callback_t callback, void *user_data)
 Add a channel.
 
int pbl_task_wdt_delete (int channel_id)
 Delete a channel.
 
int pbl_task_wdt_feed (int channel_id)
 Feed a channel, restarting its timeout.
 
void pbl_task_wdt_feed_self (void)
 Feed the channels of the calling thread, if it has any.
 
void pbl_task_wdt_feed_thread (struct pbl_thread *thread)
 Feed the channels of a thread.
 
void pbl_task_wdt_feed_all (void)
 Feed every channel.
 
void pbl_task_wdt_set_waiting (bool waiting)
 Mark the calling thread as waiting for work, or as busy again.
 
int pbl_task_wdt_add_workq (struct pbl_workq *q, uint32_t timeout_ms, pbl_task_wdt_callback_t callback, void *user_data)
 Add a channel watching the thread of a work queue.
 
void pbl_task_wdt_suspend (uint32_t timeout_ms)
 Keep every channel fed for a while, for phases where stalls are expected.
 
void pbl_task_wdt_resume (void)
 End a suspension; every channel restarts with a full timeout.
 

Detailed Description

Software watchdog that catches stuck threads and turns them into core dumps.

Each channel watches one thread that has to keep proving it makes progress by feeding the channel within its timeout, unless it is waiting for work (see pbl_task_wdt_set_waiting()). A thread at the highest priority checks the channels every CONFIG_TASK_WDT_CHECK_PERIOD_MS and feeds the hardware watchdog. A channel that expires is logged and reported with pbl_task_wdt_stalled(), and its callback gets a chance to recover the thread; once it stays expired for CONFIG_TASK_WDT_GRACE_MS after the check that found it expired, pbl_task_wdt_expired() resets the system (with CONFIG_WATCHDOG; otherwise it only logs). The pool holds CONFIG_TASK_WDT_CHANNELS channels.

pbl_task_wdt_stalled(), pbl_task_wdt_recovered() and pbl_task_wdt_expired() have defaults an application overrides to record what was stuck and to reset its own way.

static void prv_worker(void *arg) {
int ch = pbl_task_wdt_add(NULL, CONFIG_TASK_WDT_TIMEOUT_MS, NULL, NULL);
while (running) {
wait_for_work();
do_work();
}
}
int pbl_task_wdt_add(struct pbl_thread *thread, uint32_t timeout_ms, pbl_task_wdt_callback_t callback, void *user_data)
Add a channel.
int pbl_task_wdt_delete(int channel_id)
Delete a channel.
int pbl_task_wdt_feed(int channel_id)
Feed a channel, restarting its timeout.

Data Structure Documentation

◆ pbl_task_wdt_stall

struct pbl_task_wdt_stall

Expired channels, as reported with pbl_task_wdt_stalled().

Data Fields
uint8_t active_mask Channels in use, one bit per channel.
uint8_t fed_mask Channels fed in time, one bit per channel.
uintptr_t lr Link register of that thread.
uintptr_t pc Program counter of the highest-priority stuck thread, the most likely culprit.
void * work What its channel callback returned, or NULL.

Typedef Documentation

◆ pbl_task_wdt_callback_t

typedef void *(* pbl_task_wdt_callback_t) (int channel_id, void *user_data)

Callback of an expired channel.

Runs on the watchdog thread on every check while the channel stays expired, before the system resets. It may try to unblock the watched thread.

Parameters
channel_idExpired channel.
user_dataData given to pbl_task_wdt_add().
Returns
Pointer naming the work the thread was running, reported with pbl_task_wdt_stalled(), or NULL.

Function Documentation

◆ pbl_task_wdt_add()

int pbl_task_wdt_add ( struct pbl_thread *  thread,
uint32_t  timeout_ms,
pbl_task_wdt_callback_t  callback,
void *  user_data 
)

Add a channel.

The thread must delete the channel before it exits.

Parameters
threadThread to watch, NULL for the calling thread.
timeout_msMaximum time between feeds, in milliseconds.
callbackCallback when the channel expires, or NULL.
user_dataData passed to callback.
Returns
Channel id, or -ENOMEM when every channel is in use.

◆ pbl_task_wdt_add_workq()

int pbl_task_wdt_add_workq ( struct pbl_workq *  q,
uint32_t  timeout_ms,
pbl_task_wdt_callback_t  callback,
void *  user_data 
)

Add a channel watching the thread of a work queue.

The thread waits while the queue is idle, and its channel is fed after each item, so it expires when an item runs for too long or the queue does not get to run. This takes the queue's hooks.

Parameters
qQueue, whose thread must exist.
timeout_msMaximum time between items, in milliseconds.
callbackCallback when the channel expires, or NULL to name the function being run.
user_dataData passed to callback.
Returns
Channel id, -EINVAL when the queue has no thread yet, or -ENOMEM when every channel is in use.

◆ pbl_task_wdt_delete()

int pbl_task_wdt_delete ( int  channel_id)

Delete a channel.

Parameters
channel_idChannel.
Return values
0Success.
-EINVALThe channel is not in use.

◆ pbl_task_wdt_expired()

void pbl_task_wdt_expired ( void  )

Reset the system: channels stayed expired for the grace period.

Called on the watchdog thread with interrupts locked. The default panics.

◆ pbl_task_wdt_feed()

int pbl_task_wdt_feed ( int  channel_id)

Feed a channel, restarting its timeout.

Parameters
channel_idChannel.
Return values
0Success.
-EINVALThe channel is not in use.

◆ pbl_task_wdt_feed_all()

void pbl_task_wdt_feed_all ( void  )

Feed every channel.

For long operations that hold locks other watched threads wait on, such as flash erases.

◆ pbl_task_wdt_feed_self()

void pbl_task_wdt_feed_self ( void  )

Feed the channels of the calling thread, if it has any.

◆ pbl_task_wdt_feed_thread()

void pbl_task_wdt_feed_thread ( struct pbl_thread *  thread)

Feed the channels of a thread.

Parameters
threadThread; NULL is a no-op.

◆ pbl_task_wdt_init()

void pbl_task_wdt_init ( void  )

Start the watchdog thread.

Every channel added so far starts with a full timeout.

◆ pbl_task_wdt_recovered()

void pbl_task_wdt_recovered ( void  )

Report that the channels found expired have all been fed again.

Called on the watchdog thread. The default does nothing.

◆ pbl_task_wdt_resume()

void pbl_task_wdt_resume ( void  )

End a suspension; every channel restarts with a full timeout.

◆ pbl_task_wdt_set_waiting()

void pbl_task_wdt_set_waiting ( bool  waiting)

Mark the calling thread as waiting for work, or as busy again.

A thread blocked waiting for work cannot be stuck, so its channels do not expire while it waits; going back to busy restarts their timeouts. A thread that only blocks to wait for work then needs no periodic feeding. Once woken, a waiting thread that does not get to run is starved: its timeouts restart from the check that first finds it runnable.

while (running) {
wait_for_work();
do_work();
}
void pbl_task_wdt_set_waiting(bool waiting)
Mark the calling thread as waiting for work, or as busy again.
Parameters
waitingTrue before blocking for work, false once there is work to do.

◆ pbl_task_wdt_stalled()

void pbl_task_wdt_stalled ( const struct pbl_task_wdt_stall *  stall)

Report expired channels.

Called on the watchdog thread on every check that finds channels expired. The default does nothing.

Parameters
stallWhat is stuck.

◆ pbl_task_wdt_suspend()

void pbl_task_wdt_suspend ( uint32_t  timeout_ms)

Keep every channel fed for a while, for phases where stalls are expected.

A new call replaces the suspension in progress.

Parameters
timeout_msDuration in milliseconds, or 0 to last until pbl_task_wdt_resume().