Task watchdog

Software watchdog that catches stuck threads and turns them into core dumps. More...

Typedefs
typedef void *(*	pbl_task_wdt_callback_t) (int channel_id, void *user_data)
	Callback of an expired channel.

Functions
void	pbl_task_wdt_init (void)
	Start the watchdog thread.
int	pbl_task_wdt_add (struct pbl_thread *thread, uint32_t timeout_ms, pbl_task_wdt_callback_t callback, void *user_data)
	Add a channel.
int	pbl_task_wdt_delete (int channel_id)
	Delete a channel.
int	pbl_task_wdt_feed (int channel_id)
	Feed a channel, restarting its timeout.
void	pbl_task_wdt_feed_self (void)
	Feed the channels of the calling thread, if it has any.
void	pbl_task_wdt_feed_thread (struct pbl_thread *thread)
	Feed the channels of a thread.
void	pbl_task_wdt_feed_all (void)
	Feed every channel.
void	pbl_task_wdt_set_waiting (bool waiting)
	Mark the calling thread as waiting for work, or as busy again.
void	pbl_task_wdt_suspend (uint32_t timeout_ms)
	Keep every channel fed for a while, for phases where stalls are expected.
void	pbl_task_wdt_resume (void)
	End a suspension; every channel restarts with a full timeout.

Detailed Description

Software watchdog that catches stuck threads and turns them into core dumps.

Each channel watches one thread that has to keep proving it makes progress by feeding the channel within its timeout, unless it is waiting for work (see pbl_task_wdt_set_waiting()). A thread at the highest priority checks the channels every CONFIG_TASK_WDT_CHECK_PERIOD_MS and feeds the hardware watchdog. A channel that expires is logged and recorded in the reboot reason, and its callback gets a chance to recover the thread; once it stays expired for CONFIG_TASK_WDT_GRACE_MS after the check that found it expired, the system resets with a core dump (with CONFIG_WATCHDOG; otherwise it only logs). The pool holds CONFIG_TASK_WDT_CHANNELS channels.

static void prv_worker(void *arg) {
  int ch = pbl_task_wdt_add(NULL, CONFIG_TASK_WDT_TIMEOUT_MS, NULL, NULL);
 
  while (running) {
    wait_for_work();
    do_work();
    pbl_task_wdt_feed(ch);
  }
 
  pbl_task_wdt_delete(ch);
}

Typedef Documentation

pbl_task_wdt_callback_t

typedef void *(* pbl_task_wdt_callback_t) (int channel_id, void *user_data)

Callback of an expired channel.

Runs on the watchdog thread on every check while the channel stays expired, before the system resets. It may try to unblock the watched thread.

Parameters

channel_id	Expired channel.
user_data	Data given to pbl_task_wdt_add().

Returns

Pointer naming the work the thread was running, recorded in the reboot reason, or NULL.

Function Documentation

pbl_task_wdt_add()

int pbl_task_wdt_add	(	struct pbl_thread *	thread,
		uint32_t	timeout_ms,
		pbl_task_wdt_callback_t	callback,
		void *	user_data
	)

Add a channel.

The thread must delete the channel before it exits.

Parameters

thread	Thread to watch, NULL for the calling thread.
timeout_ms	Maximum time between feeds, in milliseconds.
callback	Callback when the channel expires, or NULL.
user_data	Data passed to callback.

Returns

Channel id, or -ENOMEM when every channel is in use.

pbl_task_wdt_delete()

int pbl_task_wdt_delete

(

int

channel_id

)

Delete a channel.

Parameters

channel_id

Channel.

Return values

0	Success.
-EINVAL	The channel is not in use.

pbl_task_wdt_feed()

int pbl_task_wdt_feed

(

int

channel_id

)

Feed a channel, restarting its timeout.

Parameters

channel_id

Channel.

Return values

0	Success.
-EINVAL	The channel is not in use.

pbl_task_wdt_feed_all()

void pbl_task_wdt_feed_all

(

void

)

Feed every channel.

For long operations that hold locks other watched threads wait on, such as flash erases.

pbl_task_wdt_feed_self()

void pbl_task_wdt_feed_self

(

void

)

Feed the channels of the calling thread, if it has any.

pbl_task_wdt_feed_thread()

void pbl_task_wdt_feed_thread

(

struct pbl_thread *

thread

)

Feed the channels of a thread.

Parameters

thread

Thread; NULL is a no-op.

pbl_task_wdt_init()

void pbl_task_wdt_init

(

void

)

Start the watchdog thread.

Every channel added so far starts with a full timeout.

pbl_task_wdt_resume()

void pbl_task_wdt_resume

(

void

)

End a suspension; every channel restarts with a full timeout.

pbl_task_wdt_set_waiting()

void pbl_task_wdt_set_waiting

(

bool

waiting

)

Mark the calling thread as waiting for work, or as busy again.

A thread blocked waiting for work cannot be stuck, so its channels do not expire while it waits; going back to busy restarts their timeouts. A thread that only blocks to wait for work then needs no periodic feeding.

while (running) {
  pbl_task_wdt_set_waiting(true);
  wait_for_work();
  pbl_task_wdt_set_waiting(false);
  do_work();
}

Parameters

waiting

True before blocking for work, false once there is work to do.

pbl_task_wdt_suspend()

void pbl_task_wdt_suspend

(

uint32_t

timeout_ms

)

Keep every channel fed for a while, for phases where stalls are expected.

A new call replaces the suspension in progress.

Parameters

timeout_ms

Duration in milliseconds, or 0 to last until pbl_task_wdt_resume().