Settings files

Atomic key-value store on top of PFS. More...

Modules
	Settings file raw iterator
	On-flash format of settings files and a cursor over their records.

Data Structures
struct	SettingsFile
	Open settings file. More...
struct	SettingsRecordInfo
	Record being visited by settings_file_each() or settings_file_rewrite(). More...

Macros
#define	DELETED_LIFETIME   (0 * PBL_SEC_PER_DAY)
	Minimum time in seconds a deleted record is kept before compaction drops it.

Typedefs
typedef void(*	SettingsFileChangeCallback) (SettingsFile *file, const void *key, int key_len, time_t last_modified)
	Callback invoked after a record is written.
typedef void(*	SettingsFileGetter) (SettingsFile *file, void *buf, size_t buf_len)
	Read the key or value of the record being visited.
typedef bool(*	SettingsFileEachCallback) (SettingsFile *file, SettingsRecordInfo *info, void *context)
	Callback for settings_file_each().
typedef void(*	SettingsFileRewriteCallback) (SettingsFile *old_file, SettingsFile *new_file, SettingsRecordInfo *info, void *context)
	Callback for settings_file_rewrite().
typedef bool(*	SettingsFileRewriteFilterCallback) (void *key, size_t key_len, void *value, size_t value_len, void *context)
	Filter callback for settings_file_rewrite_filtered().

Functions
status_t	settings_file_open (SettingsFile *file, const char *name, int max_used_space)
	Open or create a settings file.
status_t	settings_file_open_growable (SettingsFile *file, const char *name, int max_used_space, int initial_alloc_size)
	Open or create a settings file that grows on demand.
void	settings_file_close (SettingsFile *file)
	Close a settings file.
bool	settings_file_exists (SettingsFile *file, const void *key, size_t key_len)
	Check whether a key holds a non-empty value.
status_t	settings_file_delete (SettingsFile *file, const void *key, size_t key_len)
	Delete a record.
int	settings_file_get_len (SettingsFile *file, const void *key, size_t key_len)
	Get the length of a value.
status_t	settings_file_get (SettingsFile *file, const void *key, size_t key_len, void *val_out, size_t val_out_len)
	Read a value.
status_t	settings_file_set (SettingsFile *file, const void *key, size_t key_len, const void *val, size_t val_len)
	Write a value, timestamped with the current time.
status_t	settings_file_set_with_timestamp (SettingsFile *file, const void *key, size_t key_len, const void *val, size_t val_len, uint32_t timestamp)
	Write a value with a given timestamp instead of the current time.
status_t	settings_file_mark_synced (SettingsFile *file, const void *key, size_t key_len)
	Mark a record as synced.
status_t	settings_file_mark_all_dirty (SettingsFile *file)
	Mark all records as not synced, to trigger a full sync.
void	settings_file_set_change_callback (SettingsFileChangeCallback callback)
	Register the callback invoked after any record of any settings file is written.
status_t	settings_file_set_byte (SettingsFile *file, const void *key, size_t key_len, size_t offset, uint8_t byte)
	Write a single byte of a value in place.
status_t	settings_file_each (SettingsFile *file, SettingsFileEachCallback cb, void *context)
	Call a callback for every valid record.
status_t	settings_file_rewrite (SettingsFile *file, SettingsFileRewriteCallback cb, void *context)
	Rewrite a settings file record by record.
status_t	settings_file_rewrite_filtered (SettingsFile *file, SettingsFileRewriteFilterCallback filter_cb, void *context)
	Rewrite a settings file, keeping only the records a filter accepts.
status_t	settings_file_compact (SettingsFile *file)
	Compact a settings file.

Detailed Description

Atomic key-value store on top of PFS.

A settings file is a log-structured binary key-value store kept in a single PFS file. Keys (up to SETTINGS_KEY_MAX_LEN bytes) and values (up to SETTINGS_VAL_MAX_LEN bytes) are arbitrary bytes. Every update is atomic: after a reboot in the middle of a write, the record holds either the old or the new value. Records carry a modification timestamp and a synced flag used for synchronization with the phone.

Settings files are not thread-safe: callers serialize access to a file themselves, and a file must not be open twice at the same time.

SettingsFile file;
uint32_t key = 1;
uint8_t value = 42;
 
if (settings_file_open(&file, "myprefs", 1024) == S_SUCCESS) {
  settings_file_set(&file, &key, sizeof(key), &value, sizeof(value));
  settings_file_get(&file, &key, sizeof(key), &value, sizeof(value));
  settings_file_close(&file);
}

---

Data Structure Documentation

SettingsFile

struct SettingsFile

Open settings file.

Settings file backing a store.

The fields are internal to the settings file implementation.

Data Fields
int	alloc_used_space	Current allocation budget. Grows from the initial size toward max_used_space for growable files, equals max_used_space otherwise.
int	cur_record_pos	Position of the record being visited by settings_file_each() or settings_file_rewrite(), so that other records can be read from the callback. 0 when not iterating.
int	dead_space	Space taken by overwritten records, reclaimed by compaction.
SettingsRawIter	iter	Raw iterator over the underlying file.
uint32_t	last_modified	Most recent modification time of any record.
int	max_space_total	Space the file may take before a compaction is forced, at least max_used_space.
int	max_used_space	Space valid records may use; writes beyond it fail with E_OUT_OF_STORAGE.
int	min_alloc_used_space	Lower bound of alloc_used_space when compacting.
char *	name	File name, heap allocated.
int	used_space	Space taken by valid records.

SettingsRecordInfo

struct SettingsRecordInfo

Record being visited by settings_file_each() or settings_file_rewrite().

Data Fields
bool	dirty	Record has not been marked as synced.
SettingsFileGetter	get_key	Reads the key.
SettingsFileGetter	get_val	Reads the value.
int	key_len	Key length in bytes.
uint32_t	last_modified	Modification timestamp.
int	val_len	Value length in bytes, 0 for a deleted record.

Macro Definition Documentation

DELETED_LIFETIME

#define DELETED_LIFETIME   (0 * PBL_SEC_PER_DAY)

Minimum time in seconds a deleted record is kept before compaction drops it.

Keeping the key around gives the deletion time to propagate to synchronized devices.

Typedef Documentation

SettingsFileChangeCallback

typedef void(* SettingsFileChangeCallback) (SettingsFile *file, const void *key, int key_len, time_t last_modified)

Callback invoked after a record is written.

Parameters

file	Settings file that was modified.
key	Key that was written.
key_len	Length of key in bytes.
last_modified	Timestamp of the change.

SettingsFileEachCallback

typedef bool(* SettingsFileEachCallback) (SettingsFile *file, SettingsRecordInfo *info, void *context)

Callback for settings_file_each().

Parameters

file	Settings file being iterated.
info	Current record.
context	Context passed to settings_file_each().

Returns

true to continue iterating, false to stop.

SettingsFileGetter

typedef void(* SettingsFileGetter) (SettingsFile *file, void *buf, size_t buf_len)

Read the key or value of the record being visited.

Parameters

	file	Settings file.
[out]	buf	Destination buffer.
	buf_len	Bytes to read, at most the key or value length.

SettingsFileRewriteCallback

typedef void(* SettingsFileRewriteCallback) (SettingsFile *old_file, SettingsFile *new_file, SettingsRecordInfo *info, void *context)

Callback for settings_file_rewrite().

Parameters

old_file	Original file, to read the record from.
new_file	New file, to write records that should be kept to.
info	Current record of old_file.
context	Context passed to settings_file_rewrite().

SettingsFileRewriteFilterCallback

typedef bool(* SettingsFileRewriteFilterCallback) (void *key, size_t key_len, void *value, size_t value_len, void *context)

Filter callback for settings_file_rewrite_filtered().

Must not call any other settings file function.

Parameters

key	Record key.
key_len	Length of key in bytes.
value	Record value.
value_len	Length of value in bytes.
context	Context passed to settings_file_rewrite_filtered().

Returns

true to keep the record, false to drop it.

Function Documentation

settings_file_close()

void settings_file_close

(

SettingsFile *

file

)

Close a settings file.

Parameters

file	Settings file.

settings_file_compact()

status_t settings_file_compact

(

SettingsFile *

file

)

Compact a settings file.

Rewrites all valid records, dropping dead space. Growable files shrink their allocation toward SettingsFile::min_alloc_used_space.

Parameters

file	Settings file.

Return values

S_SUCCESS

File compacted.

Returns

Negative status code on failure.

settings_file_delete()

status_t settings_file_delete	(	SettingsFile *	file,
		const void *	key,
		size_t	key_len
	)

Delete a record.

Writes an empty value, which marks the record as deleted.

Parameters

file	Settings file.
key	Key.
key_len	Length of key in bytes.

Returns

Same as settings_file_set().

settings_file_each()

status_t settings_file_each	(	SettingsFile *	file,
		SettingsFileEachCallback	cb,
		void *	context
	)

Call a callback for every valid record.

The callback may read other records but must not modify the file; use settings_file_rewrite() for that.

Parameters

file	Settings file.
cb	Callback.
context	Context passed to cb.

Return values

S_SUCCESS

Always.

settings_file_exists()

bool settings_file_exists	(	SettingsFile *	file,
		const void *	key,
		size_t	key_len
	)

Check whether a key holds a non-empty value.

Parameters

file	Settings file.
key	Key.
key_len	Length of key in bytes.

Returns

true if the record exists and is not deleted.

settings_file_get()

status_t settings_file_get	(	SettingsFile *	file,
		const void *	key,
		size_t	key_len,
		void *	val_out,
		size_t	val_out_len
	)

Read a value.

Reads the first val_out_len bytes of the value. On failure val_out is zeroed.

Parameters

	file	Settings file.
	key	Key.
	key_len	Length of key in bytes.
[out]	val_out	Value buffer.
	val_out_len	Bytes to read, at most the stored value length.

Return values

S_SUCCESS	Value read.
E_DOES_NOT_EXIST	No such record, or it was deleted.
E_RANGE	val_out_len exceeds the stored value length.

settings_file_get_len()

int settings_file_get_len	(	SettingsFile *	file,
		const void *	key,
		size_t	key_len
	)

Get the length of a value.

Parameters

file	Settings file.
key	Key.
key_len	Length of key in bytes.

Returns

Value length in bytes, 0 if the record does not exist or is deleted.

settings_file_mark_all_dirty()

status_t settings_file_mark_all_dirty

(

SettingsFile *

file

)

Mark all records as not synced, to trigger a full sync.

Rewrites the whole file, preserving timestamps, which can be slow for large files.

Parameters

file	Settings file.

Returns

Same as settings_file_rewrite().

settings_file_mark_synced()

status_t settings_file_mark_synced	(	SettingsFile *	file,
		const void *	key,
		size_t	key_len
	)

Mark a record as synced.

The flag stays until the record is overwritten.

Parameters

file	Settings file.
key	Key, at most SETTINGS_KEY_MAX_LEN bytes.
key_len	Length of key in bytes.

Return values

S_SUCCESS	Record marked.
E_RANGE	Key too long.
E_DOES_NOT_EXIST	No such record.

settings_file_open()

status_t settings_file_open	(	SettingsFile *	file,
		const char *	name,
		int	max_used_space
	)

Open or create a settings file.

Corrupt files, files with an unknown version and files whose recovery fails are removed and recreated empty. A file created with a smaller size is rewritten to the requested size.

Persist files need max_used_space of at least 5317 bytes to always fit all records in the worst case (all values booleans).

Parameters

[out]	file	Settings file to initialize.
	name	PFS file name.
	max_used_space	Space valid records may use, in bytes.

Return values

S_SUCCESS

File opened.

Returns

Negative status code from PFS on failure.

settings_file_open_growable()

status_t settings_file_open_growable	(	SettingsFile *	file,
		const char *	name,
		int	max_used_space,
		int	initial_alloc_size
	)

Open or create a settings file that grows on demand.

Like settings_file_open(), but the file starts with room for initial_alloc_size bytes of records and doubles its allocation as needed, up to max_used_space.

Parameters

[out]	file	Settings file to initialize.
	name	PFS file name.
	max_used_space	Space valid records may use, in bytes.
	initial_alloc_size	Initial allocation in bytes, greater than 0.

Return values

S_SUCCESS

File opened.

Returns

Negative status code from PFS on failure.

settings_file_rewrite()

status_t settings_file_rewrite	(	SettingsFile *	file,
		SettingsFileRewriteCallback	cb,
		void *	context
	)

Rewrite a settings file record by record.

Opens a new file with the same name in overwrite mode and calls cb for each record of the original one. Only records the callback writes to the new file are kept. The new file replaces the original atomically, and file is reopened on it.

Parameters

file	Settings file.
cb	Callback.
context	Context passed to cb.

Return values

S_SUCCESS	File rewritten.
E_OUT_OF_MEMORY	Allocation failed.

Returns

Other negative status codes if opening the new file fails.

settings_file_rewrite_filtered()

status_t settings_file_rewrite_filtered	(	SettingsFile *	file,
		SettingsFileRewriteFilterCallback	filter_cb,
		void *	context
	)

Rewrite a settings file, keeping only the records a filter accepts.

Much faster than settings_file_rewrite() when records are only being dropped.

Parameters

file	Settings file.
filter_cb	Filter, or NULL to keep every valid record.
context	Context passed to filter_cb.

Return values

S_SUCCESS	File rewritten.
E_OUT_OF_MEMORY	Allocation failed.

Returns

Other negative status codes on failure.

settings_file_set()

status_t settings_file_set	(	SettingsFile *	file,
		const void *	key,
		size_t	key_len,
		const void *	val,
		size_t	val_len
	)

Write a value, timestamped with the current time.

Atomic: after a reboot the record holds either the old or the new value. Compacts or grows the file when needed. Must not be called from a settings_file_each() callback. Invokes the change callback, if any, on success.

Parameters

file	Settings file.
key	Key, at most SETTINGS_KEY_MAX_LEN bytes.
key_len	Length of key in bytes.
val	Value.
val_len	Length of val in bytes, at most SETTINGS_VAL_MAX_LEN. 0 deletes the record.

Return values

S_SUCCESS	Value written.
E_RANGE	Key or value too long.
E_OUT_OF_STORAGE	Not enough space for valid records.

Returns

Other negative status codes if compaction fails.

settings_file_set_byte()

status_t settings_file_set_byte	(	SettingsFile *	file,
		const void *	key,
		size_t	key_len,
		size_t	offset,
		uint8_t	byte
	)

Write a single byte of a value in place.

Writes flash directly, so it can only clear bits. Only atomic for a single byte: do not use it to modify several bytes in a row.

Parameters

file	Settings file.
key	Key, at most SETTINGS_KEY_MAX_LEN bytes.
key_len	Length of key in bytes.
offset	Offset within the value, less than the value length.
byte	Byte to write.

Return values

S_SUCCESS	Byte written.
E_RANGE	Key too long.
E_DOES_NOT_EXIST	No such record, or it was deleted.

settings_file_set_change_callback()

void settings_file_set_change_callback

(

SettingsFileChangeCallback

callback

)

Register the callback invoked after any record of any settings file is written.

Only one callback is supported; registering replaces the previous one.

Parameters

callback

Callback, or NULL to unregister.

settings_file_set_with_timestamp()

status_t settings_file_set_with_timestamp	(	SettingsFile *	file,
		const void *	key,
		size_t	key_len,
		const void *	val,
		size_t	val_len,
		uint32_t	timestamp
	)

Write a value with a given timestamp instead of the current time.

Used when rewriting files to preserve the original timestamps.

Parameters

file	Settings file.
key	Key.
key_len	Length of key in bytes.
val	Value.
val_len	Length of val in bytes.
timestamp	Modification time to record.

Returns

Same as settings_file_set().