Looseleaf User Manual 0.0.4
The looseleaf server is an HTTP-accessible key/value store with a focus on minimalism, a
small footprint, and reliability. The server has the following notable features:
The server does not and is unlikely to ever have the following features:
The
looseleaf server provides a persistent key/value database with
ACID semantics. Keys and
values are arbitrary UTF-8 strings, and a fine-grained role-based access control system
is provided
to control how users are allowed to perform operations on the database.
A
value is an arbitrary
UTF-8 encoded string.
A
key is a unique name for a
value. The core functionality of the
looseleaf server is a persistent database of keys and values that can
be updated atomically. A key is a
UTF-8 encoded string
with an associated UTF-8 string value. Key names must begin with
/
and consist of an arbitrarily long sequence of segments separated with
/ characters. Repeated sequences of
/
characters are automatically reduced to a single
/ character. Key names,
effectively, resemble
absolute UNIX
paths. Key names are ordered lexicographically, so the path
/a is considered to be
less than
/b.
A
key expression describes a set of keys. The syntax of key expressions
is identical to
keys, except that
a
key expression is allowed to end with a
wildcard
character
*. A key expression
/x/y/z matches
exactly one key:
/x/y/z, whilst the key expression
/x/y/z* matches all keys that begin with
/x/y/z.
An action defines an operation that may be performed upon a key. An
action
may be READ or WRITE, where READ
means that the value of a given key may be retrieved, and
WRITE
means that the value of a given key may be modified, or the key created and/or removed
as desired.
A
grant combines a
key expression
and an
action, and effectively declares that the given
action
is allowed to be performed on the keys described by the
key expression. For example, a grant with action
READ and
key expression
/x/y/*
allows reading the values of any keys that begin with
/x/y/. A grant
with action
WRITE and key expression
/a/b/c allows the key
/a/b/c to be created, modified, and/or deleted.
Any action not explicitly allowed by a grant is denied.
A
role is a uniquely named (with respect to other roles) container that defines a
set of
grants. A
user
may have zero or more roles.
A
user is a uniquely named (with respect to other users) container that combines
a set of
roles with a
password. All operations in the
looseleaf server
are conducted on behalf of authenticated
users; no operations are anonymous.
All operations that occur in the looseleaf server occur in terms of
single atomic read-update-delete operations, or RUDs.
An RUD operation specifies a set of keys that will be read,
followed by a set of keys that will be updated, followed by a set of keys that
will be deleted. The reads, updates, and deletes are specified to occur in that
order, and the operation as whole will occur atomically; either all read/update/delete
operations succeed,
or it is as if none of them took place at all. More formally, an RUD can be
specified as a 3-tuple (R,U,D), where R
is the set of keys to read, U is the set of keys to update, and
D is the set of keys to delete. If any of the keys specified in
U do not exist, then they will be created by the update operation.
It follows that:
As mentioned previously, operations are specified to occur in R → U → D
order. This means that, for example, the operation (∅,k,k) for any
k is a no-op.
The looseleaf package is available from several sources:
Regardless of the distribution method, the looseleaf package
will contain a command named looseleaf that acts as the main
entrypoint to all of the package's functionality. The looseleaf
command expects an environment variable named LOOSELEAF_HOME
to be defined that points to the installation directory. See the documentation for
the
installation methods below for details.
The
looseleaf package can be installed from
Docker Hub
using
docker or
podman.
The image is configured such that LOOSELEAF_HOME=/looseleaf,
with the looseleaf command existing at
/looseleaf/bin/looseleaf. The image is configured with the
expectation that users will mount a volume at /looseleaf/etc
containing a configuration file and space for the looseleaf database file. The container
can
otherwise be run without any privileges, and with a read-only root filesystem.
The looseleaf command requires that a Java 17+ compatible
JVM be accessible via /usr/bin/env java.
Verify the integrity of the distribution zip file:
Unzip the zip file, and set LOOSELEAF_HOME appropriately:
The looseleaf server accepts a JSON configuration file with a very strictly-defined
format. The configuration file is a JSON object with the following properties:
The %schema property must be present and set to the value
"https://www.io7m.com/software/looseleaf/looseleaf-config-1.json".
The addresses property specifies an array of addresses to which the server will
bind. An address is a JSON object with a mandatory string-typed host and mandatory
integer-typed port property. For example, for a server that listens on
172.17.0.1
port
20000
and
fe80::42:31ff:fe0a:119a
port 20001, the following definitions would be used:
The
looseleaf server does not support
TLS
and it is expected that the server will be configured to listen on
localhost
behind a reverse proxy that provides TLS such as
nginx.
The databaseFile property specifies the path of the database that the server
will use to store key/value data. The file will be created if it does not already
exist.
The
roles property is an array-typed property that defines a set of
roles.
The following example defines a role read-xy that allows any user that
has the role to read keys that begin with /x/y/:
Similarly, the following write-xy definition allows any user that
has the write-xy role to create, update, and/or delete keys that begin with
/x/y/:
The
users property is an array-typed property that defines a set of
users. Each element of the array is a JSON object
that specifies a user name, a
hashed password, and a set of
role names that reference roles declared in the
roles property.
A
hashed password declares an
algorithm identifier,
an uppercase hex-encoded
salt value and an uppercase hex-encoded
hash value. Currently, the only supported
algorithm identifier is
PBKDF2WithHmacSHA256:n:256, which states that passwords are hashed
with
PBKDF2 using a
SHA-256 HMAC, with
n rounds of hashing, using a
256 bit key.
The
create-password command can be used
to hash a password suitable for use in a configuration file.
The following example defines a hashed password for a user:
The following example defines a user called someone, with a hashed
password and a set of roles:
A full configuration file example is as follows:
The configuration defines two roles read-xy and
write-xy
which allow reading of keys
/x/y/*
and writing of keys /x/y/*, respectively. The configuration defines a single user
named someone with both roles assigned. The configuration specifies that the local
database will be created a /looseleaf/etc/data.db, and that the server will listen
on http://localhost:20000.
The
JSON schema that defines the configuration
file format is as follows:
This section of the documentation describes the usage of the looseleaf
server.
Given a correctly defined
configuration file
in
config.json, the server can be started with the following command:
The looseleaf command does not fork into the background,
and is designed to run under process supervision.
The
looseleaf server exposes a number of HTTP endpoints. For the sake of
example, we will assume throughout the rest of this documentation that the server
was configured to listen
on
localhost on port
20000. We will also assume
that a user exists with name
grouch and password
12345678. The
looseleaf server
requires
basic
authentication for all requests except for the
root
endpoint and the
health endpoint.
The API versions exposed by the server can be inspected directly:
The above output indicates that the server only supports
protocol v1, with all the protocol's
endpoints based at
http://localhost:20000/v1. Please see the reference
pages for the endpoints for precise descriptions of inputs and outputs, and usage
examples.
The / endpoint displays the protocols supported by the server.
The following command displays the supported protocols:
/health - The health endpoint
The /health endpoint is intended to be used as a health check by load balancers.
The endpoint returns HTTP status 200, and a small JSON object containing a
few statistics.
The following command displays the health of the server:
On failure conditions that are unrelated to problems on the server (such as hardware
or software failure, bugs,
and etc), all endpoints return a 4** HTTP status code, and a JSON object of type
Errors. An Errors object contains an array-typed
errors property such that each element is of type Error.
An Error object contains an errorCode property and
a human-readable message property.
See the
schema for the exact types of
all objects that can be returned by the endpoints.
/v1/rud - Atomically read, update, and delete sets of keys
The
/v1/rud endpoint performs a
read-update-delete operation. The
endpoint requires a JSON object containing three properties:
read,
update, and
delete. The
read property is an array-typed property containing a list of
key names that will be read. The
update property is a JSON object
where the properties names are key names, and the property values are key values;
the key names
mentioned will be set to the corresponding values. The
delete
property is an array-typed property containing a list of key names to be deleted.
On success, the endpoint returns a JSON object with a values property
that contains the values of the keys specified by the original read property.
The following command reads the value of /x/y/z, then sets the value of
/x/y/z to "after":
The following command deletes /x/y/z:
The following command updates/creates all of /x/y/a,
/x/y/b, and /x/y/c atomically:
/v1/read
- Directly read the value of a single key
The
/v1/read endpoint is a convenience endpoint that directly returns the value of a
key, assuming the key exists. It is equivalent to calling the
/v1/rud
endpoint with empty
update
and
delete sets, and a
read
set that contains a single key, and then extracting the key value from the resulting
values
property and returning it directly. The endpoint is suffixed with the name of the
target key, so to read
key
/x/y/z, the endpoint is called as
/v1/read/x/y/z. The content type of the returned data is
text/plain.
The following command reads the value of /x/y/z:
/v1/update - Directly update the value of a single key
The
/v1/update endpoint is a convenience endpoint that directly updates the value of a
key, assuming the key exists. It is equivalent to calling the
/v1/rud
endpoint with empty
read
and
delete sets, and an
update
set that contains a single key. The endpoint is suffixed with the name of the target
key, so
to update key
/x/y/z, the endpoint is called as
/v1/update/x/y/z.
The following command updates the value of /x/y/z to "a
new value":
/v1/delete - Directly update the value of a single key
The
/v1/delete endpoint is a convenience endpoint that directly updates the value of a
key, assuming the key exists. It is equivalent to calling the
/v1/rud
endpoint with empty
update
and
read sets, and a
delete
set that contains a single key. The endpoint is suffixed with the name of the target
key, so
to update key
/x/y/z, the endpoint is called as
/v1/delete/x/y/z.
The following command deletes the key /x/y/z:
The JSON schema that defines all JSON messages exchanged between the client and server
in this
version of the protocol is as follows:
The looseleaf package provides a command-line interface for performing tasks such as
starting the server, checking configuration files, hashing passwords, and etc. The
base
looseleaf command is broken into a number of subcommands which are documented
over the following sections.
All of the command-line functionality is implemented using the standard
looseleaf APIs.
All subcommands accept a --verbose parameter that may be set to one of
trace, debug, info,
warn, or error. This parameter sets the lower bound for
the severity of messages that will be logged. For example, at debug verbosity, only
messages of severity debug and above will be logged. Setting the verbosity to
trace
level effectively causes everything to be logged, and will produce large volumes of
debugging output.
The
looseleaf command-line tool uses
jcommander
to parse command-line arguments, and therefore supports placing command-line arguments
into a file,
one argument per line, and then referencing that file with
@. For example:
All subcommands, unless otherwise specified, yield an exit code of 0 on success, and
a non-zero exit code on failure.
check-configuration - Validate configuration files
The check-configuration command validates configuration files.
The check-configuration command will validate the configuration file specified with
--file.
If the command encounters no errors or warnings, it will not print anything.
create-password
- Validate configuration files
The create-password command validates configuration files.
The create-password command will validate the configuration file specified with
--file.
If the command encounters no errors or warnings, it will not print anything.
Note that, as the command accepts a password directly on the command line, the command
should not be executed on
systems where hostile users may try to read the process environments of other users
on the system. This would
allow the hostile users to capture the unhashed passwords being created. If this is
of concern, the password
can be placed into a file temporarily using
@ syntax.
server
- Start the looseleaf server.
The server command starts a server instance.
The server command will start the server based on the configuration given in the
configuration file specified with
--file. The command does not fork into the background,
and is suitable for use under process supervision.
version - Display the looseleaf version
The version command displays the current version of the command-line tool.
Documentation for the
looseleaf APIs are provided in the
form of
JavaDoc.