The jsycamore package provides...
This section of the documentation describes the theory of operation of the jsycamore
package; the basic concepts and elements that make up the system as a whole.
The top-level object in the
jsycamore package is the
screen. Screens act as containers for all UI elements,
and individual screens are effectively isolated UI instances. Most applications will
only create a single
screen.
Within a screen, the top-level objects are
windows.
Windows provide all the typical windowing system actions such as opening, closing,
maximizing, and resizing.
Inside each window is a
content area within which is placed a tree of
components. Components provide a standard set of UI
elements such as
buttons,
text fields,
images, etc.
The visual appearances of windows, and the components contained within, are completely
decided by
configurable
themes. Components and windows decide
how UI elements
behave and where they are placed, whilst themes are solely responsible
for determining how UI elements
look.
User interaction is implemented using
events. The
application hosting the
screens is responsible for
delivering events to screens, and the screen instances will relay those events to
the relevant windows and
components contained within.
The positions of UI elements are specified as values in strongly-typed
coordinate systems, and the
jsycamore package as a whole works entirely in terms of platform-independent
coordinates. The
jsycamore UI can be embedded into operating systems and used
with graphics standards that use completely different conventions as long as events
are submitted to the UI
using the
jsycamore coordinate systems; extensive static typing is used to make
it difficult to use values in the wrong coordinate system at any point.
A
screen is the top-level object in the
jsycamore package.
A screen acts as a container for
windows, and provides
access to various services such as facilities for opening
menus, handling
text selections, etc.
Most applications will only create a single screen, although there are no restrictions
on creating multiple
screens. Windows (and the components contained within) are confined to a single screen
instance, and cannot
be moved between screen instances.
Screen instances expose
service directories.
Component implementations on a given screen
can obtain references to various services via the screen's service directory. Screens
are required to
provide at least the following mandatory services:
When any of the methods on the
SyScreenMouseEventsType
and
SyScreenKeyEventsType
interfaces are called, the screen will dispatch appropriate events to
components
inside
windows
that are visible on the screen.
When dispatching
mouse events, the process
is as follows for an arbitrary mouse event
e:
Typically, successfully dispatching a mouse event to a window will result in that
window
becoming
focused.
A screen holds a reference to the current
theme.
A screen defines a
viewport within which are placed
windows. The viewport has an explicitly-defined
size, but this may be freely changed at any point.
Applications embedding
jsycamore inside fullscreen applications, or projected
into 3D spaces, are responsible for translating the platform-specific coordinates
produced by keyboard and
mouse input systems into the correct
viewport coordinate space. For example,
jsycamore can be embedded into 3D scenes
by casting rays in whatever is the host 3D engine's coordinate space, and then converting
the ray/plane
intersection coordinates to
jsycamore's viewport space:
A
window is an interactive, resizable, rectangular region that acts as a container for
components.
Each window is assigned an abstract order value by the screen in which it resides. The
order encodes the notion of windows being in a stack, with the window at the top of
the stack being the
frontmost or focused window. When the user clicks on any component within a window,
that window is moved to the top of the stack and becomes focused.
Windows are either
visible, or
invisible. An invisible window
is not rendered, and does not receive
events.
When a window's visibility changes,
events
of type
SyWindowBecameInvisible
or
SyWindowBecameVisible
will be published.
Windows can be grouped into layers. A layer is an integer value
N
such that all windows with a layer value less than
N
are considered to be below windows with layer value N. All
windows with a layer value greater than N are considered to be
above
windows with layer value N.
Within an individual layer,
window order decides
which windows are above or below other windows in that layer.
Window layers are a somewhat advanced feature, and most applications will not need
to use them directly. They
are primarily useful for implementing UI elements that must temporarily appear above
all other elements, such as
notifications or
menus. They are also useful for
implementing user interfaces for games and other simulations; the game's HUD can be
placed on one layer, and a
"pause" screen UI can be placed on a separate layer above the HUD layer, and the layers
can individually be
shown or hidden as is appropriate.
The positions and sizes of windows can be configured to snap to given values. For
example, if a window's positionSnapping attribute is set to
16, whenever the position of a window is set (either manually, or via the user
dragging the window), the position on both axes will snap to the nearest multiple
of
16.
Windows are created with a standard set of attached components. These standard components
allow users to perform
actions such as closing windows, maximizing windows, and moving and resizing windows.
Whether these components
are visible or not is dependent upon both the window's
theme, and other per-window configuration settings.
By default, the standard components are visible if the window is configured as being
decorated.
As an example, the window shown in the above diagram appears as follows when marked
as
undecorated:
Windows can be switched between being decorated or
undecorated
at any time by setting the value of the
decorated
attribute. Note that, without window decorations, the user has very little control
over the window: They cannot
directly close it, move it, or resize it.
The
close button, when clicked, closes the window. The button is a
standard button
and therefore follows the same behaviours with respect to keyboard and mouse interaction.
For a visible window, the close button is visible iff the following conditions hold:
When a window is closed, an event of type SyWindowClosed will be published.
The
maximize button, when clicked, resizes the window to match the size of the
screen's
viewport. If the window has already been
maximized, clicking the button again returns the window to its previous size. The
button is a
standard button
and therefore follows the same behaviours with respect to keyboard and mouse interaction.
For a visible window, the maximize button is visible iff the following conditions
hold:
When a window is maximized, an event of type SyWindowMaximized will be published.
When a window is returned to normal size after having been maximized, an event of
type
SyWindowUnmaximized
will be published.
The
menu button, when clicked, opens a
menu
containing a list of actions that can otherwise be performed by the other standard
window buttons.
Hypothetically, the menu button is capable of subsuming the functionality of the other
window buttons at the
cost of having the user go through an intermediate menu to perform actions. The button
is a
standard button
and therefore follows the same behaviours with respect to keyboard and mouse interaction.
For a visible window, the menu button is visible iff the following conditions hold:
Resize handles
are typically placed on the edges and corners of windows (this is dependent upon the
theme
used), and allow the user to click and drag the handles to resize the windows.
There are up to eight visible resize handles, each corresponding to the eight compass
directions, and each
implementing a specific kind of resize behaviour as detailed in the following sections.
The behaviour of each
handle is specified in terms of the user dragging the handles on the X and Y axes
independently; the resulting
effect on the window should be understood to be the sum of the effects along both
axes.
For a visible window, resize handle H is visible iff the following conditions
hold:
Note that it is not technically a requirement for a theme to allow
any resize handle
to be visible. However, making all handles invisible would be fairly user-hostile.
Some applications may
choose to design themes where only the
southeast
resize handle is visible, as this matches the design of many traditional windowing
systems.
Dragging the northwest handle N units along the Y axis
towards negative infinity will move the window N units towards negative infinity
on the Y axis whilst also enlarging the window by N units on the Y axis.
Dragging the northwest handle M units along the X axis
towards negative infinity will move the window M units towards negative infinity
on the X axis whilst also enlarging the window by M units on the X axis.
Dragging the northwest handle P units along the Y axis
towards positive infinity will move the window P units towards positive infinity
on the Y axis whilst also shrinking the window by P units on the Y axis.
Dragging the northwest handle Q units along the X axis
towards positive infinity will move the window Q units towards positive infinity
on the X axis whilst also shrinking the window by Q units on the X axis.
Dragging the north handle N units along the Y axis
towards negative infinity will move the window N units towards negative infinity
on the Y axis whilst also enlarging the window by N units on the Y axis.
Dragging the north handle M units along the Y axis
towards positive infinity will move the window M units towards positive infinity
on the Y axis whilst also shrinking the window by M units on the Y axis.
Dragging the north handle along the X axis has no effect on the window size or
position.
Dragging the northeast handle N units along the Y axis
towards negative infinity will move the window N units towards negative infinity
on the Y axis whilst also enlarging the window by N units on the Y axis.
Dragging the northeast handle M units along the X axis
towards negative infinity will shrink the window by M units on the X axis.
Dragging the northeast handle P units along the Y axis
towards positive infinity will move the window P units towards positive infinity
on the Y axis whilst also shrinking the window by P units on the Y axis.
Dragging the northeast handle Q units along the X axis
towards positive infinity will enlarge the window by M units on the X axis.
Dragging the east handle M units along the X axis
towards negative infinity will shrink the window by M units on the X axis.
Dragging the northeast handle Q units along the X axis
towards positive infinity will enlarge the window by M units on the X axis.
Dragging the east handle along the Y axis has no effect on the window size or
position.
Dragging the southeast handle N units along the Y axis
towards negative infinity will shrink the window by N units on the Y axis.
Dragging the southeast handle M units along the X axis
towards negative infinity will shrink the window by M units on the X axis.
Dragging the southeast handle P units along the Y axis
towards positive infinity will enlarge the window by P units on the Y axis.
Dragging the southeast handle Q units along the X axis
towards positive infinity will enlarge the window by M units on the X axis.
Dragging the south handle N units along the Y axis
towards negative infinity will shrink the window by N units on the Y axis.
Dragging the south handle P units along the Y axis
towards positive infinity will enlarge the window by P units on the Y axis.
Dragging the south handle along the X axis has no effect on the window size or
position.
Dragging the southwest handle N units along the Y axis
towards negative infinity will shrink the window by N units on the Y axis.
Dragging the southwest handle M units along the X axis
towards negative infinity will move the window M units towards negative infinity
on the X axis whilst also enlarging the window by M units on the X axis.
Dragging the southwest handle P units along the Y axis
towards positive infinity will enlarge the window by P units on the Y axis.
Dragging the southwest handle Q units along the X axis
towards positive infinity will move the window Q units towards positive infinity
on the X axis whilst also shrinking the window by Q units on the X axis.
Dragging the west handle M units along the X axis
towards negative infinity will move the window M units towards negative infinity
on the X axis whilst also enlarging the window by M units on the X axis.
Dragging the west handle Q units along the X axis
towards positive infinity will move the window Q units towards positive infinity
on the X axis whilst also shrinking the window by Q units on the X axis.
Dragging the west handle along the Y axis has no effect on the window size or
position.
The title bar displays the contents of the window's
title
attribute, and allows the user to move the window by dragging with the mouse.
Dragging the
title bar
M
units along the X axis will move the window M units along the X axis.
Dragging the
title bar
N
units along the Y axis will move the window N units along the Y axis.
For a visible window, the title bar is visible iff the following conditions hold:
The
content area is the rectangular region dedicated to the actual
content
of the window; it is the component that serves as the parent component of components
added to the window for
display. The content area itself is effectively invisible; themes should not render
the content area as being
any particular colour or shape. The content area does not consume
any
events.
For a visible window, the content area is always visible.
Components
are the elements that make up the user interface inside
windows. Concretely, a component is any object that
implements the
SyComponentType
interface. The package provides the abstract
SyComponentAbstract class that provides
sensible default implementations of all the required methods.
Components form a tree structure, with each component having an arbitrary number of
child components, and at most
one parent component. A component is said to be
detached
if it has no parent component, and it is also not the
root component for a window. A
detached component is not visited during
layout
operations or during
event dispatch.
A component may be active or inactive. A component can be set to
active or inactive by setting the value of the component's
activity
attribute. Inactivity is implicitly inherited; if the parent component P of a
component
C
is inactive, then C is considered to be inactive.
An inactive component will
not receive events during
event dispatch. Additionally,
themes
are encouraged to render inactive components in a greyed-out form. For example, the
button labelled "Button 1" in
the following image is inactive and so cannot be clicked:
Events
are the primary means by which the
jsycamore
package accepts input from external code, and by which it publishes notifications
on state changes.
Input events
are events that represent input to the UI. In order to remain fully platform-independent,
the
jsycamore
package does not directly read from the keyboard and mouse. Instead, users are expected
to be running
jsycamore
inside an existing system such as
AWT, or perhaps
embedded in a GPU context via
GLFW, and are therefore responsible for translating
input events coming from those systems into method calls defined on the
SyScreenMouseEventsType
and
SyScreenKeyEventsType
interfaces. When these methods are called, the screen will
dispatch
input events internally to any relevant
components.
Output events
are events that indicate state changes that have occurred. For example, closing a
window will publish
an
event.
Users may obtain access to a stream of output events from each
screen.
A
theme is an entity responsible for deciding the visual appearance of
windows
and
components. Primarily, themes can be considered to be
functions from components to sets of
render nodes,
although they do have some other responsibilities with regard to
window components.
Windows
come with a set of predefined decoration
components. Themes are responsible for deciding, given
a set of
layout constraints, where each of these components are
to be placed within the constraints.
A theme contains a set of
theme component functions that, given a component
c, will return a set of
render nodes
that describe how
c
should be rendered. The actual theme component function evaluated for any given component
is chosen based on
the
theme classes
defined for the component.
A theme class is a textual string that is used by themes to locate theme information for
specific components. The process is as follows for a given theme T:
The fallback component function is a function that will yield render nodes if no other
component function can handle the given component and style classes. The fallback
function is defined internally
to the theme, and will typically return render nodes that unambiguously indicate that
something is wrong; the
fallback function should not be evaluated unless something is wrong with the theme
or the component is defined in
a manner that makes it incompatible with the theme.
As components return a non-empty list of style classes in priority order, this allows
components to essentially
instruct themes on how to handle the components including potential fallback behaviour.
For example, a component
can return the list of style classes [AcmeMagicButton, Button]. This is the
same as stating "If you have support for
AcmeMagicButtons, then render this component as one of those. Otherwise, if you
only have support for ordinary buttons, then render this component as one of those."
The jsycamore package defines a set of standard theme classes for which all
themes should attempt to support at a minimum. These names are provided as part of
the
SyThemeClassNameStandard enum.
The jsycamore package defines numerous coordinate systems.
All 2D coordinate systems are defined such that the origin (0, 0) represents the
top left point of the coordinate system, with the X axis pointing
rightwards towards positive infinity, and the Y axis pointing
downwards towards positive infinity.
Positions in an
N-dimensional coordinate system
S with components of type
T in
jsycamore are defined as values of immutable vectors of type
PVectorNT<S>.
The
phantom-type parameter
S on the vector type statically
prevents users of the package from mixing up coordinates in different coordinate systems
and getting unexpected
results. As a concrete example, the
parent-relative
space is a two-dimensional coordinate system with integer component values, and therefore
positions in this
coordinate system are given as values of type
PVector2I<SySpaceParentRelativeType>.
The following sections describe the coordinate spaces currently defined by the
jsycamore package.
The
component-relative space is a two-dimensional integer coordinate system where the
origin
(0, 0) represents the top-left corner of a
component.
The
parent-relative space is a two-dimensional integer coordinate system where the
origin
(0, 0) represents the top-left corner of a parent
component.
The positions of all
components are
specified in this space.
The rgba-pre space is a four-dimensional real coordinate system describing linear,
alpha-premultiplied colours. For example, a pure opaque black colour is represented
by the point
(0,0,0,1).
The
text-aligned space is a two-dimensional integer coordinate system used to address
text. It takes into account the alignment applied to
text for left-to-right and right-to-left text.
The
viewport space is a two-dimensional integer coordinate system where the
origin
(0, 0) represents the top-left corner of a
screen.
The
window space is a two-dimensional integer coordinate system where the
origin
(0, 0) represents the top-left corner of a
window.
A
menu is a
component
that provides a tree-based structure from which users can select items.
The
text
components
are a family of components that provide the ability to display, format, select, and
modify
text.
The jsycamore package works in terms of immutable
text sections. The concept of a text section is rather loosely defined, but it can be
most easily thought of as representing a single paragraph of text. Each text section has
an associated text direction indicating that the text is in a
left-to-right
language (such as English), or that the text is in a
right-to-left
language (such as Arabic, or Hebrew). There is currently no support for mixing directions
within a single text
section, although having many text sections with varying directions inside a single
text view is fully supported.
A font defines the appearance and (ultimately) the layout of text. A font combines
a typeface, a size, and a
style
(such as bold, italic, etc).
Fonts contain all the information required to determine what the bounding box will
be for a given piece of text,
and much more detailed information not limited to the widths and heights of individual
characters. It is for this
reason that fonts are considered to be an essential part of the
layout
of a given piece of text rather than just part of its appearance: A text view might
be sized correctly for a
specific piece of text using font
F, but might be woefully undersized for the same
piece of text using font
G. The font interface exposes text formatting methods that
can take a
text section and a
page width, and return a set of one or more
measured lines that represent the original
text section broken into lines using the given page width as a maximum line length.
This line wrapping
functionality is used by
multi-line text models.
Fonts are represented as instances of the abstract
SyFontType interface. The
jsycamore
package provides implementations that are backed by standard Java
AWT
fonts, but any font system that can provide the information required by the
SyFontType
interface could theoretically be used.
Font instances are obtained by providing an immutable
font description
to a
font directory service
represented by the
SyFontDirectoryServiceType
interface. A font directory service aggregates
font services, represented by the
SyFontServiceType
interface, typically by loading instances of these services from
ServiceLoader.
Applications are required to ensure that they include font modules for the fonts they
use. There is no support for
reading fonts directly from the host system as this would make applications behave
differently on different host
systems.
The jsycamore package includes the following optional font modules:
A
measured line is an immutable value containing a single line of text augmented with
information derived from an associated
font. It
contains enough information to answer questions such as "If the user clicks the text
at the
parent-relative
position
(x, y), which character within the text did they click on?".
The text location information returned by measured line queries includes
information not limited to the character at the given location, whether the location
is at the leading edge of
a character, the index of the character within the given line, and a bounding box
for use in rendering a
caret.
A text model is a mutable structure that handles wrapping,
editing, and selection operations for text. A text model may
either be single-line or multi-line.
A multi-line text model is a mutable structure that maintains a list of
text sections, and transforms the text sections
into a potentially much larger list of
measured lines
based upon a given
page width
value.
The work required to implement
text selection in a multi-line model is relatively
complex. The model must correctly handle
forward selection and
backward selection, and must do so in the presence of
text sections
having different text directions within the same model.
In order to reason about text selections, it is necessary to describe the property
of logical
ordering for characters in text.
A character C is considered logically earlier in a
given piece of text than a character B if C
would be encountered before B in terms of reading order for the text.
For example, in the English sentence "The quick brown fox jumped over the lazy dogs",
the character
q is logically earlier than the character
z when the sentence is read. Conveniently, this order matches the order in
which the characters are actually stored as strings in memory, and the order does
not change regardless
of the text direction of the language in which the text is written. Accordingly, the
logically later property is simply the inverse of the
logically earlier property.
The output of the text selection progress is the inclusive lower and inclusive upper
bounds of the
selection in logical order, and a set of
selection region bounding boxes
that can be used during
rendering to render
selection boxes around the relevant text. The selection regions are trivial to create;
the selection
region for a given line is the union of the bounding boxes of the characters that
are selected on that line
(taken from the associated
font),
and the set of regions for all lines spanned by the selection is the set of per-line
regions.
Forward selection
is the process of the user dragging the mouse across text, starting from an initial
position in the text and
moving the mouse to a logically
later
position in the text. For example:
In the diagram above, the user is performing a forward selection in left-to-right
text. The text consists of
two measured lines. The dotted arrows represent the mouse dragging directions. The
user placed the cursor
before the letter q in
"quick"
and dragged the cursor rightwards and then downwards. This results in a selection
that spans the two lines.
The selection is a forward selection because the selection progressed in a manner
that started from a position
that is logically earlier in the text, and progressed to a location that is logically
later in the text.
Backward selection
is the process of the user dragging the mouse across text, starting from an initial
position in the text and
moving the mouse to a logically
earlier
position in the text according to the text direction. For example:
In the diagram above, the user is performing a backward selection in left-to-right
text. The text consists of
two measured lines. The dotted arrows represent the mouse dragging directions. The
user placed the cursor
before the letter t in
"the"
and dragged the cursor upwards and then leftwards. This results in a selection that
spans the two lines. The
selection is a backward selection because the selection progressed in a manner that
started from a position
that is logically later in the text, and progressed to a location that is logically
earlier in the text.
For right-to-left text, the situation does not change in that forward selections progress
from logically
earlier positions to logically later positions, and backward selections progress from
logically later
positions to logically earlier positions, but the physical direction in which the
user drags the selection
cursor is reversed. For example:
In the diagram above, the user is performing a forward selection in right-to-left
text. The text consists of
two measured lines. The dotted arrows represent the mouse dragging directions. The
user placed the cursor
before the letter פ in
"אנציקלופדיה"
and dragged the cursor leftwards and then downwards. This results in a selection that
spans the two lines. The
selection is a forward selection because the selection progressed in a manner that
started from a position
that is logically earlier in the text, and progressed to a location that is
logically later in the text.
Conversely, for backward selection:
In the diagram above, the user is performing a backward selection in right-to-left
text. The text consists of
two measured lines. The dotted arrows represent the mouse dragging directions. The
user placed the cursor
after the letter ם in
"בדפים"
and dragged the cursor upwards and then rightwards. This results in a selection that
spans the two lines. The
selection is a backward selection because the selection progressed in a manner that
started from a position
that is logically later in the text, and progressed to a location that is logically
earlier in the text.
The process of text selection therefore takes the following steps:
During each of the above steps, the lines between the lower and upper bounds of the
selection in progress
are examined. The lower and upper bounds may span several lines, or may simply both
be on the same line.
Based on the examined lines, a set of
selection regions is generated. The algorithm
used differs based on whether the selection is a forward selection or a backward selection.
For each
selection direction, the algorithm proceeds by case analysis:
For each case in the above algorithm, we create a selection region that's relevant
to the text direction
of the given line. For example, a region that extends from the "start of the line"
in left-to-right text
is a region that starts from the leftmost edge of the text area in
parent-relative space. Conversely, a region
that extends from the "start of the line" in right-to-left text
is a region that starts from the rightmost edge of the text area in parent-relative
space.
For backward selection, the algorithm is similar:
Following the above algorithm, we can correctly build selection regions that span
multiple lines where
each line has its own text direction:
A single-line text model is very simple, and augments a single
measured line
with the information required for performing text selections.
Text selection for single-line models is simply the
K = P case for
forward
and
backward multi-line selections.
By definition, the upper and lower bounds of selections in single-line models are
always
on the same line.
A
button is a simple
component
that triggers a
listener
when the user clicks on it.
Buttons always have exactly one assigned listener. A listener is simply a parameterless
function that is evaluated by the button implementation and is expected to perform
some kind of side effect. By
default, buttons have a listener that does nothing. A listener can be assigned to
or removed from the button at
any time, and "removing" the listener simply re-assigns a default listener that does
nothing when evaluated.
Button components implement the given state machine
:
Buttons begin in the
BUTTON_IDLE state. If the button is
inactive, it will stay in the
BUTTON_IDLE
state regardless of mouse or keyboard events.
Otherwise, assuming that the button is active, if the user moves the mouse cursor
over the button, the button
transitions to the BUTTON_OVER state. If the user then moves the cursor such that it
is no longer over the button, the button transitions back to the BUTTON_IDLE state.
When in the BUTTON_OVER state, if the user presses the primary mouse button, the
button transitions to the BUTTON_PRESSED state. If the user moves the mouse cursor
such that it is no longer over the button (without releasing the primary mouse button),
the button transitions to
the
BUTTON_PRESSED_NOT_OVER
state. Otherwise, if the user leaves the mouse cursor over the button and then releases
the primary mouse button,
the button transitions to the
BUTTON_CLICKED
state. The button evaluates the associated listener and then either transitions to
the
BUTTON_OVER
state if the mouse cursor is still over the button, or transitions to the BUTTON_IDLE state
if the mouse cursor is no longer over the button.
In the BUTTON_PRESSED_NOT_OVER state, the user can either release the primary mouse
button, which returns the button to the BUTTON_IDLE state, or the user can move the
cursor such that it is back over the button again, resulting in the button transitioning
back to the
BUTTON_PRESSED
state.
The
BUTTON_CLICKED
state is the only state in which the listener will be evaluated, and the listener
will be evaluated exactly once
upon each transition to the
BUTTON_CLICKED
state.