io7m-smfj 0.5.1 Specification
1
Notational Conventions
1.1
Unicode
1
The specification makes reference to the Unicode character set which, at the time of writing, is at version 8.0.0. The specification often references specific Unicode characters, and does so using the standard notation U+NNNN, where N represents a hexadecimal digit. For example, U+03BB corresponds to the lowercase lambda symbol λ.
1.2
EBNF
1
The specification gives grammar definitions in ISO/IEC 14977:1996 Extended Backus-Naur form.
1.3
Haskell
1
Rather than rely on untyped and ambiguous mathematical notation, this documentation expresses all mathematics and type definitions in strict Haskell 2010 with no extensions. All Haskell sources are included along with the documentation and can therefore be executed from the command line GHCi tool in order to interactively check results and experiment with functions.
2
When used within prose, functions are referred to using fully qualified notation, such as (Vector3f.cross n t). This is the application of the cross function defined in the Vector3f module, to the arguments n and t.
1.4
JPRA
1
Definitions of binary structures are given as jpra language definitions. All values are considered to be in big-endian byte order unless otherwise specified.
2
Model
2.1
Overview
1
The SMF model is a minimalist, portable storage model for triangle mesh data. SMF files are intended to be consumed by 3D rendering engines directly and therefore do not contain any complex interchange features common to formats such as COLLADA [0]. The text and binary encodings of the model are designed to be trivial to parse and to permit the easy construction of extremely fast event-based parsers that do not require loading the entire file into memory for processing.
2.2
Mesh
1
A mesh in the SMF model is a set of triangles.
2.3
Attributes
1
An attribute in the SMF model is a uniquely-named array of elements of a given type. All attributes within a particular SMF file have the same number of elements.
2
Implementations are required to support attributes of at least the following types:
2.3.3 Types
  • 8/16/32/64-bit signed integer vectors of 1-4 components
  • 8/16/32/64-bit unsigned integer vectors of 1-4 components
  • 16/32/64-bit floating point vectors of 1-4 components
2.4
Vertices
1
A vertex is an abstract object consisting of exactly one element taken from each of the defined attributes. A vertex can essentially be considered to be an array index; The vertex at index n can be considered to be the aggregation of the nth elements of all of the defined attributes. Vertices are numbered starting at 0.
2.5
Triangles
1
A triangle is a 3-tuple of vertices. In the SMF model, a triangle references vertices by their numeric index.
2.6
Coordinate System
1
A coordinate system in the SMF model is a set of three axis names and a triangle winding order. The axis names specify which Cartesian coordinate system axes correspond to the right, up, and forward directions. The winding order specifies whether vertices for triangles are given in clockwise or counter-clockwise order. A coordinate system is only valid if the three chosen axes have different axis names. The axisName function relates an axis to an axis name. The axisValid function gives a formal definition of the validity of sets of axes.
2.6.2 Axis
module Axis where

data Axis
  = PositiveX
  | PositiveY
  | PositiveZ
  | NegativeX
  | NegativeY
  | NegativeZ

data AxisName
  = AxisX
  | AxisY
  | AxisZ

axisName :: Axis -> AxisName
axisName PositiveX = AxisX
axisName NegativeX = AxisX
axisName PositiveY = AxisY
axisName NegativeY = AxisY
axisName PositiveZ = AxisZ
axisName NegativeZ = AxisZ

data AxisSystem = AxisSystem {
  axis_right   :: Axis,
  axis_up      :: Axis,
  axis_forward :: Axis
}

axisValid :: (Axis, Axis, Axis) -> Bool
axisValid (right, up, forward) =
  case (axisName right, axisName up, axisName forward) of
    (AxisX, AxisY, AxisZ) -> True
    (AxisZ, AxisX, AxisY) -> True
    (AxisY, AxisZ, AxisX) -> True
    _                     -> False

data WindingOrder
  = WindingOrderClockwise
  | WindingOrderCounterClockwise

data CoordinateSystem = CoordinateSystem {
  coords_axes  :: AxisSystem,
  coords_order :: WindingOrder
}
2.7
Schema ID
1
A schema identifier is an optional identifier that can be inserted into SMF files. Because the SMF model does not predefine any particular attributes, tools that consume SMF files cannot know ahead of time if the file they have just loaded will actually contain the attributes that they are expecting. A schema identifier effectively provides a concrete name for a set of attributes so that tools that process SMF files can perform validation of attributes based on the identifier. It is somewhat analogous to XML namespace [1] declarations; The author of a particular document inserts an XML namespace identifier into their document, and validation tools use this identifier to locate schema definitions against which the document is then validated.
2
A schema identifier consists of four integer values:
2.7.3 Schema Identifier
data SchemaIdentifier = SchemaIdentifier {
  vendor_id            :: Word32,
  schema_id            :: Word32,
  schema_version_major :: Word32,
  schema_version_minor :: Word32
}
4
The vendor_id uniquely identifiers the vendor, the schema_id uniquely identifies the file type, and the schema_version_major and schema_version_minor values identify the version of the schema.
2.8
Metadata
1
The SMF model supports the inclusion of arbitrary metadata. A metadata value is an opaque data value with vendor and schema identifiers specified in the same manner as the schema identifier. Applications may use the metadata facilities to associate small amounts of extra information with mesh data. Typical uses include checksums, copyright information, and generating software package version information.
3
SMFT - Text Encoding
3.1
Overview
1
The SMFT format is a text encoding for the SMF model. The format is separated into a header and data section.
2
The format is line-based, with one command per line. Lines (excluding the very first line in the file) containing only whitespace, or lines beginning with U+0023 # are ignored.
3.2
Header
1
The first line of an SMFT file MUST consist of an smf command. After the initial command, the header consists of any number of header commands and is terminated by the data command.
2
A header command is one of the following:
3.2.3 Header Commands
3.3
Header - smf
1
The smf command specifies the major and minor version of the specification to which the rest of the file is expected to conform. The first argument specifies the major version, and the second argument specifies the minor version. Implementations are required to immediately reject files that are not of supported versions, and are required to halt processing of the rest of the file in the case of syntax errors.
2
The command takes the following arguments:
3.3.3 smf Arguments
  1. The non-negative SMF major version
  2. The non-negative SMF minor version
3.3.4 smf
header_command_smf =
  "smf" , integer-unsigned , integer-unsigned ;
3.4
Header - vertices
1
The vertices command specifies the number of vertices that will appear in the data section for each attribute.
3.4.2 vertices
header_command_vertices =
  "vertices" , integer-unsigned ;
3
The command takes the following arguments:
3.4.4 vertices Arguments
  1. The non-negative number of vertices that will appear in the file
5
The command is required to appear exactly once in the header.
3.5
Header - schema
1
The schema command specifies the schema identifier for the file.
3.5.2 schema
header_command_schema =
  "schema" , integer-unsigned-base16 , integer-unsigned-base16 , integer-unsigned , integer-unsigned ;
3
The command takes the following arguments:
3.5.4 schema Arguments
  1. The vendor ID as a hexadecimal integer
  2. The schema ID as a hexadecimal integer
  3. The non-negative schema major version
  4. The non-negative schema minor version
5
The command is allowed to appear at most once in the header. If the command is not specified, a default schema identifier consisting of all zeroes is implicitly defined.
3.6
Header - triangles
1
The triangles command specifies the number of triangles that will appear in the data section of the file, and the size of the vertex indices in bits.
3.6.2 triangles
header_command_triangles =
  "triangles" , integer-unsigned , integer-unsigned ;
3
The command takes the following arguments:
3.6.4 triangles Arguments
  1. The non-negative triangle count
  2. The non-negative size in bits of a triangle vertex index
5
The command is required to appear exactly once in the header.
3.7
Header - attribute
1
The attribute command specifies an attribute that will appear in the data section of the file.
3.7.2 attribute
attribute_name =
  ? [\p{IsAlphabetic}\p{IsDigit}_\-\.:]{1,64} ? ;

attribute_type =
  "integer-signed" | "integer-unsigned" | "float" ;

header_command_attribute =
  "attribute" , attribute_name , attribute_type , integer-unsigned , integer-unsigned ;
3
The command takes the following arguments:
3.7.4 attributes Arguments
  1. The name of the attribute
  2. The type of the components in the attribute
  3. The non-negative number of components in the attribute
  4. The non-negative size in bits of an individual component
5
The command is allowed to appear any number of times in the header, but all specified attribute names must be unique.
3.8
Header - coordinates
1
The coordinates command specifies the coordinate space of the mesh data that will follow in the data section of the file.
3.8.2 coordinates
axis =
  "+x" | "-x" | "+y" | "-y" | "+z" | "-z" ;

winding_order =
  "clockwise" | "counter-clockwise" ;

header_command_coordinates =
  "coordinates" , <axis> , <axis> , <axis> , <winding_order> ;
3
The command takes the following arguments:
3.8.4 coordinates Arguments
  1. The axis for the right direction
  2. The axis for the up direction
  3. The axis for the forward direction
  4. The triangle winding order
3.9
Header - meta
1
The meta command specifies the number of metadata values that will appear in the data section.
3.9.2 meta
header_command_meta =
  "meta" , integer-unsigned ;
3
The command takes the following arguments:
3.9.4 meta Arguments
  1. The non-negative number of metadata values that will appear in the file
5
The command is allowed to appear at most once in the header.
3.10
Data
1
The data section of the file specifies the data for attributes and triangles. The section begins with the data command and consists of both data and data commands.
2
A data command is one of the following:
3.10.3 Data Commands
3.11
Data - data
1
The data command indicates the start of the data section.
3.11.2 data
data_command_data =
  "data" ;
3
The command takes no arguments.
3.12
Data - attribute
1
The attribute command indicates the start of data for an attribute.
3.12.2 attribute
data_command_attribute =
  "attribute" , attribute_name ;
3
The command takes the following arguments:
3.12.4 attribute Arguments
  1. The name of the attribute
5
The named attribute must have been declared in the header. For an attribute a that specifies c components of type t, in a file that is specified to contain v vertices, the next v non-empty, non-commented lines will contain exactly c whitespace-separated values of type t.
3.13
Data - triangles
1
The triangles command indicates the start of triangle data.
3.13.2 triangles
data_command_triangles =
  "triangles" ;
3
The command takes no arguments.
4
In a file that is specified to contain k triangles, the next k non-empty, non-commented lines will contain exactly 3 whitespace-separated unsigned integer values each representing the index of a vertex.
3.14
Data - metadata
1
The metadata command indicates the start of metadata.
3.14.2 metadata
data_command_metadata =
  "metadata" ;
3
The command takes no arguments.
4
The command may only appear in the file after all triangles and attributes have been received.
3.15
Data - meta
1
The meta command indicates the start of the file metadata.
3.15.2 meta
data_command_meta =
  "meta" , integer-unsigned-base16 , integer-unsigned-base16 , integer-unsigned ;
3
The command takes the following arguments:
3.15.4 meta Arguments
  1. The vendor ID as a hexadecimal integer
  2. The schema ID as a hexadecimal integer
  3. The non-negative number of lines of metadata that will follow the command
5
For any v, s, and n, the command meta v s n will indicate that the next n lines will contain RFC4648 base64url encoded data of a type with vendor v and schema s.
6
In a file that has declared in the header that m values will appear in the data section, the meta command must be present exactly m times, in any order, following a single metadata command.
3.16
Example
1
The following is a complete example of an SMFT file:
3.16.2 smf
smf 1 0
schema 696f376d a0b0c0d0 1 0
meta 2
vertices 9
triangles 4 32
coordinates +x +y -z counter-clockwise
attribute "POSITION" float 3 32
attribute "NORMAL" float 3 32
attribute "UV:UVMap" float 2 32
attribute "GROUP:group0" float 1 32
data
attribute "POSITION"
0.000000000000000 0.000000000000000 0.000000000000000
1.000000000000000 0.000000000000000 0.000000000000000
0.000000000000000 0.000000000000000 -2.000000000000000
1.000000000000000 0.000000000000000 -2.000000000000000
2.000000000000000 0.000000000000000 0.000000000000000
2.000000000000000 0.000000000000000 -2.000000000000000
1.000000000000000 0.000000000000000 0.000000000000000
1.000000000000000 0.000000000000000 -2.000000000000000
1.000000000000000 0.000000000000000 0.000000000000000
attribute "NORMAL"
0.000000000000000 1.000000000000000 0.000000000000000
0.000000000000000 0.999999940395355 0.000000000000000
0.000000000000000 1.000000000000000 0.000000000000000
0.000000000000000 1.000000000000000 0.000000000000000
0.000000000000000 1.000000000000000 0.000000000000000
0.000000000000000 1.000000000000000 0.000000000000000
0.000000000000000 0.999999940395355 0.000000000000000
0.000000000000000 1.000000000000000 0.000000000000000
0.000000000000000 0.999999940395355 0.000000000000000
attribute "UV:UVMap"
0.112528264522552 0.912521243095398
0.112528264522552 0.712549567222595
0.512471735477448 0.912521243095398
0.696853816509247 0.614087224006653
0.396896183490753 0.464108407497406
0.696853816509247 0.464108407497406
0.396896213293076 0.614087224006653
0.512471735477448 0.712549448013306
0.396896213293076 0.614087224006653
attribute "GROUP:group0"
0.000000000000000
0.300000011920929
0.200000002980232
0.400000005960465
0.600000023841858
0.699999988079071
0.300000011920929
0.400000005960465
0.300000011920929
triangles
1 2 0
6 5 3
1 7 2
8 4 5
metadata
meta 696f376d a1b1c1d0 1
aGVsbG8taGVsbG8K
meta 696f376d a0b0c0d1 5
AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8gISIjJCUmJygpKissLS4vMDEyMzQ1
Njc4OTo7PD0-P0BBQkNERUZHSElKS0xNTk9QUVJTVFVWV1hZWltcXV5fYGFiY2RlZmdoaWpr
bG1ub3BxcnN0dXZ3eHl6e3x9fn-AgYKDhIWGh4iJiouMjY6PkJGSk5SVlpeYmZqbnJ2en6Ch
oqOkpaanqKmqq6ytrq-wsbKztLW2t7i5uru8vb6_wMHCw8TFxsfIycrLzM3Oz9DR0tPU1dbX
2Nna29zd3t_g4eLj5OXm5-jp6uvs7e7v8PHy8_T19vf4-fr7_P3-_w==
4
SMFB - Binary Encoding
4.1
Header
1
The SMFB format is a binary encoding for the SMF model. The format is separated into a header and data section.
2
All SMFB files begin with a fixed header. The start of the header uses a fixed magic number and a major/minor version number pair indicating the major and minor versions of the specification that defines the file structure.
4.1.3 Header (Start)
[record SMFBHeaderStart
  ([field magic           [integer unsigned 64]]
   [field version_major   [integer unsigned 32]]
   [field version_minor   [integer unsigned 32]])]
4
The magic number MUST always be 0x89534d460d0a1a0a; implementations are required to immediately reject any files that do not begin with this magic number. The derivation of this constant is taken almost verbatim from the PNG [2] file format with the characters PNG changed to SMF .
5
The rest of the header is given by the SMFBV1Header type:
4.1.6 Header (Main)
[record SMFBV1Header
  ([field schema                   SMFBV1SchemaID]
   [field vertex_count             [integer unsigned 64]]
   [field triangle_count           [integer unsigned 64]]
   [field triangle_index_size_bits [integer unsigned 32]]
   [field coordinate_system        SMFBV1CoordinateSystems]
   [padding-octets 2]
   [padding-octets 4]
   [field meta_count               [integer unsigned 32]]
   [field attribute_count          [integer unsigned 64]])]
7
The SMFBV1SchemaID type encodes the schema ID:
4.1.8 Header (schema ID)
[record SMFBV1SchemaID
  ([field vendor_id            [integer unsigned 32]]
   [field schema_id            [integer unsigned 32]]
   [field schema_version_major [integer unsigned 32]]
   [field schema_version_minor [integer unsigned 32]])]
9
The SMFBV1CoordinateSystems type encodes the coordinate system:
4.1.10 Header (Coordinate system)
[packed SMFBV1CoordinateSystems
  ([field right   [integer unsigned 3]]
   [field up      [integer unsigned 3]]
   [field forward [integer unsigned 3]]
   [field winding [integer unsigned 2]]
   [padding-bits 5])]
11
The axis and winding order values are related to integers in the binary encoding via the following functions:
4.1.12 Header (Coordinate system mapping)
import qualified Axis as A

axisOrdinal :: A.Axis -> Integer
axisOrdinal A.PositiveX = 0
axisOrdinal A.PositiveY = 1
axisOrdinal A.PositiveZ = 2
axisOrdinal A.NegativeX = 3
axisOrdinal A.NegativeY = 4
axisOrdinal A.NegativeZ = 5

windingOrdinal :: A.WindingOrder -> Integer
windingOrdinal A.WindingOrderClockwise        = 0
windingOrdinal A.WindingOrderCounterClockwise = 1
13
The header specifies the number of vertices present in the file, the number of triangles present in the file, the size in bits of the individual triangle indices, and the number of attributes present in the file.
14
The header concludes by giving definitions for exactly n attributes, where n is the attribute count specified in the previous section of the header.
4.1.15 Header (Attributes)
[record SMFBV1Attribute
  ([field name            [string 64 "UTF-8"]]
   [field component_kind  [integer unsigned 32]]
   [field component_count [integer unsigned 32]]
   [field component_size  [integer unsigned 32]])]
16
The header is constructed with no implicit padding and no particular alignment. However, the individual header structures are constructed such that their sizes are always a multiple of 8. This ensures that the data that immediately follows the header also starts on an 8 octet boundary.
4.2
Data
1
After the header, the data for each attribute is given in full in the order in which the attributes were declared in the header. Specifically, in a file containing v vertices and a list of n attributes a, the file will contain v values of the type specified in a !! 0, followed by v values of the type specified by a !! 1, and so on up to a !! n - 1.
2
The start of the data for each attribute is aligned to the next 8 octet boundary regardless of type. The alignment is achieved by inserting padding octets at the end of the previous attribute. For example, the following diagram shows how the data is aligned for a file containing three vertices with two attributes A and B. Attribute A consists of three 16-bit floating-point components, and attribute B consists of three 32-bit floating-point components. As the file has three vertices, 3 * 3 = 9 values are placed into the file first, consuming 3 * 3 * 2 = 18 octets. Because the next octet, 18, is not divisible by 8, it's necessary to insert 6 octets of padding so that the data for B will start on the next 8 octet boundary.
4.2.3 Padding
Padding for alignment
4
All values are stored in big-endian form.
5
After all triangle and attribute data has been specified, the metadata values are specified. A metadata value begins with the following structure:
4.2.6 Metadata
[record SMFBV1Metadata
  ([field vendor [integer unsigned 32]]
   [field schema [integer unsigned 32]]
   [field size   [integer unsigned 64]])]
7
The vendor and schema fields are for use by applications to identify the types of metadata. The size field records the size of the metadata in octets.
8
This structure is followed by the opaque metadata octets with any additional padding to ensure that the next metadata value occurs on an 8 octet boundary. The padding is not included in the recorded size of the metadata.
[0]
https://en.wikipedia.org/wiki/COLLADA
[1]
https://en.wikipedia.org/wiki/XML_Namespace
[2]
https://en.wikipedia.org/wiki/Portable_Network_Graphics#File_header