SPIRV-Tools/readme.md

SPIR-V Tools

Overview

The project includes an assembler, disassembler, and validator for SPIR-V, all based on a common static library. The library contains all of the implementation details and is used in the standalone tools whilst also enabling integration into other code bases directly.

Supported features

Assembler and disassembler

• Based on Revision 31 of SPIR-V
• All core instructions are supported, except:
  • Changes to operand enum values and their syntax since Rev 30 will not work.
  • Image operands (section 3.14) are not supported, for example.
  • Test coverage is very limited.
• All GLSL std450 extended instructions are supported.
• Assembler only does basic syntax checking. No cross validation of IDs or types is performed.
• OpenCL extended instructions are not supported.

Validator

The validator is incomplete. See the Future Work section for more information.

CHANGES (for tools hackers)

2015-09-09

• Avoid confusion about ownership of storage:
  • spv_binary is only used for output of the assembler, and should always be destroyed with spvBinaryDestroy.
  • spv_text is only used for output of the disassembler, and should always be destroyed with spvTextDestroy.
  • Inputs to the assembler and disassembler are provided as pointer and length arguments.
• Fixed parsing of floating point literals.
• Fixed the -p option for the disassembler executable.
• Fixed a build break on MSVC when using a ternary operator with conflicting types.
• More test coverage and other cleanups.

2015-09-04

• The parser has been overhauled
  • We use an automatically generated table to describe the syntax of each core instruction. The changes to the SPIR-V spec document generator to create this table are still being developed.
  • The parser uses a dynamically updated list of expected operand types. It is expanded as needed for variable-length lists of operands, and consumed during the parse. See the uses of spv_operand_pattern_t.
  • The syntax of enum operands and their potential operands is still hand-coded. (That might change depending on the cost-benefit tradeoff.)
• We are actively increasing test coverage.
• We have tweaked the CMake build rules to make it easier to integrate into other packages. Google is integrating SPIR-V Tools into the Vulkan conformance test suite.
• New code tends to use Google C++ style, including formatting as generated by clang-format --style=google.
• The spvBinaryToText and spvTextToBinary interfaces have been updated to remove a conceptual ambiguity that arises when cleaning up spv_binary_t and spv_text_t objects.

Where is the code?

The master branch of the repository is maintained by Kenneth Benzie k.benzie@codeplay.com.

You are looking at the google branch. Google plans to maintain a linear history of commits.

Please submit any merge requests as stated in these instructions.

Build

The project uses CMake to generate platform-specific build configurations. To generate these build files issue the following commands.

mkdir <spirv-dir>/build
cd <spirv-dir>/build
cmake [-G<platform-generator>] ..

Once the build files have been generated, build using your preferred development environment.

CMake Options

• SPIRV_USE_SANITIZER=<sanitizer> - on UNIX platforms with an appropriate version of clang this option enables the use of the sanitizers documented here, this should only be used with a debug build, disabled by default
• SPIRV_COLOR_TERMINAL=ON - enables color console output, enabled by default
• SPIRV_WARN_EVERYTHING=OFF - on UNIX platforms enable the -Weverything compiler front end option, disabled by default
• SPIRV_WERROR=OFF - on UNIX platforms enable the -Werror compiler front end option, disabled by default

Library

Usage

In order to use the library from an application, the include path should point to <spirv-dir>/include, which will enable the application to include the header <spirv-dir>/include/libspirv/libspirv.h then linking against the static library in <spirv-build-dir>/bin/libSPIRV.a or <spirv-build-dir>/bin/SPIRV.lib. The intention is for this to be a C API, however currently it relies on the generated header spirv.h meaning this is currently a C++ API.

• SPIRV - the static library CMake target outputs <spirv-dir>/lib/libSPIRV.a on Linux/Mac or <spirv-dir>/lib/SPIRV.lib on Windows.

Entry Points

There are three main entry points into the library.

• spvTextToBinary implements the assembler functionality.
• spvBinaryToText implements the disassembler functionality.
• spvValidate implements the validator functionality.

Source

In addition to the interface header <spirv-dir>/include/libspirv/libspirv.h the implementation source files reside in <spirv-dir>/source/*.

Tools

Assembler

The standalone assembler is the binary called spirv-as and is located in <spirv-build-dir>/bin/spirv-as. The functionality of the assembler is implemented by the spvTextToBinary library function.

The assembler operates on the textual form.

• spirv-as - the standalone assembler
  • <spirv-dir>/bin/spirv-as

Options

• -o <filename> is used to specify the output file, otherwise this is set to out.spv.

Format

The assembly attempts to adhere to the binary form as closely as possible using text names from that specification. Here is an example.

OpCapability Shader
OpMemoryModel Logical Simple
OpEntryPoint GLCompute %3 "main"
OpExecutionMode %3 LocalSize 64 64 1
OpTypeVoid %1
OpTypeFunction %2 %1
OpFunction %1 %3 None %2
OpLabel %4
OpReturn
OpFunctionEnd

In order to improve the text's readability, the <result-id> generated by an instruction can be moved to the beginning of that instruction and followed by an = sign. This allows us to distinguish between variable defs and uses and locate variable defs more easily. So, the above example can also be written as:

     OpCapability Shader
     OpMemoryModel Logical Simple
     OpEntryPoint GLCompute %3 "main"
     OpExecutionMode %3 LocalSize 64 64 1
%1 = OpTypeVoid
%2 = OpTypeFunction %1
%3 = OpFunction %1 None %2
%4 = OpLabel
     OpReturn
     OpFunctionEnd

Each line encapsulates one and only one instruction, or an OpCode and all of its operands. OpCodes use the names provided in section 3.28 Instructions of the SPIR-V specification, immediate values such as Addressing Model, Memory Model, etc. use the names provided in sections 3.2 Source Language through 3.27 Capability of the SPIR-V specification. Literals strings are enclosed in quotes "<string>" while literal numbers have no special formatting.

ID Definitions & Usage

An ID definition pertains to the <result-id> of an OpCode, and ID usage is any input to an OpCode. All IDs are prefixed with %. To differentiate between defs and uses, we suggest using the second format shown in the above example.

Named IDs

The assembler also supports named IDs, or virtual IDs, which greatly improves the readability of the assembly. The same ID definition and usage prefixes apply. Names must begin with an character in the range [a-z|A-Z]. The following example will result in identical SPIR-V binary as the example above.

          OpCapability Shader
          OpMemoryModel Logical Simple
          OpEntryPoint GLCompute %main "main"
          OpExecutionMode %main LocalSize 64 64 1
  %void = OpTypeVoid
%fnMain = OpTypeFunction %void
  %main = OpFunction %void None %fnMain
%lbMain = OpLabel
          OpReturn
          OpFunctionEnd

Arbitrary Integers

Warning: Not all of the following has been implemented

When writing tests it can be useful to emit an invalid 32 bit word into the binary stream at arbitrary positions within the assembly. To specify an arbitrary word into the stream the prefix ! is used, this takes the form !<integer>. Here is an example.

OpCapability !0x0000FF00

Any word in a valid assembly program may be replaced by !<integer> -- even words that dictate how the rest of the instruction is parsed. Consider, for example, the following assembly program:

%4 = OpConstant %1 123 456 789 OpExecutionMode %2 LocalSize 11 22 33
OpExecutionMode %3 InputLines

The words OpConstant, LocalSize, and InputLines may be replaced by random !<integer> values, and the assembler will still assemble an output binary with three instructions. It will not necessarily be valid SPIR-V, but it will faithfully reflect the input text.

You may wonder how the assembler recognizes the instruction structure (including instruction boundaries) in the text with certain crucial words replaced by arbitrary integers. If, say, OpConstant becomes a !<integer> whose value differs from the binary representation of OpConstant (remember that this feature is intended for fine-grain control in SPIR-V testing), the assembler generally has no idea what that value stands for. So how does it know there is exactly one <id> and three number literals following in that instruction, before the next one begins? And if LocalSize is replaced by an arbitrary !<integer>, how does it know to take the next three words (instead of zero or one, both of which are possible in the absence of certainty that LocalSize provided)? The answer is a simple rule governing the parsing of instructions with !<integer> in them:

When a word in the assembly program is a !<integer>, that integer value is emitted into the binary output, and parsing proceeds differently than before: each subsequent word not recognized as an OpCode is emitted into the binary output without any checking; when a recognizable OpCode is eventually encountered, it begins a new instruction and parsing returns to normal. (If a subsequent OpCode is never found, then this alternate parsing mode handles all the remaining words in the program. If a subsequent OpCode is in an assignment format, the ID preceding it begins a new instruction, even if that ID is itself a !<integer>.)

The assembler processes the words encountered in alternate parsing mode as follows:

• If the word is a number literal, it outputs that number as one or more words, as defined in the SPIR-V specification for Literal Number.
• If the word is a string literal, it outputs a sequence of words representing the string as defined in the SPIR-V specification for Literal String.
• If the word is an ID, it outputs the ID's internal number. If no such number exists yet, a unique new one will be generated. (Uniqueness is at the translation-unit level: no other ID in the same translation unit will have the same number.)
• If the word is another !<integer>, it outputs that integer.
• Any other word causes the assembler to quit with an error.

Note that this has some interesting consequences, including:

• When an OpCode is replaced by !<integer>, the integer value should encode the instruction's word count, as specified in the physical-layout section of the SPIR-V specification.

• Consecutive instructions may have their OpCode replaced by !<integer> and still produce valid SPIR-V. For example, !262187 %1 %2 "abc" !327739 %1 %3 6 %2 will successfully assemble into SPIR-V declaring a constant and a PrivateGlobal variable.

• Enums (such as DontInline or SubgroupMemory, for instance) are not handled by the alternate parsing mode. They must be replaced by !<integer> for successful assembly.

• The = sign cannot be processed by the alternate parsing mode if the OpCode following it is a !<integer>.

• When replacing a named ID with !<integer>, it is possible to generate unintentionally valid SPIR-V. If the integer provided happens to equal a number generated for an existing named ID, it will result in a reference to that named ID being output. This may be valid SPIR-V, contrary to the presumed intention of the writer.

Disassembler

The standalone disassembler is the binary called spirv-dis and is located in <spirv-build-dir>/bin/spirv-dis. The functionality of the disassembler is implemented by the spvBinaryToText library function.

The disassembler operates on the binary form.

• spirv-dis - the standalone disassembler
  • <spirv-dir>/bin/spirv-dis

Options

• -o <filename> is used to specify the output file, otherwise this is set to out.spvasm.
• -p prints the assembly to the console on stdout, this includes colored output on Linux, Windows, and Mac.

Validator

The standalone validator is the binary called spirv-val and is located in <spirv-build-dir>/bin/spirv-val. The functionality of the validator is implemented by the spvValidate library function.

The validator operates on the binary form.

• spirv-val - the standalone validator
  • <spirv-dir>/bin/spirv-val

Options

• -basic performs basic stream validation, currently not implemented.
• -layout performs logical layout validation as described in section 2.16 Validation Rules, currently not implemented.
• -id performs ID validation according to the instruction rules in sections 3.28.1 through 3.28.22, enabled but is a work in progress.
• -capability performs capability validation and or reporting, currently not implemented.

Tests

The project contains a number of tests, implemented in the UnitSPIRV executable, used to drive the development and correctness of the tools, these use the googletest framework. The googletest source is not provided with this project, to enable the tests place the googletest source in the <spirv-dir>/external/googletest directory, rerun CMake if you have already done so previously, CMake will detect the existence of <spirv-dir>/external/googletest then build as normal.

Future Work

Assembler and disassembler

• Support OpenCL extension libraries.
• Enforce the parsing rules.
• Likely add an option to select from different assembly language syntaxes.

Validator

• Adopt the parser strategy used by the text and binary parsers.
• Complete implementation of ID validation rules in spirv-val.
• Implement section 2.16 Validation Rules in spirv-val.
• Implement Capability validation and or report in spirv-val.
• Improve assembly output from spirv-dis.
• Improve diagnostic reports.

Known Issues

• Float constant values are parsed incorrectly.
• Improve literal parsing in the assembler, currently only decimal integers and floating-point numbers are supported as literal operands and the parser is not contextually aware of the desired width of the operand.
• Sometimes the assembler will succeed, but the disassembler will fail to disassemble the result.
• Sometimes the disassembler fails to produce any output on an error.
• Many more issues.

Licence

Copyright (c) 2015 The Khronos Group Inc.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and/or associated documentation files (the "Materials"), to deal in the Materials without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Materials, and to permit persons to whom the Materials are furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Materials.

MODIFICATIONS TO THIS FILE MAY MEAN IT NO LONGER ACCURATELY REFLECTS KHRONOS STANDARDS. THE UNMODIFIED, NORMATIVE VERSIONS OF KHRONOS SPECIFICATIONS AND HEADER INFORMATION ARE LOCATED AT https://www.khronos.org/registry/

THE MATERIALS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE MATERIALS OR THE USE OR OTHER DEALINGS IN THE MATERIALS.