Add documentation of the xlat tables library V2

The documentation describes the design of the translation tables
library version 2 used by the ARM Trusted Firmware.

The diagram file has been created with Dia version 0.97.2. This tool
can be obtained from: https://wiki.gnome.org/Apps/Dia/Download

Inkscape has been used to generate the *.png file from the *.dia file
to work around a bug in the generation of *.png files in some versions
of Dia.

Change-Id: Ie67d9998d4ae881b2c060200a318ad3ac2fa5e91
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

docs/diagrams/generate_xlat_images.sh

@@ -0,0 +1,26 @@
+#! /bin/bash
+
+#
+# This script generates the image file used in the ARM Trusted Firmware
+# Translation Tables Library V2 Design document from the 'xlat_align.dia' file.
+#
+
+set -e
+
+# Usage: generate_image <dia_filename> <layers> <image_filename>
+function generate_image
+{
+	dia				\
+		--show-layers=$2	\
+		--filter=svg		\
+		--export=$3		\
+		$1
+
+}
+
+generate_image			\
+	xlat_align.dia		\
+	bg,translations		\
+	xlat_align.svg
+
+inkscape -z xlat_align.svg -e xlat_align.png -d 45

docs/firmware-design.rst

@@ -25,6 +25,10 @@ interrupts generated in either security state. The details of the interrupt
 management framework and its design can be found in ARM Trusted Firmware
 Interrupt Management Design guide [4]_.
 
+The ARM Trusted Firmware also implements a library for setting up and managing
+the translation tables. The details of this library can be found in
+`Xlat_tables design`_.
+
 The ARM Trusted Firmware can be built to support either AArch64 or AArch32
 execution state.
 
@@ -2418,7 +2422,7 @@ References
 
 --------------
 
-*Copyright (c) 2013-2016, ARM Limited and Contributors. All rights reserved.*
+*Copyright (c) 2013-2017, ARM Limited and Contributors. All rights reserved.*
 
 .. _Reset Design: ./reset-design.rst
 .. _Porting Guide: ./porting-guide.rst
@@ -2436,5 +2440,6 @@ References
 .. _User Guide: ./user-guide.rst
 .. _SMC Calling Convention PDD: http://infocenter.arm.com/help/topic/com.arm.doc.den0028b/ARM_DEN0028B_SMC_Calling_Convention.pdf
 .. _ARM Trusted Firmware Interrupt Management Design guide: ./interrupt-framework-design.rst
+.. _Xlat_tables design: xlat-tables-lib-v2-design.rst
 
 .. |Image 1| image:: diagrams/rt-svc-descs-layout.png?raw=true

docs/user-guide.rst

@@ -73,6 +73,10 @@ In addition, the following optional packages and tools may be needed:
 
 -  For debugging, ARM `Development Studio 5 (DS-5)`_.
 
+-  To create and modify the diagram files included in the documentation, `Dia`_.
+   This tool can be found in most Linux distributions. Inkscape is needed to
+   generate the actual *.png files.
+
 Getting the Trusted Firmware source code
 ----------------------------------------
 
@@ -1739,6 +1743,7 @@ wakeup interrupt from RTC.
 .. _Instructions for using Linaro's deliverables on Juno: https://community.arm.com/dev-platforms/b/documents/posts/using-linaros-deliverables-on-juno
 .. _ARM Platforms Portal: https://community.arm.com/dev-platforms/
 .. _Development Studio 5 (DS-5): http://www.arm.com/products/tools/software-tools/ds-5/index.php
+.. _Dia: https://wiki.gnome.org/Apps/Dia/Download
 .. _here: psci-lib-integration-guide.rst
 .. _Trusted Board Boot: trusted-board-boot.rst
 .. _Secure-EL1 Payloads and Dispatchers: firmware-design.rst#user-content-secure-el1-payloads-and-dispatchers

docs/xlat-tables-lib-v2-design.rst

@@ -0,0 +1,370 @@
+Translation Tables Library Design
+=================================
+
+
+.. section-numbering::
+    :suffix: .
+
+.. contents::
+
+
+This document describes the design of the translation tables library (version 2)
+used by the ARM Trusted Firmware. This library provides APIs to create page
+tables based on a description of the memory layout, as well as setting up system
+registers related to the Memory Management Unit (MMU) and performing the
+required Translation Lookaside Buffer (TLB) maintenance operations.
+
+More specifically, some use cases that this library aims to support are:
+
+#. Statically allocate translation tables and populate them (at run-time) based
+   on a description of the memory layout. The memory layout is typically
+   provided by the platform port as a list of memory regions;
+
+#. Support for generating translation tables pertaining to a different
+   translation regime than the exception level the library code is executing at;
+
+#. Support for dynamic mapping and unmapping of regions, even while the MMU is
+   on. This can be used to temporarily map some memory regions and unmap them
+   later on when no longer needed;
+
+#. Support for non-identity virtual to physical mappings to compress the virtual
+   address space;
+
+#. Support for changing memory attributes of memory regions at run-time.
+
+
+About version 1 and version 2
+-----------------------------
+
+This document focuses on version 2 of the library, whose sources are available
+in the `lib/xlat\_tables\_v2`_ directory. Version 1 of the library can still be
+found in `lib/xlat\_tables`_ directory but it is less flexible and doesn't
+support dynamic mapping. Although potential bug fixes will be applied to both
+versions, future features enhancements will focus on version 2 and might not be
+back-ported to version 1. Therefore, it is recommended to use version 2,
+especially for new platform ports.
+
+However, please note that version 2 is still in active development and is not
+considered stable yet. Hence, compatibility breaks might be introduced.
+
+From this point onwards, this document will implicitly refer to version 2 of the
+library.
+
+
+Design concepts and interfaces
+------------------------------
+
+This section presents some of the key concepts and data structures used in the
+translation tables library.
+
+`mmap` regions
+~~~~~~~~~~~~~~
+
+An ``mmap_region`` is an abstract, concise way to represent a memory region to
+map. It is one of the key interfaces to the library. It is identified by:
+
+- its physical base address;
+- its virtual base address;
+- its size;
+- its attributes.
+
+See the ``struct mmap_region`` type in `xlat\_tables\_v2.h`_.
+
+The user usually provides a list of such mmap regions to map and lets the
+library transpose that in a set of translation tables. As a result, the library
+might create new translation tables, update or split existing ones.
+
+The region attributes specify the type of memory (for example device or cached
+normal memory) as well as the memory access permissions (read-only or
+read-write, executable or not, secure or non-secure, and so on). See the
+``mmap_attr_t`` enumeration type in `xlat\_tables\_v2.h`_.
+
+
+Translation Context
+~~~~~~~~~~~~~~~~~~~
+
+The library can create or modify translation tables pertaining to a different
+translation regime than the exception level the library code is executing at.
+For example, the library might be used by EL3 software (for instance BL31) to
+create translation tables pertaining to the S-EL1&0 translation regime.
+
+This flexibility comes from the use of *translation contexts*. A *translation
+context* constitutes the superset of information used by the library to track
+the status of a set of translation tables for a given translation regime.
+
+The library internally allocates a default translation context, which pertains
+to the translation regime of the current exception level. Additional contexts
+may be explicitly allocated and initialized using the
+``REGISTER_XLAT_CONTEXT()`` macro. Separate APIs are provided to act either on
+the default translation context or on an alternative one.
+
+To register a translation context, the user must provide the library with the
+following information:
+
+* A name.
+
+  The resulting translation context variable will be called after this name, to
+  which ``_xlat_ctx`` is appended. For example, if the macro name parameter is
+  ``foo``, the context variable name will be ``foo_xlat_ctx``.
+
+* The maximum number of `mmap` regions to map.
+
+  Should account for both static and dynamic regions, if applicable.
+
+* The number of sub-translation tables to allocate.
+
+  Number of translation tables to statically allocate for this context,
+  excluding the initial lookup level translation table, which is always
+  allocated. For example, if the initial lookup level is 1, this parameter would
+  specify the number of level-2 and level-3 translation tables to pre-allocate
+  for this context.
+
+* The size of the virtual address space.
+
+  Size in bytes of the virtual address space to map using this context. This
+  will incidentally determine the number of entries in the initial lookup level
+  translation table : the library will allocate as many entries as is required
+  to map the entire virtual address space.
+
+* The size of the physical address space.
+
+  Size in bytes of the physical address space to map using this context.
+
+The default translation context is internally initialized using information
+coming (for the most part) from platform-specific defines:
+
+- name: hard-coded to ``tf`` ; hence the name of the default context variable is
+  ``tf_xlat_ctx``;
+- number of `mmap` regions: ``MAX_MMAP_REGIONS``;
+- number of sub-translation tables: ``MAX_XLAT_TABLES``;
+- size of the virtual address space: ``PLAT_VIRT_ADDR_SPACE_SIZE``;
+- size of the physical address space: ``PLAT_PHY_ADDR_SPACE_SIZE``.
+
+Please refer to the `Porting Guide`_ for more details about these macros.
+
+
+Static and dynamic memory regions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The library optionally supports dynamic memory mapping. This feature may be
+enabled using the ``PLAT_XLAT_TABLES_DYNAMIC`` platform build flag.
+
+When dynamic memory mapping is enabled, the library categorises mmap regions as
+*static* or *dynamic*.
+
+- *Static regions* are fixed for the lifetime of the system. They can only be
+  added early on, before the translation tables are created and populated. They
+  cannot be removed afterwards.
+
+- *Dynamic regions* can be added or removed any time.
+
+When the dynamic memory mapping feature is disabled, only static regions exist.
+
+The dynamic memory mapping feature may be used to map and unmap transient memory
+areas. This is useful when the user needs to access some memory for a fixed
+period of time, after which the memory may be discarded and reclaimed. For
+example, a memory region that is only required at boot time while the system is
+initializing, or to temporarily share a memory buffer between the normal world
+and trusted world. Note that it is up to the caller to ensure that these regions
+are not accessed concurrently while the regions are being added or removed.
+
+Although this feature provides some level of dynamic memory allocation, this
+does not allow dynamically allocating an arbitrary amount of memory at an
+arbitrary memory location. The user is still required to declare at compile-time
+the limits of these allocations ; the library will deny any mapping request that
+does not fit within this pre-allocated pool of memory.
+
+
+Library APIs
+------------
+
+The external APIs exposed by this library are declared and documented in the
+`xlat\_tables\_v2.h`_ header file. This should be the reference point for
+getting information about the usage of the different APIs this library
+provides. This section just provides some extra details and clarifications.
+
+Although the ``mmap_region`` structure is a publicly visible type, it is not
+recommended to populate these structures by hand. Instead, wherever APIs expect
+function arguments of type ``mmap_region_t``, these should be constructed using
+the ``MAP_REGION*()`` family of helper macros. This is to limit the risk of
+compatibility breaks, should the ``mmap_region`` structure type evolve in the
+future.
+
+As explained earlier in this document, when the dynamic mapping feature is
+disabled, there is no notion of dynamic regions. Conceptually, there are only
+static regions. For this reason (and to retain backward compatibility with the
+version 1 of the library), the APIs that map static regions do not embed the
+word *static* in their functions names (for example ``mmap_add_region()``), in
+contrast with the dynamic regions APIs (for example
+``mmap_add_dynamic_region()``).
+
+Although the definition of static and dynamic regions is not based on the state
+of the MMU, the two are still related in some way. Static regions can only be
+added before ``init_xlat_tables()`` is called and ``init_xlat_tables()`` must be
+called while the MMU is still off. As a result, static regions cannot be added
+once the MMU has been enabled. Dynamic regions can be added with the MMU on or
+off. In practice, the usual call flow would look like this:
+
+#. The MMU is initially off.
+
+#. Add some static regions, add some dynamic regions.
+
+#. Initialize translation tables based on the list of mmap regions (using one of
+   the ``init_xlat_tables*()`` APIs).
+
+#. At this point, it is no longer possible to add static regions. Dynamic
+   regions can still be added or removed.
+
+#. Enable the MMU.
+
+#. Dynamic regions can continue to be added or removed.
+
+Because static regions are added early on at boot time and are all in the
+control of the platform initialization code, the ``mmap_add*()`` family of APIs
+are not expected to fail. They do not return any error code.
+
+Nonetheless, these APIs will check upfront whether the region can be
+successfully added before updating the translation context structure. If the
+library detects that there is insufficient memory to meet the request, or that
+the new region will overlap another one in an invalid way, or if any other
+unexpected error is encountered, they will print an error message on the UART.
+Additionally, when asserts are enabled (typically in debug builds), an assertion
+will be triggered. Otherwise, the function call will just return straight away,
+without adding the offending memory region.
+
+
+Library limitations
+-------------------
+
+Dynamic regions are not allowed to overlap each other. Static regions are
+allowed to overlap as long as one of them is fully contained inside the other
+one. This is allowed for backwards compatibility with the previous behaviour in
+the version 1 of the library.
+
+
+Implementation details
+----------------------
+
+Code structure
+~~~~~~~~~~~~~~
+
+The library is divided into 2 modules:
+
+The core module
+    Provides the main functionality of the library.
+
+    See `xlat\_tables\_internal.c`_.
+
+The architectural module
+    Provides functions that are dependent on the current execution state
+    (AArch32/AArch64), such as the functions used for TLB invalidation or MMU
+    setup.
+
+    See `aarch32/xlat\_tables\_arch.c`_ and `aarch64/xlat\_tables\_arch.c`_.
+
+Core module
+~~~~~~~~~~~
+
+All the APIs in this module work on a translation context. The translation
+context contains the list of ``mmap_region``, which holds the information of all
+the regions that are mapped at any given time. Whenever there is a request to
+map (resp. unmap) a memory region, it is added to (resp. removed from) the
+``mmap_region`` list.
+
+The mmap regions list is a conceptual way to represent the memory layout. At
+some point, the library has to convert this information into actual translation
+tables to program into the MMU.
+
+Before the ``init_xlat_tables()`` API is called, the library only acts on the
+mmap regions list. Adding a static or dynamic region at this point through one
+of the ``mmap_add*()`` APIs does not affect the translation tables in any way,
+they only get registered in the internal mmap region list. It is only when the
+user calls the ``init_xlat_tables()`` that the translation tables are populated
+in memory based on the list of mmap regions registered so far. This is an
+optimization that allows creation of the initial set of translation tables in
+one go, rather than having to edit them every time while the MMU is disabled.
+
+After the ``init_xlat_tables()`` API has been called, only dynamic regions can
+be added. Changes to the translation tables (as well as the mmap regions list)
+will take effect immediately.
+
+The mapping function is implemented as a recursive algorithm. It is however
+bound by the level of depth of the translation tables (the ARMv8-A architecture
+allows up to 4 lookup levels).
+
+By default, the algorithm will attempt to minimize the number of translation
+tables created to satisfy the user's request. It will favour mapping a region
+using the biggest possible blocks, only creating a sub-table if it is strictly
+necessary. This is to reduce the memory footprint of the firmware.
+
+The most common reason for needing a sub-table is when a specific mapping
+requires a finer granularity. Misaligned regions also require a finer
+granularity than what the user may had originally expected, using a lot more
+memory than expected. The reason is that all levels of translation are
+restricted to address translations of the same granularity as the size of the
+blocks of that level.  For example, for a 4 KiB page size, a level 2 block entry
+can only translate up to a granularity of 2 MiB. If the Physical Address is not
+aligned to 2 MiB then additional level 3 tables are also needed.
+
+Note that not every translation level allows any type of descriptor. Depending
+on the page size, levels 0 and 1 of translation may only allow table
+descriptors. If a block entry could be able to describe a translation, but that
+level does not allow block descriptors, a table descriptor will have to be used
+instead, as well as additional tables at the next level.
+
+|Alignment Example|
+
+The mmap regions are sorted in a way that simplifies the code that maps
+them. Even though this ordering is only strictly needed for overlapping static
+regions, it must also be applied for dynamic regions to maintain a consistent
+order of all regions at all times. As each new region is mapped, existing
+entries in the translation tables are checked to ensure consistency. Please
+refer to the comments in the source code of the core module for more details
+about the sorting algorithm in use.
+
+The library takes care of performing TLB maintenance operations when required.
+For example, when the user requests removing a dynamic region, the library
+invalidates all TLB entries associated to that region to ensure that these
+changes are visible to subsequent execution, including speculative execution,
+that uses the changed translation table entries.
+
+A counter-example is the initialization of translation tables. In this case,
+explicit TLB maintenance is not required. The ARMv8-A architecture guarantees
+that all TLBs are disabled from reset and their contents have no effect on
+address translation at reset [#tlb-reset-ref]_. Therefore, the TLBs invalidation
+is deferred to the ``enable_mmu*()`` family of functions, just before the MMU is
+turned on.
+
+TLB invalidation is not required when adding dynamic regions either. Dynamic
+regions are not allowed to overlap existing memory region. Therefore, if the
+dynamic mapping request is deemed legitimate, it automatically concerns memory
+that was not mapped in this translation regime and the library will have
+initialized its corresponding translation table entry to an invalid
+descriptor. Given that the TLBs are not architecturally permitted to hold any
+invalid translation table entry [#tlb-no-invalid-entry]_, this means that this
+mapping cannot be cached in the TLBs.
+
+.. [#tlb-reset-ref] See section D4.8 `Translation Lookaside Buffers (TLBs)`, subsection `TLB behavior at reset` in ARMv8-A, rev B.a.
+
+.. [#tlb-no-invalid-entry] See section D4.9.1 `General TLB maintenance requirements` in ARMv8-A, rev B.a.
+
+Architectural module
+~~~~~~~~~~~~~~~~~~~~
+
+This module contains functions that have different implementations for AArch32
+and AArch64. For example, it provides APIs to perform TLB maintenance operations,
+enable the MMU or calculate the Physical Address Space size. They do not need a
+translation context to work on.
+
+--------------
+
+*Copyright (c) 2017, ARM Limited and Contributors. All rights reserved.*
+
+.. _lib/xlat\_tables\_v2: ../lib/xlat_tables_v2
+.. _lib/xlat\_tables: ../lib/xlat_tables
+.. _xlat\_tables\_v2.h: ../include/lib/xlat_tables/xlat_tables_v2.h
+.. _xlat\_tables\_internal.c: ../lib/xlat_tables_v2/xlat_tables_internal.c
+.. _aarch32/xlat\_tables\_arch.c: ../lib/xlat_tables_v2/aarch32/xlat_tables_arch.c
+.. _aarch64/xlat\_tables\_arch.c: ../lib/xlat_tables_v2/aarch64/xlat_tables_arch.c
+.. _Porting Guide: porting-guide.rst
+.. |Alignment Example| image:: ./diagrams/xlat_align.png?raw=true