path: root/Documentation/powerpc/ultravisor.rst

.. SPDX-License-Identifier: GPL-2.0
.. _ultravisor:

============================
Protected Execution Facility
============================

.. contents::
    :depth: 3

Introduction
############

    Protected Execution Facility (PEF) is an architectural change for
    POWER 9 that enables Secure Virtual Machines (SVMs). DD2.3 chips
    (PVR=0x004e1203) or greater will be PEF-capable. A new ISA release
    will include the PEF RFC02487 changes.

    When enabled, PEF adds a new higher privileged mode, called Ultravisor
    mode, to POWER architecture. Along with the new mode there is new
    firmware called the Protected Execution Ultravisor (or Ultravisor
    for short). Ultravisor mode is the highest privileged mode in POWER
    architecture.

	+------------------+
	| Privilege States |
	+==================+
	|  Problem         |
	+------------------+
	|  Supervisor      |
	+------------------+
	|  Hypervisor      |
	+------------------+
	|  Ultravisor      |
	+------------------+

    PEF protects SVMs from the hypervisor, privileged users, and other
    VMs in the system. SVMs are protected while at rest and can only be
    executed by an authorized machine. All virtual machines utilize
    hypervisor services. The Ultravisor filters calls between the SVMs
    and the hypervisor to assure that information does not accidentally
    leak. All hypercalls except H_RANDOM are reflected to the hypervisor.
    H_RANDOM is not reflected to prevent the hypervisor from influencing
    random values in the SVM.

    To support this there is a refactoring of the ownership of resources
    in the CPU. Some of the resources which were previously hypervisor
    privileged are now ultravisor privileged.

Hardware
========

    The hardware changes include the following:

    * There is a new bit in the MSR that determines whether the current
      process is running in secure mode, MSR(S) bit 41. MSR(S)=1, process
      is in secure mode, MSR(s)=0 process is in normal mode.

    * The MSR(S) bit can only be set by the Ultravisor.

    * HRFID cannot be used to set the MSR(S) bit. If the hypervisor needs
      to return to a SVM it must use an ultracall. It can determine if
      the VM it is returning to is secure.

    * There is a new Ultravisor privileged register, SMFCTRL, which has an
      enable/disable bit SMFCTRL(E).

    * The privilege of a process is now determined by three MSR bits,
      MSR(S, HV, PR). In each of the tables below the modes are listed
      from least privilege to highest privilege. The higher privilege
      modes can access all the resources of the lower privilege modes.

      **Secure Mode MSR Settings**

      +---+---+---+---------------+
      | S | HV| PR|Privilege      |
      +===+===+===+===============+
      | 1 | 0 | 1 | Problem       |
      +---+---+---+---------------+
      | 1 | 0 | 0 | Privileged(OS)|
      +---+---+---+---------------+
      | 1 | 1 | 0 | Ultravisor    |
      +---+---+---+---------------+
      | 1 | 1 | 1 | Reserved      |
      +---+---+---+---------------+

      **Normal Mode MSR Settings**

      +---+---+---+---------------+
      | S | HV| PR|Privilege      |
      +===+===+===+===============+
      | 0 | 0 | 1 | Problem       |
      +---+---+---+---------------+
      | 0 | 0 | 0 | Privileged(OS)|
      +---+---+---+---------------+
      | 0 | 1 | 0 | Hypervisor    |
      +---+---+---+---------------+
      | 0 | 1 | 1 | Problem (Host)|
      +---+---+---+---------------+

    * Memory is partitioned into secure and normal memory. Only processes
      that are running in secure mode can access secure memory.

    * The hardware does not allow anything that is not running secure to
      access secure memory. This means that the Hypervisor cannot access
      the memory of the SVM without using an ultracall (asking the
      Ultravisor). The Ultravisor will only allow the hypervisor to see
      the SVM memory encrypted.

    * I/O systems are not allowed to directly address secure memory. This
      limits the SVMs to virtual I/O only.

    * The architecture allows the SVM to share pages of memory with the
      hypervisor that are not protected with encryption. However, this
      sharing must be initiated by the SVM.

    * When a process is running in secure mode all hypercalls
      (syscall lev=1) go to the Ultravisor.

    * When a process is in secure mode all interrupts go to the
      Ultravisor.

    * The following resources have become Ultravisor privileged and
      require an Ultravisor interface to manipulate:

      * Processor configurations registers (SCOMs).

      * Stop state information.

      * The debug registers CIABR, DAWR, and DAWRX when SMFCTRL(D) is set.
        If SMFCTRL(D) is not set they do not work in secure mode. When set,
        reading and writing requires an Ultravisor call, otherwise that
        will cause a Hypervisor Emulation Assistance interrupt.

      * PTCR and partition table entries (partition table is in secure
        memory). An attempt to write to PTCR will cause a Hypervisor
        Emulation Assitance interrupt.

      * LDBAR (LD Base Address Register) and IMC (In-Memory Collection)
        non-architected registers. An attempt to write to them will cause a
        Hypervisor Emulation Assistance interrupt.

      * Paging for an SVM, sharing of memory with Hypervisor for an SVM.
        (Including Virtual Processor Area (VPA) and virtual I/O).


Software/Microcode
==================

    The software changes include:

    * SVMs are created from normal VM using (open source) tooling supplied
      by IBM.

    * All SVMs start as normal VMs and utilize an ultracall, UV_ESM
      (Enter Secure Mode), to make the transition.

    * When the UV_ESM ultracall is made the Ultravisor copies the VM into
      secure memory, decrypts the verification information, and checks the
      integrity of the SVM. If the integrity check passes the Ultravisor
      passes control in secure mode.

    * The verification information includes the pass phrase for the
      encrypted disk associated with the SVM. This pass phrase is given
      to the SVM when requested.

    * The Ultravisor is not involved in protecting the encrypted disk of
      the SVM while at rest.

    * For external interrupts the Ultravisor saves the state of the SVM,
      and reflects the interrupt to the hypervisor for processing.
      For hypercalls, the Ultravisor inserts neutral state into all
      registers not needed for the hypercall then reflects the call to
      the hypervisor for processing. The H_RANDOM hypercall is performed
      by the Ultravisor and not reflected.

    * For virtual I/O to work bounce buffering must be done.

    * The Ultravisor uses AES (IAPM) for protection of SVM memory. IAPM
      is a mode of AES that provides integrity and secrecy concurrently.

    * The movement of data between normal and secure pages is coordinated
      with the Ultravisor by a new HMM plug-in in the Hypervisor.

    The Ultravisor offers new services to the hypervisor and SVMs. These
    are accessed through ultracalls.

Terminology
===========

    * Hypercalls: special system calls used to request services from
      Hypervisor.

    * Normal memory: Memory that is accessible to Hypervisor.

    * Normal page: Page backed by normal memory and available to
      Hypervisor.

    * Shared page: A page backed by normal memory and available to both
      the Hypervisor/QEMU and the SVM (i.e page has mappings in SVM and
      Hypervisor/QEMU).

    * Secure memory: Memory that is accessible only to Ultravisor and
      SVMs.

    * Secure page: Page backed by secure memory and only available to
      Ultravisor and SVM.

    * SVM: Secure Virtual Machine.

    * Ultracalls: special system calls used to request services from
      Ultravisor.


Ultravisor calls API
####################

    This section describes Ultravisor calls (ultracalls) needed to
    support Secure Virtual Machines (SVM)s and Paravirtualized KVM. The
    ultracalls allow the SVMs and Hypervisor to request services from the
    Ultravisor such as accessing a register or memory region that can only
    be accessed when running in Ultravisor-privileged mode.

    The specific service needed from an ultracall is specified in register
    R3 (the first parameter to the ultracall). Other parameters to the
    ultracall, if any, are specified in registers R4 through R12.

    Return value of all ultracalls is in register R3. Other output values
    from the ultracall, if any, are returned in registers R4 through R12.
    The only exception to this register usage is the ``UV_RETURN``
    ultracall described below.

    Each ultracall returns specific error codes, applicable in the context
    of the ultracall. However, like with the PowerPC Architecture Platform
    Reference (PAPR), if no specific error code is defined for a
    particular situation, then the ultracall will fallback to an erroneous
    parameter-position based code. i.e U_PARAMETER, U_P2, U_P3 etc
    depending on the ultracall parameter that may have caused the error.

    Some ultracalls involve transferring a page of data between Ultravisor
    and Hypervisor.  Secure pages that are transferred from secure memory
    to normal memory may be encrypted using dynamically generated keys.
    When the secure pages are transferred back to secure memory, they may
    be decrypted using the same dynamically generated keys. Generation and
    management of these keys will be covered in a separate document.

    For now this only covers ultracalls currently implemented and being
    used by Hypervisor and SVMs but others can be added here when it
    makes sense.

    The full specification for all hypercalls/ultracalls will eventually
    be made available in the public/OpenPower version of the PAPR
    specification.

    .. note::

        If PEF is not enabled, the ultracalls will be redirected to the
        Hypervisor which must handle/fail the calls.

Ultracalls used by Hypervisor
=============================

    This section describes the virtual memory management ultracalls used
    by the Hypervisor to manage SVMs.

UV_PAGE_OUT
-----------

    Encrypt and move the contents of a page from secure memory to normal
    memory.

Syntax
~~~~~~

.. code-block:: c

	uint64_t ultracall(const uint64_t UV_PAGE_OUT,
		uint16_t lpid,		/* LPAR ID */
		uint64_t dest_ra,	/* real address of destination page */
		uint64_t src_gpa,	/* source guest-physical-address */
		uint8_t  flags,		/* flags */
		uint64_t order)		/* page size order */

Return values
~~~~~~~~~~~~~

    One of the following values:

	* U_SUCCESS	on success.
	* U_PARAMETER	if ``lpid`` is invalid.
	* U_P2 		if ``dest_ra`` is invalid.
	* U_P3		if the ``src_gpa`` address is invalid.
	* U_P4		if any bit in the ``flags`` is unrecognized
	* U_P5		if the ``order`` parameter is unsupported.
	* U_FUNCTION	if functionality is not supported.
	* U_BUSY	if page cannot be currently paged-out.

Description
~~~~~~~~~~~

    Encrypt the contents of a secure-page and make it available to
    Hypervisor in a normal page.

    By default, the source page is unmapped from the SVM's partition-
    scoped page table. But the Hypervisor can provide a hint to the
    Ultravisor to retain the page mapping by setting the ``UV_SNAPSHOT``
    flag in ``flags`` parameter.

    If the source page is already a shared page the call returns
    U_SUCCESS, without doing anything.

Use cases
~~~~~~~~~

    #. QEMU attempts to access an address belonging to the SVM but the
       page frame for that address is not mapped into QEMU's address
       space. In this case, the Hypervisor will allocate a page frame,
       map it into QEMU's address space and issue the ``UV_PAGE_OUT``
       call to retrieve the encrypted contents of the page.

    #. When Ultravisor runs low on secure memory and it needs to page-out
       an LRU page. In this case, Ultravisor will issue the
       ``H_SVM_PAGE_OUT`` hypercall to the Hypervisor. The Hypervisor will
       then allocate a normal page and issue the ``UV_PAGE_OUT`` ultracall
       and the Ultravisor will encrypt and move the contents of the secure
       page into the normal page.

    #. When Hypervisor accesses SVM data, the Hypervisor requests the
       Ultravisor to transfer the corresponding page into a insecure page,
       which the Hypervisor can access. The data in the normal page will
       be encrypted though.

UV_PAGE_IN
----------

    Move the contents of a page from normal memory to secure memory.

Syntax
~~~~~~

.. code-block:: c

	uint64_t ultracall(const uint64_t UV_PAGE_IN,
		uint16_t lpid,		/* the LPAR ID */
		uint64_t src_ra,	/* source real address of page */
		uint64_t dest_gpa,	/* destination guest physical address */
		uint64_t flags,		/* flags */
		uint64_t order)		/* page size order */

Return values
~~~~~~~~~~~~~

    One of the following values:

	* U_SUCCESS	on success.
	* U_BUSY	if page cannot be currently paged-in.
	* U_FUNCTION	if functionality is not supported
	* U_PARAMETER	if ``lpid`` is invalid.
	* U_P2 		if ``src_ra`` is invalid.
	* U_P3		if the ``dest_gpa`` address is invalid.
	* U_P4		if any bit in the ``flags`` is unrecognized
	* U_P5		if the ``order`` parameter is unsupported.

Description