path: root/Documentation/virt/uml/user_mode_linux_howto_v2.rst

.. SPDX-License-Identifier: GPL-2.0

#########
UML HowTo
#########

.. contents:: :local:

************
Introduction
************

Welcome to User Mode Linux

User Mode Linux is the first Open Source virtualization platform (first
release date 1991) and second virtualization platform for an x86 PC.

How is UML Different from a VM using Virtualization package X?
==============================================================

We have come to assume that virtualization also means some level of
hardware emulation. In fact, it does not. As long as a virtualization
package provides the OS with devices which the OS can recognize and
has a driver for, the devices do not need to emulate real hardware.
Most OSes today have built-in support for a number of "fake"
devices used only under virtualization.
User Mode Linux takes this concept to the ultimate extreme - there
is not a single real device in sight. It is 100% artificial or if
we use the correct term 100% paravirtual. All UML devices are abstract
concepts which map onto something provided by the host - files, sockets,
pipes, etc.

The other major difference between UML and various virtualization
packages is that there is a distinct difference between the way the UML
kernel and the UML programs operate.
The UML kernel is just a process running on Linux - same as any other
program. It can be run by an unprivileged user and it does not require
anything in terms of special CPU features.
The UML userspace, however, is a bit different. The Linux kernel on the
host machine assists UML in intercepting everything the program running
on a UML instance is trying to do and making the UML kernel handle all
of its requests.
This is different from other virtualization packages which do not make any
difference between the guest kernel and guest programs. This difference
results in a number of advantages and disadvantages of UML over let's say
QEMU which we will cover later in this document.


Why Would I Want User Mode Linux?
=================================


* If User Mode Linux kernel crashes, your host kernel is still fine. It
  is not accelerated in any way (vhost, kvm, etc) and it is not trying to
  access any devices directly.  It is, in fact, a process like any other.

* You can run a usermode kernel as a non-root user (you may need to
  arrange appropriate permissions for some devices).

* You can run a very small VM with a minimal footprint for a specific
  task (for example 32M or less).

* You can get extremely high performance for anything which is a "kernel
  specific task" such as forwarding, firewalling, etc while still being
  isolated from the host kernel.

* You can play with kernel concepts without breaking things.

* You are not bound by "emulating" hardware, so you can try weird and
  wonderful concepts which are very difficult to support when emulating
  real hardware such as time travel and making your system clock
  dependent on what UML does (very useful for things like tests).

* It's fun.

Why not to run UML
==================

* The syscall interception technique used by UML makes it inherently
  slower for any userspace applications. While it can do kernel tasks
  on par with most other virtualization packages, its userspace is
  **slow**. The root cause is that UML has a very high cost of creating
  new processes and threads (something most Unix/Linux applications
  take for granted).

* UML is strictly uniprocessor at present. If you want to run an
  application which needs many CPUs to function, it is clearly the
  wrong choice.

***********************
Building a UML instance
***********************

There is no UML installer in any distribution. While you can use off
the shelf install media to install into a blank VM using a virtualization
package, there is no UML equivalent. You have to use appropriate tools on
your host to build a viable filesystem image.

This is extremely easy on Debian - you can do it using debootstrap. It is
also easy on OpenWRT - the build process can build UML images. All other
distros - YMMV.

Creating an image
=================

Create a sparse raw disk image::

   # dd if=/dev/zero of=disk_image_name bs=1 count=1 seek=16G

This will create a 16G disk image. The OS will initially allocate only one
block and will allocate more as they are written by UML. As of kernel
version 4.19 UML fully supports TRIM (as usually used by flash drives).
Using TRIM inside the UML image by specifying discard as a mount option
or by running ``tune2fs -o discard /dev/ubdXX`` will request UML to
return any unused blocks to the OS.

Create a filesystem on the disk image and mount it::

   # mkfs.ext4 ./disk_image_name && mount ./disk_image_name /mnt

This example uses ext4, any other filesystem such as ext3, btrfs, xfs,
jfs, etc will work too.

Create a minimal OS installation on the mounted filesystem::

   # debootstrap buster /mnt http://deb.debian.org/debian

debootstrap does not set up the root password, fstab, hostname or
anything related to networking. It is up to the user to do that.

Set the root password - the easiest way to do that is to chroot into the
mounted image::

   # chroot /mnt
   # passwd
   # exit

Edit key system files
=====================

UML block devices are called ubds. The fstab created by debootstrap
will be empty and it needs an entry for the root file system::

   /dev/ubd0   ext4    discard,errors=remount-ro  0       1

The image hostname will be set to the same as the host on which you
are creating its image. It is a good idea to change that to avoid
"Oh, bummer, I rebooted the wrong machine".

UML supports two classes of network devices - the older uml_net ones
which are scheduled for obsoletion. These are called ethX. It also
supports the newer vector IO devices which are significantly faster
and have support for some standard virtual network encapsulations like
Ethernet over GRE and Ethernet over L2TPv3. These are called vec0.

Depending on which one is in use, ``/etc/network/interfaces`` will
need entries like::

   # legacy UML network devices
   auto eth0
   iface eth0 inet dhcp

   # vector UML network devices
   auto vec0
   iface vec0 inet dhcp

We now have a UML image which is nearly ready to run, all we need is a
UML kernel and modules for it.

Most distributions have a UML package. Even if you intend to use your own
kernel, testing the image with a stock one is always a good start. These
packages come with a set of modules which should be copied to the target
filesystem. The location is distribution dependent. For Debian these
reside under /usr/lib/uml/modules. Copy recursively the content of this
directory to the mounted UML filesystem::

   # cp -rax /usr/lib/uml/modules /mnt/lib/modules

If you have compiled your own kernel, you need to use the usual "install
modules to a location" procedure by running::

  # make INSTALL_MOD_PATH=/mnt/lib/modules modules_install

This will install modules into /mnt/lib/modules/$(KERNELRELEASE).
To specify the full module installation path, use::

  # make MODLIB=/mnt/lib/modules modules_install

At this point the image is ready to be brought up.

*************************
Setting Up UML Networking
*************************

UML networking is designed to emulate an Ethernet connection. This
connection may be either point-to-point (similar to a connection
between machines using a back-to-back cable) or a connection to a
switch. UML supports a wide variety of means to build these
connections to all of: local machine, remote machine(s), local and
remote UML and other VM instances.


+-----------+--------+------------------------------------+------------+
| Transport |  Type  |        Capabilities                | Throughput |
+===========+========+====================================+============+
| tap       | vector | checksum, tso                      | > 8Gbit    |
+-----------+--------+------------------------------------+------------+
| hybrid    | vector | checksum, tso, multipacket rx      | > 6GBit    |
+-----------+--------+------------------------------------+------------+
| raw       | vector | checksum, tso, multipacket rx, tx" | > 6GBit    |
+-----------+--------+------------------------------------+------------+
| EoGRE     | vector | multipacket rx, tx                 | > 3Gbit    |
+-----------+--------+------------------------------------+------------+
| Eol2tpv3  | vector | multipacket rx, tx                 | > 3Gbit    |
+-----------+--------+------------------------------------+------------+
| bess      | vector | multipacket rx, tx                 | > 3Gbit    |
+-----------+--------+------------------------------------+------------+
| fd        | vector | dependent on fd type               | varies     |
+-----------+--------+------------------------------------+------------+
| vde       | vector | dep. on VDE VPN: Virt.Net Locator  | varies     |
+-----------+--------+------------------------------------+------------+
| tuntap    | legacy | none                               | ~ 500Mbit  |
+-----------+--------+------------------------------------+------------+
| daemon    | legacy | none                               | ~ 450Mbit  |
+-----------+--------+------------------------------------+------------+
| socket    | legacy | none                               | ~ 450Mbit  |
+-----------+--------+------------------------------------+------------+
| ethertap  | legacy | obsolete                           | ~ 500Mbit  |
+-----------+--------+------------------------------------+------------+
| vde       | legacy | obsolete                           | ~ 500Mbit  |
+-----------+--------+------------------------------------+------------+

* All transports which have tso and checksum offloads can deliver speeds
  approaching 10G on TCP streams.

* All transports which have multi-packet rx and/or tx can deliver pps
  rates of up to 1Mps or more.

* All legacy transports are generally limited to ~600-700MBit and 0.05Mps.

* GRE and L2TPv3 allow connections to all of: local machine, remote
  machines, remote network devices and remote UML instances.

* Socket allows connections only between UML instances.

* Daemon and bess require running a local switch. This switch may be
  connected to the host as well.


Network configuration privileges
================================

The majority of the supported networking modes need ``root`` privileges.
For example, in the legacy tuntap networking mode, users were required
to be part of the group associated with the tunnel device.

For newer network drivers like the vector transports, ``root``