• Update (2016-10-19): all has been merged upstream now!
• Update (2016-10-30): static_website_tls works (TLS,HTTP,network via tap device)!
• Update (2017-02-23): no more extra remotes, Mirage3 is released!

What?

As described earlier, MirageOS is a library operating system developed in OCaml. The code size is already pretty small, deployments are so far either as a UNIX binary or as a Xen virtual machine.

Xen is a hypervisor, providing concrete device drivers for the actual hardware of a physical machine, memory management, scheduling, etc. The initial release of Xen was done in 2003, since then the code size and code complexity of Xen is growing. It also has various different mechanisms for virtualisation, hardware assisted ones or purely software based ones, some where the guest operating system needs to cooperate others where it does not need to cooperate.

Since 2005, Intel CPUs (as well as AMD CPUs) provide hardware assistance for virtualisation (the VT-x extension), since 2008 extended page tables (EPT) are around which allow a guest to safely access the MMU. Those features gave rise to much smaller hypervisors, such as KVM (mainly Linux), bhyve (FreeBSD), xhyve (MacOSX), vmm (OpenBSD), which do not need to emulate the MMU and other things in software. The boot sequence in those hypervisors uses kexec or multiboot, instead of doing all the 16 bit, 32 bit, 64 bit mode changes manually.

MirageOS initially targeted only Xen, in 2015 there was a port to use rumpkernel (a modularised NetBSD), and 2016 solo5 emerged where you can run MirageOS on. Solo5 comes in two shapes, either as ukvm on top of KVM, or as a multiboot image using virtio interfaces (block and network, plus a serial console). Solo5 is only ~1000 lines of code (plus dlmalloc), and ISC-licensed.

A recent paper describes the advantages of a tiny virtual machine monitor in detail, namely no more venom like security issues since there is no legacy hardware emulated. Also, each virtual machine monitor can be customised to the unikernel running on top of it: if the unikernel does not need a block device, the monitor shouldn't contain code for it. The idea is to have one customised monitor for each unikernel.

While lots of people seem to like KVM and Linux, I still prefer FreeBSD, their jails, and nowadays bhyve. I finally found some time, thanks to various cleanups to the solo5 code base, to finally look into porting solo5 to FreeBSD/bhyve. It runs and can output to console.

How?

These instructions are still slightly bumpy. If you've a FreeBSD with bhyve (I use FreeBSD-CURRENT), and OCaml and opam (>=1.2.2) installed, it is pretty straightforward to get solo5 running. First, I'd suggest to use a fresh opam switch in case you work on other OCaml projects: opam switch -A 4.04.0 solo5 (followed by eval `opam config env` to setup some environment variables).

You need some software from the ports: devel/pkgconf, devel/gmake, devel/binutils, and sysutils/grub2-bhyve.

An opam install mirage mirage-logs solo5-kernel-virtio mirage-bootvar-solo5 mirage-solo5 should provide you with a basic set of libraries.

Now you can get the mirage-skeleton repository, and inside of device-usage/console, run mirage configure --no-opam --virtio followed by make. There should be a resulting mir-console.virtio.

Once that is in place, start your VM:

sudo grub-bhyve -M 128M console
> multiboot (host)/home/hannes/mirage-skeleton/console/mir-console.virtio
> boot

sudo bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -l com1,stdio -m 128M console

The following output will appear on your controlling terminal:

            |      ___|
  __|  _ \  |  _ \ __ \
\__ \ (   | | (   |  ) |
____/\___/ _|\___/____/
multiboot: Using memory: 0x100000 - 0x8000000
TSC frequency estimate is 2593803000 Hz
Solo5: new bindings
STUB: getenv() called
hello
world
hello
world
hello
world
hello
world
solo5_app_main() returned with 0
Kernel done.
Goodbye!

Network and TLS stack works as well (tested 30th October).

Open issues

• I'm not happy to require ld from the ports (but the one in base does not produce sensible binaries with -z max-page-size=0x1000 related)
• Via twitter, bhyve devs suggested to port ukvm to ubhyve. This is a great idea, to no longer depend on virtio, and get more speed. Any takers?
• Debugging via gdb should be doable somehow, bhyve has some support for gdb, but it is unclear to me what I need to do to enter the debugger (busy looping in the VM and a gdb remote to the port opened by bhyve does not work).

Conclusion

I managed to get solo5 to work with bhyve. I even use clang instead of gcc and don't need to link libgcc.a. :) It is great to see further development in hypervisors and virtual machine monitors. Especially thanks to Martin Lucina for getting things sorted.

I'm interested in feedback, either via twitter or via eMail.

Other updates in the MirageOS ecosystem

There were some busy times, several pull requests are still waiting to get merged (e.g. some cosmetics in mirage as preconditions for treemaps and dependency diagrams), I proposed to use sleep_ns : int64 -> unit io instead of the sleep : float -> unit io (nobody wants floating point numbers); also an RFC for random, Matt Gray proposed to get rid of CLOCK (and have a PCLOCK and a MCLOCK instead). Soon there will be a major MirageOS release which breaks all the previous unikernels! :)