hannes/hannes.robur.coop
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

updated from main (commit c59a208400)

Browse source
This commit is contained in:
Canopy bot 2024-10-11 09:44:49 +00:00
parent 6b114f48ad
commit 60096fea59
22 changed files with 80 additions and 91 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
About
View file

@ -1,5 +1,5 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>About</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="About" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>About</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/overview" class="tag">overview</a><a href="/tags/myself" class="tag">myself</a><a href="/tags/background" class="tag">background</a></div><span class="date">Published: 2016-04-01 (last updated: 2023-11-20)</span><article><h2 id="what-is-a-full-stack-engineer">What is a &quot;full stack engineer&quot;?</h2>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>About</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="About" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>About</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/overview" class="tag">overview</a><a href="/tags/myself" class="tag">myself</a><a href="/tags/background" class="tag">background</a></div><span class="date">Published: 2016-04-01 (last updated: 2024-10-11)</span><article><h2 id="what-is-a-full-stack-engineer">What is a &quot;full stack engineer&quot;?</h2>
<p>Analysing the word literally, we should start with silicon and some electrons,
maybe a soldering iron, and build everything all the way up to our favourite
communication system.</p>
@ -68,7 +68,7 @@ researched (I would not consider it production ready yet, needs more work on a
faster runtime, and libraries).</p>
<p>After I finished my PhD, I decided to slack off for some time to make decent
espresso. I ended up spending the winter (beginning of 2014) in Mirleft,
Morocco. A good friend of mine pointed me to <a href="https://mirage.io">MirageOS</a>, a
Morocco. A good friend of mine pointed me to <a href="https://mirageos.org">MirageOS</a>, a
clean-slate operating system written in the high-level language <a href="https://ocaml.org">OCaml</a>. I got
hooked pretty fast, after some experience with LISP machines I imagined a modern
OS written in a single functional programming language.</p>

4
Posts/ARP
View file

@ -1,5 +1,5 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Re-engineering ARP</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Re-engineering ARP" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Re-engineering ARP</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a></div><span class="date">Published: 2016-07-12 (last updated: 2021-11-19)</span><article><h2 id="what-is-arp">What is ARP?</h2>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Re-engineering ARP</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Re-engineering ARP" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Re-engineering ARP</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a></div><span class="date">Published: 2016-07-12 (last updated: 2024-10-11)</span><article><h2 id="what-is-arp">What is ARP?</h2>
<p>ARP is the <em>A</em>ddress <em>R</em>esolution <em>P</em>rotocol, widely used in legacy IP networks (which support only IPv4). It is responsible to translate an IPv4 address to an Ethernet address. It is strictly more general, abstracting over protocol and hardware addresses. It is basically DNS (the domain name system) on a different layer.</p>
<p>ARP is link-local: ARP frames are not routed into other networks, all stay in the same broadcast domain. Thus there is no need for a hop limit (time-to-live). A reverse lookup mechanism (hardware address to protocol) is also available, named reverse ARP ;).</p>
<p>I will focus on ARP in this article, as used widely to translate IPv4 addresses into Ethernet addresses. There are two operations in ARP: request and response. A request is usually broadcasted to all hosts (by setting the destination to the broadcast Ethernet address, <code>ff:ff:ff:ff:ff:ff</code>), while a reply is send via unicast (to the host which requested that information).</p>
@ -13,7 +13,7 @@
<p>That's it. Pretty straightforward.</p>
<h2 id="design">Design</h2>
<p>Back in 2008, together with Andreas Bogk, we just used a hash table and installed expiration and retransmission timers when needed. Certainly timers sometimes needed to be cancelled, and testing the code was cumbersome. It were only <a href="https://github.com/dylan-hackers/network-night-vision/blob/master/network/ip-stack/layers/network/arp/arp.dylan">250 lines of Dylan code</a> plus some <a href="https://github.com/dylan-hackers/network-night-vision/blob/master/protocols/ipv4.dylan">wire format definition</a>.</p>
<p>Nowadays, after some years of doing formal verification and typed functional programming, I try to have effects, including mutable state, isolated and explicitly annotated. The code should not contain surprises, but straightforward to understand. The core protocol logic should not be convoluted with side effects, rather a small wrapper around it should. Once this is achieved, testing is straightforward. If the fashion of the asynchronous task library changes (likely with OCaml multicore), the core logic can be reused. It can also be repurposed to run as a test oracle. You can read more marketing of this style in our <a href="https://usenix15.nqsb.io">Usenix security paper</a>.</p>
<p>Nowadays, after some years of doing formal verification and typed functional programming, I try to have effects, including mutable state, isolated and explicitly annotated. The code should not contain surprises, but straightforward to understand. The core protocol logic should not be convoluted with side effects, rather a small wrapper around it should. Once this is achieved, testing is straightforward. If the fashion of the asynchronous task library changes (likely with OCaml multicore), the core logic can be reused. It can also be repurposed to run as a test oracle. You can read more marketing of this style in our <a href="https://www.usenix.org/conference/usenixsecurity15/technical-sessions/presentation/kaloper-mersinjak">Usenix security paper</a>.</p>
<p>My proposed style and hash tables are not good friends, since hash tables in OCaml are imperative structures. Instead, a <em>Map</em> (<a href="http://caml.inria.fr/pub/docs/manual-ocaml/libref/Map.html">documentation</a>) is a functional data structure for associating keys with values. Its underlying data structure is a balanced binary tree.</p>
<p>Our ARP handler certainly has some <em>state</em>, at least its IPv4 address, its Ethernet address, and the map containing entries.</p>
<p>We have to deal with the various effects mentioned earlier:</p>

4
Posts/BadRecordMac
View file

@ -1,5 +1,5 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Catch the bug, walking through the stack</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Catch the bug, walking through the stack" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Catch the bug, walking through the stack</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/security" class="tag">security</a></div><span class="date">Published: 2016-05-03 (last updated: 2021-11-19)</span><article><h2 id="bad-record-mac">BAD RECORD MAC</h2>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Catch the bug, walking through the stack</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Catch the bug, walking through the stack" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Catch the bug, walking through the stack</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/security" class="tag">security</a></div><span class="date">Published: 2016-05-03 (last updated: 2024-10-11)</span><article><h2 id="bad-record-mac">BAD RECORD MAC</h2>
<p>Roughly 2 weeks ago, <a href="https://github.com/Engil">Engil</a> informed me that a TLS alert pops up in his browser sometimes when he reads this website. His browser reported that the <a href="https://en.wikipedia.org/wiki/Message_authentication_code">message authentication code</a> was wrong. From <a href="https://tools.ietf.org/html/rfc5246">RFC 5246</a>: This message is always fatal and should never be observed in communication between proper implementations (except when messages were corrupted in the network).</p>
<p>I tried hard, but could not reproduce, but was very worried and was eager to find the root cause (some little fear remained that it was in our TLS stack). I setup this website with some TLS-level tracing (extending the code from our <a href="https://tls.openmirage.org">TLS handshake server</a>). We tried to reproduce the issue with traces and packet captures (both on client and server side) in place from our computer labs office with no success. Later, Engil tried from his home and after 45MB of wire data, ran into this issue. Finally, evidence! Isolating the TCP flow with the alert resulted in just about 200KB of packet capture data (TLS ASCII trace around 650KB).</p>
<p><img src="/static/img/encrypted-alert.png" alt="encrypted alert" /></p>
@ -51,7 +51,7 @@
</li>
<li><a href="https://github.com/mirage/mirage/pull/534">Mirage 2.9.0</a> was released, which integrates support of the logs library (now already used in <a href="https://github.com/mirage/mirage-net-xen/pull/43">mirage-net-xen</a> and <a href="https://github.com/mirage/mirage-tcpip/pull/199">mirage-tcpip</a>)
</li>
<li>This blog post has an accompanied <a href="https://mirage.io/blog/MSA00">MirageOS security advisory</a>
<li>This blog post has an accompanied <a href="https://mirageos.org/blog/MSA00">MirageOS security advisory</a>
</li>
<li>cfcs documented some <a href="https://github.com/cfcs/mirage-examples">basic unikernels</a>
</li>

6
Posts/DNS
View file

@ -1,10 +1,10 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>My 2018 contains robur and starts with re-engineering DNS</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="My 2018 contains robur and starts with re-engineering DNS" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>My 2018 contains robur and starts with re-engineering DNS</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a></div><span class="date">Published: 2018-01-11 (last updated: 2023-11-20)</span><article><h2 id="section">2018</h2>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>My 2018 contains robur and starts with re-engineering DNS</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="My 2018 contains robur and starts with re-engineering DNS" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>My 2018 contains robur and starts with re-engineering DNS</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a></div><span class="date">Published: 2018-01-11 (last updated: 2024-10-11)</span><article><h2 id="section">2018</h2>
<p>At the end of 2017, I resigned from my PostDoc position at University of
Cambridge (in the <a href="https://www.cl.cam.ac.uk/~pes20/rems/">rems</a> project). Early
December 2017 I organised the <a href="https://mirage.io/blog/2017-winter-hackathon-roundup">4th MirageOS hack
December 2017 I organised the <a href="https://mirageos.org/blog/2017-winter-hackathon-roundup">4th MirageOS hack
retreat</a>, with which I'm
very satisfied. In March 2018 the <a href="http://retreat.mirage.io">5th retreat</a> will
very satisfied. In March 2018 the <a href="http://retreat.mirageos.org">5th retreat</a> will
happen (please sign up!).</p>
<p>In 2018 I moved to Berlin and started to work for the (non-profit) <a href="https://techcultivation.org">Center for
the cultivation of technology</a> with our

4
Posts/DnsServer
View file

@ -1,5 +1,5 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Deploying authoritative OCaml-DNS servers as MirageOS unikernels</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Deploying authoritative OCaml-DNS servers as MirageOS unikernels" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Deploying authoritative OCaml-DNS servers as MirageOS unikernels</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a><a href="/tags/deployment" class="tag">deployment</a></div><span class="date">Published: 2019-12-23 (last updated: 2023-03-02)</span><article><h2 id="goal">Goal</h2>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Deploying authoritative OCaml-DNS servers as MirageOS unikernels</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Deploying authoritative OCaml-DNS servers as MirageOS unikernels" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Deploying authoritative OCaml-DNS servers as MirageOS unikernels</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a><a href="/tags/deployment" class="tag">deployment</a></div><span class="date">Published: 2019-12-23 (last updated: 2024-10-11)</span><article><h2 id="goal">Goal</h2>
<p>Have your domain served by OCaml-DNS authoritative name servers. Data is stored in a git remote, and let's encrypt certificates can be requested to DNS. This software is deployed since more than two years for several domains such as <code>nqsb.io</code> and <code>robur.coop</code>. This present the authoritative server side, and certificate library of the OCaml-DNS implementation formerly known as <a href="/Posts/DNS">µDNS</a>.</p>
<h2 id="prerequisites">Prerequisites</h2>
<p>You need to own a domain, and be able to delegate the name service to your own servers.
@ -173,7 +173,7 @@ personal._update.mirage. DNSKEY 0 3 163 kJJqipaQHQWqZL31Raar6uPnepGFIdtpjkXot9rv
10.0.42.2.10.0.42.3._transfer.mirage. DNSKEY 0 3 163 cDK6sKyvlt8UBerZlmxuD84ih2KookJGDagJlLVNo20=
git-repo&gt; git commit -m &quot;updates&quot; . &amp;&amp; git push
</code></pre>
<p>Ok, the git repository is ready, now we need to compile the unikernels for the virtualisation target (see <a href="https://mirage.io/wiki/hello-world#Building-for-Another-Backend">other targets</a> for further information).</p>
<p>Ok, the git repository is ready, now we need to compile the unikernels for the virtualisation target (see <a href="https://mirageos.org/wiki/hello-world#Building-for-Another-Backend">other targets</a> for further information).</p>
<pre><code class="language-shell"># back to primary
$ cd ../dns-primary-git
$ mirage configure -t hvt # or e.g. -t spt (and solo5-spt below)

4
Posts/EC
View file

@ -1,8 +1,8 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Cryptography updates in OCaml and MirageOS</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Cryptography updates in OCaml and MirageOS" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Cryptography updates in OCaml and MirageOS</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/security" class="tag">security</a><a href="/tags/tls" class="tag">tls</a></div><span class="date">Published: 2021-04-23 (last updated: 2021-11-19)</span><article><h2 id="introduction">Introduction</h2>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Cryptography updates in OCaml and MirageOS</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Cryptography updates in OCaml and MirageOS" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Cryptography updates in OCaml and MirageOS</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/security" class="tag">security</a><a href="/tags/tls" class="tag">tls</a></div><span class="date">Published: 2021-04-23 (last updated: 2024-10-11)</span><article><h2 id="introduction">Introduction</h2>
<p>Tl;DR: mirage-crypto-ec, with x509 0.12.0, and tls 0.13.0, provide fast and secure elliptic curve support in OCaml and MirageOS - using the verified <a href="https://github.com/mit-plv/fiat-crypto/">fiat-crypto</a> stack (Coq to OCaml to executable which generates C code that is interfaced by OCaml). In x509, a long standing issue (countryName encoding), and archive (PKCS 12) format is now supported, in addition to EC keys. In tls, ECDH key exchanges are supported, and ECDSA and EdDSA certificates.</p>
<h2 id="elliptic-curve-cryptography">Elliptic curve cryptography</h2>
<p><a href="https://mirage.io/blog/tls-1-3-mirageos">Since May 2020</a>, our <a href="https://usenix15.nqsb.io">OCaml-TLS</a> stack supports TLS 1.3 (since tls version 0.12.0 on opam).</p>
<p><a href="https://mirageos.org/blog/tls-1-3-mirageos">Since May 2020</a>, our <a href="https://www.usenix.org/conference/usenixsecurity15/technical-sessions/presentation/kaloper-mersinjak">OCaml-TLS</a> stack supports TLS 1.3 (since tls version 0.12.0 on opam).</p>
<p>TLS 1.3 requires elliptic curve cryptography - which was not available in <a href="https://github.com/mirage/mirage-crypto">mirage-crypto</a> (the maintained fork of <a href="https://github.com/mirleft/ocaml-nocrypto">nocrypto</a>).</p>
<p>There are two major uses of elliptic curves: <a href="https://en.wikipedia.org/wiki/Elliptic-curve_Diffie%E2%80%93Hellman">key exchange (ECDH)</a> for establishing a shared secret over an insecure channel, and <a href="https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm">digital signature (ECDSA)</a> for authentication, integrity, and non-repudiation. (Please note that the construction of digital signatures on Edwards curves (Curve25519, Ed448) is called EdDSA instead of ECDSA.)</p>
<p>Elliptic curve cryptoraphy is <a href="https://eprint.iacr.org/2020/615">vulnerable</a> <a href="https://raccoon-attack.com/">to</a> <a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-5407">various</a> <a href="https://github.com/mimoo/timing_attack_ecdsa_tls">timing</a> <a href="https://minerva.crocs.fi.muni.cz/">attacks</a> - have a read of the <a href="https://blog.trailofbits.com/2020/06/11/ecdsa-handle-with-care/">overview article on ECDSA</a>. When implementing elliptic curve cryptography, it is best to avoid these known attacks. Gladly, there are some projects which address these issues by construction.</p>

8
Posts/Functoria
View file

@ -1,11 +1,11 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Configuration DSL step-by-step</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Configuration DSL step-by-step" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Configuration DSL step-by-step</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/background" class="tag">background</a></div><span class="date">Published: 2016-05-10 (last updated: 2021-11-19)</span><article><p>Sorry for being late again with this article, I had other ones planned, but am not yet satisfied with content and code, will have to wait another week.</p>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Configuration DSL step-by-step</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Configuration DSL step-by-step" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Configuration DSL step-by-step</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/background" class="tag">background</a></div><span class="date">Published: 2016-05-10 (last updated: 2024-10-11)</span><article><p>Sorry for being late again with this article, I had other ones planned, but am not yet satisfied with content and code, will have to wait another week.</p>
<h2 id="mirageos-configuration">MirageOS configuration</h2>
<p>As described in an <a href="/Posts/OperatingSystem">earlier post</a>, MirageOS is a library operating system which generates single address space custom kernels (so called unikernels) for each application. The application code is (mostly) independent on the used backend. To achieve this, the language which expresses the configuration of a MirageOS unikernel is rather complex, and has to deal with package dependencies, setup of layers (network stack starting at the (virtual) ethernet device, or sockets), logging, tracing.</p>
<p>The abstraction over concrete implementation of e.g. the network stack is done by providing a module signature in the <a href="https://github.com/mirage/mirage/tree/master/types">mirage-types</a> package. The socket-based network stack, the tap device based network stack, and the Xen virtual network device based network stack implement this signature (depending on other module signatures). The unikernel contains code which applies those dependent modules to instantiate a custom-tailored network stack for the specific configuration. A developer should only describe what their requirements are, the user who wants to deploy it should provide the concrete configuration. And the developer should not need to manually instantiate the network stack for all possible configurations, this is what the mirage tool should embed.</p>
<p>Initially, MirageOS contained an adhoc system which relied on concatenation of strings representing OCaml code. This turned out to be error prone. In 2015 <a href="https://github.com/Drup">Drup</a> developed <a href="https://github.com/mirage/functoria">Functoria</a>, a domain-specific language (DSL) to organize functor applications, primarily for MirageOS. It has been introduced in <a href="https://mirage.io/blog/introducing-functoria">a blog post</a>. It is not limited to MirageOS (although this is the primary user right now).</p>
<p>Initially, MirageOS contained an adhoc system which relied on concatenation of strings representing OCaml code. This turned out to be error prone. In 2015 <a href="https://github.com/Drup">Drup</a> developed <a href="https://github.com/mirage/functoria">Functoria</a>, a domain-specific language (DSL) to organize functor applications, primarily for MirageOS. It has been introduced in <a href="https://mirageos.org/blog/introducing-functoria">a blog post</a>. It is not limited to MirageOS (although this is the primary user right now).</p>
<p>Functoria has been included in MirageOS since its <a href="https://github.com/mirage/mirage/releases/tag/v2.7.0">2.7.0 release</a> at the end of February 2016. Functoria provides support for command line arguments which can then either be passed at configuration time or at boot time to the unikernel (such as IP address configuration) using the <a href="http://erratique.ch/software/cmdliner">cmdliner library</a> underneath (and includes dynamic man pages, help, sensible command line parsing, and even visualisation (<code>mirage describe</code>) of the configuration and data dependencies).</p>
<p>I won't go into details about command line arguments in here, please have a look at the <a href="https://mirage.io/blog/introducing-functoria">functoria blog post</a> in case you're interested. Instead, I'll describe how to define a Functoria device which inserts content as code at configuration time into a MirageOS unikernel (<a href="http://marrakech2016.mirage.io">running here</a>, <a href="https://github.com/mirage/marrakech2016">source</a>). Using this approach, no external data (using crunch or a file system image) is needed, while the content can still be modified using markdown. Also, no markdown to HTML converter is needed at runtime, but this step is completely done at compile time (the result is a small (still too large) unikernel, 4.6MB).</p>
<p>I won't go into details about command line arguments in here, please have a look at the <a href="https://mirageos.org/blog/introducing-functoria">functoria blog post</a> in case you're interested. Instead, I'll describe how to define a Functoria device which inserts content as code at configuration time into a MirageOS unikernel (<a href="http://retreat.mirageos.org">running here</a>, <a href="https://github.com/mirage/retreat.mirageos.org">source</a>). Using this approach, no external data (using crunch or a file system image) is needed, while the content can still be modified using markdown. Also, no markdown to HTML converter is needed at runtime, but this step is completely done at compile time (the result is a small (still too large) unikernel, 4.6MB).</p>
<h3 id="unikernel">Unikernel</h3>
<p>Similar to <a href="/Posts/nqsbWebsite">my nqsb.io website post</a>, this unikernel only has a single resource and thus does not need to do any parsing (or even call <code>read</code>). The main function is <code>start</code>:</p>
<pre><code class="language-OCaml">let start stack _ =
@ -111,7 +111,7 @@ end
</li>
<li><a href="https://github.com/mirage/mirage-platform/pull/165">mirage-platform now has 4.03 support</a> and <a href="https://github.com/mirage/mirage-platform/issues/118">strtod</a> (finally :)
</li>
<li><a href="https://mirage.io/blog/2016-spring-hackathon">blog posts about retreat in marrakech</a>
<li><a href="https://mirageos.org/blog/2016-spring-hackathon">blog posts about retreat in marrakech</a>
</li>
<li><a href="https://github.com/Cumulus/Syndic">syndic 1.5.0 release</a> now using ptime instead of calendar
</li>

8
Posts/Jackline
View file

@ -1,6 +1,6 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Jackline, a secure terminal-based XMPP client</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Jackline, a secure terminal-based XMPP client" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Jackline, a secure terminal-based XMPP client</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/UI" class="tag">UI</a><a href="/tags/security" class="tag">security</a></div><span class="date">Published: 2017-01-30 (last updated: 2021-09-08)</span><article><p><img src="/static/img/jackline2.png" alt="screenshot" /></p>
<p>Back in 2014, when we implemented <a href="https://nqsb.io">TLS</a> in OCaml, at some point
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Jackline, a secure terminal-based XMPP client</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Jackline, a secure terminal-based XMPP client" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Jackline, a secure terminal-based XMPP client</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/UI" class="tag">UI</a><a href="/tags/security" class="tag">security</a></div><span class="date">Published: 2017-01-30 (last updated: 2024-10-11)</span><article><p><img src="/static/img/jackline2.png" alt="screenshot" /></p>
<p>Back in 2014, when we implemented <a href="https://github.com/mirleft/ocaml-tls">TLS</a> in OCaml, at some point
I was bored with TLS. I usually need at least two projects (but not more than 5) at the same time to
procrastinate the one I should do with the other one - it is always more fun to
do what you're not supposed to do. I started to implement another security
@ -139,7 +139,7 @@ friends logged in multiple times with wrongly set priorities - and end-to-end
encryption. I don't need inline HTML, avatar images, my currently running
music, leaking timezone information, etc. I explicitly don't want to import any
private key material from other clients and libraries, because I want to ensure
that the key was generated by a good random number generator (read <a href="https://mirage.io/blog/mirage-entropy">David's blog article</a> on randomness and entropy).</p>
that the key was generated by a good random number generator (read <a href="https://mirageos.org/blog/mirage-entropy">David's blog article</a> on randomness and entropy).</p>
<p>The security story is crucial: always do strict certificate validation, fail
hard, make it noticable by the user if they're doing insecure communication.
Only few people are into reading out loud their OTR public key fingerprint, and
@ -244,7 +244,7 @@ things I'd like to have as well).</p>
<li>
<p>The <a href="https://github.com/ermine/xmpp">XMPP library</a> makes heavy use of
functors (to abstract over the concrete IO, etc.), and embeds IO deep inside it.
I do prefer (see e.g. <a href="https://usenix15.nqsb.io">our TLS paper</a>, or <a href="/Posts/ARP">my ARP post</a>) these days to have a pure interface for
I do prefer (see e.g. <a href="https://www.usenix.org/conference/usenixsecurity15/technical-sessions/presentation/kaloper-mersinjak">our TLS paper</a>, or <a href="/Posts/ARP">my ARP post</a>) these days to have a pure interface for
the protocol implementation, providing explicit input (state, event, data), and
output (state, action, potentially data to send on network, potentially data to
process by the application). The <a href="https://github.com/hannesm/xmpp/blob/eee18bd3dd343550169969c0b45548eafd51cfe1/src/sasl.ml">sasl implementation</a>

8
Posts/NGI
View file

@ -1,5 +1,5 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>The road ahead for MirageOS in 2021</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="The road ahead for MirageOS in 2021" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>The road ahead for MirageOS in 2021</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a></div><span class="date">Published: 2021-01-25 (last updated: 2021-11-19)</span><article><h2 id="introduction">Introduction</h2>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>The road ahead for MirageOS in 2021</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="The road ahead for MirageOS in 2021" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>The road ahead for MirageOS in 2021</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a></div><span class="date">Published: 2021-01-25 (last updated: 2024-10-11)</span><article><h2 id="introduction">Introduction</h2>
<p>2020 was an intense year. I hope you're healthy and keep being healthy. I am privileged (as lots of software engineers and academics are) to be able to work from home during the pandemic. Let's not forget people in less privileged situations, and let’s try to give them as much practical, psychological and financial support as we can these days. And as much joy as possible to everyone around :)</p>
<p>I cancelled the autumn MirageOS retreat due to the pandemic. Instead I collected donations for our hosts in Marrakech - they were very happy to receive our financial support, since they had a difficult year, since their income is based on tourism. I hope that in autumn 2021 we'll have an on-site retreat again.</p>
<p>For 2021, we (at <a href="https://robur.coop">robur</a>) got a grant from the EU (via <a href="https://pointer.ngi.eu">NGI pointer</a>) for &quot;Deploying MirageOS&quot; (more details below), and another grant from <a href="https://ocaml-sf.org">OCaml software foundation</a> for securing the opam supply chain (using <a href="https://github.com/hannesm/conex">conex</a>). Some long-awaited releases for MirageOS libraries, namely a <a href="https://discuss.ocaml.org/t/ann-first-release-of-awa-ssh">ssh implementation</a> and a rewrite of our <a href="https://discuss.ocaml.org/t/ann-release-of-ocaml-git-v3-0-duff-encore-decompress-etc/">git implementation</a> have already been published.</p>
@ -13,9 +13,9 @@
<p><a href="https://github.com/mirage/mirage-crypto">Mirage-crypto</a> provides the underlying cryptographic primitives, initially released in March 2020 as a fork of <a href="https://github.com/mirleft/ocaml-nocrypto">nocrypto</a> -- huge thanks to <a href="https://github.com/pqwy">pqwy</a> for his great work. Mirage-crypto detects <a href="https://github.com/mirage/mirage-crypto/pull/53">CPU features at runtime</a> (thanks to <a href="https://github.com/Julow">Julow</a>) (<a href="https://github.com/mirage/mirage-crypto/pull/96">bugfix for bswap</a>), using constant time modular exponentation (powm_sec) and hardens against Lenstra's CRT attack, supports <a href="https://github.com/mirage/mirage-crypto/pull/39">compilation on Windows</a> (thanks to <a href="https://github.com/avsm">avsm</a>), <a href="https://github.com/mirage/mirage-crypto/pull/90">async entropy harvesting</a> (thanks to <a href="https://github.com/seliopou">seliopou</a>), <a href="https://github.com/mirage/mirage-crypto/pull/65">32 bit support</a>, <a href="https://github.com/mirage/mirage-crypto/pull/72">chacha20/poly1305</a> (thanks to <a href="https://github.com/abeaumont">abeaumont</a>), <a href="https://github.com/mirage/mirage-crypto/pull/84">cross-compilation</a> (thanks to <a href="https://github.com/EduardoRFS">EduardoRFS</a>) and <a href="https://github.com/mirage/mirage-crypto/pull/78">various</a> <a href="https://github.com/mirage/mirage-crypto/pull/81">bug</a> <a href="https://github.com/mirage/mirage-crypto/pull/83">fixes</a>, even <a href="https://github.com/mirage/mirage-crypto/pull/95">memory leak</a> (thanks to <a href="https://github.com/talex5">talex5</a> for reporting several of these issues), and <a href="https://github.com/mirage/mirage-crypto/pull/99">RSA</a> <a href="https://github.com/mirage/mirage-crypto/pull/100">interoperability</a> (thanks to <a href="https://github.com/psafont">psafont</a> for investigation and <a href="https://github.com/mattjbray">mattjbray</a> for reporting). This library feels very mature now - being used by multiple stakeholders, and lots of issues have been fixed in 2020.</p>
<h3 id="qubes-firewall">Qubes Firewall</h3>
<p>The <a href="https://github.com/mirage/qubes-mirage-firewall/">MirageOS based Qubes firewall</a> is the most widely used MirageOS unikernel. And it got major updates: in May <a href="https://github.com/linse">Steffi</a> <a href="https://groups.google.com/g/qubes-users/c/Xzplmkjwa5Y">announced</a> her and <a href="https://github.com/yomimono">Mindy's</a> work on improving it for Qubes 4.0 - including <a href="https://www.qubes-os.org/doc/vm-interface/#firewall-rules-in-4x">dynamic firewall rules via QubesDB</a>. Thanks to <a href="https://prototypefund.de/project/portable-firewall-fuer-qubesos/">prototypefund</a> for sponsoring.</p>
<p>In October 2020, we released <a href="https://mirage.io/blog/announcing-mirage-39-release">Mirage 3.9</a> with PVH virtualization mode (thanks to <a href="https://github.com/mato">mato</a>). There's still a <a href="https://github.com/mirage/qubes-mirage-firewall/issues/120">memory leak</a> to be investigated and fixed.</p>
<p>In October 2020, we released <a href="https://mirageos.org/blog/announcing-mirage-39-release">Mirage 3.9</a> with PVH virtualization mode (thanks to <a href="https://github.com/mato">mato</a>). There's still a <a href="https://github.com/mirage/qubes-mirage-firewall/issues/120">memory leak</a> to be investigated and fixed.</p>
<h3 id="ipv6">IPv6</h3>
<p>In December, with <a href="https://mirage.io/blog/announcing-mirage-310-release">Mirage 3.10</a> we got the IPv6 code up and running. Now MirageOS unikernels have a dual stack available, besides IPv4-only and IPv6-only network stacks. Thanks to <a href="https://github.com/nojb">nojb</a> for the initial code and <a href="https://github.com/MagnusS">MagnusS</a>.</p>
<p>In December, with <a href="https://mirageos.org/blog/announcing-mirage-310-release">Mirage 3.10</a> we got the IPv6 code up and running. Now MirageOS unikernels have a dual stack available, besides IPv4-only and IPv6-only network stacks. Thanks to <a href="https://github.com/nojb">nojb</a> for the initial code and <a href="https://github.com/MagnusS">MagnusS</a>.</p>
<p>Turns out this blog, but also robur services, are now available via IPv6 :)</p>
<h3 id="albatross">Albatross</h3>
<p>Also in December, I pushed an initial release of <a href="https://github.com/roburio/albatross">albatross</a>, a unikernel orchestration system with remote access. <em>Deploy your unikernel via a TLS handshake -- the unikernel image is embedded in the TLS client certificates.</em></p>
@ -23,7 +23,7 @@
<h3 id="ca-certs">CA certs</h3>
<p>For several years I postponed the problem of how to actually use the operating system trust anchors for OCaml-TLS connections. Thanks to <a href="https://github.com/emillon">emillon</a> for initial code, there are now <a href="https://github.com/mirage/ca-certs">ca-certs</a> and <a href="https://github.com/mirage/ca-certs-nss">ca-certs-nss</a> opam packages (see <a href="https://discuss.ocaml.org/t/ann-ca-certs-and-ca-certs-nss">release announcement</a>) which fills this gap.</p>
<h2 id="unikernels">Unikernels</h2>
<p>I developed several useful unikernels in 2020, and also pushed <a href="https://mirage.io/wiki/gallery">a unikernel gallery</a> to the Mirage website:</p>
<p>I developed several useful unikernels in 2020, and also pushed <a href="https://mirageos.org/wiki/gallery">a unikernel gallery</a> to the Mirage website:</p>
<h3 id="traceroute-in-mirageos">Traceroute in MirageOS</h3>
<p>I already wrote about <a href="/Posts/Traceroute">traceroute</a> which traces the routing to a given remote host.</p>
<h3 id="unipi---static-website-hosting">Unipi - static website hosting</h3>

6
Posts/OCaml
View file

@ -1,5 +1,5 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Why OCaml</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Why OCaml" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Why OCaml</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/overview" class="tag">overview</a><a href="/tags/background" class="tag">background</a></div><span class="date">Published: 2016-04-17 (last updated: 2021-11-19)</span><article><h2 id="programming">Programming</h2>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Why OCaml</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Why OCaml" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Why OCaml</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/overview" class="tag">overview</a><a href="/tags/background" class="tag">background</a></div><span class="date">Published: 2016-04-17 (last updated: 2024-10-11)</span><article><h2 id="programming">Programming</h2>
<p>For me, programming is fun. I enjoy doing it, every single second. All the way
from designing over experimenting to debugging why it does not do what I want.
In the end, the computer is dumb and executes only what you (or code from
@ -72,7 +72,7 @@ on a regular schedule. Daniel wrote <a href="http://erratique.ch/software/rresu
The <a href="https://opam.ocaml.org/packages/">opam repository</a> contains over 1000
libraries. The quality varies, I personally like the small libraries done by
<a href="http://erratique.ch/software">Daniel Bünzli</a>, as well as our
<a href="https://nqsb.io">nqsb</a> libraries (see <a href="https://github.com/mirleft">mirleft org</a>),
not quite so broken libraries (see <a href="https://github.com/mirleft">mirleft org</a>),
<a href="https://github.com/pqwy/notty">notty</a>.
A concise library (not much code),
including tests, documentation, etc. is
@ -101,7 +101,7 @@ developed within MirageOS. Because I'm impatient, I setup an <a href="https://g
feed</a>
which watches lots of MirageOS-related repositories.</p>
<p>I hope I gave some insight into OCaml, and why I currently enjoy it. A longer read on applicability of OCaml is our Usenix 2015 paper
<a href="https://nqsb.io/nqsbtls-usenix-security15.pdf">Not-quite-so-broken TLS: lessons in re-engineering a security protocol
<a href="https://www.usenix.org/conference/usenixsecurity15/technical-sessions/presentation/kaloper-mersinjak">Not-quite-so-broken TLS: lessons in re-engineering a security protocol
specification and
implementation</a>. I'm interested in feedback, either via
<a href="https://twitter.com/h4nnes">twitter</a> or via eMail.</p>

6
Posts/OpamMirror
View file

@ -1,5 +1,5 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Mirroring the opam repository and all tarballs</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Mirroring the opam repository and all tarballs" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Mirroring the opam repository and all tarballs</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/deployment" class="tag">deployment</a><a href="/tags/opam" class="tag">opam</a></div><span class="date">Published: 2022-09-29 (last updated: 2023-11-20)</span><article><p>We at <a href="https://robur.coop">robur</a> developed <a href="https://git.robur.coop/robur/opam-mirror">opam-mirror</a> in the last month and run a public opam mirror at https://opam.robur.coop (updated hourly).</p>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Mirroring the opam repository and all tarballs</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Mirroring the opam repository and all tarballs" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Mirroring the opam repository and all tarballs</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/deployment" class="tag">deployment</a><a href="/tags/opam" class="tag">opam</a></div><span class="date">Published: 2022-09-29 (last updated: 2024-10-11)</span><article><p>We at <a href="https://robur.coop">robur</a> developed <a href="https://git.robur.coop/robur/opam-mirror">opam-mirror</a> in the last month and run a public opam mirror at https://opam.robur.coop (updated hourly).</p>
<h1 id="what-is-opam-and-why-should-i-care">What is opam and why should I care?</h1>
<p><a href="https://opam.ocaml.org">Opam</a> is the OCaml package manager (also used by other projects such as <a href="https://coq.inria.fr">coq</a>). It is a source based system: the so-called repository contains the metadata (url to source tarballs, build dependencies, author, homepage, development repository) of all packages. The main repository is hosted on GitHub as <a href="https://github.com/ocaml/opam-repository">ocaml/opam-repository</a>, where authors of OCaml software can contribute (as pull request) their latest releases.</p>
<p>When opening a pull request, automated systems attempt to build not only the newly released package on various platforms and OCaml versions, but also all reverse dependencies, and also with dependencies with the lowest allowed version numbers. That's crucial since neither semantic versioning has been adapted across the OCaml ecosystem (which is tricky, for example due to local opens any newly introduced binding will lead to a major version bump), neither do many people add upper bounds of dependencies when releasing a package (nobody is keen to state &quot;my package will not work with <a href="https://erratique.ch/software/cmdliner">cmdliner</a> in version 1.2.0&quot;).</p>
@ -10,9 +10,9 @@
<p>Opam, after initialisation, downloads the <code>index.tar.gz</code> from <code>https://opam.ocaml.org/index.tar.gz</code>, and uses this as the local opam universe. An <code>opam install cmdliner</code> will resolve the dependencies, and download all required tarballs. The download is first tried from the cache, and if that failed, the URL in the package file is used. The download from the cache uses the base url, appends the archive-mirror, followed by the hash algorithm, the first two characters of the has of the tarball, and the hex encoded hash of the archive, i.e. for cmdliner 1.1.1 which specifies its sha512: <code>https://opam.ocaml.org/cache/sha512/54/5478ad833da254b5587b3746e3a8493e66e867a081ac0f653a901cc8a7d944f66e4387592215ce25d939be76f281c4785702f54d4a74b1700bc8838a62255c9e</code>.</p>
<h1 id="how-does-the-opam-repository-work">How does the opam repository work?</h1>
<p>According to DNS, opam.ocaml.org is a machine at amazon. It likely, apart from the website, uses <code>opam admin index</code> periodically to create the index tarball and the cache. There's an observable delay between a package merge in the opam-repository and when it shows up at opam.ocaml.org. Recently, there was <a href="https://discuss.ocaml.org/t/opam-ocaml-org-is-currently-down-is-that-where-indices-are-kept-still/">a reported downtime</a>.</p>
<p>Apart from being a single point of failure, if you're compiling a lot of opam projects (e.g. a continuous integration / continuous build system), it makes sense from a network usage (and thus sustainability perspective) to move the cache closer to where you need the source archives. We're also organising the MirageOS <a href="http://retreat.mirage.io">hack retreats</a> in a northern African country with poor connectivity - so if you gather two dozen camels you better bring your opam repository cache with you to reduce the bandwidth usage (NB: this requires at the moment cooperation of all participants to configure their default opam repository accordingly).</p>
<p>Apart from being a single point of failure, if you're compiling a lot of opam projects (e.g. a continuous integration / continuous build system), it makes sense from a network usage (and thus sustainability perspective) to move the cache closer to where you need the source archives. We're also organising the MirageOS <a href="http://retreat.mirageos.org">hack retreats</a> in a northern African country with poor connectivity - so if you gather two dozen camels you better bring your opam repository cache with you to reduce the bandwidth usage (NB: this requires at the moment cooperation of all participants to configure their default opam repository accordingly).</p>
<h1 id="re-developing-opam-admin-create-as-mirageos-unikernel">Re-developing &quot;opam admin create&quot; as MirageOS unikernel</h1>
<p>The need for a local opam cache at our <a href="https://builds.robur.coop">reproducible build infrastructure</a> and the retreats, we decided to develop <a href="https://git.robur.coop/robur/opam-mirror">opam-mirror</a> as a <a href="https://mirage.io">MirageOS unikernel</a>. Apart from a useful showcase using persistent storage (that won't fit into memory), and having fun while developing it, our aim was to reduce our time spent on system administration (the <code>opam admin index</code> is only one part of the story, it needs a Unix system and a webserver next to it - plus remote access for doing software updates - which has quite some attack surface.</p>
<p>The need for a local opam cache at our <a href="https://builds.robur.coop">reproducible build infrastructure</a> and the retreats, we decided to develop <a href="https://git.robur.coop/robur/opam-mirror">opam-mirror</a> as a <a href="https://mirageos.org">MirageOS unikernel</a>. Apart from a useful showcase using persistent storage (that won't fit into memory), and having fun while developing it, our aim was to reduce our time spent on system administration (the <code>opam admin index</code> is only one part of the story, it needs a Unix system and a webserver next to it - plus remote access for doing software updates - which has quite some attack surface.</p>
<p>Another reason for re-developing the functionality was that the opam code (what opam admin index actually does) is part of the opam source code, which totals to 50_000 lines of code -- looking up whether one or all checksums are verified before adding the tarball to the cache, was rather tricky.</p>
<p>In earlier years, we avoided persistent storage and block devices in MirageOS (by embedding it into the source code with <a href="https://github.com/mirage/ocaml-crunch">crunch</a>, or using a remote git repository), but recent development, e.g. of <a href="https://somerandomidiot.com/blog/2022/03/04/chamelon/">chamelon</a> sparked some interest in actually using file systems and figuring out whether MirageOS is ready in that area. A month ago we started the opam-mirror project.</p>
<p>Opam-mirror takes a remote repository URL, and downloads all referenced archives. It serves as a cache and opam-repository - and does periodic updates from the remote repository. The idea is to validate all available checksums and store the tarballs only once, and store overlays (as maps) from the other hash algorithms.</p>

6
Posts/OperatingSystem
View file

@ -1,5 +1,5 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Operating systems</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Operating systems" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Operating systems</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/overview" class="tag">overview</a><a href="/tags/operating system" class="tag">operating system</a><a href="/tags/mirageos" class="tag">mirageos</a></div><span class="date">Published: 2016-04-09 (last updated: 2021-11-19)</span><article><p>Sorry to be late with this entry, but I had to fix some issues.</p>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Operating systems</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Operating systems" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Operating systems</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/overview" class="tag">overview</a><a href="/tags/operating system" class="tag">operating system</a><a href="/tags/mirageos" class="tag">mirageos</a></div><span class="date">Published: 2016-04-09 (last updated: 2024-10-11)</span><article><p>Sorry to be late with this entry, but I had to fix some issues.</p>
<h2 id="what-is-an-operating-system">What is an operating system?</h2>
<p>Wikipedia says: &quot;An operating system (OS) is system software that manages
computer hardware and software resources and provides common services for
@ -57,7 +57,7 @@ Dylan), and as mentioned earlier I went into formal methods and mechanised
proofs of full functional correctness properties.</p>
<h3 id="mirageos">MirageOS</h3>
<p>At the end of 2013, David pointed me to
<a href="https://mirage.io">MirageOS</a>, an operating system developed from scratch in the
<a href="https://mirageos.org">MirageOS</a>, an operating system developed from scratch in the
functional and statically typed language <a href="https://ocaml.org">OCaml</a>. I've not
used much OCaml before, but some other functional programming languages.
Since then, I spend nearly every day on developing OCaml libraries (with varying success on being happy
@ -153,7 +153,7 @@ how MirageOS fits into the picture. I'm interested in feedback, either via
</li>
<li>I was travelling
</li>
<li>good news: it now works on Xen, and there is <a href="https://hannes.nqsb.io/atom">an atom feed</a>
<li>good news: it now works on Xen, and there is <a href="https://hannes.robur.coop/atom">an atom feed</a>
</li>
<li>life of an &quot;eat your own dogfood&quot; full stack engineer ;)
</li>

12
Posts/Pinata
View file

@ -1,21 +1,21 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>The Bitcoin Piñata - no candy for you</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="The Bitcoin Piñata - no candy for you" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>The Bitcoin Piñata - no candy for you</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/security" class="tag">security</a><a href="/tags/bitcoin" class="tag">bitcoin</a></div><span class="date">Published: 2018-04-18 (last updated: 2021-11-19)</span><article><h2 id="history">History</h2>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>The Bitcoin Piñata - no candy for you</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="The Bitcoin Piñata - no candy for you" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>The Bitcoin Piñata - no candy for you</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/security" class="tag">security</a><a href="/tags/bitcoin" class="tag">bitcoin</a></div><span class="date">Published: 2018-04-18 (last updated: 2024-10-11)</span><article><h2 id="history">History</h2>
<p>On February 10th 2015 David Kaloper-Meršinjak and Hannes Mehnert
<a href="https://mirage.io/announcing-bitcoin-pinata">launched</a> (read also <a href="http://amirchaudhry.com/bitcoin-pinata">Amir's
<a href="https://mirageos.org/announcing-bitcoin-pinata">launched</a> (read also <a href="http://amirchaudhry.com/bitcoin-pinata">Amir's
description</a>) our <a href="https://en.wikipedia.org/wiki/Bug_bounty_program">bug bounty
program</a> in the form of our
<a href="http://ownme.ipredator.se">Bitcoin Piñata</a> MirageOS unikernel. Thanks again to
<a href="https://ipredator.se">IPredator</a> for both hosting our services and lending us
the 10 Bitcoins! We <a href="https://mirage.io/blog/bitcoin-pinata-results">analysed</a> a
the 10 Bitcoins! We <a href="https://mirageos.org/blog/bitcoin-pinata-results">analysed</a> a
bit more in depth after running it for five months. Mindy recently wrote about
<a href="https://somerandomidiot.com/blog/2018/04/17/whacking-the-bitcoin-pinata/">whacking the Bitcoin
Piñata</a>.</p>
<p>On March 18th 2018, after more than three years, IPredator, the lender of the Bitcoins, repurposed the 10 Bitcoins for other projects. Initially, we thought that the Piñata would maybe run for a month or two, but IPredator, David, and I decided to keep it running. The update of the Piñata's bounty is a good opportunity to reflect on the project.</p>
<p>The 10 Bitcoin in the Piñata were fluctuating in price over time, at peak worth 165000€.</p>
<p>From the start of the Piñata project, we published the <a href="https://github.com/mirleft/btc-pinata">source code</a>, the virtual machine image, and the versions of the used libraries in a git repository. Everybody could develop their exploits locally before launching them against our Piñata. The Piñata provides TLS endpoints, which require private keys and certificates. These are generated by the Piñata at startup, and the secret for the Bitcoin wallet is provided as a command line argument.</p>
<p>Initially the Piñata was deployed on a Linux/Xen machine, later it was migrated to a FreeBSD host using BHyve and VirtIO with <a href="https://github.com/solo5/solo5">solo5</a>, and in December 2017 it was migrated to native BHyve (<a href="/Posts/Solo5">using <code>ukvm-bin</code> and solo5</a>). We also changed the Piñata code to accomodate for updates, such as the <a href="https://mirage.io/blog/announcing-mirage-30-release">MirageOS 3.0 release</a>, and the discontinuation of floating point numbers for timestamps (asn1-combinators 0.2.0, x509 0.6.0, tls 0.9.0).</p>
<p>Initially the Piñata was deployed on a Linux/Xen machine, later it was migrated to a FreeBSD host using BHyve and VirtIO with <a href="https://github.com/solo5/solo5">solo5</a>, and in December 2017 it was migrated to native BHyve (<a href="/Posts/Solo5">using <code>ukvm-bin</code> and solo5</a>). We also changed the Piñata code to accomodate for updates, such as the <a href="https://mirageos.org/blog/announcing-mirage-30-release">MirageOS 3.0 release</a>, and the discontinuation of floating point numbers for timestamps (asn1-combinators 0.2.0, x509 0.6.0, tls 0.9.0).</p>
<h2 id="motivation">Motivation</h2>
<p>We built the Piñata for many purposes: to attract security professionals to evaluate our <a href="https://mirage.io/blog/introducing-ocaml-tls">from-scratch developed TLS stack</a>, to gather empirical data for our <a href="https://usenix15.nqsb.io">Usenix Security 15 paper</a>, and as an improvement to current bug bounty programs.</p>
<p>We built the Piñata for many purposes: to attract security professionals to evaluate our <a href="https://mirageos.org/blog/introducing-ocaml-tls">from-scratch developed TLS stack</a>, to gather empirical data for our <a href="https://www.usenix.org/conference/usenixsecurity15/technical-sessions/presentation/kaloper-mersinjak">Usenix Security 15 paper</a>, and as an improvement to current bug bounty programs.</p>
<p>Most bug bounty programs require communication via forms and long wait times for
human experts to evaluate the potential bug. This evaluation is subjective,
intransparent, and often requires signing of non-disclosure agreements (NDA),
@ -32,7 +32,7 @@ that the access to the wallet is actually inside. But trust us, it is!</p>
<p>In April 2016 we stumbled upon an <a href="/Posts/BadRecordMac">information disclosure in the virtual network
device driver for Xen in MirageOS</a>. Given enough
bandwidth, this could have been used to access the private wallet key. We
upgraded the Piñata and released the <a href="https://mirage.io/blog/MSA00">MirageOS Security Advisory
upgraded the Piñata and released the <a href="https://mirageos.org/blog/MSA00">MirageOS Security Advisory
00</a>.</p>
<p>We analysed the Piñata's access logs to the and bucketed them into website traffic and bounty connections. We are still wondering what happened in July 2015 and July 2017 where the graph shows spikes. Could it be a presentation mentioning the Piñata, or a new automated tool which tests for TLS vulnerabilities, or an increase in market price for Bitcoins?</p>
<p><img src="/static/img/pinata_access_20180403.png" alt="Piñata access" /> <img src="/static/img/pinata_access_cumulative_20180403.png" alt="Piñata access cumulative" /></p>

6
Posts/ReproducibleOPAM
View file

@ -1,11 +1,11 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Reproducible MirageOS unikernel builds</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Reproducible MirageOS unikernel builds" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Reproducible MirageOS unikernel builds</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/security" class="tag">security</a><a href="/tags/package signing" class="tag">package signing</a></div><span class="date">Published: 2019-12-16 (last updated: 2021-11-19)</span><article><h2 id="reproducible-builds-summit">Reproducible builds summit</h2>
<p>I'm just back from the <a href="https://reproducible-builds.org/events/Marrakesh2019/">Reproducible builds summit 2019</a>. In 2018, several people developing <a href="https://ocaml.org">OCaml</a> and <a href="https://opam.ocaml.org">opam</a> and <a href="https://mirage.io">MirageOS</a>, attended <a href="https://reproducible-builds.org/events/paris2018/">the Reproducible builds summit in Paris</a>. The notes from last year on <a href="https://reproducible-builds.org/events/paris2018/report/#Toc11410_331763073">opam reproducibility</a> and <a href="https://reproducible-builds.org/events/paris2018/report/#Toc11681_331763073">MirageOS reproducibility</a> are online. After last years workshop, Raja started developing the opam reproducibilty builder <a href="https://github.com/rjbou/orb">orb</a>, which I extended at and after this years summit. This year before and after the facilitated summit there were hacking days, which allowed further interaction with participants, writing some code and conduct experiments. I had this year again an exciting time at the summit and hacking days, thanks to our hosts, organisers, and all participants.</p>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Reproducible MirageOS unikernel builds</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Reproducible MirageOS unikernel builds" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Reproducible MirageOS unikernel builds</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/security" class="tag">security</a><a href="/tags/package signing" class="tag">package signing</a></div><span class="date">Published: 2019-12-16 (last updated: 2024-10-11)</span><article><h2 id="reproducible-builds-summit">Reproducible builds summit</h2>
<p>I'm just back from the <a href="https://reproducible-builds.org/events/Marrakesh2019/">Reproducible builds summit 2019</a>. In 2018, several people developing <a href="https://ocaml.org">OCaml</a> and <a href="https://opam.ocaml.org">opam</a> and <a href="https://mirageos.org">MirageOS</a>, attended <a href="https://reproducible-builds.org/events/paris2018/">the Reproducible builds summit in Paris</a>. The notes from last year on <a href="https://reproducible-builds.org/events/paris2018/report/#Toc11410_331763073">opam reproducibility</a> and <a href="https://reproducible-builds.org/events/paris2018/report/#Toc11681_331763073">MirageOS reproducibility</a> are online. After last years workshop, Raja started developing the opam reproducibilty builder <a href="https://github.com/rjbou/orb">orb</a>, which I extended at and after this years summit. This year before and after the facilitated summit there were hacking days, which allowed further interaction with participants, writing some code and conduct experiments. I had this year again an exciting time at the summit and hacking days, thanks to our hosts, organisers, and all participants.</p>
<h2 id="goal">Goal</h2>
<p>Stepping back a bit, first look on the <a href="https://reproducible-builds.org/">goal of reproducible builds</a>: when compiling source code multiple times, the produced binaries should be identical. It should be sufficient if the binaries are behaviourally equal, but this is pretty hard to check. It is much easier to check <strong>bit-wise identity of binaries</strong>, and relaxes the burden on the checker -- checking for reproducibility is reduced to computing the hash of the binaries. Let's stick to the bit-wise identical binary definition, which also means software developers have to avoid non-determinism during compilation in their toolchains, dependent libraries, and developed code.</p>
<p>A <a href="https://reproducible-builds.org/docs/test-bench/">checklist</a> of potential things leading to non-determinism has been written up by the reproducible builds project. Examples include recording the build timestamp into the binary, ordering of code and embedded data. The reproducible builds project also developed <a href="https://packages.debian.org/sid/disorderfs">disorderfs</a> for testing reproducibility and <a href="https://diffoscope.org/">diffoscope</a> for comparing binaries with file-dependent readers, falling back to <code>objdump</code> and <code>hexdump</code>. A giant <a href="https://tests.reproducible-builds.org/">test infrastructure</a> with <a href="https://tests.reproducible-builds.org/debian/index_variations.html">lots of variations</a> between the builds, mostly using Debian, has been setup over the years.</p>
<p>Reproducibility is a precondition for trustworthy binaries. See <a href="https://reproducible-builds.org/#why-does-it-matter">why does it matter</a>. If there are no instructions how to get from the published sources to the exact binary, why should anyone trust and use the binary which claims to be the result of the sources? It may as well contain different code, including a backdoor, bitcoin mining code, outputting the wrong results for specific inputs, etc. Reproducibility does not imply the software is free of security issues or backdoors, but instead of a audit of the binary - which is tedious and rarely done - the source code can be audited - but the toolchain (compiler, linker, ..) used for compilation needs to be taken into account, i.e. trusted or audited to not be malicious. <strong>I will only ever publish binaries if they are reproducible</strong>.</p>
<p>My main interest at the summit was to enhance existing tooling and conduct some experiments about the reproducibility of <a href="https://mirage.io">MirageOS unikernels</a> -- a unikernel is a statically linked ELF binary to be run as Unix process or <a href="https://github.com/solo5/solo5">virtual machine</a>. MirageOS heavily uses <a href="https://ocaml.org">OCaml</a> and <a href="https://opam.ocaml.org">opam</a>, the OCaml package manager, and is an opam package itself. Thus, <em>checking reproducibility of a MirageOS unikernel is the same problem as checking reproducibility of an opam package</em>.</p>
<p>My main interest at the summit was to enhance existing tooling and conduct some experiments about the reproducibility of <a href="https://mirageos.org">MirageOS unikernels</a> -- a unikernel is a statically linked ELF binary to be run as Unix process or <a href="https://github.com/solo5/solo5">virtual machine</a>. MirageOS heavily uses <a href="https://ocaml.org">OCaml</a> and <a href="https://opam.ocaml.org">opam</a>, the OCaml package manager, and is an opam package itself. Thus, <em>checking reproducibility of a MirageOS unikernel is the same problem as checking reproducibility of an opam package</em>.</p>
<h2 id="reproducible-builds-with-opam">Reproducible builds with opam</h2>
<p>Testing for reproducibility is achieved by taking the sources and compile them twice independently. Afterwards the equality of the resulting binaries can be checked. In trivial projects, the sources is just a single file, or originate from a single tarball. In OCaml, opam uses <a href="https://github.com/ocaml/opam-repository">a community repository</a> where OCaml developers publish their package releases to, but can also use custom repositores, and in addition pin packages to git remotes (url including branch or commit), or a directory on the local filesystem. Manually tracking and updating all dependent packages of a MirageOS unikernel is not feasible: our hello-world compiled for hvt (kvm/BHyve) already has 79 opam dependencies, including the OCaml compiler which is distribued as opam package. The unikernel serving this website depends on 175 opam packages.</p>
<p>Conceptually there should be two tools, the <em>initial builder</em>, which takes the latest opam packages which do not conflict, and exports exact package versions used during the build, as well as hashes of binaries. The other tool is a <em>rebuilder</em>, which imports the export, conducts a build, and outputs the hashes of the produced binaries.</p>

4
Posts/Retreat2024
View file

@ -1,7 +1,7 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>The MirageOS retreat 2024</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="The MirageOS retreat 2024" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>The MirageOS retreat 2024</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/community" class="tag">community</a></div><span class="date">Published: 2024-05-15 (last updated: 2024-05-16)</span><article><p>End of April I spent a week in Marrakech to work on MirageOS at our annual <a href="https://retreat.mirage.io">retreat</a>. This was the 12th time when it actually took place, and it was amazing. We were a total of 17 people attending, of which 5 people used ground transportation. I have respect for them, to take the 3 or more day trip from Berlin or Glasgow or Paris or whereever to Marrakech - with a ferry in between (which availability depends on the wind etc.). This time, I didn't take ground transportation since I had appointments just before and just after the retreat nearby Berlin where I had to be in person. The food an weather was really nice and motivating.</p>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>The MirageOS retreat 2024</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="The MirageOS retreat 2024" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>The MirageOS retreat 2024</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/community" class="tag">community</a></div><span class="date">Published: 2024-05-15 (last updated: 2024-10-11)</span><article><p>End of April I spent a week in Marrakech to work on MirageOS at our annual <a href="https://retreat.mirageos.org">retreat</a>. This was the 12th time when it actually took place, and it was amazing. We were a total of 17 people attending, of which 5 people used ground transportation. I have respect for them, to take the 3 or more day trip from Berlin or Glasgow or Paris or whereever to Marrakech - with a ferry in between (which availability depends on the wind etc.). This time, I didn't take ground transportation since I had appointments just before and just after the retreat nearby Berlin where I had to be in person. The food an weather was really nice and motivating.</p>
<p>Of the 17 people there were lots of new faces. It is really nice to see an active community which is welcoming to new people, and also that there's change: a lot of fresh ideas are brought by these people and we rethink what we are doing and why.</p>
<p>Our daily routine included breakfast, circle, lunch, dinner, (sometimes) presentations. There wasn't more formal structure, and there was no need for it since people kept themselves busy working on various projects. We had a bigger discussion about &quot;deploying MirageOS&quot;, which is a reoccurring theme. Even running a MirageOS unikernel on a laptop is for people who are not very familiar with networking pretty hard. In addition, our <a href="https://mirage.io">website</a> only briefly described network setup, and included DHCP very early (which is overwhelming to setup and use on your laptop).</p>
<p>Our daily routine included breakfast, circle, lunch, dinner, (sometimes) presentations. There wasn't more formal structure, and there was no need for it since people kept themselves busy working on various projects. We had a bigger discussion about &quot;deploying MirageOS&quot;, which is a reoccurring theme. Even running a MirageOS unikernel on a laptop is for people who are not very familiar with networking pretty hard. In addition, our <a href="https://mirageos.org">website</a> only briefly described network setup, and included DHCP very early (which is overwhelming to setup and use on your laptop).</p>
<p>I went to the retreat with some ideas in my head what I want to achieve (for example, replace all the bigarray allocations by bytes &amp; string), but the discussion about deployment made me prioritise this more.</p>
<h2 id="operator-handbook">Operator handbook</h2>
<p>I sat down with some people to start off a <a href="https://mirage.github.io/operator-handbook/index.html">MirageOS operator handbook</a> - we talked about the scope and the early chapters (and what is not in scope). This is still very much work in progress, <a href="https://github.com/mirage/operator-handbook">issues</a> and pull requests are welcome. The target audience is someone who wants to execute a MirageOS unikernel with hvt on a laptop or server. It will not be necessary to install OCaml, but the focus is really on the operational aspects with binaries that we provide from our <a href="https://builds.robur.coop">reproducible build infrastructure</a>.</p>

4
Posts/Summer2019
View file

@ -1,9 +1,9 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Summer 2019</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Summer 2019" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Summer 2019</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/security" class="tag">security</a><a href="/tags/package signing" class="tag">package signing</a><a href="/tags/tls" class="tag">tls</a><a href="/tags/monitoring" class="tag">monitoring</a><a href="/tags/deployment" class="tag">deployment</a></div><span class="date">Published: 2019-07-08 (last updated: 2021-11-19)</span><article><h2 id="working-at-robur">Working at <a href="https://robur.io">robur</a></h2>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Summer 2019</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Summer 2019" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Summer 2019</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/security" class="tag">security</a><a href="/tags/package signing" class="tag">package signing</a><a href="/tags/tls" class="tag">tls</a><a href="/tags/monitoring" class="tag">monitoring</a><a href="/tags/deployment" class="tag">deployment</a></div><span class="date">Published: 2019-07-08 (last updated: 2024-10-11)</span><article><h2 id="working-at-robur">Working at <a href="https://robur.io">robur</a></h2>
<p>As announced <a href="/Posts/DNS">previously</a>, I started to work at robur early 2018. We're a collective of five people, distributed around Europe and the US, with the goal to deploy MirageOS unikernels. We do this by developing bespoke MirageOS unikernels which provide useful services, and deploy them for ourselves. We also develop new libraries and enhance existing ones and other components of MirageOS. Example unikernels include <a href="https://robur.io">our website</a> which uses <a href="https://github.com/Engil/Canopy">Canopy</a>, a <a href="https://robur.io/Our%20Work/Projects#CalDAV-Server">CalDAV server that stores entries in a git remote</a>, and <a href="https://github.com/roburio/unikernels">DNS servers</a> (the latter two are further described below).</p>
<p>Robur is part of the non-profit company <a href="https://techcultivation.org">Center for the Cultivation of Technology</a>, who are managing the legal and administrative sides for us. We're ourselves responsible to acquire funding to pay ourselves reasonable salaries. We received funding for CalDAV from <a href="https://prototypefund.de">prototypefund</a> and further funding from <a href="https://tarides.com">Tarides</a>, for TLS 1.3 from <a href="http://ocamllabs.io/">OCaml Labs</a>; security-audited an OCaml codebase, and received <a href="https://robur.io/Donate">donations</a>, also in the form of Bitcoins. We're looking for further funded collaborations and also contracting, mail us at <code>team@robur.io</code>. Please <a href="https://robur.io/Donate">donate</a> (tax-deductible in EU), so we can accomplish our goal of putting robust and sustainable MirageOS unikernels into production, replacing insecure legacy system that emit tons of CO<span style="vertical-align: baseline; position: relative;bottom: -0.4em;">2</span>.</p>
<h2 id="deploying-mirageos-unikernels">Deploying MirageOS unikernels</h2>
<p>While several examples are running since years (the <a href="https://mirage.io">MirageOS website</a>, <a href="http://ownme.ipredator.se">Bitcoin Piñata</a>, <a href="https://tls.nqsb.io">TLS demo server</a>, etc.), and some shell-scripts for cloud providers are floating around, it is not (yet) streamlined.</p>
<p>While several examples are running since years (the <a href="https://mirageos.org">MirageOS website</a>, <a href="http://ownme.ipredator.se">Bitcoin Piñata</a>, TLS demo server, etc.), and some shell-scripts for cloud providers are floating around, it is not (yet) streamlined.</p>
<p>Service deployment is complex: you have to consider its configuration, exfiltration of logs and metrics, provisioning with valid key material (TLS certificate, hmac shared secret) and authenticators (CA certificate, ssh key fingerprint). Instead of requiring millions lines of code during orchestration (such as Kubernetes), creating the images (docker), or provisioning (ansible), why not minimise the required configuration and dependencies?</p>
<p><a href="/Posts/VMM">Earlier in this blog I introduced Albatross</a>, which serves in an enhanced version as our deployment platform on a physical machine (running 15 unikernels at the moment), I won't discuss more detail thereof in this article.</p>
<h2 id="caldav">CalDAV</h2>

4
Posts/Syslog
View file

@ -1,5 +1,5 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Exfiltrating log data using syslog</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Exfiltrating log data using syslog" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Exfiltrating log data using syslog</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a><a href="/tags/logging" class="tag">logging</a></div><span class="date">Published: 2016-11-05 (last updated: 2021-11-19)</span><article><p>It has been a while since my last entry... I've been busy working on too many
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Exfiltrating log data using syslog</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Exfiltrating log data using syslog" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Exfiltrating log data using syslog</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a><a href="/tags/logging" class="tag">logging</a></div><span class="date">Published: 2016-11-05 (last updated: 2024-10-11)</span><article><p>It has been a while since my last entry... I've been busy working on too many
projects in parallel, and was also travelling on several continents. I hope to
get back to a biweekly cycle.</p>
<h2 id="what-is-syslog">What is syslog?</h2>
@ -61,7 +61,7 @@ MirageOS, and a more reliable log stream (what about authentication and
encryption?). I'll cover both topics in the rest of this article.</p>
<h3 id="mirageos-integration">MirageOS integration</h3>
<p>Since Mirage3, syslog is integrated (see
<a href="http://docs.mirage.io/mirage/Mirage/index.html#type-syslog_config">documentation</a>).
<a href="http://docs.mirageos.org/mirage/Mirage/index.html#type-syslog_config">documentation</a>).
Some additions to your <code>config.ml</code> are needed, see <a href="https://github.com/hannesm/ns.nqsb.io/blob/master/config.ml">ns
example</a> or
<a href="https://github.com/mirage/marrakech2017/blob/master/config.ml">marrakech

6
Posts/TCP-ns
View file

@ -1,15 +1,15 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Redeveloping TCP from the ground up</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Redeveloping TCP from the ground up" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Redeveloping TCP from the ground up</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a><a href="/tags/tcp" class="tag">tcp</a></div><span class="date">Published: 2023-11-28 (last updated: 2023-11-29)</span><article><p>The <a href="https://en.wikipedia.org/wiki/Transmission_Control_Protocol">Transmission Control Protocol (TCP)</a> is one of the main Internet protocols. Usually spoken on top of the Internet Protocol (legacy version 4 or version 6), it provides a reliable, ordered, and error-checked stream of octets. When an application uses TCP, they get these properties for free (in contrast to UDP).</p>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Redeveloping TCP from the ground up</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Redeveloping TCP from the ground up" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Redeveloping TCP from the ground up</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a><a href="/tags/tcp" class="tag">tcp</a></div><span class="date">Published: 2023-11-28 (last updated: 2024-10-11)</span><article><p>The <a href="https://en.wikipedia.org/wiki/Transmission_Control_Protocol">Transmission Control Protocol (TCP)</a> is one of the main Internet protocols. Usually spoken on top of the Internet Protocol (legacy version 4 or version 6), it provides a reliable, ordered, and error-checked stream of octets. When an application uses TCP, they get these properties for free (in contrast to UDP).</p>
<p>As common for Internet protocols, TCP is specified in a series of so-called requests for comments (RFC). The latest revised version from August 2022 is <a href="https://datatracker.ietf.org/doc/html/rfc9293">RFC 9293</a>; the initial one was <a href="https://datatracker.ietf.org/doc/html/rfc793">RFC 793</a> from September 1981.</p>
<h1 id="my-brief-personal-tcp-story">My brief personal TCP story</h1>
<p>My interest in TCP started back in 2006 when we worked on a <a href="https://github.com/dylan-hackers/network-night-vision">network stack in Dylan</a> (these days abandoned). Ever since, I wanted to understand the implementation tradeoffs in more detail, including attacks and how to prevent a TCP stack from being vulnerable.</p>
<p>In 2012, I attended ICFP in Copenhagen while a PhD student at ITU Copenhagen. There, <a href="https://www.cl.cam.ac.uk/~pes20/">Peter Sewell</a> gave an invited talk &quot;Tales from the jungle&quot; about rigorous methods for real-world infrastructure (C semantics, hardware (concurrency) behaviour of CPUs, TCP/IP, and likely more). Working on formal specifications myself in (<a href="https://en.itu.dk/-/media/EN/Research/PhD-Programme/PhD-defences/2013/130731-Hannes-Mehnert-PhD-dissertation-finalpdf.pdf">my dissertation</a>), and having a strong interest in real systems, I was immediately hooked by his perspective.</p>
<p>To dive a bit more into <a href="https://www.cl.cam.ac.uk/~pes20/Netsem/">network semantics</a>, the work done on TCP by Peter Sewell, et al., is a formal specification (or a model) of TCP/IP and the Unix sockets API developed in HOL4. It is a label transition system with nondeterministic choices, and the model itself is executable. It has been validated with the real world by collecting thousands of traces on Linux, Windows, and FreeBSD, which have been checked by the model for validity. This copes with the different implementations of the English prose of the RFCs. The network semantics research found several issues in existing TCP stacks and reported them upstream to have them fixed (though, there still is some special treatment, e.g., for the &quot;BSD listen bug&quot;).</p>
<p>In 2014, I joined Peter's research group in Cambridge to continue their work on the model: updating to more recent versions of HOL4 and PolyML, revising the test system to use DTrace, updating to a more recent FreeBSD network stack (from FreeBSD 4.6 to FreeBSD 10), and finally getting the <a href="https://dl.acm.org/doi/10.1145/3243650">journal paper</a> (<a href="http://www.cl.cam.ac.uk/~pes20/Netsem/paper3.pdf">author's copy</a>) published. At the same time, the <a href="https://mirage.io">MirageOS</a> melting pot was happening at University of Cambridge, where I contributed with David OCaml-TLS and other things.</p>
<p>In 2014, I joined Peter's research group in Cambridge to continue their work on the model: updating to more recent versions of HOL4 and PolyML, revising the test system to use DTrace, updating to a more recent FreeBSD network stack (from FreeBSD 4.6 to FreeBSD 10), and finally getting the <a href="https://dl.acm.org/doi/10.1145/3243650">journal paper</a> (<a href="http://www.cl.cam.ac.uk/~pes20/Netsem/paper3.pdf">author's copy</a>) published. At the same time, the <a href="https://mirageos.org">MirageOS</a> melting pot was happening at University of Cambridge, where I contributed with David OCaml-TLS and other things.</p>
<p>My intention was to understand TCP better and use the specification as a basis for a TCP stack for MirageOS. The <a href="https://github.com/mirage/mirage-tcpip">existing one</a> (which is still used) has technical debt: a high issue to number of lines ratio. The Lwt monad is ubiquitous, which makes testing and debugging pretty hard, and also utilising multiple cores with OCaml Multicore won't be easy. Plus it has various resource leaks, and there is no active maintainer. But honestly, it works fine on a local network, and with well-behaved traffic. It doesn't work that well on the wild Internet with a variety of broken implementations. Apart from resource leakage, which made me implement things such as restart-on-failure in <a href="https://github.com/robur-coop/albatross">Albatross</a>, there are certain connection states which will never be exited.</p>
<h1 id="the-rise-of-µtcp">The rise of <a href="https://github.com/robur-coop/utcp">µTCP</a></h1>
<p>Back in Cambridge, I didn't manage to write a TCP stack based on the model, but in 2019, I restarted that work and got µTCP (the formal model manually translated to OCaml) to compile and do TCP session setup and teardown. Since it was a model that uses nondeterminism, this couldn't be translated one-to-one into an executable program, but there are places where decisions have to be made. Due to other projects, I worked only briefly in 2021 and 2022 on µTCP, but finally in the Summer of 2023, I motivated myself to push µTCP into a usable state. So far I've spend 25 days in 2023 on µTCP. Thanks to <a href="https://tarides.com">Tarides</a> for supporting my work.</p>
<p>Since late August, we have been running some unikernels using µTCP, e.g., the <a href="https://retreat.mirage.io">retreat</a> website. This allows us to observe µTCP and find and solve issues that occur in the real world. It turned out that the model is not always correct (i.e., there is no retransmit timer in the close wait state, which avoids proper session teardowns). We report statistics about how many TCP connections are in which state to an Influx time series database and view graphs rendered by Grafana. If there are connections that are stuck for multiple hours, this indicates a resource leak that should be addressed. Grafana was tremendously helpful to find out where to look for resource leaks. Still, there's work to understand the behaviour, look at what the model does, what µTCP does, what the RFC says, and eventually what existing deployed TCP stacks do.</p>
<p>Since late August, we have been running some unikernels using µTCP, e.g., the <a href="https://retreat.mirageos.org">retreat</a> website. This allows us to observe µTCP and find and solve issues that occur in the real world. It turned out that the model is not always correct (i.e., there is no retransmit timer in the close wait state, which avoids proper session teardowns). We report statistics about how many TCP connections are in which state to an Influx time series database and view graphs rendered by Grafana. If there are connections that are stuck for multiple hours, this indicates a resource leak that should be addressed. Grafana was tremendously helpful to find out where to look for resource leaks. Still, there's work to understand the behaviour, look at what the model does, what µTCP does, what the RFC says, and eventually what existing deployed TCP stacks do.</p>
<h1 id="the-secondary-nameserver-issue">The secondary nameserver issue</h1>
<p>One of our secondary nameservers attempts to receive zones (via AXFR using TCP) from another nameserver that is currently not running. Thus it replies to each SYN packet a corresponding RST. Below I graphed the network utilisation (send data/packets is positive y-axis, receive part on the negative) over time (on the x-axis) on the left and memory usage (bytes on y-axis) over time (x-axis) on the right of our nameserver. You can observe that both increases over time, and roughly every 3 hours, the unikernel hits its configured memory limit (64 MB), crashes with <em>out of memory</em>, and is restarted. The graph below is using the <code>mirage-tcpip</code> stack.</p>
<p><a href="/static/img/a.ns.mtcp.png"><img src="/static/img/a.ns.mtcp.png" width="750" /></a></p>

4
Posts/Traceroute
View file

@ -1,5 +1,5 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Traceroute</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Traceroute" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Traceroute</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a></div><span class="date">Published: 2020-06-24 (last updated: 2021-11-19)</span><article><h2 id="traceroute">Traceroute</h2>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Traceroute</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Traceroute" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Traceroute</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a></div><span class="date">Published: 2020-06-24 (last updated: 2024-10-11)</span><article><h2 id="traceroute">Traceroute</h2>
<p>Is a diagnostic utility which displays the route and measures transit delays of
packets across an Internet protocol (IP) network.</p>
<pre><code class="language-bash">$ doas solo5-hvt --net:service=tap0 -- traceroute.hvt --ipv4=10.0.42.2/24 --ipv4-gateway=10.0.42.1 --host=198.167.222.207
@ -331,7 +331,7 @@ let () =
</code></pre>
<p>And voila, that's all the code. If you copy it together (or download the two
files from <a href="https://github.com/roburio/traceroute">the GitHub repository</a>),
and have OCaml, opam, and <a href="https://mirage.io/wiki/install">mirage (&gt;= 3.8.0)</a> installed,
and have OCaml, opam, and <a href="https://mirageos.org/wiki/install">mirage (&gt;= 3.8.0)</a> installed,
you should be able to:</p>
<pre><code class="language-bash">$ mirage configure -t hvt
$ make depend

4
Posts/X50907
View file

@ -1,8 +1,8 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>X509 0.7</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="X509 0.7" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>X509 0.7</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/security" class="tag">security</a><a href="/tags/tls" class="tag">tls</a></div><span class="date">Published: 2019-08-15 (last updated: 2021-11-19)</span><article><h2 id="cryptographic-material">Cryptographic material</h2>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>X509 0.7</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="X509 0.7" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>X509 0.7</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/security" class="tag">security</a><a href="/tags/tls" class="tag">tls</a></div><span class="date">Published: 2019-08-15 (last updated: 2024-10-11)</span><article><h2 id="cryptographic-material">Cryptographic material</h2>
<p>Once a private and public key pair is generated (doesn't matter whether it is plain RSA, DSA, ECC on any curve), this is fine from a scientific point of view, and can already be used for authenticating and encrypting. From a practical point of view, the public parts need to be exchanged and verified (usually a fingerprint or hash thereof). This leads to the struggle how to encode this cryptographic material, and how to embed an identity (or multiple), capabilities, and other information into it. <a href="https://en.wikipedia.org/wiki/X.509">X.509</a> is a standard to solve this encoding and embedding, and provides more functionality, such as establishing chains of trust and revocation of invalidated or compromised material. X.509 uses certificates, which contain the public key, and additional information (in a extensible key-value store), and are signed by an issuer, either the private key corresponding to the public key - a so-called self-signed certificate - or by a different private key, an authority one step up the chain. A rather long, but very good introduction to certificates by Mike Malone is <a href="https://smallstep.com/blog/everything-pki.html">available here</a>.</p>
<h2 id="ocaml-ecosystem-evolving">OCaml ecosystem evolving</h2>
<p>More than 5 years ago David Kaloper and I <a href="https://mirage.io/blog/introducing-x509">released the initial ocaml-x509</a> package as part of our <a href="https://nqsb.io">TLS stack</a>, which contained code for decoding and encoding certificates, and path validation of a certificate chain (as described in <a href="https://tools.ietf.org/html/rfc6125">RFC 5280</a>). The validation logic and the decoder/encoder, based on the ASN.1 grammar specified in the RFC, implemented using David's <a href="https://github.com/mirleft/ocaml-asn1-combinators">asn1-combinators</a> library changed much over time.</p>
<p>More than 5 years ago David Kaloper and I <a href="https://mirageos.org/blog/introducing-x509">released the initial ocaml-x509</a> package as part of our <a href="https://github.com/mirleft/ocaml-tls">TLS stack</a>, which contained code for decoding and encoding certificates, and path validation of a certificate chain (as described in <a href="https://tools.ietf.org/html/rfc6125">RFC 5280</a>). The validation logic and the decoder/encoder, based on the ASN.1 grammar specified in the RFC, implemented using David's <a href="https://github.com/mirleft/ocaml-asn1-combinators">asn1-combinators</a> library changed much over time.</p>
<p>The OCaml ecosystem evolved over the years, which lead to some changes:</p>
<ul>
<li>Camlp4 deprecation - we used camlp4 for stream parsers of PEM-encoded certificates, and sexplib.syntax to derive s-expression decoders and encoders;

19
Posts/nqsbWebsite
View file

@ -1,7 +1,7 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Fitting the things together</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Fitting the things together" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Fitting the things together</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/http" class="tag">http</a><a href="/tags/tls" class="tag">tls</a><a href="/tags/protocol" class="tag">protocol</a></div><span class="date">Published: 2016-04-24 (last updated: 2021-11-19)</span><article><h2 id="task">Task</h2>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Fitting the things together</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Fitting the things together" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Fitting the things together</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/http" class="tag">http</a><a href="/tags/tls" class="tag">tls</a><a href="/tags/protocol" class="tag">protocol</a></div><span class="date">Published: 2016-04-24 (last updated: 2024-10-11)</span><article><h2 id="task">Task</h2>
<p>Our task is to build a small unikernel which provides a project website. On our way we will wade through various layers using code examples. The website itself contains a few paragraphs of text, some link lists, and our published papers in pdf form.</p>
<p><em>Spoiler alert</em> final result can be seen <a href="https://nqsb.io">here</a>, the full code <a href="https://github.com/mirleft/nqsb.io">here</a>.</p>
<p><em>Spoiler alert</em> final result can be seen (no longer available), the full code <a href="https://github.com/mirleft/nqsb.io">here</a>.</p>
<h2 id="a-first-idea">A first idea</h2>
<p>We could go all the way to use <a href="https://github.com/mirage/ocaml-conduit">conduit</a> for wrapping connections, and <a href="https://github.com/mirage/mirage-http">mirage-http</a> (using <a href="https://github.com/mirage/ocaml-cohttp">cohttp</a>, a very lightweight HTTP server). We'd just need to write routing code which in the end reads from a virtual file system, and some HTML and CSS for the actual site.</p>
<p>Turns out, the conduit library is already 1.7 MB in size and depends on 34 libraries, cohttp is another 3.7 MB and 40 dependent libraries.
@ -170,25 +170,14 @@ the <a href="http://ownme.ipredator.se">BTC Piñata</a>: just send the resource
S.listen_tcpv4 stack ~port:443 (tls_accept f config) ;
S.listen stack
</code></pre>
<p>That's it, the <a href="https://nqsb.io">nqsb.io</a> contains slightly more code to log onto a console, and to redirect requests on port 80 (HTTP) to port 443 (by signaling a <code>301 Moved permanently</code> HTTP status code).</p>
<p>That's it, it contains slightly more code to log onto a console, and to redirect requests on port 80 (HTTP) to port 443 (by signaling a <code>301 Moved permanently</code> HTTP status code).</p>
<h2 id="conclusion">Conclusion</h2>
<p>A comparison using Firefox builtin network diagnostics shows that the waiting before receiving data is minimal (3ms, even spotted 0ms).</p>
<p><a href="/static/img/performance-nqsbio.png"><img src="/static/img/performance-nqsbio.png" title="latency of our website" width="50%" /></a></p>
<p>We do not render HTML for each request, we do not splice data together, we <em>don't even read the client request</em>. And I'm sure we can improve the performance even more by profiling.</p>
<p>We saw a journey from typed XML over key value stores, HTTP, TLS, and HTTPS. The actual application code of our unikernel serving <a href="https://nqsb.io">nqsb,io</a> is less than 100 lines of OCaml. We used MirageOS for our minimal HTTPS website, serving a <em>single resource per hostname</em>. We depend (directly) on the tyxml library, the mirage tool and network stack, and the tls library. That's it.</p>
<p>We saw a journey from typed XML over key value stores, HTTP, TLS, and HTTPS. The actual application code of our unikernel serving nqsb.io is less than 100 lines of OCaml. We used MirageOS for our minimal HTTPS website, serving a <em>single resource per hostname</em>. We depend (directly) on the tyxml library, the mirage tool and network stack, and the tls library. That's it.</p>
<p>There is a long list of potential features, such as full HTTP protocol compliance (caching, favicon, ...), logging, natively getting let's encrypt certificates -- but in the web out there it is sufficient to get picked up by search engines, and the maintenance is marginal.</p>
<p>For a start in MirageOS unikernels, look into our <a href="https://github.com/mirage/mirage-skeleton">mirage-skeleton</a> project, and into the <a href="https://github.com/mattgray/devwinter2016/">/dev/winter</a> presentation by Matt Gray.</p>
<p>I'm interested in feedback, either via
<a href="https://twitter.com/h4nnes">twitter</a> or via eMail.</p>
<h2 id="other-updates-in-the-mirageos-ecosystem">Other updates in the MirageOS ecosystem</h2>
<ul>
<li><a href="https://github.com/Engil/Canopy">Canopy</a> improvements: <a href="https://github.com/Engil/Canopy/pull/26">no bower anymore</a>, <a href="https://github.com/Engil/Canopy/pull/27">HTTP caching support (via etags)</a>, <a href="https://github.com/Engil/Canopy/pull/31">listings now include dates</a>, <a href="https://github.com/Engil/Canopy/pull/32">dates are now in big-endian (y-m-d)</a>
</li>
<li><a href="http://canopy.mirage.io/irclogs/20-04-2016">MirageOS call irclog from 2014-04-20</a>
</li>
<li>blog article about <a href="https://abailly.github.io/posts/mirage-os-newbie.html">naive authentication service using MirageOS</a>
</li>
<li><a href="https://lwn.net/SubscriberLink/684128/1436601f401c1f09/">OCaml 4.03 post</a>
</li>
</ul>
</article></div></div></main></body></html>

40
atom
View file

@ -1,7 +1,7 @@
<feed xmlns="http://www.w3.org/2005/Atom"><link href="https://hannes.robur.coop/atom" rel="self"/><id>urn:uuid:981361ca-e71d-4997-a52c-baeee78e4156</id><title type="text">full stack engineer</title><updated>2024-05-16T14:19:12-00:00</updated><entry><summary type="html">&lt;p&gt;My involvement and experience of the MirageOS retreat 2024&lt;/p&gt;
</summary><published>2024-05-15T14:38:51-00:00</published><link href="/Posts/Retreat2024" rel="alternate"/><content type="html">&lt;p&gt;End of April I spent a week in Marrakech to work on MirageOS at our annual &lt;a href=&quot;https://retreat.mirage.io&quot;&gt;retreat&lt;/a&gt;. This was the 12th time when it actually took place, and it was amazing. We were a total of 17 people attending, of which 5 people used ground transportation. I have respect for them, to take the 3 or more day trip from Berlin or Glasgow or Paris or whereever to Marrakech - with a ferry in between (which availability depends on the wind etc.). This time, I didn't take ground transportation since I had appointments just before and just after the retreat nearby Berlin where I had to be in person. The food an weather was really nice and motivating.&lt;/p&gt;
<feed xmlns="http://www.w3.org/2005/Atom"><link href="https://hannes.robur.coop/atom" rel="self"/><id>urn:uuid:981361ca-e71d-4997-a52c-baeee78e4156</id><title type="text">full stack engineer</title><updated>2024-10-11T09:43:26-00:00</updated><entry><summary type="html">&lt;p&gt;My involvement and experience of the MirageOS retreat 2024&lt;/p&gt;
</summary><published>2024-05-15T14:38:51-00:00</published><link href="/Posts/Retreat2024" rel="alternate"/><content type="html">&lt;p&gt;End of April I spent a week in Marrakech to work on MirageOS at our annual &lt;a href=&quot;https://retreat.mirageos.org&quot;&gt;retreat&lt;/a&gt;. This was the 12th time when it actually took place, and it was amazing. We were a total of 17 people attending, of which 5 people used ground transportation. I have respect for them, to take the 3 or more day trip from Berlin or Glasgow or Paris or whereever to Marrakech - with a ferry in between (which availability depends on the wind etc.). This time, I didn't take ground transportation since I had appointments just before and just after the retreat nearby Berlin where I had to be in person. The food an weather was really nice and motivating.&lt;/p&gt;
&lt;p&gt;Of the 17 people there were lots of new faces. It is really nice to see an active community which is welcoming to new people, and also that there's change: a lot of fresh ideas are brought by these people and we rethink what we are doing and why.&lt;/p&gt;
&lt;p&gt;Our daily routine included breakfast, circle, lunch, dinner, (sometimes) presentations. There wasn't more formal structure, and there was no need for it since people kept themselves busy working on various projects. We had a bigger discussion about &amp;quot;deploying MirageOS&amp;quot;, which is a reoccurring theme. Even running a MirageOS unikernel on a laptop is for people who are not very familiar with networking pretty hard. In addition, our &lt;a href=&quot;https://mirage.io&quot;&gt;website&lt;/a&gt; only briefly described network setup, and included DHCP very early (which is overwhelming to setup and use on your laptop).&lt;/p&gt;
&lt;p&gt;Our daily routine included breakfast, circle, lunch, dinner, (sometimes) presentations. There wasn't more formal structure, and there was no need for it since people kept themselves busy working on various projects. We had a bigger discussion about &amp;quot;deploying MirageOS&amp;quot;, which is a reoccurring theme. Even running a MirageOS unikernel on a laptop is for people who are not very familiar with networking pretty hard. In addition, our &lt;a href=&quot;https://mirageos.org&quot;&gt;website&lt;/a&gt; only briefly described network setup, and included DHCP very early (which is overwhelming to setup and use on your laptop).&lt;/p&gt;
&lt;p&gt;I went to the retreat with some ideas in my head what I want to achieve (for example, replace all the bigarray allocations by bytes &amp;amp; string), but the discussion about deployment made me prioritise this more.&lt;/p&gt;
&lt;h2 id=&quot;operator-handbook&quot;&gt;Operator handbook&lt;/h2&gt;
&lt;p&gt;I sat down with some people to start off a &lt;a href=&quot;https://mirage.github.io/operator-handbook/index.html&quot;&gt;MirageOS operator handbook&lt;/a&gt; - we talked about the scope and the early chapters (and what is not in scope). This is still very much work in progress, &lt;a href=&quot;https://github.com/mirage/operator-handbook&quot;&gt;issues&lt;/a&gt; and pull requests are welcome. The target audience is someone who wants to execute a MirageOS unikernel with hvt on a laptop or server. It will not be necessary to install OCaml, but the focus is really on the operational aspects with binaries that we provide from our &lt;a href=&quot;https://builds.robur.coop&quot;&gt;reproducible build infrastructure&lt;/a&gt;.&lt;/p&gt;
@ -21,18 +21,18 @@
&lt;p&gt;I'm looking forward to other writeups and future retreats, or other events where I can say hi to attendees.&lt;/p&gt;
&lt;p&gt;We at &lt;a href=&quot;https://robur.coop&quot;&gt;Robur&lt;/a&gt; are working as a collective since 2018 on public funding, commercial contracts, and donations. Our mission is to get sustainable, robust, and secure MirageOS unikernels developed and deployed. Running your own digital communication infrastructure should be easy, including trustworthy binaries and smooth upgrades. You can help us continue our work by &lt;a href=&quot;https://aenderwerk.de/donate/&quot;&gt;donating&lt;/a&gt; (select Robur from the drop-down or put &amp;quot;donation Robur&amp;quot; in the purpose of the bank transfer).&lt;/p&gt;
&lt;p&gt;If you have any questions, reach us best via eMail to team AT robur DOT coop.&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/community" term="community"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:2c9e2634-78c5-5957-b6f2-d0706f80d7eb</id><title type="text">The MirageOS retreat 2024</title><updated>2024-05-16T14:19:12-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;Core Internet protocols require operational experiments, even if formally specified&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/community" term="community"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:2c9e2634-78c5-5957-b6f2-d0706f80d7eb</id><title type="text">The MirageOS retreat 2024</title><updated>2024-10-11T09:43:26-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;Core Internet protocols require operational experiments, even if formally specified&lt;/p&gt;
</summary><published>2023-11-28T21:17:01-00:00</published><link href="/Posts/TCP-ns" rel="alternate"/><content type="html">&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Transmission_Control_Protocol&quot;&gt;Transmission Control Protocol (TCP)&lt;/a&gt; is one of the main Internet protocols. Usually spoken on top of the Internet Protocol (legacy version 4 or version 6), it provides a reliable, ordered, and error-checked stream of octets. When an application uses TCP, they get these properties for free (in contrast to UDP).&lt;/p&gt;
&lt;p&gt;As common for Internet protocols, TCP is specified in a series of so-called requests for comments (RFC). The latest revised version from August 2022 is &lt;a href=&quot;https://datatracker.ietf.org/doc/html/rfc9293&quot;&gt;RFC 9293&lt;/a&gt;; the initial one was &lt;a href=&quot;https://datatracker.ietf.org/doc/html/rfc793&quot;&gt;RFC 793&lt;/a&gt; from September 1981.&lt;/p&gt;
&lt;h1 id=&quot;my-brief-personal-tcp-story&quot;&gt;My brief personal TCP story&lt;/h1&gt;
&lt;p&gt;My interest in TCP started back in 2006 when we worked on a &lt;a href=&quot;https://github.com/dylan-hackers/network-night-vision&quot;&gt;network stack in Dylan&lt;/a&gt; (these days abandoned). Ever since, I wanted to understand the implementation tradeoffs in more detail, including attacks and how to prevent a TCP stack from being vulnerable.&lt;/p&gt;
&lt;p&gt;In 2012, I attended ICFP in Copenhagen while a PhD student at ITU Copenhagen. There, &lt;a href=&quot;https://www.cl.cam.ac.uk/~pes20/&quot;&gt;Peter Sewell&lt;/a&gt; gave an invited talk &amp;quot;Tales from the jungle&amp;quot; about rigorous methods for real-world infrastructure (C semantics, hardware (concurrency) behaviour of CPUs, TCP/IP, and likely more). Working on formal specifications myself in (&lt;a href=&quot;https://en.itu.dk/-/media/EN/Research/PhD-Programme/PhD-defences/2013/130731-Hannes-Mehnert-PhD-dissertation-finalpdf.pdf&quot;&gt;my dissertation&lt;/a&gt;), and having a strong interest in real systems, I was immediately hooked by his perspective.&lt;/p&gt;
&lt;p&gt;To dive a bit more into &lt;a href=&quot;https://www.cl.cam.ac.uk/~pes20/Netsem/&quot;&gt;network semantics&lt;/a&gt;, the work done on TCP by Peter Sewell, et al., is a formal specification (or a model) of TCP/IP and the Unix sockets API developed in HOL4. It is a label transition system with nondeterministic choices, and the model itself is executable. It has been validated with the real world by collecting thousands of traces on Linux, Windows, and FreeBSD, which have been checked by the model for validity. This copes with the different implementations of the English prose of the RFCs. The network semantics research found several issues in existing TCP stacks and reported them upstream to have them fixed (though, there still is some special treatment, e.g., for the &amp;quot;BSD listen bug&amp;quot;).&lt;/p&gt;
&lt;p&gt;In 2014, I joined Peter's research group in Cambridge to continue their work on the model: updating to more recent versions of HOL4 and PolyML, revising the test system to use DTrace, updating to a more recent FreeBSD network stack (from FreeBSD 4.6 to FreeBSD 10), and finally getting the &lt;a href=&quot;https://dl.acm.org/doi/10.1145/3243650&quot;&gt;journal paper&lt;/a&gt; (&lt;a href=&quot;http://www.cl.cam.ac.uk/~pes20/Netsem/paper3.pdf&quot;&gt;author's copy&lt;/a&gt;) published. At the same time, the &lt;a href=&quot;https://mirage.io&quot;&gt;MirageOS&lt;/a&gt; melting pot was happening at University of Cambridge, where I contributed with David OCaml-TLS and other things.&lt;/p&gt;
&lt;p&gt;In 2014, I joined Peter's research group in Cambridge to continue their work on the model: updating to more recent versions of HOL4 and PolyML, revising the test system to use DTrace, updating to a more recent FreeBSD network stack (from FreeBSD 4.6 to FreeBSD 10), and finally getting the &lt;a href=&quot;https://dl.acm.org/doi/10.1145/3243650&quot;&gt;journal paper&lt;/a&gt; (&lt;a href=&quot;http://www.cl.cam.ac.uk/~pes20/Netsem/paper3.pdf&quot;&gt;author's copy&lt;/a&gt;) published. At the same time, the &lt;a href=&quot;https://mirageos.org&quot;&gt;MirageOS&lt;/a&gt; melting pot was happening at University of Cambridge, where I contributed with David OCaml-TLS and other things.&lt;/p&gt;
&lt;p&gt;My intention was to understand TCP better and use the specification as a basis for a TCP stack for MirageOS. The &lt;a href=&quot;https://github.com/mirage/mirage-tcpip&quot;&gt;existing one&lt;/a&gt; (which is still used) has technical debt: a high issue to number of lines ratio. The Lwt monad is ubiquitous, which makes testing and debugging pretty hard, and also utilising multiple cores with OCaml Multicore won't be easy. Plus it has various resource leaks, and there is no active maintainer. But honestly, it works fine on a local network, and with well-behaved traffic. It doesn't work that well on the wild Internet with a variety of broken implementations. Apart from resource leakage, which made me implement things such as restart-on-failure in &lt;a href=&quot;https://github.com/robur-coop/albatross&quot;&gt;Albatross&lt;/a&gt;, there are certain connection states which will never be exited.&lt;/p&gt;
&lt;h1 id=&quot;the-rise-of-µtcp&quot;&gt;The rise of &lt;a href=&quot;https://github.com/robur-coop/utcp&quot;&gt;µTCP&lt;/a&gt;&lt;/h1&gt;
&lt;p&gt;Back in Cambridge, I didn't manage to write a TCP stack based on the model, but in 2019, I restarted that work and got µTCP (the formal model manually translated to OCaml) to compile and do TCP session setup and teardown. Since it was a model that uses nondeterminism, this couldn't be translated one-to-one into an executable program, but there are places where decisions have to be made. Due to other projects, I worked only briefly in 2021 and 2022 on µTCP, but finally in the Summer of 2023, I motivated myself to push µTCP into a usable state. So far I've spend 25 days in 2023 on µTCP. Thanks to &lt;a href=&quot;https://tarides.com&quot;&gt;Tarides&lt;/a&gt; for supporting my work.&lt;/p&gt;
&lt;p&gt;Since late August, we have been running some unikernels using µTCP, e.g., the &lt;a href=&quot;https://retreat.mirage.io&quot;&gt;retreat&lt;/a&gt; website. This allows us to observe µTCP and find and solve issues that occur in the real world. It turned out that the model is not always correct (i.e., there is no retransmit timer in the close wait state, which avoids proper session teardowns). We report statistics about how many TCP connections are in which state to an Influx time series database and view graphs rendered by Grafana. If there are connections that are stuck for multiple hours, this indicates a resource leak that should be addressed. Grafana was tremendously helpful to find out where to look for resource leaks. Still, there's work to understand the behaviour, look at what the model does, what µTCP does, what the RFC says, and eventually what existing deployed TCP stacks do.&lt;/p&gt;
&lt;p&gt;Since late August, we have been running some unikernels using µTCP, e.g., the &lt;a href=&quot;https://retreat.mirageos.org&quot;&gt;retreat&lt;/a&gt; website. This allows us to observe µTCP and find and solve issues that occur in the real world. It turned out that the model is not always correct (i.e., there is no retransmit timer in the close wait state, which avoids proper session teardowns). We report statistics about how many TCP connections are in which state to an Influx time series database and view graphs rendered by Grafana. If there are connections that are stuck for multiple hours, this indicates a resource leak that should be addressed. Grafana was tremendously helpful to find out where to look for resource leaks. Still, there's work to understand the behaviour, look at what the model does, what µTCP does, what the RFC says, and eventually what existing deployed TCP stacks do.&lt;/p&gt;
&lt;h1 id=&quot;the-secondary-nameserver-issue&quot;&gt;The secondary nameserver issue&lt;/h1&gt;
&lt;p&gt;One of our secondary nameservers attempts to receive zones (via AXFR using TCP) from another nameserver that is currently not running. Thus it replies to each SYN packet a corresponding RST. Below I graphed the network utilisation (send data/packets is positive y-axis, receive part on the negative) over time (on the x-axis) on the left and memory usage (bytes on y-axis) over time (x-axis) on the right of our nameserver. You can observe that both increases over time, and roughly every 3 hours, the unikernel hits its configured memory limit (64 MB), crashes with &lt;em&gt;out of memory&lt;/em&gt;, and is restarted. The graph below is using the &lt;code&gt;mirage-tcpip&lt;/code&gt; stack.&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;/static/img/a.ns.mtcp.png&quot;&gt;&lt;img src=&quot;/static/img/a.ns.mtcp.png&quot; width=&quot;750&quot; /&gt;&lt;/a&gt;&lt;/p&gt;
@ -58,7 +58,7 @@
&lt;p&gt;We'll take more time to investigate issues of µTCP in production, plan to write further documentation and blog posts, and hopefully soon will be ready for an initial public release. In the meantime, you can follow our development repository.&lt;/p&gt;
&lt;p&gt;We at &lt;a href=&quot;https://robur.coop&quot;&gt;Robur&lt;/a&gt; are working as a collective since 2018 on public funding, commercial contracts, and donations. Our mission is to get sustainable, robust, and secure MirageOS unikernels developed and deployed. Running your own digital communication infrastructure should be easy, including trustworthy binaries and smooth upgrades. You can help us continue our work by &lt;a href=&quot;https://aenderwerk.de/donate/&quot;&gt;donating&lt;/a&gt; (select Robur from the drop-down or put &amp;quot;donation Robur&amp;quot; in the purpose of the bank transfer).&lt;/p&gt;
&lt;p&gt;If you have any questions, reach us best via eMail to team AT robur DOT coop.&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/tcp" term="tcp"/><category scheme="https://hannes.robur.coop/tags/protocol" term="protocol"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:96688956-0808-5d44-b795-1d64cbb4f947</id><title type="text">Redeveloping TCP from the ground up</title><updated>2023-11-29T13:31:13-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;fleet management for MirageOS unikernels using a mutually authenticated TLS handshake&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/tcp" term="tcp"/><category scheme="https://hannes.robur.coop/tags/protocol" term="protocol"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:96688956-0808-5d44-b795-1d64cbb4f947</id><title type="text">Redeveloping TCP from the ground up</title><updated>2024-10-11T09:43:26-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;fleet management for MirageOS unikernels using a mutually authenticated TLS handshake&lt;/p&gt;
</summary><published>2022-11-17T12:41:11-00:00</published><link href="/Posts/Albatross" rel="alternate"/><content type="html">&lt;p&gt;EDIT (2023-05-16): Updated with albatross release version 2.0.0.&lt;/p&gt;
&lt;h2 id=&quot;deploying-mirageos-unikernels&quot;&gt;Deploying MirageOS unikernels&lt;/h2&gt;
&lt;p&gt;More than five years ago, I posted &lt;a href=&quot;/Posts/VMM&quot;&gt;how to deploy MirageOS unikernels&lt;/a&gt;. My motivation to work on this topic is that I'm convinced of reduced complexity, improved security, and more sustainable resource footprint of MirageOS unikernels, and want to ease deployment thereof. More than one year ago, I described &lt;a href=&quot;/Posts/Deploy&quot;&gt;how to deploy reproducible unikernels&lt;/a&gt;.&lt;/p&gt;
@ -152,9 +152,9 @@ $ pkg install solo5 albatross
&lt;p&gt;Opam, after initialisation, downloads the &lt;code&gt;index.tar.gz&lt;/code&gt; from &lt;code&gt;https://opam.ocaml.org/index.tar.gz&lt;/code&gt;, and uses this as the local opam universe. An &lt;code&gt;opam install cmdliner&lt;/code&gt; will resolve the dependencies, and download all required tarballs. The download is first tried from the cache, and if that failed, the URL in the package file is used. The download from the cache uses the base url, appends the archive-mirror, followed by the hash algorithm, the first two characters of the has of the tarball, and the hex encoded hash of the archive, i.e. for cmdliner 1.1.1 which specifies its sha512: &lt;code&gt;https://opam.ocaml.org/cache/sha512/54/5478ad833da254b5587b3746e3a8493e66e867a081ac0f653a901cc8a7d944f66e4387592215ce25d939be76f281c4785702f54d4a74b1700bc8838a62255c9e&lt;/code&gt;.&lt;/p&gt;
&lt;h1 id=&quot;how-does-the-opam-repository-work&quot;&gt;How does the opam repository work?&lt;/h1&gt;
&lt;p&gt;According to DNS, opam.ocaml.org is a machine at amazon. It likely, apart from the website, uses &lt;code&gt;opam admin index&lt;/code&gt; periodically to create the index tarball and the cache. There's an observable delay between a package merge in the opam-repository and when it shows up at opam.ocaml.org. Recently, there was &lt;a href=&quot;https://discuss.ocaml.org/t/opam-ocaml-org-is-currently-down-is-that-where-indices-are-kept-still/&quot;&gt;a reported downtime&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Apart from being a single point of failure, if you're compiling a lot of opam projects (e.g. a continuous integration / continuous build system), it makes sense from a network usage (and thus sustainability perspective) to move the cache closer to where you need the source archives. We're also organising the MirageOS &lt;a href=&quot;http://retreat.mirage.io&quot;&gt;hack retreats&lt;/a&gt; in a northern African country with poor connectivity - so if you gather two dozen camels you better bring your opam repository cache with you to reduce the bandwidth usage (NB: this requires at the moment cooperation of all participants to configure their default opam repository accordingly).&lt;/p&gt;
&lt;p&gt;Apart from being a single point of failure, if you're compiling a lot of opam projects (e.g. a continuous integration / continuous build system), it makes sense from a network usage (and thus sustainability perspective) to move the cache closer to where you need the source archives. We're also organising the MirageOS &lt;a href=&quot;http://retreat.mirageos.org&quot;&gt;hack retreats&lt;/a&gt; in a northern African country with poor connectivity - so if you gather two dozen camels you better bring your opam repository cache with you to reduce the bandwidth usage (NB: this requires at the moment cooperation of all participants to configure their default opam repository accordingly).&lt;/p&gt;
&lt;h1 id=&quot;re-developing-opam-admin-create-as-mirageos-unikernel&quot;&gt;Re-developing &amp;quot;opam admin create&amp;quot; as MirageOS unikernel&lt;/h1&gt;
&lt;p&gt;The need for a local opam cache at our &lt;a href=&quot;https://builds.robur.coop&quot;&gt;reproducible build infrastructure&lt;/a&gt; and the retreats, we decided to develop &lt;a href=&quot;https://git.robur.coop/robur/opam-mirror&quot;&gt;opam-mirror&lt;/a&gt; as a &lt;a href=&quot;https://mirage.io&quot;&gt;MirageOS unikernel&lt;/a&gt;. Apart from a useful showcase using persistent storage (that won't fit into memory), and having fun while developing it, our aim was to reduce our time spent on system administration (the &lt;code&gt;opam admin index&lt;/code&gt; is only one part of the story, it needs a Unix system and a webserver next to it - plus remote access for doing software updates - which has quite some attack surface.&lt;/p&gt;
&lt;p&gt;The need for a local opam cache at our &lt;a href=&quot;https://builds.robur.coop&quot;&gt;reproducible build infrastructure&lt;/a&gt; and the retreats, we decided to develop &lt;a href=&quot;https://git.robur.coop/robur/opam-mirror&quot;&gt;opam-mirror&lt;/a&gt; as a &lt;a href=&quot;https://mirageos.org&quot;&gt;MirageOS unikernel&lt;/a&gt;. Apart from a useful showcase using persistent storage (that won't fit into memory), and having fun while developing it, our aim was to reduce our time spent on system administration (the &lt;code&gt;opam admin index&lt;/code&gt; is only one part of the story, it needs a Unix system and a webserver next to it - plus remote access for doing software updates - which has quite some attack surface.&lt;/p&gt;
&lt;p&gt;Another reason for re-developing the functionality was that the opam code (what opam admin index actually does) is part of the opam source code, which totals to 50_000 lines of code -- looking up whether one or all checksums are verified before adding the tarball to the cache, was rather tricky.&lt;/p&gt;
&lt;p&gt;In earlier years, we avoided persistent storage and block devices in MirageOS (by embedding it into the source code with &lt;a href=&quot;https://github.com/mirage/ocaml-crunch&quot;&gt;crunch&lt;/a&gt;, or using a remote git repository), but recent development, e.g. of &lt;a href=&quot;https://somerandomidiot.com/blog/2022/03/04/chamelon/&quot;&gt;chamelon&lt;/a&gt; sparked some interest in actually using file systems and figuring out whether MirageOS is ready in that area. A month ago we started the opam-mirror project.&lt;/p&gt;
&lt;p&gt;Opam-mirror takes a remote repository URL, and downloads all referenced archives. It serves as a cache and opam-repository - and does periodic updates from the remote repository. The idea is to validate all available checksums and store the tarballs only once, and store overlays (as maps) from the other hash algorithms.&lt;/p&gt;
@ -171,7 +171,7 @@ $ pkg install solo5 albatross
&lt;h1 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h1&gt;
&lt;p&gt;To conclude, we managed within a month to develop this opam-mirror cache from scratch. It has a reasonable footprint (CPU and memory-wise), is easy to maintain and easy to update - if you want to use it, we also provide &lt;a href=&quot;https://builds.robur.coop/job/opam-mirror&quot;&gt;reproducible binaries&lt;/a&gt; for solo5-hvt. You can use our opam mirror with &lt;code&gt;opam repository set-url default https://opam.robur.coop&lt;/code&gt; (revert to the other with &lt;code&gt;opam repository set-url default https://opam.ocaml.org&lt;/code&gt;) or use it as a backup with &lt;code&gt;opam repository add robur --rank 2 https://opam.robur.coop&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Please reach out to us (at team AT robur DOT coop) if you have feedback and suggestions. We are a non-profit company, and rely on &lt;a href=&quot;https://robur.coop/Donate&quot;&gt;donations&lt;/a&gt; for doing our work - everyone can contribute.&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/opam" term="opam"/><category scheme="https://hannes.robur.coop/tags/deployment" term="deployment"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:0dbd251f-32c7-57bd-8e8f-7392c0833a09</id><title type="text">Mirroring the opam repository and all tarballs</title><updated>2023-11-20T16:58:35-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;How to monitor your MirageOS unikernel with albatross and monitoring-experiments&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/opam" term="opam"/><category scheme="https://hannes.robur.coop/tags/deployment" term="deployment"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:0dbd251f-32c7-57bd-8e8f-7392c0833a09</id><title type="text">Mirroring the opam repository and all tarballs</title><updated>2024-10-11T09:43:26-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;How to monitor your MirageOS unikernel with albatross and monitoring-experiments&lt;/p&gt;
</summary><published>2022-03-08T11:26:31-00:00</published><link href="/Posts/Monitoring" rel="alternate"/><content type="html">&lt;h1 id=&quot;introduction-to-monitoring&quot;&gt;Introduction to monitoring&lt;/h1&gt;
&lt;p&gt;At &lt;a href=&quot;https://robur.coop&quot;&gt;robur&lt;/a&gt; we use a range of MirageOS unikernels. Recently, we worked on improving the operations story thereof. One part is shipping binaries using our &lt;a href=&quot;https://builds.robur.coop&quot;&gt;reproducible builds infrastructure&lt;/a&gt;. Another part is, once deployed we want to observe what is going on.&lt;/p&gt;
&lt;p&gt;I first got into touch with monitoring - collecting and graphing metrics - with &lt;a href=&quot;https://oss.oetiker.ch/mrtg/&quot;&gt;MRTG&lt;/a&gt; and &lt;a href=&quot;https://munin-monitoring.org/&quot;&gt;munin&lt;/a&gt; - and the simple network management protocol &lt;a href=&quot;https://en.wikipedia.org/wiki/Simple_Network_Management_Protocol&quot;&gt;SNMP&lt;/a&gt;. From the whole system perspective, I find it crucial that the monitoring part of a system does not add pressure. This favours a push-based design, where reporting is done at the disposition of the system.&lt;/p&gt;
@ -347,7 +347,7 @@ _stack.V4V6) (_ : sig end) (Management : Mirage_stack.V4V6) = struct
</summary><published>2021-04-23T13:33:06-00:00</published><link href="/Posts/EC" rel="alternate"/><content type="html">&lt;h2 id=&quot;introduction&quot;&gt;Introduction&lt;/h2&gt;
&lt;p&gt;Tl;DR: mirage-crypto-ec, with x509 0.12.0, and tls 0.13.0, provide fast and secure elliptic curve support in OCaml and MirageOS - using the verified &lt;a href=&quot;https://github.com/mit-plv/fiat-crypto/&quot;&gt;fiat-crypto&lt;/a&gt; stack (Coq to OCaml to executable which generates C code that is interfaced by OCaml). In x509, a long standing issue (countryName encoding), and archive (PKCS 12) format is now supported, in addition to EC keys. In tls, ECDH key exchanges are supported, and ECDSA and EdDSA certificates.&lt;/p&gt;
&lt;h2 id=&quot;elliptic-curve-cryptography&quot;&gt;Elliptic curve cryptography&lt;/h2&gt;
&lt;p&gt;&lt;a href=&quot;https://mirage.io/blog/tls-1-3-mirageos&quot;&gt;Since May 2020&lt;/a&gt;, our &lt;a href=&quot;https://usenix15.nqsb.io&quot;&gt;OCaml-TLS&lt;/a&gt; stack supports TLS 1.3 (since tls version 0.12.0 on opam).&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://mirageos.org/blog/tls-1-3-mirageos&quot;&gt;Since May 2020&lt;/a&gt;, our &lt;a href=&quot;https://www.usenix.org/conference/usenixsecurity15/technical-sessions/presentation/kaloper-mersinjak&quot;&gt;OCaml-TLS&lt;/a&gt; stack supports TLS 1.3 (since tls version 0.12.0 on opam).&lt;/p&gt;
&lt;p&gt;TLS 1.3 requires elliptic curve cryptography - which was not available in &lt;a href=&quot;https://github.com/mirage/mirage-crypto&quot;&gt;mirage-crypto&lt;/a&gt; (the maintained fork of &lt;a href=&quot;https://github.com/mirleft/ocaml-nocrypto&quot;&gt;nocrypto&lt;/a&gt;).&lt;/p&gt;
&lt;p&gt;There are two major uses of elliptic curves: &lt;a href=&quot;https://en.wikipedia.org/wiki/Elliptic-curve_Diffie%E2%80%93Hellman&quot;&gt;key exchange (ECDH)&lt;/a&gt; for establishing a shared secret over an insecure channel, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm&quot;&gt;digital signature (ECDSA)&lt;/a&gt; for authentication, integrity, and non-repudiation. (Please note that the construction of digital signatures on Edwards curves (Curve25519, Ed448) is called EdDSA instead of ECDSA.)&lt;/p&gt;
&lt;p&gt;Elliptic curve cryptoraphy is &lt;a href=&quot;https://eprint.iacr.org/2020/615&quot;&gt;vulnerable&lt;/a&gt; &lt;a href=&quot;https://raccoon-attack.com/&quot;&gt;to&lt;/a&gt; &lt;a href=&quot;https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-5407&quot;&gt;various&lt;/a&gt; &lt;a href=&quot;https://github.com/mimoo/timing_attack_ecdsa_tls&quot;&gt;timing&lt;/a&gt; &lt;a href=&quot;https://minerva.crocs.fi.muni.cz/&quot;&gt;attacks&lt;/a&gt; - have a read of the &lt;a href=&quot;https://blog.trailofbits.com/2020/06/11/ecdsa-handle-with-care/&quot;&gt;overview article on ECDSA&lt;/a&gt;. When implementing elliptic curve cryptography, it is best to avoid these known attacks. Gladly, there are some projects which address these issues by construction.&lt;/p&gt;
@ -387,7 +387,7 @@ _stack.V4V6) (_ : sig end) (Management : Mirage_stack.V4V6) = struct
&lt;p&gt;Another potential project that is very close now is a certificate authority developed in MirageOS - now that EC keys, PKCS 12, revocation lists, ... are implemented.&lt;/p&gt;
&lt;h2 id=&quot;footer&quot;&gt;Footer&lt;/h2&gt;
&lt;p&gt;If you want to support our work on MirageOS unikernels, please &lt;a href=&quot;https://robur.coop/Donate&quot;&gt;donate to robur&lt;/a&gt;. I'm interested in feedback, either via &lt;a href=&quot;https://twitter.com/h4nnes&quot;&gt;twitter&lt;/a&gt;, &lt;a href=&quot;https://mastodon.social/@hannesm&quot;&gt;hannesm@mastodon.social&lt;/a&gt; or via eMail.&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/tls" term="tls"/><category scheme="https://hannes.robur.coop/tags/security" term="security"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:16427713-5da1-50cd-b17c-ca5b5cca431d</id><title type="text">Cryptography updates in OCaml and MirageOS</title><updated>2021-11-19T18:04:52-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;Home office, MirageOS unikernels, 2020 recap, 2021 tbd&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/tls" term="tls"/><category scheme="https://hannes.robur.coop/tags/security" term="security"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:16427713-5da1-50cd-b17c-ca5b5cca431d</id><title type="text">Cryptography updates in OCaml and MirageOS</title><updated>2024-10-11T09:43:26-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;Home office, MirageOS unikernels, 2020 recap, 2021 tbd&lt;/p&gt;
</summary><published>2021-01-25T12:45:54-00:00</published><link href="/Posts/NGI" rel="alternate"/><content type="html">&lt;h2 id=&quot;introduction&quot;&gt;Introduction&lt;/h2&gt;
&lt;p&gt;2020 was an intense year. I hope you're healthy and keep being healthy. I am privileged (as lots of software engineers and academics are) to be able to work from home during the pandemic. Let's not forget people in less privileged situations, and let’s try to give them as much practical, psychological and financial support as we can these days. And as much joy as possible to everyone around :)&lt;/p&gt;
&lt;p&gt;I cancelled the autumn MirageOS retreat due to the pandemic. Instead I collected donations for our hosts in Marrakech - they were very happy to receive our financial support, since they had a difficult year, since their income is based on tourism. I hope that in autumn 2021 we'll have an on-site retreat again.&lt;/p&gt;
@ -402,9 +402,9 @@ _stack.V4V6) (_ : sig end) (Management : Mirage_stack.V4V6) = struct
&lt;p&gt;&lt;a href=&quot;https://github.com/mirage/mirage-crypto&quot;&gt;Mirage-crypto&lt;/a&gt; provides the underlying cryptographic primitives, initially released in March 2020 as a fork of &lt;a href=&quot;https://github.com/mirleft/ocaml-nocrypto&quot;&gt;nocrypto&lt;/a&gt; -- huge thanks to &lt;a href=&quot;https://github.com/pqwy&quot;&gt;pqwy&lt;/a&gt; for his great work. Mirage-crypto detects &lt;a href=&quot;https://github.com/mirage/mirage-crypto/pull/53&quot;&gt;CPU features at runtime&lt;/a&gt; (thanks to &lt;a href=&quot;https://github.com/Julow&quot;&gt;Julow&lt;/a&gt;) (&lt;a href=&quot;https://github.com/mirage/mirage-crypto/pull/96&quot;&gt;bugfix for bswap&lt;/a&gt;), using constant time modular exponentation (powm_sec) and hardens against Lenstra's CRT attack, supports &lt;a href=&quot;https://github.com/mirage/mirage-crypto/pull/39&quot;&gt;compilation on Windows&lt;/a&gt; (thanks to &lt;a href=&quot;https://github.com/avsm&quot;&gt;avsm&lt;/a&gt;), &lt;a href=&quot;https://github.com/mirage/mirage-crypto/pull/90&quot;&gt;async entropy harvesting&lt;/a&gt; (thanks to &lt;a href=&quot;https://github.com/seliopou&quot;&gt;seliopou&lt;/a&gt;), &lt;a href=&quot;https://github.com/mirage/mirage-crypto/pull/65&quot;&gt;32 bit support&lt;/a&gt;, &lt;a href=&quot;https://github.com/mirage/mirage-crypto/pull/72&quot;&gt;chacha20/poly1305&lt;/a&gt; (thanks to &lt;a href=&quot;https://github.com/abeaumont&quot;&gt;abeaumont&lt;/a&gt;), &lt;a href=&quot;https://github.com/mirage/mirage-crypto/pull/84&quot;&gt;cross-compilation&lt;/a&gt; (thanks to &lt;a href=&quot;https://github.com/EduardoRFS&quot;&gt;EduardoRFS&lt;/a&gt;) and &lt;a href=&quot;https://github.com/mirage/mirage-crypto/pull/78&quot;&gt;various&lt;/a&gt; &lt;a href=&quot;https://github.com/mirage/mirage-crypto/pull/81&quot;&gt;bug&lt;/a&gt; &lt;a href=&quot;https://github.com/mirage/mirage-crypto/pull/83&quot;&gt;fixes&lt;/a&gt;, even &lt;a href=&quot;https://github.com/mirage/mirage-crypto/pull/95&quot;&gt;memory leak&lt;/a&gt; (thanks to &lt;a href=&quot;https://github.com/talex5&quot;&gt;talex5&lt;/a&gt; for reporting several of these issues), and &lt;a href=&quot;https://github.com/mirage/mirage-crypto/pull/99&quot;&gt;RSA&lt;/a&gt; &lt;a href=&quot;https://github.com/mirage/mirage-crypto/pull/100&quot;&gt;interoperability&lt;/a&gt; (thanks to &lt;a href=&quot;https://github.com/psafont&quot;&gt;psafont&lt;/a&gt; for investigation and &lt;a href=&quot;https://github.com/mattjbray&quot;&gt;mattjbray&lt;/a&gt; for reporting). This library feels very mature now - being used by multiple stakeholders, and lots of issues have been fixed in 2020.&lt;/p&gt;
&lt;h3 id=&quot;qubes-firewall&quot;&gt;Qubes Firewall&lt;/h3&gt;
&lt;p&gt;The &lt;a href=&quot;https://github.com/mirage/qubes-mirage-firewall/&quot;&gt;MirageOS based Qubes firewall&lt;/a&gt; is the most widely used MirageOS unikernel. And it got major updates: in May &lt;a href=&quot;https://github.com/linse&quot;&gt;Steffi&lt;/a&gt; &lt;a href=&quot;https://groups.google.com/g/qubes-users/c/Xzplmkjwa5Y&quot;&gt;announced&lt;/a&gt; her and &lt;a href=&quot;https://github.com/yomimono&quot;&gt;Mindy's&lt;/a&gt; work on improving it for Qubes 4.0 - including &lt;a href=&quot;https://www.qubes-os.org/doc/vm-interface/#firewall-rules-in-4x&quot;&gt;dynamic firewall rules via QubesDB&lt;/a&gt;. Thanks to &lt;a href=&quot;https://prototypefund.de/project/portable-firewall-fuer-qubesos/&quot;&gt;prototypefund&lt;/a&gt; for sponsoring.&lt;/p&gt;
&lt;p&gt;In October 2020, we released &lt;a href=&quot;https://mirage.io/blog/announcing-mirage-39-release&quot;&gt;Mirage 3.9&lt;/a&gt; with PVH virtualization mode (thanks to &lt;a href=&quot;https://github.com/mato&quot;&gt;mato&lt;/a&gt;). There's still a &lt;a href=&quot;https://github.com/mirage/qubes-mirage-firewall/issues/120&quot;&gt;memory leak&lt;/a&gt; to be investigated and fixed.&lt;/p&gt;
&lt;p&gt;In October 2020, we released &lt;a href=&quot;https://mirageos.org/blog/announcing-mirage-39-release&quot;&gt;Mirage 3.9&lt;/a&gt; with PVH virtualization mode (thanks to &lt;a href=&quot;https://github.com/mato&quot;&gt;mato&lt;/a&gt;). There's still a &lt;a href=&quot;https://github.com/mirage/qubes-mirage-firewall/issues/120&quot;&gt;memory leak&lt;/a&gt; to be investigated and fixed.&lt;/p&gt;
&lt;h3 id=&quot;ipv6&quot;&gt;IPv6&lt;/h3&gt;
&lt;p&gt;In December, with &lt;a href=&quot;https://mirage.io/blog/announcing-mirage-310-release&quot;&gt;Mirage 3.10&lt;/a&gt; we got the IPv6 code up and running. Now MirageOS unikernels have a dual stack available, besides IPv4-only and IPv6-only network stacks. Thanks to &lt;a href=&quot;https://github.com/nojb&quot;&gt;nojb&lt;/a&gt; for the initial code and &lt;a href=&quot;https://github.com/MagnusS&quot;&gt;MagnusS&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;In December, with &lt;a href=&quot;https://mirageos.org/blog/announcing-mirage-310-release&quot;&gt;Mirage 3.10&lt;/a&gt; we got the IPv6 code up and running. Now MirageOS unikernels have a dual stack available, besides IPv4-only and IPv6-only network stacks. Thanks to &lt;a href=&quot;https://github.com/nojb&quot;&gt;nojb&lt;/a&gt; for the initial code and &lt;a href=&quot;https://github.com/MagnusS&quot;&gt;MagnusS&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Turns out this blog, but also robur services, are now available via IPv6 :)&lt;/p&gt;
&lt;h3 id=&quot;albatross&quot;&gt;Albatross&lt;/h3&gt;
&lt;p&gt;Also in December, I pushed an initial release of &lt;a href=&quot;https://github.com/roburio/albatross&quot;&gt;albatross&lt;/a&gt;, a unikernel orchestration system with remote access. &lt;em&gt;Deploy your unikernel via a TLS handshake -- the unikernel image is embedded in the TLS client certificates.&lt;/em&gt;&lt;/p&gt;
@ -412,7 +412,7 @@ _stack.V4V6) (_ : sig end) (Management : Mirage_stack.V4V6) = struct
&lt;h3 id=&quot;ca-certs&quot;&gt;CA certs&lt;/h3&gt;
&lt;p&gt;For several years I postponed the problem of how to actually use the operating system trust anchors for OCaml-TLS connections. Thanks to &lt;a href=&quot;https://github.com/emillon&quot;&gt;emillon&lt;/a&gt; for initial code, there are now &lt;a href=&quot;https://github.com/mirage/ca-certs&quot;&gt;ca-certs&lt;/a&gt; and &lt;a href=&quot;https://github.com/mirage/ca-certs-nss&quot;&gt;ca-certs-nss&lt;/a&gt; opam packages (see &lt;a href=&quot;https://discuss.ocaml.org/t/ann-ca-certs-and-ca-certs-nss&quot;&gt;release announcement&lt;/a&gt;) which fills this gap.&lt;/p&gt;
&lt;h2 id=&quot;unikernels&quot;&gt;Unikernels&lt;/h2&gt;
&lt;p&gt;I developed several useful unikernels in 2020, and also pushed &lt;a href=&quot;https://mirage.io/wiki/gallery&quot;&gt;a unikernel gallery&lt;/a&gt; to the Mirage website:&lt;/p&gt;
&lt;p&gt;I developed several useful unikernels in 2020, and also pushed &lt;a href=&quot;https://mirageos.org/wiki/gallery&quot;&gt;a unikernel gallery&lt;/a&gt; to the Mirage website:&lt;/p&gt;
&lt;h3 id=&quot;traceroute-in-mirageos&quot;&gt;Traceroute in MirageOS&lt;/h3&gt;
&lt;p&gt;I already wrote about &lt;a href=&quot;/Posts/Traceroute&quot;&gt;traceroute&lt;/a&gt; which traces the routing to a given remote host.&lt;/p&gt;
&lt;h3 id=&quot;unipi---static-website-hosting&quot;&gt;Unipi - static website hosting&lt;/h3&gt;
@ -453,7 +453,7 @@ _stack.V4V6) (_ : sig end) (Management : Mirage_stack.V4V6) = struct
&lt;p&gt;Mirage so far still uses ocamlbuild and ocamlfind for compiling the virtual machine binary. But the switch to dune is &lt;a href=&quot;https://github.com/mirage/mirage/issues/1195&quot;&gt;close&lt;/a&gt;, a lot of effort has been done. This will make the developer experience of MirageOS much more smooth, with a per-unikernel monorepo workflow where you can push your changes to the individual libraries.&lt;/p&gt;
&lt;h2 id=&quot;footer&quot;&gt;Footer&lt;/h2&gt;
&lt;p&gt;If you want to support our work on MirageOS unikernels, please &lt;a href=&quot;https://robur.coop/Donate&quot;&gt;donate to robur&lt;/a&gt;. I'm interested in feedback, either via &lt;a href=&quot;https://twitter.com/h4nnes&quot;&gt;twitter&lt;/a&gt;, &lt;a href=&quot;https://mastodon.social/@hannesm&quot;&gt;hannesm@mastodon.social&lt;/a&gt; or via eMail.&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:bc7675a5-47d0-5ce1-970c-01ed07fdf404</id><title type="text">The road ahead for MirageOS in 2021</title><updated>2021-11-19T18:04:52-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;A MirageOS unikernel which traces the path between itself and a remote host.&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:bc7675a5-47d0-5ce1-970c-01ed07fdf404</id><title type="text">The road ahead for MirageOS in 2021</title><updated>2024-10-11T09:43:26-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;A MirageOS unikernel which traces the path between itself and a remote host.&lt;/p&gt;
</summary><published>2020-06-24T10:38:10-00:00</published><link href="/Posts/Traceroute" rel="alternate"/><content type="html">&lt;h2 id=&quot;traceroute&quot;&gt;Traceroute&lt;/h2&gt;
&lt;p&gt;Is a diagnostic utility which displays the route and measures transit delays of
packets across an Internet protocol (IP) network.&lt;/p&gt;
@ -786,7 +786,7 @@ let () =
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;And voila, that's all the code. If you copy it together (or download the two
files from &lt;a href=&quot;https://github.com/roburio/traceroute&quot;&gt;the GitHub repository&lt;/a&gt;),
and have OCaml, opam, and &lt;a href=&quot;https://mirage.io/wiki/install&quot;&gt;mirage (&amp;gt;= 3.8.0)&lt;/a&gt; installed,
and have OCaml, opam, and &lt;a href=&quot;https://mirageos.org/wiki/install&quot;&gt;mirage (&amp;gt;= 3.8.0)&lt;/a&gt; installed,
you should be able to:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;$ mirage configure -t hvt
$ make depend
@ -798,7 +798,7 @@ $ solo5-hvt --net:service=tap0 -- traceroute.hvt ...
&lt;p&gt;If you develop enhancements you'd like to share, please sent a pull request to the git repository.&lt;/p&gt;
&lt;p&gt;Motivation for this traceroute unikernel was while talking with &lt;a href=&quot;https://twitter.com/networkservice&quot;&gt;Aaron&lt;/a&gt; and &lt;a href=&quot;https://github.com/phaer&quot;&gt;Paul&lt;/a&gt;, who contributed several patches to the IP stack which pass the ttl through.&lt;/p&gt;
&lt;p&gt;If you want to support our work on MirageOS unikernels, please &lt;a href=&quot;https://robur.coop/Donate&quot;&gt;donate to robur&lt;/a&gt;. I'm interested in feedback, either via &lt;a href=&quot;https://twitter.com/h4nnes&quot;&gt;twitter&lt;/a&gt;, &lt;a href=&quot;https://mastodon.social/@hannesm&quot;&gt;hannesm@mastodon.social&lt;/a&gt; or via eMail.&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/protocol" term="protocol"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:ed3036f6-83d2-5e80-b3da-4ccbedb5ae9e</id><title type="text">Traceroute</title><updated>2021-11-19T18:04:52-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;A tutorial how to deploy authoritative name servers, let's encrypt, and updating entries from unix services.&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/protocol" term="protocol"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:ed3036f6-83d2-5e80-b3da-4ccbedb5ae9e</id><title type="text">Traceroute</title><updated>2024-10-11T09:43:26-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;A tutorial how to deploy authoritative name servers, let's encrypt, and updating entries from unix services.&lt;/p&gt;
</summary><published>2019-12-23T21:30:53-00:00</published><link href="/Posts/DnsServer" rel="alternate"/><content type="html">&lt;h2 id=&quot;goal&quot;&gt;Goal&lt;/h2&gt;
&lt;p&gt;Have your domain served by OCaml-DNS authoritative name servers. Data is stored in a git remote, and let's encrypt certificates can be requested to DNS. This software is deployed since more than two years for several domains such as &lt;code&gt;nqsb.io&lt;/code&gt; and &lt;code&gt;robur.coop&lt;/code&gt;. This present the authoritative server side, and certificate library of the OCaml-DNS implementation formerly known as &lt;a href=&quot;/Posts/DNS&quot;&gt;µDNS&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id=&quot;prerequisites&quot;&gt;Prerequisites&lt;/h2&gt;
@ -973,7 +973,7 @@ personal._update.mirage. DNSKEY 0 3 163 kJJqipaQHQWqZL31Raar6uPnepGFIdtpjkXot9rv
10.0.42.2.10.0.42.3._transfer.mirage. DNSKEY 0 3 163 cDK6sKyvlt8UBerZlmxuD84ih2KookJGDagJlLVNo20=
git-repo&amp;gt; git commit -m &amp;quot;updates&amp;quot; . &amp;amp;&amp;amp; git push
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Ok, the git repository is ready, now we need to compile the unikernels for the virtualisation target (see &lt;a href=&quot;https://mirage.io/wiki/hello-world#Building-for-Another-Backend&quot;&gt;other targets&lt;/a&gt; for further information).&lt;/p&gt;
&lt;p&gt;Ok, the git repository is ready, now we need to compile the unikernels for the virtualisation target (see &lt;a href=&quot;https://mirageos.org/wiki/hello-world#Building-for-Another-Backend&quot;&gt;other targets&lt;/a&gt; for further information).&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-shell&quot;&gt;# back to primary
$ cd ../dns-primary-git
$ mirage configure -t hvt # or e.g. -t spt (and solo5-spt below)
@ -1073,4 +1073,4 @@ $ ocertify 10.0.42.2 foo.mirage
&lt;p&gt;There are further steps to take, such as monitoring (&lt;code&gt;mirage configure --monitoring&lt;/code&gt;), which use a second network interface for reporting syslog and metrics to telegraf / influx / grafana. Some DNS features are still missing, most prominently DNSSec.&lt;/p&gt;
&lt;p&gt;I'd like to thank all people involved in this software stack, without other key components, including &lt;a href=&quot;https://github.com/mirage/ocaml-git&quot;&gt;git&lt;/a&gt;, &lt;a href=&quot;https://github.com/mirage/mirage-crypto&quot;&gt;mirage-crypto&lt;/a&gt;, &lt;a href=&quot;https://github.com/mirage/awa-ssh&quot;&gt;awa-ssh&lt;/a&gt;, &lt;a href=&quot;https://github.com/solo5/sol5&quot;&gt;solo5&lt;/a&gt;, &lt;a href=&quot;https://github.com/mirage/mirage&quot;&gt;mirage&lt;/a&gt;, &lt;a href=&quot;https://github.com/mmaker/ocaml-letsencrypt&quot;&gt;ocaml-letsencrypt&lt;/a&gt;, and more.&lt;/p&gt;
&lt;p&gt;If you want to support our work on MirageOS unikernels, please &lt;a href=&quot;https://robur.coop/Donate&quot;&gt;donate to robur&lt;/a&gt;. I'm interested in feedback, either via &lt;a href=&quot;https://twitter.com/h4nnes&quot;&gt;twitter&lt;/a&gt;, &lt;a href=&quot;https://mastodon.social/@hannesm&quot;&gt;hannesm@mastodon.social&lt;/a&gt; or via eMail.&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/deployment" term="deployment"/><category scheme="https://hannes.robur.coop/tags/protocol" term="protocol"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:e3d4fd9e-e379-5c86-838e-46034ddd435d</id><title type="text">Deploying authoritative OCaml-DNS servers as MirageOS unikernels</title><updated>2023-03-02T17:20:44-00:00</updated><author><name>hannes</name></author></entry></feed>
</content><category scheme="https://hannes.robur.coop/tags/deployment" term="deployment"/><category scheme="https://hannes.robur.coop/tags/protocol" term="protocol"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:e3d4fd9e-e379-5c86-838e-46034ddd435d</id><title type="text">Deploying authoritative OCaml-DNS servers as MirageOS unikernels</title><updated>2024-10-11T09:43:26-00:00</updated><author><name>hannes</name></author></entry></feed>