updated from main (commit e386567813)
This commit is contained in:
Canopy bot
parent
9c032cedd2
commit
a02d294215
12 changed files with 88 additions and 59 deletions
38
Posts/TCP-ns
Normal file
38
Posts/TCP-ns
Normal file
|
|
@ -0,0 +1,38 @@
|
||||||
|
<!DOCTYPE html>
|
||||||
|
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Re-developing TCP from the grounds up</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Re-developing TCP from the grounds up" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Re-developing TCP from the grounds up</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a><a href="/tags/tcp" class="tag">tcp</a></div><span class="date">Published: 2023-11-28 (last updated: 2023-11-28)</span><article><p>The <a href="https://en.wikipedia.org/wiki/Transmission_Control_Protocol">Transmission Control Protocol (TCP)</a> is one of the main Internet protocols. Usually spoken on top of the Internet Protocol (legacy version 4 or version 6), it provides a reliable, ordered, and error-checked stream of octets. When an application uses TCP, they get these properties for free (in contrast to UDP).</p>
|
||||||
|
<p>As common for Internet protocols, also TCP is specified in a series of so-called requests for comments (RFC), the latest revised version from August 2022 is <a href="https://datatracker.ietf.org/doc/html/rfc9293">RFC 9293</a>, the initial one was <a href="https://datatracker.ietf.org/doc/html/rfc793">RFC 793</a> from September 1981.</p>
|
||||||
|
<h1 id="my-brief-personal-tcp-story">My brief personal TCP story</h1>
|
||||||
|
<p>My interest in TCP started back in 2006 when we worked on a <a href="https://github.com/dylan-hackers/network-night-vision">network stack in Dylan</a> (these days abandoned) - ever since then I wanted to understand the implementation tradeoffs in more detail - including attacks and how to prevent a TCP stack from being vulnerable.</p>
|
||||||
|
<p>In 2012 I attended ICFP in Copenhagen - while being a PhD student at ITU Copenhagen - and there <a href="https://www.cl.cam.ac.uk/~pes20/">Peter Sewell</a> gave an invited talk "Tales from the jungle" about rigorous methods for real-world infrastructure (C semantics, hardware (concurrency) behaviour of CPUs, TCP/IP, and likely more). Working on formal specifications myself (<a href="https://en.itu.dk/-/media/EN/Research/PhD-Programme/PhD-defences/2013/130731-Hannes-Mehnert-PhD-dissertation-finalpdf.pdf">my dissertation</a>), and having a strong interest in real systems, I was immediately hooked by his perspective.</p>
|
||||||
|
<p>To dive a bit more into <a href="https://www.cl.cam.ac.uk/~pes20/Netsem/">network semantics</a> - the work done on TCP by Peter Sewell et al - is a formal specification (or a model) of TCP/IP and the Unix sockets API developed in HOL4. It is a label transition system with non-deterministic choices, and the model itself is executable. It has been validated with the real world by collecting thousands of traces on Linux, Windows, and FreeBSD - which have been checked by the model for validity - this copes with the different implementations of the English prose of the RFCs. The network semantics research found several issues in existing TCP stack and reported them upstream to have them fixed (though, there still is some special treatment, e.g. for the "BSD listen bug").</p>
|
||||||
|
<p>In 2014 I joined Peter's research group in Cambridge to continue their work on the model: updating to more recent versions of HOL4 and PolyML, revising the test system to use DTrace, updating to a more recent FreeBSD network stack (from FreeBSD 4.6 to FreeBSD 10), and finally getting the <a href="https://dl.acm.org/doi/10.1145/3243650">journal paper</a> (<a href="http://www.cl.cam.ac.uk/~pes20/Netsem/paper3.pdf">author's copy</a>) published. At the same time the <a href="https://mirage.io">MirageOS</a> melting pot was happening at University of Cambridge, where I contributed OCaml-TLS etc. with David.</p>
|
||||||
|
<p>My intention was to understand TCP better, and use the specification as a basis for a TCP stack for MirageOS - the <a href="https://github.com/mirage/mirage-tcpip">existing one</a> (which is still used) has technical debt: a high issue to number of lines ratio, the lwt monad is ubiquitous which makes testing and debugging pretty hard, utilizing multiple cores with OCaml multicore won't be easy, it has various resource leaks, and there is no active maintainer. But honestly, it works fine on a local network, and with well-behaved traffic. It doesn't work that well on the wild Internet with a variety of broken implementations. Apart from resource leakage, which made me to implement things such as restart-on-failure in albatross, there are certain connection states which will never be exited.</p>
|
||||||
|
<h1 id="the-rise-of-µtcp">The rise of <a href="https://github.com/robur-coop/utcp">µTCP</a></h1>
|
||||||
|
<p>Back in Cambridge I didn't manage to write a TCP stack based on the model, but in 2019 I re-started that work and got µTCP (the formal model manually translated to OCaml) to compile and do TCP session setup and teardown. Since it was a model that uses non-determinism, this couldn't be translated one-to-one into an executable program, but there are places where decisions have to be done. Due to other projects, I worked only briefly in 2021 and 2022 on µTCP, but finally in the summer 2023 I motivated myself to push µTCP into a usable state - so far I spend 25 days in 2023 on µTCP. Thanks to <a href="https://tarides.com">Tarides</a> for supporting my work.</p>
|
||||||
|
<p>Since late August we are running some unikernels using µTCP, e.g. the <a href="https://retreat.mirage.io">retreat</a> website. This allows us to observe µTCP and find and solve issues that occur in the real world. It turned out that the model is not always correct (i.e. in the model there is no retransmit timer in the close wait state - which avoids proper session teardowns). We report statistics about how many TCP connections are in which state to an influx time series database and view graphs rendered by grafana. If there are connections that are stuck for multiple hours, this indicates a resource leak that should be addressed. Grafana was tremendously helpful for us to find out where to look for resource leaks. Still, there's work to understand the behaviour, look at what the model does, what µTCP does, what the RFC says, and eventually what existing deployed TCP stacks do.</p>
|
||||||
|
<h1 id="the-secondary-nameserver-issue">The secondary nameserver issue</h1>
|
||||||
|
<p>One of our secondary nameservers attempts to receive zones (via AXFR using TCP) from another nameserver that is currently not running. Thus it replies to each SYN packet a corresponding RST. Below I graphed the network utilization (send data/packets is positive y-axis, receive part on the negative) over time (on the x-axis) on the left and memory usage (bytes on y-axis) over time (x-axis) on the right of our nameserver - you can observe that both increases over time, and roughly every 3 hours the unikernel hits its configured memory limit (64 MB), crashes with out of memory, and is restarted. The graph below is using the mirage-tcpip stack.</p>
|
||||||
|
<p><img src="/static/img/a.ns.mtcp.png" alt="" /></p>
|
||||||
|
<p>Now, after switching over to µTCP, graphed below, there's much fewer network utilization and the memory limit is only reached after 36 hours, which is a great result. Though, still it is not very satisfying that the unikernel leaks memory. Both graphs contain on their left side a few hours of mirage-tcpip, and shortly after 20:00 on Nov 23rd µTCP got deployed.</p>
|
||||||
|
<p><img src="/static/img/a.ns.mtcp-utcp.png" alt="" /></p>
|
||||||
|
<p>Investigating the involved parts showed that a TCP connection that was never established has been registered at the MirageOS layer, but the pure core does not expose an event from the received RST that the connection has been cancelled. This means the MirageOS layer piles up all the connection attempts, and doesn't inform the application that the connection couldn't be established. Once this was well understood, developing the <a href="https://github.com/robur-coop/utcp/commit/67fc49468e6b75b96a481ebe44dd11ce4bb76e6c">required code changes</a> was straightforward. The graph shows that the fix was deployed at 15:25. The memory usage is constant afterwards, but the network utilization increased enormously.</p>
|
||||||
|
<p><img src="/static/img/a.ns.utcp-utcp.png" alt="" /></p>
|
||||||
|
<p>Now, the network utilization is unwanted. This was hidden by the application waiting forever that the TCP connection getting established. Our bugfix uncovered another issue, a tight loop:</p>
|
||||||
|
<ul>
|
||||||
|
<li>the nameserver attempts to connect to the other nameserver (<code>request</code>);
|
||||||
|
</li>
|
||||||
|
<li>this results in a <code>TCP.create_connection</code> which errors after one roundtrip;
|
||||||
|
</li>
|
||||||
|
<li>this leads to a <code>close</code>, which attempts a <code>request</code> again.
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
<p>This is unnecessary since the DNS server code has a timer to attempt to connect to the remote nameserver periodically (but takes a break between attempts). After understanding this behaviour, we worked on <a href="https://github.com/mirage/ocaml-dns/pull/347">the fix</a> and re-deployed the nameserver again. The graph has on the left edge the tight loop (so you have a comparison), at 16:05 we deployed the fix - since then it looks pretty smooth, both in memory usage and in network utilization.</p>
|
||||||
|
<p><img src="/static/img/a.ns.utcp-fixed.png" alt="" /></p>
|
||||||
|
<p>To give you the entire picture, below is the graph where you can spot the mirage-tcpip stack (lots of network, restarting every 3 hours), µTCP-without-informing-application (run for 3 * ~36 hours), dns-server-high-network-utilization (which only lasted for a brief period, thus it is more a point in the graph), and finally the unikernel with both fixes applied.</p>
|
||||||
|
<p><img src="/static/img/a.ns.all.png" alt="" /></p>
|
||||||
|
<h1 id="conclusion">Conclusion</h1>
|
||||||
|
<p>What can we learn from that? Choosing convenient tooling is crucial for effective debugging. Also, fixing one issue may uncover other issues. And of course, the mirage-tcpip was running with the dns-server that had a tight reconnect loop. But, below the line: should such an application lead to memory leaks? I don't think so. My approach is that all core network libraries should work in a non-resource-leaky way with any kind of application on top of it. When one TCP connection returns an error (and thus is destroyed), the TCP stack should have no more resources used for that connection.</p>
|
||||||
|
<p>We'll take some more time to investigate issues of µTCP in production, plan to write further documentation and blog posts, and hopefully soon are ready for an initial public release. In the meantime, you can follow our development repository.</p>
|
||||||
|
<p>We at <a href="https://robur.coop">robur</a> are working as a collective since 2018, on public funding, commercial contracts, and donations. Our mission is to get sustainable, robust and secure MirageOS unikernels developed and deployed. Running your own digital communication infrastructure should be easy, including trustworthy binaries and smooth upgrades. You can help us continuing our work by <a href="https://aenderwerk.de/donate/">donating</a> (select robur from the drop-down, or put "donation robur" in the purpose of the bank transfer).</p>
|
||||||
|
<p>If you have any questions, reach us best via eMail to team AT robur DOT coop.</p>
|
||||||
|
</article></div></div></main></body></html>
|
||||||
|
|
@ -1,5 +1,6 @@
|
||||||
<!DOCTYPE html>
|
<!DOCTYPE html>
|
||||||
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>full stack engineer</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="full stack engineer" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="flex-container"><div class="list-group listing"><a href="/Posts/Albatross" class="list-group-item"><h2 class="list-group-item-heading">Deploying reproducible unikernels with albatross</h2><span class="author">Written by hannes</span> <time>2022-11-17</time><br/><p class="list-group-item-text abstract"><p>fleet management for MirageOS unikernels using a mutually authenticated TLS handshake</p>
|
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>full stack engineer</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="full stack engineer" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="flex-container"><div class="list-group listing"><a href="/Posts/TCP-ns" class="list-group-item"><h2 class="list-group-item-heading">Re-developing TCP from the grounds up</h2><span class="author">Written by hannes</span> <time>2023-11-28</time><br/><p class="list-group-item-text abstract"><p>Core Internet protocols require operational experiments, even if formally specified</p>
|
||||||
|
</p></a><a href="/Posts/Albatross" class="list-group-item"><h2 class="list-group-item-heading">Deploying reproducible unikernels with albatross</h2><span class="author">Written by hannes</span> <time>2022-11-17</time><br/><p class="list-group-item-text abstract"><p>fleet management for MirageOS unikernels using a mutually authenticated TLS handshake</p>
|
||||||
</p></a><a href="/Posts/OpamMirror" class="list-group-item"><h2 class="list-group-item-heading">Mirroring the opam repository and all tarballs</h2><span class="author">Written by hannes</span> <time>2022-09-29</time><br/><p class="list-group-item-text abstract"><p>Re-developing an opam cache from scratch, as a MirageOS unikernel</p>
|
</p></a><a href="/Posts/OpamMirror" class="list-group-item"><h2 class="list-group-item-heading">Mirroring the opam repository and all tarballs</h2><span class="author">Written by hannes</span> <time>2022-09-29</time><br/><p class="list-group-item-text abstract"><p>Re-developing an opam cache from scratch, as a MirageOS unikernel</p>
|
||||||
</p></a><a href="/Posts/Monitoring" class="list-group-item"><h2 class="list-group-item-heading">All your metrics belong to influx</h2><span class="author">Written by hannes</span> <time>2022-03-08</time><br/><p class="list-group-item-text abstract"><p>How to monitor your MirageOS unikernel with albatross and monitoring-experiments</p>
|
</p></a><a href="/Posts/Monitoring" class="list-group-item"><h2 class="list-group-item-heading">All your metrics belong to influx</h2><span class="author">Written by hannes</span> <time>2022-03-08</time><br/><p class="list-group-item-text abstract"><p>How to monitor your MirageOS unikernel with albatross and monitoring-experiments</p>
|
||||||
</p></a><a href="/Posts/Deploy" class="list-group-item"><h2 class="list-group-item-heading">Deploying binary MirageOS unikernels</h2><span class="author">Written by hannes</span> <time>2021-06-30</time><br/><p class="list-group-item-text abstract"><p>Finally, we provide reproducible binary MirageOS unikernels together with packages to reproduce them and setup your own builder</p>
|
</p></a><a href="/Posts/Deploy" class="list-group-item"><h2 class="list-group-item-heading">Deploying binary MirageOS unikernels</h2><span class="author">Written by hannes</span> <time>2021-06-30</time><br/><p class="list-group-item-text abstract"><p>Finally, we provide reproducible binary MirageOS unikernels together with packages to reproduce them and setup your own builder</p>
|
||||||
|
|
|
||||||
94
atom
94
atom
|
|
@ -1,4 +1,41 @@
|
||||||
<feed xmlns="http://www.w3.org/2005/Atom"><link href="https://hannes.robur.coop/atom" rel="self"/><id>urn:uuid:981361ca-e71d-4997-a52c-baeee78e4156</id><title type="text">full stack engineer</title><updated>2023-11-20T16:58:35-00:00</updated><entry><summary type="html"><p>fleet management for MirageOS unikernels using a mutually authenticated TLS handshake</p>
|
<feed xmlns="http://www.w3.org/2005/Atom"><link href="https://hannes.robur.coop/atom" rel="self"/><id>urn:uuid:981361ca-e71d-4997-a52c-baeee78e4156</id><title type="text">full stack engineer</title><updated>2023-11-28T21:17:01-00:00</updated><entry><summary type="html"><p>Core Internet protocols require operational experiments, even if formally specified</p>
|
||||||
|
</summary><published>2023-11-28T21:17:01-00:00</published><link href="/Posts/TCP-ns" rel="alternate"/><content type="html"><p>The <a href="https://en.wikipedia.org/wiki/Transmission_Control_Protocol">Transmission Control Protocol (TCP)</a> is one of the main Internet protocols. Usually spoken on top of the Internet Protocol (legacy version 4 or version 6), it provides a reliable, ordered, and error-checked stream of octets. When an application uses TCP, they get these properties for free (in contrast to UDP).</p>
|
||||||
|
<p>As common for Internet protocols, also TCP is specified in a series of so-called requests for comments (RFC), the latest revised version from August 2022 is <a href="https://datatracker.ietf.org/doc/html/rfc9293">RFC 9293</a>, the initial one was <a href="https://datatracker.ietf.org/doc/html/rfc793">RFC 793</a> from September 1981.</p>
|
||||||
|
<h1 id="my-brief-personal-tcp-story">My brief personal TCP story</h1>
|
||||||
|
<p>My interest in TCP started back in 2006 when we worked on a <a href="https://github.com/dylan-hackers/network-night-vision">network stack in Dylan</a> (these days abandoned) - ever since then I wanted to understand the implementation tradeoffs in more detail - including attacks and how to prevent a TCP stack from being vulnerable.</p>
|
||||||
|
<p>In 2012 I attended ICFP in Copenhagen - while being a PhD student at ITU Copenhagen - and there <a href="https://www.cl.cam.ac.uk/~pes20/">Peter Sewell</a> gave an invited talk &quot;Tales from the jungle&quot; about rigorous methods for real-world infrastructure (C semantics, hardware (concurrency) behaviour of CPUs, TCP/IP, and likely more). Working on formal specifications myself (<a href="https://en.itu.dk/-/media/EN/Research/PhD-Programme/PhD-defences/2013/130731-Hannes-Mehnert-PhD-dissertation-finalpdf.pdf">my dissertation</a>), and having a strong interest in real systems, I was immediately hooked by his perspective.</p>
|
||||||
|
<p>To dive a bit more into <a href="https://www.cl.cam.ac.uk/~pes20/Netsem/">network semantics</a> - the work done on TCP by Peter Sewell et al - is a formal specification (or a model) of TCP/IP and the Unix sockets API developed in HOL4. It is a label transition system with non-deterministic choices, and the model itself is executable. It has been validated with the real world by collecting thousands of traces on Linux, Windows, and FreeBSD - which have been checked by the model for validity - this copes with the different implementations of the English prose of the RFCs. The network semantics research found several issues in existing TCP stack and reported them upstream to have them fixed (though, there still is some special treatment, e.g. for the &quot;BSD listen bug&quot;).</p>
|
||||||
|
<p>In 2014 I joined Peter's research group in Cambridge to continue their work on the model: updating to more recent versions of HOL4 and PolyML, revising the test system to use DTrace, updating to a more recent FreeBSD network stack (from FreeBSD 4.6 to FreeBSD 10), and finally getting the <a href="https://dl.acm.org/doi/10.1145/3243650">journal paper</a> (<a href="http://www.cl.cam.ac.uk/~pes20/Netsem/paper3.pdf">author's copy</a>) published. At the same time the <a href="https://mirage.io">MirageOS</a> melting pot was happening at University of Cambridge, where I contributed OCaml-TLS etc. with David.</p>
|
||||||
|
<p>My intention was to understand TCP better, and use the specification as a basis for a TCP stack for MirageOS - the <a href="https://github.com/mirage/mirage-tcpip">existing one</a> (which is still used) has technical debt: a high issue to number of lines ratio, the lwt monad is ubiquitous which makes testing and debugging pretty hard, utilizing multiple cores with OCaml multicore won't be easy, it has various resource leaks, and there is no active maintainer. But honestly, it works fine on a local network, and with well-behaved traffic. It doesn't work that well on the wild Internet with a variety of broken implementations. Apart from resource leakage, which made me to implement things such as restart-on-failure in albatross, there are certain connection states which will never be exited.</p>
|
||||||
|
<h1 id="the-rise-of-µtcp">The rise of <a href="https://github.com/robur-coop/utcp">µTCP</a></h1>
|
||||||
|
<p>Back in Cambridge I didn't manage to write a TCP stack based on the model, but in 2019 I re-started that work and got µTCP (the formal model manually translated to OCaml) to compile and do TCP session setup and teardown. Since it was a model that uses non-determinism, this couldn't be translated one-to-one into an executable program, but there are places where decisions have to be done. Due to other projects, I worked only briefly in 2021 and 2022 on µTCP, but finally in the summer 2023 I motivated myself to push µTCP into a usable state - so far I spend 25 days in 2023 on µTCP. Thanks to <a href="https://tarides.com">Tarides</a> for supporting my work.</p>
|
||||||
|
<p>Since late August we are running some unikernels using µTCP, e.g. the <a href="https://retreat.mirage.io">retreat</a> website. This allows us to observe µTCP and find and solve issues that occur in the real world. It turned out that the model is not always correct (i.e. in the model there is no retransmit timer in the close wait state - which avoids proper session teardowns). We report statistics about how many TCP connections are in which state to an influx time series database and view graphs rendered by grafana. If there are connections that are stuck for multiple hours, this indicates a resource leak that should be addressed. Grafana was tremendously helpful for us to find out where to look for resource leaks. Still, there's work to understand the behaviour, look at what the model does, what µTCP does, what the RFC says, and eventually what existing deployed TCP stacks do.</p>
|
||||||
|
<h1 id="the-secondary-nameserver-issue">The secondary nameserver issue</h1>
|
||||||
|
<p>One of our secondary nameservers attempts to receive zones (via AXFR using TCP) from another nameserver that is currently not running. Thus it replies to each SYN packet a corresponding RST. Below I graphed the network utilization (send data/packets is positive y-axis, receive part on the negative) over time (on the x-axis) on the left and memory usage (bytes on y-axis) over time (x-axis) on the right of our nameserver - you can observe that both increases over time, and roughly every 3 hours the unikernel hits its configured memory limit (64 MB), crashes with out of memory, and is restarted. The graph below is using the mirage-tcpip stack.</p>
|
||||||
|
<p><img src="/static/img/a.ns.mtcp.png" alt="" /></p>
|
||||||
|
<p>Now, after switching over to µTCP, graphed below, there's much fewer network utilization and the memory limit is only reached after 36 hours, which is a great result. Though, still it is not very satisfying that the unikernel leaks memory. Both graphs contain on their left side a few hours of mirage-tcpip, and shortly after 20:00 on Nov 23rd µTCP got deployed.</p>
|
||||||
|
<p><img src="/static/img/a.ns.mtcp-utcp.png" alt="" /></p>
|
||||||
|
<p>Investigating the involved parts showed that a TCP connection that was never established has been registered at the MirageOS layer, but the pure core does not expose an event from the received RST that the connection has been cancelled. This means the MirageOS layer piles up all the connection attempts, and doesn't inform the application that the connection couldn't be established. Once this was well understood, developing the <a href="https://github.com/robur-coop/utcp/commit/67fc49468e6b75b96a481ebe44dd11ce4bb76e6c">required code changes</a> was straightforward. The graph shows that the fix was deployed at 15:25. The memory usage is constant afterwards, but the network utilization increased enormously.</p>
|
||||||
|
<p><img src="/static/img/a.ns.utcp-utcp.png" alt="" /></p>
|
||||||
|
<p>Now, the network utilization is unwanted. This was hidden by the application waiting forever that the TCP connection getting established. Our bugfix uncovered another issue, a tight loop:</p>
|
||||||
|
<ul>
|
||||||
|
<li>the nameserver attempts to connect to the other nameserver (<code>request</code>);
|
||||||
|
</li>
|
||||||
|
<li>this results in a <code>TCP.create_connection</code> which errors after one roundtrip;
|
||||||
|
</li>
|
||||||
|
<li>this leads to a <code>close</code>, which attempts a <code>request</code> again.
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
<p>This is unnecessary since the DNS server code has a timer to attempt to connect to the remote nameserver periodically (but takes a break between attempts). After understanding this behaviour, we worked on <a href="https://github.com/mirage/ocaml-dns/pull/347">the fix</a> and re-deployed the nameserver again. The graph has on the left edge the tight loop (so you have a comparison), at 16:05 we deployed the fix - since then it looks pretty smooth, both in memory usage and in network utilization.</p>
|
||||||
|
<p><img src="/static/img/a.ns.utcp-fixed.png" alt="" /></p>
|
||||||
|
<p>To give you the entire picture, below is the graph where you can spot the mirage-tcpip stack (lots of network, restarting every 3 hours), µTCP-without-informing-application (run for 3 * ~36 hours), dns-server-high-network-utilization (which only lasted for a brief period, thus it is more a point in the graph), and finally the unikernel with both fixes applied.</p>
|
||||||
|
<p><img src="/static/img/a.ns.all.png" alt="" /></p>
|
||||||
|
<h1 id="conclusion">Conclusion</h1>
|
||||||
|
<p>What can we learn from that? Choosing convenient tooling is crucial for effective debugging. Also, fixing one issue may uncover other issues. And of course, the mirage-tcpip was running with the dns-server that had a tight reconnect loop. But, below the line: should such an application lead to memory leaks? I don't think so. My approach is that all core network libraries should work in a non-resource-leaky way with any kind of application on top of it. When one TCP connection returns an error (and thus is destroyed), the TCP stack should have no more resources used for that connection.</p>
|
||||||
|
<p>We'll take some more time to investigate issues of µTCP in production, plan to write further documentation and blog posts, and hopefully soon are ready for an initial public release. In the meantime, you can follow our development repository.</p>
|
||||||
|
<p>We at <a href="https://robur.coop">robur</a> are working as a collective since 2018, on public funding, commercial contracts, and donations. Our mission is to get sustainable, robust and secure MirageOS unikernels developed and deployed. Running your own digital communication infrastructure should be easy, including trustworthy binaries and smooth upgrades. You can help us continuing our work by <a href="https://aenderwerk.de/donate/">donating</a> (select robur from the drop-down, or put &quot;donation robur&quot; in the purpose of the bank transfer).</p>
|
||||||
|
<p>If you have any questions, reach us best via eMail to team AT robur DOT coop.</p>
|
||||||
|
</content><category scheme="https://hannes.robur.coop/tags/tcp" term="tcp"/><category scheme="https://hannes.robur.coop/tags/protocol" term="protocol"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:96688956-0808-5d44-b795-1d64cbb4f947</id><title type="text">Re-developing TCP from the grounds up</title><updated>2023-11-28T21:17:01-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html"><p>fleet management for MirageOS unikernels using a mutually authenticated TLS handshake</p>
|
||||||
</summary><published>2022-11-17T12:41:11-00:00</published><link href="/Posts/Albatross" rel="alternate"/><content type="html"><p>EDIT (2023-05-16): Updated with albatross release version 2.0.0.</p>
|
</summary><published>2022-11-17T12:41:11-00:00</published><link href="/Posts/Albatross" rel="alternate"/><content type="html"><p>EDIT (2023-05-16): Updated with albatross release version 2.0.0.</p>
|
||||||
<h2 id="deploying-mirageos-unikernels">Deploying MirageOS unikernels</h2>
|
<h2 id="deploying-mirageos-unikernels">Deploying MirageOS unikernels</h2>
|
||||||
<p>More than five years ago, I posted <a href="/Posts/VMM">how to deploy MirageOS unikernels</a>. My motivation to work on this topic is that I'm convinced of reduced complexity, improved security, and more sustainable resource footprint of MirageOS unikernels, and want to ease deployment thereof. More than one year ago, I described <a href="/Posts/Deploy">how to deploy reproducible unikernels</a>.</p>
|
<p>More than five years ago, I posted <a href="/Posts/VMM">how to deploy MirageOS unikernels</a>. My motivation to work on this topic is that I'm convinced of reduced complexity, improved security, and more sustainable resource footprint of MirageOS unikernels, and want to ease deployment thereof. More than one year ago, I described <a href="/Posts/Deploy">how to deploy reproducible unikernels</a>.</p>
|
||||||
|
|
@ -1055,57 +1092,4 @@ $ orb build --twice --keep-build-dir --diffoscope &lt;your-favourite-opam-pa
|
||||||
<p>What was fun was to compare the unikernel when built on Linux with gcc against a built on FreeBSD with clang and lld - spoiler: they emit debug sections with different dwarf versions, it is pretty big. Other fun differences were between OCaml compiler versions: the difference between minor versions (4.08.0 vs 4.08.1) is pretty small (~100kB as human-readable output), while the difference between major version (4.08.1 vs 4.09.0) is rather big (~900kB as human-readable diff).</p>
|
<p>What was fun was to compare the unikernel when built on Linux with gcc against a built on FreeBSD with clang and lld - spoiler: they emit debug sections with different dwarf versions, it is pretty big. Other fun differences were between OCaml compiler versions: the difference between minor versions (4.08.0 vs 4.08.1) is pretty small (~100kB as human-readable output), while the difference between major version (4.08.1 vs 4.09.0) is rather big (~900kB as human-readable diff).</p>
|
||||||
<p>An item on my list for the future is to distribute the opam export, build hashes and build environment artifacts in a authenticated way. I want to integrate this as <a href="https://in-toto.io/">in-toto</a> style into <a href="https://github.com/hannesm/conex">conex</a>, my not-yet-deployed implementation of <a href="https://theupdateframework.github.io/">tuf</a> for opam that needs further development and a test installation, hopefully in 2020.</p>
|
<p>An item on my list for the future is to distribute the opam export, build hashes and build environment artifacts in a authenticated way. I want to integrate this as <a href="https://in-toto.io/">in-toto</a> style into <a href="https://github.com/hannesm/conex">conex</a>, my not-yet-deployed implementation of <a href="https://theupdateframework.github.io/">tuf</a> for opam that needs further development and a test installation, hopefully in 2020.</p>
|
||||||
<p>If you want to support our work on MirageOS unikernels, please <a href="https://robur.coop/Donate">donate to robur</a>. I'm interested in feedback, either via <a href="https://twitter.com/h4nnes">twitter</a>, <a href="https://mastodon.social/@hannesm">hannesm@mastodon.social</a> or via eMail.</p>
|
<p>If you want to support our work on MirageOS unikernels, please <a href="https://robur.coop/Donate">donate to robur</a>. I'm interested in feedback, either via <a href="https://twitter.com/h4nnes">twitter</a>, <a href="https://mastodon.social/@hannesm">hannesm@mastodon.social</a> or via eMail.</p>
|
||||||
</content><category scheme="https://hannes.robur.coop/tags/package%20signing" term="package signing"/><category scheme="https://hannes.robur.coop/tags/security" term="security"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:09922d6b-56c8-595d-8086-5aef9656cbc4</id><title type="text">Reproducible MirageOS unikernel builds</title><updated>2021-11-19T18:04:52-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html"><p>Five years since ocaml-x509 initial release, it has been reworked and used more widely</p>
|
</content><category scheme="https://hannes.robur.coop/tags/package%20signing" term="package signing"/><category scheme="https://hannes.robur.coop/tags/security" term="security"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:09922d6b-56c8-595d-8086-5aef9656cbc4</id><title type="text">Reproducible MirageOS unikernel builds</title><updated>2021-11-19T18:04:52-00:00</updated><author><name>hannes</name></author></entry></feed>
|
||||||
</summary><published>2019-08-15T11:21:30-00:00</published><link href="/Posts/X50907" rel="alternate"/><content type="html"><h2 id="cryptographic-material">Cryptographic material</h2>
|
|
||||||
<p>Once a private and public key pair is generated (doesn't matter whether it is plain RSA, DSA, ECC on any curve), this is fine from a scientific point of view, and can already be used for authenticating and encrypting. From a practical point of view, the public parts need to be exchanged and verified (usually a fingerprint or hash thereof). This leads to the struggle how to encode this cryptographic material, and how to embed an identity (or multiple), capabilities, and other information into it. <a href="https://en.wikipedia.org/wiki/X.509">X.509</a> is a standard to solve this encoding and embedding, and provides more functionality, such as establishing chains of trust and revocation of invalidated or compromised material. X.509 uses certificates, which contain the public key, and additional information (in a extensible key-value store), and are signed by an issuer, either the private key corresponding to the public key - a so-called self-signed certificate - or by a different private key, an authority one step up the chain. A rather long, but very good introduction to certificates by Mike Malone is <a href="https://smallstep.com/blog/everything-pki.html">available here</a>.</p>
|
|
||||||
<h2 id="ocaml-ecosystem-evolving">OCaml ecosystem evolving</h2>
|
|
||||||
<p>More than 5 years ago David Kaloper and I <a href="https://mirage.io/blog/introducing-x509">released the initial ocaml-x509</a> package as part of our <a href="https://nqsb.io">TLS stack</a>, which contained code for decoding and encoding certificates, and path validation of a certificate chain (as described in <a href="https://tools.ietf.org/html/rfc6125">RFC 5280</a>). The validation logic and the decoder/encoder, based on the ASN.1 grammar specified in the RFC, implemented using David's <a href="https://github.com/mirleft/ocaml-asn1-combinators">asn1-combinators</a> library changed much over time.</p>
|
|
||||||
<p>The OCaml ecosystem evolved over the years, which lead to some changes:</p>
|
|
||||||
<ul>
|
|
||||||
<li>Camlp4 deprecation - we used camlp4 for stream parsers of PEM-encoded certificates, and sexplib.syntax to derive s-expression decoders and encoders;
|
|
||||||
</li>
|
|
||||||
<li>Avoiding brittle ppx converters - which we used for s-expression decoders and encoders of certificates after camlp4 was deprecated;
|
|
||||||
</li>
|
|
||||||
<li>Build and release system iterations - initially oasis and a packed library, then topkg and ocamlbuild, now dune;
|
|
||||||
</li>
|
|
||||||
<li>Introduction of the <code>result</code> type in the standard library - we used to use <code>[ `Ok of certificate option | `Fail of failure ]</code>;
|
|
||||||
</li>
|
|
||||||
<li>No more leaking exceptions in the public API;
|
|
||||||
</li>
|
|
||||||
<li>Usage of pretty-printers, esp with the <a href="https://erratique.ch/software/fmt">fmt</a> library <code>val pp : Format.formatter -&gt; 'a -&gt; unit</code>, instead of <code>val to_string : t -&gt; string</code> functions;
|
|
||||||
</li>
|
|
||||||
<li>Release of <a href="https://erratique.ch/software/ptime">ptime</a>, a platform-independent POSIX time support;
|
|
||||||
</li>
|
|
||||||
<li>Release of <a href="https://erratique.ch/software/rresult">rresult</a>, which includes combinators for computation <code>result</code>s;
|
|
||||||
</li>
|
|
||||||
<li>Release of <a href="https://github.com/hannesm/gmap">gmap</a>, a <code>Map</code> whose value types depend on the key, used for X.509 extensions, GeneralName, DistinguishedName, etc.;
|
|
||||||
</li>
|
|
||||||
<li>Release of <a href="https://github.com/hannesm/domain-name">domain-name</a>, a library for domain name operations (as specified in <a href="https://tools.ietf.org/html/rfc1035">RFC 1035</a>) - used for name validation;
|
|
||||||
</li>
|
|
||||||
<li>Usage of the <a href="https://github.com/mirage/alcotest">alcotest</a> unit testing framework (instead of oUnit).
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
<h2 id="more-use-cases-for-x.509">More use cases for X.509</h2>
|
|
||||||
<p>Initially, we designed and used ocaml-x509 for providing TLS server endpoints and validation in TLS clients - mostly on the public web, where each operating system ships a set of ~100 trust anchors to validate any web server certificate against. But once you have a X.509 implementation, every authentication problem can be solved by applying it.</p>
|
|
||||||
<h3 id="authentication-with-path-building">Authentication with path building</h3>
|
|
||||||
<p>It turns out that the trust anchor sets are not equal across operating systems and versions, thus some web servers serve sets, instead of chains, of certificates - as described in <a href="https://tools.ietf.org/html/rfc4158">RFC 4158</a>, where the client implementation needs to build valid paths and accept a connection if any path can be validated. The path building was initially in 0.5.2 slightly wrong, but fixed quickly in <a href="https://github.com/mirleft/ocaml-x509/commit/1a1476308d24bdcc49d45c4cd9ef539ca57461d2">0.5.3</a>.</p>
|
|
||||||
<h3 id="fingerprint-authentication">Fingerprint authentication</h3>
|
|
||||||
<p>The chain of trust validation is useful for the open web, where you as software developer don't know to which remote endpoint your software will ever connect to - as long as the remote has a certificate signed (via intermediates) by any of the trust anchors. In the early days, before <a href="https://letsencrypt.org/">let's encrypt</a> was launched and embedded as trust anchors (or cross-signed by already deployed trust anchors), operators needed to pay for a certificate - a business model where some CAs did not bother to check the authenticity of a certificate signing request, and thus random people owning valid certificates for microsoft.com or google.com.</p>
|
|
||||||
<p>Instead of using the set of trust anchors, the fingerprint of the server certificate, or preferably the fingerprint of the public key of the certificate, can be used for authentication, as optionally done since some years in <a href="https://github.com/hannesm/jackline/commit/a1e6f3159be1e45e6b690845e1b29366c41239a2">jackline</a>, an XMPP client. Support for this certificate / public key pinning was added in x509 0.2.1 / 0.5.0.</p>
|
|
||||||
<h3 id="certificate-signing-requests">Certificate signing requests</h3>
|
|
||||||
<p>Until x509 0.4.0 there was no support for generating certificate signing requests (CSR), as defined in PKCS 10, which are self-signed blobs containing a public key, an identity, and possibly extensions. Such as CSR is sent to the certificate authority, and after validation of ownership of the identity and paying a fee, the certificate is issued. Let's encrypt specified the ACME protocol which automates the proof of ownership: they provide a HTTP API for requesting a challenge, providing the response (the proof of ownership) via HTTP or DNS, and then allow the submission of a CSR and downloading the signed certificate. The ocaml-x509 library provides operations for creating such a CSR, and also for signing a CSR to generate a certificate.</p>
|
|
||||||
<p>Mindy developed the command-line utility <a href="https://github.com/yomimono/ocaml-certify/">certify</a> which uses these operations from the ocaml-x509 library and acts as a swiss-army knife purely in OCaml for these required operations.</p>
|
|
||||||
<p>Maker developed a <a href="https://github.com/mmaker/ocaml-letsencrypt">let's encrypt library</a> which implements the above mentioned ACME protocol for provisioning CSR to certificates, also using our ocaml-x509 library.</p>
|
|
||||||
<p>To complete the required certificate authority functionality, in x509 0.6.0 certificate revocation lists, both validation and signing, was implemented.</p>
|
|
||||||
<h3 id="deploying-unikernels">Deploying unikernels</h3>
|
|
||||||
<p>As <a href="/Posts/VMM">described in another post</a>, I developed <a href="https://github.com/hannesm/albatross">albatross</a>, an orchestration system for MirageOS unikernels. This uses ASN.1 for internal socket communication and allows remote management via a TLS connection which is mutually authenticated with a X.509 client certificate. To encrypt the X.509 client certificate, first a TLS handshake where the server authenticates itself to the client is established, and over that connection another TLS handshake is established where the client certificate is requested. Note that this mechanism can be dropped with TLS 1.3, since there the certificates are transmitted over an already encrypted channel.</p>
|
|
||||||
<p>The client certificate already contains the command to execute remotely - as a custom extension, being it &quot;show me the console output&quot;, or &quot;destroy the unikernel with name = YYY&quot;, or &quot;deploy the included unikernel image&quot;. The advantage is that the commands are already authenticated, and there is no need for developing an ad-hoc protocol on top of the TLS session. The resource limits, assigned by the authority, are also part of the certificate chain - i.e. the number of unikernels, access to network bridges, available accumulated memory, accumulated size for block devices, are constrained by the certificate chain presented to the server, and currently running unikernels. The names of the chain are used for access control - if Alice and Bob have intermediate certificates from the same CA, neither Alice may manage Bob's unikernels, nor Bob may manage Alice's unikernels. I'm using albatross since 2.5 years in production on two physical machines with ~20 unikernels total (multiple users, multiple administrative domains), and it works stable and is much nicer to deal with than <code>scp</code> and custom hacked shell scripts.</p>
|
|
||||||
<h2 id="why-0.7">Why 0.7?</h2>
|
|
||||||
<p>There are still some missing pieces in our ocaml-x509 implementation, namely modern ECC certificates (depending on elliptic curve primitives not yet available in OCaml), RSA-PSS signing (should be straightforward), PKCS 12 (there is a <a href="https://github.com/mirleft/ocaml-x509/pull/114">pull request</a>, but this should wait until asn1-combinators supports the <code>ANY defined BY</code> construct to cleanup the code), ...
|
|
||||||
Once these features are supported, the library should likely be named PKCS since it supports more than X.509, and released as 1.0.</p>
|
|
||||||
<p>The 0.7 release series moved a lot of modules and function names around, thus it is a major breaking release. By using a map instead of lists for extensions, GeneralName, ..., the API was further revised - invariants that each extension key (an ASN.1 object identifier) may occur at most once are now enforced. By not leaking exceptions through the public interface, the API is easier to use safely - see <a href="https://github.com/mmaker/ocaml-letsencrypt/commit/dc53518f46310f384c9526b1d96a8e8f815a09c7">let's encrypt</a>, <a href="https://git.robur.io/?p=openvpn.git;a=commitdiff;h=929c53116c1438ba1214f53df7506d32da566ccc">openvpn</a>, <a href="https://github.com/yomimono/ocaml-certify/pull/17">certify</a>, <a href="https://github.com/mirleft/ocaml-tls/pull/394">tls</a>, <a href="https://github.com/mirage/capnp-rpc/pull/158">capnp</a>, <a href="https://github.com/hannesm/albatross/commit/50ed6a8d1ead169b3e322aaccb469e870ad72acc">albatross</a>.</p>
|
|
||||||
<p>I intended in 0.7.0 to have much more precise types, esp. for the SubjectAlternativeName (SAN) extension that uses a GeneralName, but it turns out the GeneralName is as well used for NameConstraints (NC) in a different way -- IP in SAN is an IPv4 or IPv6 address, in CN it is the IP/netmask; DNS is a domain name in SAN, in CN it is a name starting with a leading dot (i.e. &quot;.example.com&quot;), which is not a valid domain name. In 0.7.1, based on a bug report, I had to revert these variants and use less precise types.</p>
|
|
||||||
<h2 id="conclusion">Conclusion</h2>
|
|
||||||
<p>The work on X.509 was sponsored by <a href="http://ocamllabs.io/">OCaml Labs</a>. You can support our work at robur by a <a href="https://robur.io/Donate">donation</a>, which we will use to work on our OCaml and MirageOS projects. You can also reach out to us to realize commercial products.</p>
|
|
||||||
<p>I'm interested in feedback, either via <strike><a href="https://twitter.com/h4nnes">twitter</a></strike> <a href="https://mastodon.social/@hannesm">hannesm@mastodon.social</a> or via eMail.</p>
|
|
||||||
</content><category scheme="https://hannes.robur.coop/tags/tls" term="tls"/><category scheme="https://hannes.robur.coop/tags/security" term="security"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:f2cf2a6a-8eef-5c2c-be03-d81a1bf0f066</id><title type="text">X509 0.7</title><updated>2021-11-19T18:04:52-00:00</updated><author><name>hannes</name></author></entry></feed>
|
|
||||||
|
|
@ -1,5 +1,6 @@
|
||||||
<!DOCTYPE html>
|
<!DOCTYPE html>
|
||||||
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>full stack engineer</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="full stack engineer" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="flex-container"><div class="list-group listing"><a href="/Posts/Albatross" class="list-group-item"><h2 class="list-group-item-heading">Deploying reproducible unikernels with albatross</h2><span class="author">Written by hannes</span> <time>2022-11-17</time><br/><p class="list-group-item-text abstract"><p>fleet management for MirageOS unikernels using a mutually authenticated TLS handshake</p>
|
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>full stack engineer</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="full stack engineer" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="flex-container"><div class="list-group listing"><a href="/Posts/TCP-ns" class="list-group-item"><h2 class="list-group-item-heading">Re-developing TCP from the grounds up</h2><span class="author">Written by hannes</span> <time>2023-11-28</time><br/><p class="list-group-item-text abstract"><p>Core Internet protocols require operational experiments, even if formally specified</p>
|
||||||
|
</p></a><a href="/Posts/Albatross" class="list-group-item"><h2 class="list-group-item-heading">Deploying reproducible unikernels with albatross</h2><span class="author">Written by hannes</span> <time>2022-11-17</time><br/><p class="list-group-item-text abstract"><p>fleet management for MirageOS unikernels using a mutually authenticated TLS handshake</p>
|
||||||
</p></a><a href="/Posts/OpamMirror" class="list-group-item"><h2 class="list-group-item-heading">Mirroring the opam repository and all tarballs</h2><span class="author">Written by hannes</span> <time>2022-09-29</time><br/><p class="list-group-item-text abstract"><p>Re-developing an opam cache from scratch, as a MirageOS unikernel</p>
|
</p></a><a href="/Posts/OpamMirror" class="list-group-item"><h2 class="list-group-item-heading">Mirroring the opam repository and all tarballs</h2><span class="author">Written by hannes</span> <time>2022-09-29</time><br/><p class="list-group-item-text abstract"><p>Re-developing an opam cache from scratch, as a MirageOS unikernel</p>
|
||||||
</p></a><a href="/Posts/Monitoring" class="list-group-item"><h2 class="list-group-item-heading">All your metrics belong to influx</h2><span class="author">Written by hannes</span> <time>2022-03-08</time><br/><p class="list-group-item-text abstract"><p>How to monitor your MirageOS unikernel with albatross and monitoring-experiments</p>
|
</p></a><a href="/Posts/Monitoring" class="list-group-item"><h2 class="list-group-item-heading">All your metrics belong to influx</h2><span class="author">Written by hannes</span> <time>2022-03-08</time><br/><p class="list-group-item-text abstract"><p>How to monitor your MirageOS unikernel with albatross and monitoring-experiments</p>
|
||||||
</p></a><a href="/Posts/Deploy" class="list-group-item"><h2 class="list-group-item-heading">Deploying binary MirageOS unikernels</h2><span class="author">Written by hannes</span> <time>2021-06-30</time><br/><p class="list-group-item-text abstract"><p>Finally, we provide reproducible binary MirageOS unikernels together with packages to reproduce them and setup your own builder</p>
|
</p></a><a href="/Posts/Deploy" class="list-group-item"><h2 class="list-group-item-heading">Deploying binary MirageOS unikernels</h2><span class="author">Written by hannes</span> <time>2021-06-30</time><br/><p class="list-group-item-text abstract"><p>Finally, we provide reproducible binary MirageOS unikernels together with packages to reproduce them and setup your own builder</p>
|
||||||
|
|
|
||||||
BIN
static/img/a.ns.all.png
Normal file
BIN
static/img/a.ns.all.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 239 KiB |
BIN
static/img/a.ns.mtcp-utcp.png
Normal file
BIN
static/img/a.ns.mtcp-utcp.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 190 KiB |
BIN
static/img/a.ns.mtcp.png
Normal file
BIN
static/img/a.ns.mtcp.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 360 KiB |
BIN
static/img/a.ns.utcp-ev.png
Normal file
BIN
static/img/a.ns.utcp-ev.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 151 KiB |
BIN
static/img/a.ns.utcp-fixed.png
Normal file
BIN
static/img/a.ns.utcp-fixed.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 130 KiB |
|
|
@ -1,5 +1,6 @@
|
||||||
<!DOCTYPE html>
|
<!DOCTYPE html>
|
||||||
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>full stack engineer</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="full stack engineer" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="flex-container"><div class="list-group listing"><a href="/Posts/Albatross" class="list-group-item"><h2 class="list-group-item-heading">Deploying reproducible unikernels with albatross</h2><span class="author">Written by hannes</span> <time>2022-11-17</time><br/><p class="list-group-item-text abstract"><p>fleet management for MirageOS unikernels using a mutually authenticated TLS handshake</p>
|
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>full stack engineer</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="full stack engineer" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="flex-container"><div class="list-group listing"><a href="/Posts/TCP-ns" class="list-group-item"><h2 class="list-group-item-heading">Re-developing TCP from the grounds up</h2><span class="author">Written by hannes</span> <time>2023-11-28</time><br/><p class="list-group-item-text abstract"><p>Core Internet protocols require operational experiments, even if formally specified</p>
|
||||||
|
</p></a><a href="/Posts/Albatross" class="list-group-item"><h2 class="list-group-item-heading">Deploying reproducible unikernels with albatross</h2><span class="author">Written by hannes</span> <time>2022-11-17</time><br/><p class="list-group-item-text abstract"><p>fleet management for MirageOS unikernels using a mutually authenticated TLS handshake</p>
|
||||||
</p></a><a href="/Posts/OpamMirror" class="list-group-item"><h2 class="list-group-item-heading">Mirroring the opam repository and all tarballs</h2><span class="author">Written by hannes</span> <time>2022-09-29</time><br/><p class="list-group-item-text abstract"><p>Re-developing an opam cache from scratch, as a MirageOS unikernel</p>
|
</p></a><a href="/Posts/OpamMirror" class="list-group-item"><h2 class="list-group-item-heading">Mirroring the opam repository and all tarballs</h2><span class="author">Written by hannes</span> <time>2022-09-29</time><br/><p class="list-group-item-text abstract"><p>Re-developing an opam cache from scratch, as a MirageOS unikernel</p>
|
||||||
</p></a><a href="/Posts/Monitoring" class="list-group-item"><h2 class="list-group-item-heading">All your metrics belong to influx</h2><span class="author">Written by hannes</span> <time>2022-03-08</time><br/><p class="list-group-item-text abstract"><p>How to monitor your MirageOS unikernel with albatross and monitoring-experiments</p>
|
</p></a><a href="/Posts/Monitoring" class="list-group-item"><h2 class="list-group-item-heading">All your metrics belong to influx</h2><span class="author">Written by hannes</span> <time>2022-03-08</time><br/><p class="list-group-item-text abstract"><p>How to monitor your MirageOS unikernel with albatross and monitoring-experiments</p>
|
||||||
</p></a><a href="/Posts/Deploy" class="list-group-item"><h2 class="list-group-item-heading">Deploying binary MirageOS unikernels</h2><span class="author">Written by hannes</span> <time>2021-06-30</time><br/><p class="list-group-item-text abstract"><p>Finally, we provide reproducible binary MirageOS unikernels together with packages to reproduce them and setup your own builder</p>
|
</p></a><a href="/Posts/Deploy" class="list-group-item"><h2 class="list-group-item-heading">Deploying binary MirageOS unikernels</h2><span class="author">Written by hannes</span> <time>2021-06-30</time><br/><p class="list-group-item-text abstract"><p>Finally, we provide reproducible binary MirageOS unikernels together with packages to reproduce them and setup your own builder</p>
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,6 @@
|
||||||
<!DOCTYPE html>
|
<!DOCTYPE html>
|
||||||
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>full stack engineer</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="full stack engineer" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="flex-container"><div class="list-group listing"><a href="/Posts/Traceroute" class="list-group-item"><h2 class="list-group-item-heading">Traceroute</h2><span class="author">Written by hannes</span> <time>2020-06-24</time><br/><p class="list-group-item-text abstract"><p>A MirageOS unikernel which traces the path between itself and a remote host.</p>
|
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>full stack engineer</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="full stack engineer" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="flex-container"><div class="list-group listing"><a href="/Posts/TCP-ns" class="list-group-item"><h2 class="list-group-item-heading">Re-developing TCP from the grounds up</h2><span class="author">Written by hannes</span> <time>2023-11-28</time><br/><p class="list-group-item-text abstract"><p>Core Internet protocols require operational experiments, even if formally specified</p>
|
||||||
|
</p></a><a href="/Posts/Traceroute" class="list-group-item"><h2 class="list-group-item-heading">Traceroute</h2><span class="author">Written by hannes</span> <time>2020-06-24</time><br/><p class="list-group-item-text abstract"><p>A MirageOS unikernel which traces the path between itself and a remote host.</p>
|
||||||
</p></a><a href="/Posts/DnsServer" class="list-group-item"><h2 class="list-group-item-heading">Deploying authoritative OCaml-DNS servers as MirageOS unikernels</h2><span class="author">Written by hannes</span> <time>2019-12-23</time><br/><p class="list-group-item-text abstract"><p>A tutorial how to deploy authoritative name servers, let's encrypt, and updating entries from unix services.</p>
|
</p></a><a href="/Posts/DnsServer" class="list-group-item"><h2 class="list-group-item-heading">Deploying authoritative OCaml-DNS servers as MirageOS unikernels</h2><span class="author">Written by hannes</span> <time>2019-12-23</time><br/><p class="list-group-item-text abstract"><p>A tutorial how to deploy authoritative name servers, let's encrypt, and updating entries from unix services.</p>
|
||||||
</p></a><a href="/Posts/DNS" class="list-group-item"><h2 class="list-group-item-heading">My 2018 contains robur and starts with re-engineering DNS</h2><span class="author">Written by hannes</span> <time>2018-01-11</time><br/><p class="list-group-item-text abstract"><p>New year brings new possibilities and a new environment. I've been working on the most Widely deployed key-value store, the domain name system. Primary and secondary name services are available, including dynamic updates, notify, and tsig authentication.</p>
|
</p></a><a href="/Posts/DNS" class="list-group-item"><h2 class="list-group-item-heading">My 2018 contains robur and starts with re-engineering DNS</h2><span class="author">Written by hannes</span> <time>2018-01-11</time><br/><p class="list-group-item-text abstract"><p>New year brings new possibilities and a new environment. I've been working on the most Widely deployed key-value store, the domain name system. Primary and secondary name services are available, including dynamic updates, notify, and tsig authentication.</p>
|
||||||
</p></a><a href="/Posts/Syslog" class="list-group-item"><h2 class="list-group-item-heading">Exfiltrating log data using syslog</h2><span class="author">Written by hannes</span> <time>2016-11-05</time><br/><p class="list-group-item-text abstract"><p>sometimes preservation of data is useful</p>
|
</p></a><a href="/Posts/Syslog" class="list-group-item"><h2 class="list-group-item-heading">Exfiltrating log data using syslog</h2><span class="author">Written by hannes</span> <time>2016-11-05</time><br/><p class="list-group-item-text abstract"><p>sometimes preservation of data is useful</p>
|
||||||
|
|
|
||||||
3
tags/tcp
Normal file
3
tags/tcp
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
<!DOCTYPE html>
|
||||||
|
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>full stack engineer</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="full stack engineer" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="flex-container"><div class="list-group listing"><a href="/Posts/TCP-ns" class="list-group-item"><h2 class="list-group-item-heading">Re-developing TCP from the grounds up</h2><span class="author">Written by hannes</span> <time>2023-11-28</time><br/><p class="list-group-item-text abstract"><p>Core Internet protocols require operational experiments, even if formally specified</p>
|
||||||
|
</p></a></div></div></div></main></body></html>
|
||||||
Loading…
Reference in a new issue