hannes/hannes.robur.coop
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

updated from main (commit 5316960bcb)

Browse source
This commit is contained in:
Canopy bot 2023-11-29 12:45:31 +00:00
parent a03a114b3f
commit 1a8e2df853
2 changed files with 5 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
Posts/TCP-ns
View file

@ -1,5 +1,5 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Re-developing TCP from the grounds up</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Re-developing TCP from the grounds up" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Re-developing TCP from the grounds up</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a><a href="/tags/tcp" class="tag">tcp</a></div><span class="date">Published: 2023-11-28 (last updated: 2023-11-28)</span><article><p>The <a href="https://en.wikipedia.org/wiki/Transmission_Control_Protocol">Transmission Control Protocol (TCP)</a> is one of the main Internet protocols. Usually spoken on top of the Internet Protocol (legacy version 4 or version 6), it provides a reliable, ordered, and error-checked stream of octets. When an application uses TCP, they get these properties for free (in contrast to UDP).</p>
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Re-developing TCP from the grounds up</title><meta charset="UTF-8"/><link rel="stylesheet" href="/static/css/style.css"/><link rel="stylesheet" href="/static/css/highlight.css"/><script src="/static/js/highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script><link rel="alternate" href="/atom" title="Re-developing TCP from the grounds up" type="application/atom+xml"/><meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover"/></head><body><nav class="navbar navbar-default navbar-fixed-top"><div class="container"><div class="navbar-header"><a class="navbar-brand" href="/Posts">full stack engineer</a></div><div class="collapse navbar-collapse collapse"><ul class="nav navbar-nav navbar-right"><li><a href="/About"><span>About</span></a></li><li><a href="/Posts"><span>Posts</span></a></li></ul></div></div></nav><main><div class="flex-container"><div class="post"><h2>Re-developing TCP from the grounds up</h2><span class="author">Written by hannes</span><br/><div class="tags">Classified under: <a href="/tags/mirageos" class="tag">mirageos</a><a href="/tags/protocol" class="tag">protocol</a><a href="/tags/tcp" class="tag">tcp</a></div><span class="date">Published: 2023-11-28 (last updated: 2023-11-29)</span><article><p>The <a href="https://en.wikipedia.org/wiki/Transmission_Control_Protocol">Transmission Control Protocol (TCP)</a> is one of the main Internet protocols. Usually spoken on top of the Internet Protocol (legacy version 4 or version 6), it provides a reliable, ordered, and error-checked stream of octets. When an application uses TCP, they get these properties for free (in contrast to UDP).</p>
<p>As common for Internet protocols, also TCP is specified in a series of so-called requests for comments (RFC), the latest revised version from August 2022 is <a href="https://datatracker.ietf.org/doc/html/rfc9293">RFC 9293</a>, the initial one was <a href="https://datatracker.ietf.org/doc/html/rfc793">RFC 793</a> from September 1981.</p>
<h1 id="my-brief-personal-tcp-story">My brief personal TCP story</h1>
<p>My interest in TCP started back in 2006 when we worked on a <a href="https://github.com/dylan-hackers/network-night-vision">network stack in Dylan</a> (these days abandoned) - ever since then I wanted to understand the implementation tradeoffs in more detail - including attacks and how to prevent a TCP stack from being vulnerable.</p>
@ -15,7 +15,7 @@
<p><a href="/static/img/a.ns.mtcp.png"><img src="/static/img/a.ns.mtcp.png" width="750" /></a></p>
<p>Now, after switching over to µTCP, graphed below, there's much fewer network utilization and the memory limit is only reached after 36 hours, which is a great result. Though, still it is not very satisfying that the unikernel leaks memory. Both graphs contain on their left side a few hours of mirage-tcpip, and shortly after 20:00 on Nov 23rd µTCP got deployed.</p>
<p><a href="/static/img/a.ns.mtcp-utcp.png"><img src="/static/img/a.ns.mtcp-utcp.png" width="750" /></a></p>
<p>Investigating the involved parts showed that a TCP connection that was never established has been registered at the MirageOS layer, but the pure core does not expose an event from the received RST that the connection has been cancelled. This means the MirageOS layer piles up all the connection attempts, and doesn't inform the application that the connection couldn't be established. Once this was well understood, developing the <a href="https://github.com/robur-coop/utcp/commit/67fc49468e6b75b96a481ebe44dd11ce4bb76e6c">required code changes</a> was straightforward. The graph shows that the fix was deployed at 15:25. The memory usage is constant afterwards, but the network utilization increased enormously.</p>
<p>Investigating the involved parts showed that a TCP connection that was never established has been registered at the MirageOS layer, but the pure core does not expose an event from the received RST that the connection has been cancelled. This means the MirageOS layer piles up all the connection attempts, and doesn't inform the application that the connection couldn't be established. Note that the MirageOS layer is not code derived from the formal model, but boilerplate for (a) effectful side-effects (IO) and (b) meeting the needs of the <a href="https://github.com/mirage/mirage-tcpip/blob/v8.0.0/src/core/tcp.ml">TCP.S</a> module type (so that µTCP can be used as a drop-in replacement for mirage-tcpip). Once this was well understood, developing the <a href="https://github.com/robur-coop/utcp/commit/67fc49468e6b75b96a481ebe44dd11ce4bb76e6c">required code changes</a> was straightforward. The graph shows that the fix was deployed at 15:25. The memory usage is constant afterwards, but the network utilization increased enormously.</p>
<p><a href="/static/img/a.ns.utcp-ev.png"><img src="/static/img/a.ns.utcp-ev.png" width="750" /></a></p>
<p>Now, the network utilization is unwanted. This was hidden by the application waiting forever that the TCP connection getting established. Our bugfix uncovered another issue, a tight loop:</p>
<ul>

6
atom
View file

@ -1,4 +1,4 @@
<feed xmlns="http://www.w3.org/2005/Atom"><link href="https://hannes.robur.coop/atom" rel="self"/><id>urn:uuid:981361ca-e71d-4997-a52c-baeee78e4156</id><title type="text">full stack engineer</title><updated>2023-11-28T21:26:50-00:00</updated><entry><summary type="html">&lt;p&gt;Core Internet protocols require operational experiments, even if formally specified&lt;/p&gt;
<feed xmlns="http://www.w3.org/2005/Atom"><link href="https://hannes.robur.coop/atom" rel="self"/><id>urn:uuid:981361ca-e71d-4997-a52c-baeee78e4156</id><title type="text">full stack engineer</title><updated>2023-11-29T12:45:29-00:00</updated><entry><summary type="html">&lt;p&gt;Core Internet protocols require operational experiments, even if formally specified&lt;/p&gt;
</summary><published>2023-11-28T21:17:01-00:00</published><link href="/Posts/TCP-ns" rel="alternate"/><content type="html">&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Transmission_Control_Protocol&quot;&gt;Transmission Control Protocol (TCP)&lt;/a&gt; is one of the main Internet protocols. Usually spoken on top of the Internet Protocol (legacy version 4 or version 6), it provides a reliable, ordered, and error-checked stream of octets. When an application uses TCP, they get these properties for free (in contrast to UDP).&lt;/p&gt;
&lt;p&gt;As common for Internet protocols, also TCP is specified in a series of so-called requests for comments (RFC), the latest revised version from August 2022 is &lt;a href=&quot;https://datatracker.ietf.org/doc/html/rfc9293&quot;&gt;RFC 9293&lt;/a&gt;, the initial one was &lt;a href=&quot;https://datatracker.ietf.org/doc/html/rfc793&quot;&gt;RFC 793&lt;/a&gt; from September 1981.&lt;/p&gt;
&lt;h1 id=&quot;my-brief-personal-tcp-story&quot;&gt;My brief personal TCP story&lt;/h1&gt;
@ -15,7 +15,7 @@
&lt;p&gt;&lt;a href=&quot;/static/img/a.ns.mtcp.png&quot;&gt;&lt;img src=&quot;/static/img/a.ns.mtcp.png&quot; width=&quot;750&quot; /&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Now, after switching over to µTCP, graphed below, there's much fewer network utilization and the memory limit is only reached after 36 hours, which is a great result. Though, still it is not very satisfying that the unikernel leaks memory. Both graphs contain on their left side a few hours of mirage-tcpip, and shortly after 20:00 on Nov 23rd µTCP got deployed.&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;/static/img/a.ns.mtcp-utcp.png&quot;&gt;&lt;img src=&quot;/static/img/a.ns.mtcp-utcp.png&quot; width=&quot;750&quot; /&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Investigating the involved parts showed that a TCP connection that was never established has been registered at the MirageOS layer, but the pure core does not expose an event from the received RST that the connection has been cancelled. This means the MirageOS layer piles up all the connection attempts, and doesn't inform the application that the connection couldn't be established. Once this was well understood, developing the &lt;a href=&quot;https://github.com/robur-coop/utcp/commit/67fc49468e6b75b96a481ebe44dd11ce4bb76e6c&quot;&gt;required code changes&lt;/a&gt; was straightforward. The graph shows that the fix was deployed at 15:25. The memory usage is constant afterwards, but the network utilization increased enormously.&lt;/p&gt;
&lt;p&gt;Investigating the involved parts showed that a TCP connection that was never established has been registered at the MirageOS layer, but the pure core does not expose an event from the received RST that the connection has been cancelled. This means the MirageOS layer piles up all the connection attempts, and doesn't inform the application that the connection couldn't be established. Note that the MirageOS layer is not code derived from the formal model, but boilerplate for (a) effectful side-effects (IO) and (b) meeting the needs of the &lt;a href=&quot;https://github.com/mirage/mirage-tcpip/blob/v8.0.0/src/core/tcp.ml&quot;&gt;TCP.S&lt;/a&gt; module type (so that µTCP can be used as a drop-in replacement for mirage-tcpip). Once this was well understood, developing the &lt;a href=&quot;https://github.com/robur-coop/utcp/commit/67fc49468e6b75b96a481ebe44dd11ce4bb76e6c&quot;&gt;required code changes&lt;/a&gt; was straightforward. The graph shows that the fix was deployed at 15:25. The memory usage is constant afterwards, but the network utilization increased enormously.&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;/static/img/a.ns.utcp-ev.png&quot;&gt;&lt;img src=&quot;/static/img/a.ns.utcp-ev.png&quot; width=&quot;750&quot; /&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Now, the network utilization is unwanted. This was hidden by the application waiting forever that the TCP connection getting established. Our bugfix uncovered another issue, a tight loop:&lt;/p&gt;
&lt;ul&gt;
@ -35,7 +35,7 @@
&lt;p&gt;We'll take some more time to investigate issues of µTCP in production, plan to write further documentation and blog posts, and hopefully soon are ready for an initial public release. In the meantime, you can follow our development repository.&lt;/p&gt;
&lt;p&gt;We at &lt;a href=&quot;https://robur.coop&quot;&gt;robur&lt;/a&gt; are working as a collective since 2018, on public funding, commercial contracts, and donations. Our mission is to get sustainable, robust and secure MirageOS unikernels developed and deployed. Running your own digital communication infrastructure should be easy, including trustworthy binaries and smooth upgrades. You can help us continuing our work by &lt;a href=&quot;https://aenderwerk.de/donate/&quot;&gt;donating&lt;/a&gt; (select robur from the drop-down, or put &amp;quot;donation robur&amp;quot; in the purpose of the bank transfer).&lt;/p&gt;
&lt;p&gt;If you have any questions, reach us best via eMail to team AT robur DOT coop.&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/tcp" term="tcp"/><category scheme="https://hannes.robur.coop/tags/protocol" term="protocol"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:96688956-0808-5d44-b795-1d64cbb4f947</id><title type="text">Re-developing TCP from the grounds up</title><updated>2023-11-28T21:26:50-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;fleet management for MirageOS unikernels using a mutually authenticated TLS handshake&lt;/p&gt;
</content><category scheme="https://hannes.robur.coop/tags/tcp" term="tcp"/><category scheme="https://hannes.robur.coop/tags/protocol" term="protocol"/><category scheme="https://hannes.robur.coop/tags/mirageos" term="mirageos"/><id>urn:uuid:96688956-0808-5d44-b795-1d64cbb4f947</id><title type="text">Re-developing TCP from the grounds up</title><updated>2023-11-29T12:45:29-00:00</updated><author><name>hannes</name></author></entry><entry><summary type="html">&lt;p&gt;fleet management for MirageOS unikernels using a mutually authenticated TLS handshake&lt;/p&gt;
</summary><published>2022-11-17T12:41:11-00:00</published><link href="/Posts/Albatross" rel="alternate"/><content type="html">&lt;p&gt;EDIT (2023-05-16): Updated with albatross release version 2.0.0.&lt;/p&gt;
&lt;h2 id=&quot;deploying-mirageos-unikernels&quot;&gt;Deploying MirageOS unikernels&lt;/h2&gt;
&lt;p&gt;More than five years ago, I posted &lt;a href=&quot;/Posts/VMM&quot;&gt;how to deploy MirageOS unikernels&lt;/a&gt;. My motivation to work on this topic is that I'm convinced of reduced complexity, improved security, and more sustainable resource footprint of MirageOS unikernels, and want to ease deployment thereof. More than one year ago, I described &lt;a href=&quot;/Posts/Deploy&quot;&gt;how to deploy reproducible unikernels&lt;/a&gt;.&lt;/p&gt;