[talk] APU2 post-mortem options

Fri May 12 17:33:31 EDT 2023

Last I checked, Protectli was just rebranded Yanbling/Minisys hardware.  You can find them on AliExpress.
Qotom is a different Chinese ODM who you can also find on AliExpress.

Patrick at STH has been doing a good job detailing what’s available.   I’d check STH if you’re looking for an inexpensive platform from China.

If you’re looking just a bit into the future, there is a new Atom coming: 
https://www.intel.com/content/www/us/en/products/sku/229206/intel-atom-processor-c1110-6m-cache-2-10-ghz/specifications.html

While I can’t say much about it, Silicom has leaked that this is based on Gracemont cores (this is the ‘little’ core in Alder Lake).
https://www.silicom-usa.com/pr/4g-5g-products/4g-5g-appliances/valencia-network-appliance-series/

This is not your father’s Atom.  It’s not what you’re used to for performance from an Atom.  Four cores of Gracemont deliver 80% more performance than two Skylake cores.

https://chipsandcheese.com/2021/12/21/gracemont-revenge-of-the-atom-cores/
https://www.anandtech.com/show/16881/a-deep-dive-into-intels-alder-lake-microarchitectures/4
[…]

Four cores of Gracemont and four i226 2.5gbps Ethernet devices will run fanless at over 40C ambient.

In addition to a much more performant front-end and scheduler, the C1110 also supports AVX/AVX2 and even VAES/VPCMULQDQ if you encode the instructions using VEX.  This makes AES-GCM, AES-CBC/SHA-256 and ChaCha20/Poly1305 go real fast.

As an example: Using VPP Intel will show it doing IPsec (AES-128-GCM) at … well 14.99gbps on a single core.  But you don’t care about VPP, you want to hear about FreeBSD performance, right?

I’m not going to quote benchmarking on the C1110 (yet), but as an illustration of why you might want to consider having AVX/AVX2 in your next platform:

On a Ryzen 5 5600, using 4 FreeBSD VMs, each with 4 guest-CPUs, 4GB RAM, 2 SR-IOV passthrough ConnectX virtual function NICs
connected via a Mellanox ConnectX-5EN 25Gbps PCIe-3 which features an embedded switch capable of delivering 50Gbps traffic between the virtual functions and a single TCP stream via iperf:

OCF below is what you can get with FreeBSD
IIMB is the BSD-licensed Intel IPsec-MB library, ported to the OCF framework.

IPsec-ESP (AES-128-GCM)
- OCF: sync: 4.88Gbps async: 4.46Gbps
- IIMB: sync: 5.48Gbps async: 7.68Gbps

IPsec-ESP (AES-256-CBC,HMAC-SHA256)
- OCF async: 4.2Gbps
- IIMB async: 5.3Gbps

ChaCha20/Poly1305 also gets a big lift from AVX/AVX2.  
WireGuard
- OCF: 4.4Gbps
- IIMB: 6.0Gbps

Notes:
- WireGuard consumes all CPUs with low idle time (<15%), whereas IPsec and OpenVPN uses 3 CPUs for network and encryption workloads
(20% idle) and are otherwise 60% idle during this test.

- Linux native WireGuard attained 7.5Gbps on the same virtual setup

As a bit more illustration of what is possible with some software work:

OpenVPN DCO (AES-256-GCM):
- OCF: sync: 4.1Gbps async: 6.0Gbps
- IIMB: sync: 4.9Gbps async: 10Gbps (*)

OpenVPN DCO (ChaCha20-Poly1305):
- IIMB: sync: 2.9Gbps async: 5.1Gbps

(*) peak throughput for OpenVPN DCO was observed to be 12Gbps 

This is a big reason we did the work to bring IPsec-MB to pfSense Plus. https://redmine.pfsense.org/issues/14291
There was a talk at AsiaBSDCon about this, by Leon Dang, presented by Kristof Provost
https://2023.asiabsdcon.org/program.html.en

Some of the results above are from that paper.

I’m not saying you want an Alder Lake for your server, you don’t, in part because the FreeBSD scheduler doesn’t do well on a big/little architecture, and there isn’t anyone in the project to work on that.  But if you cut the big (Arizona Beach) *or* little cores (Sapphire Rapids) off, then FreeBSD can run very well on the result.

Responding to Charles Sprickman:
> I recall in IRC there was some talk that ARM, while in theory is "open", is
> chock-full of proprietary stuff when you get down to trying to buy anything
> that looks like a commodity mini-PC, right?

They’re not PCs, if that’s what you mean, but it’s straight-forward to get FreeBSD running on them.  Then the work of writing drivers for that vendor’s devices begins.  They tend to ship drivers for linux, but not for anything else.

Jim

> On May 12, 2023, at 2:39 PM, George Rosamond <george at ceetonetechnology.com> wrote:
> 
> On 5/12/23 15:59, Isaac (.ike) Levy wrote:
>> I have one option in hand,
>> 
>>> On May 12, 2023, at 2:09 PM, George Rosamond <george at ceetonetechnology.com> wrote:
>>> 
>>> ideal hardware options after APU2/PCEngines go away
>> 
>> I've been hoarding PCEngines boards for years, please ping me off list and have bank routing numbers in hand, and don't get sassy when I state my prices.
>> 
>> I won't begin selling my APU2's until I depelete my supply of APU/i386 boards.
>> 
>> I may have a Soekris or two left as well, if you're lucky.  And if you'd like, I am able to paint the APU machines pea green, as an add on.
>> 
>> If you've read this far and don't yet understand I'm joking, please know I write this holding back tears that my favorite compute platform of all time is gone...  And Pascal is the best.
> 
> Ooops. You were banned after I read the first line.  Hope this email
> finds you well!
> 
> Also in seriousness.. I do remember the last analog-based trading phone
> systems slowly dying many years ago. A few people in NYC had decent
> supplies to keep them going. Everyone went digital obviously, but at
> that point there was a reliability in copper pairs over anything
> internet based. Dropping packets was not an option for traders.
> 
> I didn't stop using Soekris for at least console servers until a few
> years ago... and I assume my APU2s will keep chugging away for many
> years. But just like the Soekris' specs were a drag, the APU2s were
> showing it for a while already.
> 
> Getting those mSATAs was rough, and the underpowered CPUs were killing
> me. Plus the heat dissipation was problematic. If anyone hasn't heard
> the story, one production APU2 I had hit 113C.. Don't tell me "you could
> fry an egg on it".. since it would burn too quickly.
> 
> I have a hard time opting for a replacement system meant for the
> desktop. I don't want your wifi/bluetooth/HDMI or even VGA. I don't want
> cabinet space wasted with those useless extras. And give me db9 or give
> me death.
> 
> I think I speak for many others when I say it became natural and
> comfortable to just order a bunch of APU2s when in need, and they always
> seemed to come in cheaper than their quoted estimate, and they arrived
> faster than expected.
> 
> And I had interacted with Pascal a number of times going way back, and
> really appreciated what one engineer could do to satisfy the needs for
> so many of our types. And PCEngines was a sponsor at NYCBSDCon 2008*
> 
> g
> 
> * https://www.google.com/url?q=http://www.nycbsdcon.org/2008/&source=gmail-imap&ust=1684528879000000&usg=AOvVaw21__LKxrafaNPaaMbpwgus
> 
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org <mailto:talk at lists.nycbug.org>
> https://www.google.com/url?q=https://lists.nycbug.org:8443/mailman/listinfo/talk&source=gmail-imap&ust=1684528879000000&usg=AOvVaw3GLllRw5MRdhQkahcaQOr_

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20230512/06507b31/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-1.png
Type: image/png
Size: 43856 bytes
Desc: not available
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20230512/06507b31/attachment.png>