the BSDs in the AI Age

[Pete Wright]

On 4/2/26 10:16, Justin Sherrill wrote:
> On Thu, Apr 2, 2026 at 9:15 AM George Rosamond 
> <george at ceetonetechnology.com <mailto:george at ceetonetechnology.com>> wrote:
> 
>     I want to initiate a thread on the "BSDs and AI today."
> 
>     I'm looking to do a presentation on this in the summer for NYC*BUG.
> 
> 
> Two quantifiable measures, though they will change by the time you are 
> doing a summer presentation:
> 
> - What models and software run on BSDs?  There's all sorts of tooling 
> for accessing LLMs, but how much have made it to BSD?
> 

its not the model per-se but drivers and library support between freebsd 
and most likely the expensive Nvidia GPU you can't afford to perform the 
math against the model.  i would not recommend trying to do this with 
any bsd at this point, while it may be possible you'll be swimming 
upstream best case scenario.

regarding the tooling and ecosystem in general, there is plenty of 
opportunity there.  most people use python to interact with models - i 
use freebsd and hugging face as well as other popular libraries to 
interact with LLM api's.  they work mostly ok but require more effort to 
maintain then linux, this is mostly due to developers integrating rust 
into python so you'll need to do lots of compiling.

then there are things like vector databases that may be well suited to 
freebsd (these tend to be opensearch based things, but can also be 
hosted in postgresql for example too).


at the end of the day its just computers once you remove the hype...and 
lots of immature code lol.

-pete

-- 
Pete Wright
pete at nomadlogic.org