From jhb at FreeBSD.org  Thu Apr  7 13:32:18 2022
From: jhb at FreeBSD.org (John Baldwin)
Date: Thu, 7 Apr 2022 10:32:18 -0700
Subject: [talk] Does swap still matter?
In-Reply-To: <20220316162535.GA14202@castrovalva.Ivy.NET>
References: <7e41922c-0b0a-fe65-a1c2-81291e5c4201@ceetonetechnology.com>
 <20220316162535.GA14202@castrovalva.Ivy.NET>
Message-ID: <302a8134-c558-5df3-6029-07343f9ccad5@FreeBSD.org>

On 3/16/22 9:25 AM, Miles Nordin wrote:
>> When RAM never seemed adequate for a system in the past, the utility of
>> swap was obvious.
> 
> idea #1:
> 
> The measure of used RAM isn't obvious.  RAM allocated by sbrk or mmap is
> not used until it's written to for the first time.  (or maybe read, I am
> not sure)
> 
> what's more interesting wrt swap is, on some mallocs, like at least older
> GNU malloc, freed RAM is never returned to the OS.  tcmalloc does return
> RAM to the OS, but on some kind of lazy background thread or something.  so
> if a C program uses a whole bunch of RAM at startup then frees it, the physical
> RAM can only actually be freed by writing it to swap (then never reading it),
> or by the process exiting.

[ I know this is an old thread. ]

This is not quite correct in modern allocators.  While allocators generally
do not return virtual address space back to the OS via munmap(), they do
use madvise() (e.g. MADV_FREE on FreeBSD, OS X and Linux have similar
extensions) to let the OS reclaim the backing RAM for free'd memory.

FWIW, I still configure swap on systems I install, but really it's just to
allocate space to hold kernel crashdumps, not because I really need/want
swap.

Swap can save you a bit by not letting the OOM start killing processes
right away but give you time to login and be more selective before the
kernels starts killing processes itself.  For example, if you have a nagios
alert or the like for when swapping begins then having some swap available
might give you time to more gracefully deal with the problem.

-- 
John Baldwin



From spork at bway.net  Thu Apr  7 16:46:40 2022
From: spork at bway.net (Charles Sprickman)
Date: Thu, 7 Apr 2022 16:46:40 -0400
Subject: [talk] Does swap still matter?
In-Reply-To: <302a8134-c558-5df3-6029-07343f9ccad5@FreeBSD.org>
References: <7e41922c-0b0a-fe65-a1c2-81291e5c4201@ceetonetechnology.com>
 <20220316162535.GA14202@castrovalva.Ivy.NET>
 <302a8134-c558-5df3-6029-07343f9ccad5@FreeBSD.org>
Message-ID: <5692714D-B12C-443E-B164-CCD4F015B83A@bway.net>



> On Apr 7, 2022, at 1:32 PM, John Baldwin <jhb at FreeBSD.org> wrote:
> 
> 
> FWIW, I still configure swap on systems I install, but really it's just to
> allocate space to hold kernel crashdumps, not because I really need/want
> swap.

I?m in the same habit, and I?m generally fairly generous on anything where space isn?t scarce (but on the odd ?cloud? VM, I?m much more stingy as storage is pricey).

A few questions, as I?ve been at this long enough that some of my information is probably outdated:

- Is there a formula for how much swap space to allocate to be able to always fit the full kernel dump? I have one server at home that?s had a few issues, and with 8GB of RAM (which generally sees at least half of that being used by zfs ARC), my vmcore files are around 4GB. Curious what I?d see on a host with 64GB or 128GB of RAM.

- With SSDs being so common, especially for ?boot? drives where swap would likely live, are there any special considerations? Do swap partitions get TRIM support? I also assume SSDs make a system going into swap due to physical memory exhaustion suffer far less than it would with spinny drives that are thrashing around between swap at the beginning of the drive and other data being spread out all over the remainder of the drive (but could also eat up your precious write cycles if swap usage is not being monitored).

- Anywhere I have multiple drives, I always mirror my swap, using gmirror. Any drawbacks to mirroring? Is gmirror still the only proper option for this?

- Is swap on zfs advisable in any situations yet?

Charles

-- 
Charles Sprickman
NetEng/SysAdmin
Bway.net - New York's Best Internet www.bway.net
spork at bway.net - 212.982.9800

> 
> -- 
> John Baldwin
> 
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> https://lists.nycbug.org:8443/mailman/listinfo/talk



From jhb at FreeBSD.org  Thu Apr  7 20:16:29 2022
From: jhb at FreeBSD.org (John Baldwin)
Date: Thu, 7 Apr 2022 17:16:29 -0700
Subject: [talk] Does swap still matter?
In-Reply-To: <5692714D-B12C-443E-B164-CCD4F015B83A@bway.net>
References: <7e41922c-0b0a-fe65-a1c2-81291e5c4201@ceetonetechnology.com>
 <20220316162535.GA14202@castrovalva.Ivy.NET>
 <302a8134-c558-5df3-6029-07343f9ccad5@FreeBSD.org>
 <5692714D-B12C-443E-B164-CCD4F015B83A@bway.net>
Message-ID: <740657ba-6fb5-9971-eeb0-a996cefb34d3@FreeBSD.org>

On 4/7/22 1:46 PM, Charles Sprickman wrote:
> 
> 
>> On Apr 7, 2022, at 1:32 PM, John Baldwin <jhb at FreeBSD.org> wrote:
>>
>>
>> FWIW, I still configure swap on systems I install, but really it's just to
>> allocate space to hold kernel crashdumps, not because I really need/want
>> swap.
> 
> I?m in the same habit, and I?m generally fairly generous on anything where space isn?t scarce (but on the odd ?cloud? VM, I?m much more stingy as storage is pricey).
> 
> A few questions, as I?ve been at this long enough that some of my information is probably outdated:
> 
> - Is there a formula for how much swap space to allocate to be able to always fit the full kernel dump? I have one server at home that?s had a few issues, and with 8GB of RAM (which generally sees at least half of that being used by zfs ARC), my vmcore files are around 4GB. Curious what I?d see on a host with 64GB or 128GB of RAM.

Hummm, I don't know of a good one.  minidumps should mean that a crashdump
is less than all of RAM, but it can be a sizable chunk of it, especially
with ZFS.  The last machine I installed from scratch is a laptop with 16G
of RAM and I went with 16G of swap.  I recently upgraded my desktop to 64G
of RAM (good for all the compiling I do), but still went with 16G of swap
one each disk (and a core can't span multiple disks, so it's the single
disk value that counts).  I haven't had a panic on my new desktop yet to
see if it will fit. :-/

> - With SSDs being so common, especially for ?boot? drives where swap would likely live, are there any special considerations? Do swap partitions get TRIM support? I also assume SSDs make a system going into swap due to physical memory exhaustion suffer far less than it would with spinny drives that are thrashing around between swap at the beginning of the drive and other data being spread out all over the remainder of the drive (but could also eat up your precious write cycles if swap usage is not being monitored).

Hmmmmm, I don't know if we try to TRIM swap partitions.  It is true that
SSDs make swapping less terrible.  An anecdote I heard about iPhones
early on is that they swap like mad, but using flash storage means it
still performs ok.  I'm not sure if that is still true of newer versions,
but it wouldn't surprise me as the amount of DRAM in mobile devices has
a power cost (and power is the primary "currency" of a mobile BOM).

In terms of write cycles, in my experience to date (and of folks I've talked
to at other places), that seems a bit overblown as SSDs seem to last a
fairly long time such that it's not really a practical worry.

> - Anywhere I have multiple drives, I always mirror my swap, using gmirror. Any drawbacks to mirroring? Is gmirror still the only proper option for this?

So I don't use gmirror for swap, I just let add them as individual swap
devices in /etc/fstab.  The swap pager in FreeBSD will spread swap
access across multiple devices just fine, and it's not clear that
mirroring is a win for a swap workload to me (you are doing double writes,
so swapping out it slower vs multiple swap devices which can potentially
schedule writes of different data to the disk).  Maybe mirroring would
help with reading the data back in faster perhaps, but usually if you are
reading enough from swap for that to matter you are thrashing and already
hosed.

> - Is swap on zfs advisable in any situations yet?

I create a separate GPT partition for swap separate from ZFS rather than
trying to swap to ZVOLs (assuming that is what you mean).  I suspect that
will perform better generally speaking that a ZVOL for swap.  Also, swap
gets used when you are in a low-memory situation and you ideally want to
avoid lots of complexity in the I/O path to swap to avoid potential low
memory deadlocks (i.e. you need to allocate something like duplicating an
I/O request to cross a gmirror device, or allocating some data structures
in ZFS to deal with writes ZFS needs to do to a zvol) vs just a dumb
write straight to a partition on a disk that can generally avoid malloc.

-- 
John Baldwin



From justin at shiningsilence.com  Tue Apr 19 19:06:26 2022
From: justin at shiningsilence.com (Justin Sherrill)
Date: Tue, 19 Apr 2022 19:06:26 -0400
Subject: [talk] Me, presenting at SEMIBUG
Message-ID: <CA+YR8=3UAaeTR4VDT_c7Tz+=YjGh55vx3qqr5UynVz_b6t4YTA@mail.gmail.com>

I should have written this earlier, but I'm talking about DragonFly and the
Digest for SEMIBUG riiiiight about now, on Jitsi:

https://meet.jit.si/SEMI-BUGDragonFlyBSD
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20220419/e9b8f892/attachment.htm>

From george at ceetonetechnology.com  Tue Apr 19 19:11:24 2022
From: george at ceetonetechnology.com (George Rosamond)
Date: Tue, 19 Apr 2022 19:11:24 -0400
Subject: [talk] Me, presenting at SEMIBUG
In-Reply-To: <CA+YR8=3UAaeTR4VDT_c7Tz+=YjGh55vx3qqr5UynVz_b6t4YTA@mail.gmail.com>
References: <CA+YR8=3UAaeTR4VDT_c7Tz+=YjGh55vx3qqr5UynVz_b6t4YTA@mail.gmail.com>
Message-ID: <900fd880-5a7e-3047-8fea-30dc6ed85846@ceetonetechnology.com>

On 4/19/22 19:06, Justin Sherrill wrote:
> I should have written this earlier, but I'm talking about DragonFly and the
> Digest for SEMIBUG riiiiight about now, on Jitsi:
> 
> https://meet.jit.si/SEMI-BUGDragonFlyBSD
> 

Nice Justin... thanks for the early notice!

You'll have to redo the talk for us first Wednesday in May then...

On May the Fourth Be With You Day.  645 PM EDT.

See you then!

g



From george at ceetonetechnology.com  Tue Apr 19 23:08:42 2022
From: george at ceetonetechnology.com (George Rosamond)
Date: Tue, 19 Apr 2022 23:08:42 -0400
Subject: [talk] Gemini
Message-ID: <e969711b-e4b0-2355-3142-decb35b8e2bb@ceetonetechnology.com>

Anyone tinker with this project?

https://gemini.circumlunar.space/docs/faq.gmi

A lot about it piques my interest...

g



From steve.b at osfda.org  Tue Apr 19 23:32:37 2022
From: steve.b at osfda.org (steve.b at osfda.org)
Date: Tue, 19 Apr 2022 23:32:37 -0400
Subject: [talk] Gemini
In-Reply-To: <e969711b-e4b0-2355-3142-decb35b8e2bb@ceetonetechnology.com>
References: <e969711b-e4b0-2355-3142-decb35b8e2bb@ceetonetechnology.com>
Message-ID: <3a9ea03f-a229-daa7-ad13-054d058f5682@osfda.org>

A devil's argument I came across that pans it...

    https://xn--gckvb8fzb.com/gemini-is-solutionism-at-its-worst

On 4/19/22 11:08 PM, George Rosamond wrote:
> Anyone tinker with this project?
>
> https://gemini.circumlunar.space/docs/faq.gmi
>
> A lot about it piques my interest...
>
> g
>
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> https://lists.nycbug.org:8443/mailman/listinfo/talk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20220419/2a27941b/attachment.htm>

From ike at blackskyresearch.net  Wed Apr 20 15:12:13 2022
From: ike at blackskyresearch.net (Isaac (.ike) Levy)
Date: Wed, 20 Apr 2022 15:12:13 -0400
Subject: [talk] Gemini
In-Reply-To: <3a9ea03f-a229-daa7-ad13-054d058f5682@osfda.org>
References: <e969711b-e4b0-2355-3142-decb35b8e2bb@ceetonetechnology.com>
 <3a9ea03f-a229-daa7-ad13-054d058f5682@osfda.org>
Message-ID: <cd2767de-fe6e-4db6-b1b9-0cea0fca404e@www.fastmail.com>

On Tue, Apr 19, 2022, at 11:32 PM, steve.b at osfda.org wrote:
> A devil's argument I came across that pans it...
>
>> https://xn--gckvb8fzb.com/gemini-is-solutionism-at-its-worst
>> 
> On 4/19/22 11:08 PM, George Rosamond wrote:
>> Anyone tinker with this project? 
>> 
>> https://gemini.circumlunar.space/docs/faq.gmi 
>> 
>> A lot about it piques my interest... 
>> 
>> g 

Warning, meandering ramble typed while eating lunch:

Super interesting, both posts on this Gemini thing.

That scathing review is *very* interesting read by itself, a breath of fresh air for people (like me), not super excited about the state of the internet.

Yet, as I read this screed, I find myself thinking the *same* things I thought about a super-dumb "website" back in the day called Twitter.  (My disparaging thoughts about Twitter have not changed, but Twitter don't care one bit about me and has done just great by it's own measure...)

Gopher for the 21st centry?
At first glance, open mind, the Gemini project seems interesting, as a way to eject from the ubiquity of 'the web', and simply want to use the internet to distribute "files".  Which, as an idea, I can actually see as *very* powerful- (however, not the same ubiquitious benefit the web has brought).  If you look at the Gemini markup/protocol/blah as merely utility indexes for file sharing, I can imagine some great applied uses.

Example: I grew up with punk/music zines as a teen.  Photocoped cheaply distributed "files", often found in record stores, always cheap.  I can imagine some new world of "Zines distributed as PDF's or even just folders full of jpeg pages."  A new world opens up here for PDF "magazine" distribution, Podcasts are just MP3 files, or video files, whatever.
I mean heck, in a world where the internet delivers content "files" for proprietary platforms, (Netfilix, Hulu, and other big-brand video platforms), I can certainly see room for Gopher-like distribution of files.

I can really dig the idea of "forget the web, but long live the internet", but now on to see how this technology stands up and try it...  hrm...

Best,
.ike



From steve.b at osfda.org  Wed Apr 20 15:37:34 2022
From: steve.b at osfda.org (steve.b at osfda.org)
Date: Wed, 20 Apr 2022 15:37:34 -0400
Subject: [talk] Gemini
In-Reply-To: <cd2767de-fe6e-4db6-b1b9-0cea0fca404e@www.fastmail.com>
References: <e969711b-e4b0-2355-3142-decb35b8e2bb@ceetonetechnology.com>
 <3a9ea03f-a229-daa7-ad13-054d058f5682@osfda.org>
 <cd2767de-fe6e-4db6-b1b9-0cea0fca404e@www.fastmail.com>
Message-ID: <ce524cda-4a2c-44ad-8455-9e39ab56098c@osfda.org>

Full disclosure: I detest Twitter culture. I don't see it as the 
monopolies that Google, Facebook, and Microsoft are (and should be 
addressed; presently only the EU is doing that...); but I also don't see 
it as making for positive constructive cultural development either.

My take: It appears the critic is saying it's a protest move; an attempt 
to decouple and make a clean breast from the commercial Internet.

On 4/20/22 03:12 PM, Isaac (.ike) Levy wrote:
> On Tue, Apr 19, 2022, at 11:32 PM, steve.b at osfda.org wrote:
>> A devil's argument I came across that pans it...
>>
>>> https://xn--gckvb8fzb.com/gemini-is-solutionism-at-its-worst
>>>
>> On 4/19/22 11:08 PM, George Rosamond wrote:
>>> Anyone tinker with this project?
>>>
>>> https://gemini.circumlunar.space/docs/faq.gmi
>>>
>>> A lot about it piques my interest...
>>>
>>> g
> Warning, meandering ramble typed while eating lunch:
>
> Super interesting, both posts on this Gemini thing.
>
> That scathing review is *very* interesting read by itself, a breath of fresh air for people (like me), not super excited about the state of the internet.
>
> Yet, as I read this screed, I find myself thinking the *same* things I thought about a super-dumb "website" back in the day called Twitter.  (My disparaging thoughts about Twitter have not changed, but Twitter don't care one bit about me and has done just great by it's own measure...)
>
> Gopher for the 21st centry?
> At first glance, open mind, the Gemini project seems interesting, as a way to eject from the ubiquity of 'the web', and simply want to use the internet to distribute "files".  Which, as an idea, I can actually see as *very* powerful- (however, not the same ubiquitious benefit the web has brought).  If you look at the Gemini markup/protocol/blah as merely utility indexes for file sharing, I can imagine some great applied uses.
>
> Example: I grew up with punk/music zines as a teen.  Photocoped cheaply distributed "files", often found in record stores, always cheap.  I can imagine some new world of "Zines distributed as PDF's or even just folders full of jpeg pages."  A new world opens up here for PDF "magazine" distribution, Podcasts are just MP3 files, or video files, whatever.
> I mean heck, in a world where the internet delivers content "files" for proprietary platforms, (Netfilix, Hulu, and other big-brand video platforms), I can certainly see room for Gopher-like distribution of files.
>
> I can really dig the idea of "forget the web, but long live the internet", but now on to see how this technology stands up and try it...  hrm...
>
> Best,
> .ike
>
> _______________________________________________
> talk mailing list
> talk at lists.nycbug.org
> https://lists.nycbug.org:8443/mailman/listinfo/talk