[nycbug-talk] make -jN performance was: Re: talk Digest, Vol 14, Issue 9
Pete Wright
pete
Tue Jan 18 17:59:15 EST 2005
On Thu, Jan 13, 2005 at 08:02:42PM -0500, Bob Ippolito wrote:
>
> On Jan 13, 2005, at 19:45, Pete Wright wrote:
>
> >
> >>>
> >>># make -j5 buildworld KERNCONF=MY_KERNEL
> >>>
> >>>this will launch 5 make jobs, which will speed up some of the make
> >>>tasks.
> >>
> >>Typically it's recommended to use -jN where N is the number of
> >>processors you actually have. For values of N greater than the number
> >>of CPUs you have, it should actually go slower... I would think that
> >>-j5 is slower than -j4 on a dual CPU system (though -j3 or -j4 might
> >>be
> >>a little faster than -j2 for some kinds of source, -j2 is always
> >>faster
> >>than -j1 on a dual processor system).
> >
> >Just to prove that I'm not pulling this outta my a*s. From the
> >handbook
> >(granted it's from make buildworld...altho I've found this to work
> >fine on
> >kernels for a long time now):
> >
> >"It is now possible to specify a -j option to make which will cause it
> >to
> >spawn several simultaneous processes. This is most useful on multi-CPU
> >machines. However, since much of the compiling process is IO bound
> >rather
> >than CPU bound it is also useful on single CPU machines.
> >
> >On a typical single-CPU machine you would run:
> >
> ># make -j4 buildworld
> >
> >make(1) will then have up to 4 processes running at any one time.
> >Empirical evidence posted to the mailing lists shows this generally
> >gives
> >the best performance benefit.
> >
> >If you have a multi-CPU machine and you are using an SMP configured
> >kernel
> >try values between 6 and 10 and see how they speed things up.
> >
> >Be aware that this is still somewhat experimental, and commits to the
> >source tree may occasionally break this feature. If the world fails to
> >compile using this parameter try again without it before you report any
> >problems."
>
> Ok, I said "some kinds of source" for that reason. It really depends
> (but it's universally true that N should be *at least* the number of
> CPUs you have). In this case, they've tried it, and large numbers are
> good. When I've tried -jN, I usually find that the optimal numbers
> aren't that large, but I'm compiling different things..
>
> -bob
>
Hey bob all,
so instead of relying on blind faith I ran some tests to see the truth
in the make -jN statements in the FreeBSD handbook. While I agree with what
bob stated regarding the "-j" flag I thought that maybe there was something in
the makeworld process that got around these issues. Well I hacked up a small
shell script to test this out and the findings were not what I expected (and
frankly not what I remember experiencing in the past). The make times flattened
out after make -j2 buildworld on an SMP system running 5.3. I've posted a pretty detailed
post to freebsd-stable@ and hope to get some feedback. I'll keep you all posted
here as well if you all are interested.
-pete
--
~~oO00Oo~~
Peter Wright
pete at nomadlogic.org
www.nomadlogic.org/~pete
917.415.9866
More information about the talk
mailing list