[nycbug-talk] make -jN performance was: Re: talk Digest, Vol 14, Issue 9

Pete Wright pete
Tue Jan 18 17:59:15 EST 2005

On Thu, Jan 13, 2005 at 08:02:42PM -0500, Bob Ippolito wrote:
> On Jan 13, 2005, at 19:45, Pete Wright wrote:
> >
> >>>
> >>># make -j5 buildworld KERNCONF=MY_KERNEL
> >>>
> >>>this will launch 5 make jobs, which will speed up some of the make
> >>>tasks.
> >>
> >>Typically it's recommended to use -jN where N is the number of
> >>processors you actually have.  For values of N greater than the number
> >>of CPUs you have, it should actually go slower...  I would think that
> >>-j5 is slower than -j4 on a dual CPU system (though -j3 or -j4 might 
> >>be
> >>a little faster than -j2 for some kinds of source, -j2 is always 
> >>faster
> >>than -j1 on a dual processor system).
> >
> >Just to prove that I'm not pulling this outta my a*s.  From the 
> >handbook
> >(granted it's from make buildworld...altho I've found this to work 
> >fine on
> >kernels for a long time now):
> >
> >"It is now possible to specify a -j option to make which will cause it 
> >to
> >spawn several simultaneous processes. This is most useful on multi-CPU
> >machines. However, since much of the compiling process is IO bound 
> >rather
> >than CPU bound it is also useful on single CPU machines.
> >
> >On a typical single-CPU machine you would run:
> >
> ># make -j4 buildworld
> >
> >make(1) will then have up to 4 processes running at any one time.
> >Empirical evidence posted to the mailing lists shows this generally 
> >gives
> >the best performance benefit.
> >
> >If you have a multi-CPU machine and you are using an SMP configured 
> >kernel
> >try values between 6 and 10 and see how they speed things up.
> >
> >Be aware that this is still somewhat experimental, and commits to the
> >source tree may occasionally break this feature. If the world fails to
> >compile using this parameter try again without it before you report any
> >problems."
> Ok, I said "some kinds of source" for that reason.  It really depends 
> (but it's universally true that N should be *at least* the number of 
> CPUs you have).  In this case, they've tried it, and large numbers are 
> good.  When I've tried -jN, I usually find that the optimal numbers 
> aren't that large, but I'm compiling different things..
> -bob

Hey bob all,
	so instead of relying on blind faith I ran some tests to see the truth
in the make -jN statements in the FreeBSD handbook.  While I agree with what 
bob stated regarding the "-j" flag I thought that maybe there was something in
the makeworld process that got around these issues.  Well I hacked up a small
shell script to test this out and the findings were not what I expected (and 
frankly not what I remember experiencing in the past).  The make times flattened
out after make -j2 buildworld on an SMP system running 5.3.  I've posted a pretty detailed
post to freebsd-stable@ and hope to get some feedback.  I'll keep you all posted
here as well if you all are interested.


Peter Wright
pete at nomadlogic.org

More information about the talk mailing list