[nycbug-talk] Increasing memory for an application

[Francisco Reyes]

Marc Spitzer writes:

> Do you get different behavior if you run as root?

No.
 
> How much RAM is installed on the box?

2GB.

 
> How much is free?

I knew exactly I would not have the other read about memory going..
But see if this helps.

Top memory:
Mem: 433M Active, 974M Inact, 242M Wired, 95M Cache, 112M Buf, 259M Free
 
 
> are you running the psql client on the same box as the server?

Yes.
 
> How much swap space is there?

Device          1K-blocks     Used    Avail Capacity
/dev/da0s1b       4194304      116  4194188     0%


 
> How much is avalable?

Almost all of it.

 
> Does this work else where?

Don't know.
 
> should the result set be this big?

Yes it should.
This is not a postgresql issue. The result set is 6 million records and yes 
it, unfortunately, needs to be that big. I will try and talk to the Bacula 
devs about perhaps doing a better job.. instead of doing a straight select 
against a large table
   
> what version of postgress are you running?

I really don't believe it is an issue, but it's 8.1.3
I already got an answer from the postgreSQL list. The psql client is running 
out of memory.
   
> are you running a custom kernel?

Yes. 

 
> other questions with answeres to help you solve your problem should go here

This all started with a Bacula setup I am getting done with Dan Langille's 
help. When a restore job failed, we tracked it down to running out of memory 
at the process level.

The bacula program (either the director or the storage daemon) ran out of 
memory on a query. I tried the same query on psql with the same result. That 
gives us two different programs having the same issue, running out of memory 
on a particular query.

    
> What you have done to test should go here

Narrowed the problem down to what seems FreeBSD placing a limit on a 
process.
  
> the more information you give us the more likely it is we can and will
> help you.

Does the above help?

 
> Please also keep in mind the quality of the problem report is seen by
> me, and possibly others, as how much you respect the people on the
> list who can and are trying to fix your problem for free.

I actually did not report a problem. I asked a question if someone knew 
offhand how to increase the memory limit for an app.

I do agree that more background info would have been helpfull. Will most 
definitely keep that in mind. However, I personally don't like to write long 
emails when asking questions. I try to find a balance between giving all the 
info possible and not making the email too long. I try my best to respect 
people's time. Some people will know about particular problems. Others 
wont'.. but both type of people may spend the time reading an email, so I 
don't want to take too much time away from someone who may not know the 
answer anyways.


> am see in your requests you do not appear to be putting in much effort
> befor bothering the rest of us and I feel that that is quite rude. 

I did not ask anyone to troubleshoot my problem.
I did my best over 3 days, possibly 10+ hours searching archives.. trying 
different keywords, trying different search engines. After all that time and 
a few people's advice all it came down to is what seems some time of 
per process limit. On top of that when I didn't know yet it was a FreeBSD 
issue spent some other 2+ hours going over the PostgreSQL docs trying to 
increase the server memory; increasing shared buffers, making sure it was 
not some type of server resource or cursor resource that needed to be 
increased. ONLY after I exausted all options I could find I asked in the 
PostgreSQL answer. One of the PostgreSQL developers responded it was a 
client issue.

Given that both psql and the bacula process ran out of memory it seemed to 
make sense that it was not a server issue. Therefore I have been 
concentrating on the OS.

Other simmilar reports on other operating systems indicated also a per 
process type of limit, but didn't pay as much attention to those emails 
because the settings would likely not be the same on a BSD (the emails were 
for Linux).   

> Now if you have been working on the problem it would be best to give a
> detailed list of what happened and what you did to try to fix it and
> what the results of those actions were

That sounds like asking someone how to get to broadway.. and they asking you 
why yo want to go there.

I agree that more info than I originaly gave would not have been a bad 
thing, however after I (hopefully) have given more info.. we are very likely 
exactly at the same place than when I wrote the initial question.. looking 
for a way to increase the per process limit of an app.

I do agree I should try other machines.. perhaps something on that machine 
needs to be changed.. it is 5.4.. perhaps a 6.X will not have the problem.

>if you can not write a decient list then why are you bothering us?

Perhaps I don't understand the purpose of a mailing list like this.
Isn't the purpose of this list for people with a common interest to pool 
their knowledge and try to help each other? 

If someone writes an email and it is obvious to you what the problem is that 
someone is having shouldn't we help that person?

If we don't know and have no idea, then we can just move on to the next 
email. It is not like this list is so busy that we have to censor emails.. 
specially if they are related to one of the BSD.
    
> Now I am not writting this based on just one thread but after at least
> three threads that I have been involved in with you in the last week
> or so, two of them tonight.

Is there a quota on the number of questions we can post?
I know about the memory question, which your previous post basically 
implies that I didn't bother to read the man page. I did. And also the man 
page for vmstat.. and several freebsd books I have.. read the sections on 
memory.

What seems to be obvious to you regarding top memory display is not obvious 
to me.. specially when I see different machines with different values..etc, 
etc.. I will reply about that on that other email, instead of getting 
things convoluted.

Moreover, that subject was NOT an urgent issue that I am trying to get other 
people to "solve for me". It is basically a search for more info on a topic, 
which I believe would be of interest to others once I find better answers 
which I will obviously share.

   
> I am here to discus things with my peers and to help people become my
> peers, so we can talk about interesting stuff.

If you find my posts boring, or a lazy attemp to get others to do my job, 
then please just ignore them. I can't recall asking a question before I have 
spent at least 4+ hours researching the topic and often times have been 
researching the topic for days.

You are also forgetting the psycology of a problem.. some times after a 
person has been looking at a problem for a week or two, they may not be in 
the most objective or logical state that you wish people where. People 
sometimes are tired of a problem, or even desperate.. so their questions may 
not make the most sense to the rest of the world.. It is on those times that 
others can be more helpfull and direct the person to give the right info 
instead of making assumptions about how much/little they have researched.



>I am not here to provide free consulting, on a professional level I do not care about
> your problems.

I am so looking forward to other's feedback on this thread. :-)


> The reason for this is you do not pay me to care about
> your problems.  You are acting like a client and are not paying my
> rate, please fix the first part or the second.

I really must not understand Open Source lists and user group lists then.. I 
really though it was about people helping each other. I must have really 
missed "the memo" where it was dictated people  HAD to help others. If I 
don't know or don't want to help someone.. I don't reply to their email. 
Personally I feel anything else is a waste of list member's time.