How to increase the limit of “maximum ope…
apple Verified ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
The default limit for the max open files on Mac OS X is 256 (ulimit -n) and my application needs about 400 file handlers.
I tried to change the limit with setrlimit() but even if the function executes correctly, i'm still limited to 256.
Here is the test program I use (compiled with gcc):
#include <stdio.h>
#include <sys/resource.h>
main()
{
struct rlimit rlp;
FILE *fp[10000];
int i;
getrlimit(RLIMIT_NOFILE, &rlp);
printf("before %d %d ", rlp.rlim_cur, rlp.rlim_max);
rlp.rlim_cur = 10000;
setrlimit(RLIMIT_NOFILE, &rlp);
getrlimit(RLIMIT_NOFILE, &rlp);
printf("after %d %d ", rlp.rlim_cur, rlp.rlim_max);
for(i=0;i<10000;i++) {
fp = fopen("a.out", "r");
if(fp==0) { printf("failed after %d ", i); break; }
}
}
and the output is:
before 256 -1
after 10000 -1
failed after 253
I cannot ask the people who use my application to poke inside a /etc file or something. I need the application to do it by itself.
15 replies
Sign in or join Beachball Help to reply.
- 0
etresoft ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
acemtp wrote:
The default limit for the max open files on Mac OS X is 256 (ulimit -n) and my application needs about 400 file handlers.Why?
- 0
acemtp ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
It seems there's a bug in the stdio since open() works after the setrlimit, but not the fopen() function.
- 0
Jongware ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
I think etre rather meant, why does an application need 400 files opened +at the same time+. The default of 256 probably was defined as "let's take a large number that no-one ought to need".
- 0
acemtp ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
The problem is that we have a function 'setrlimit' that doesn't work on Mac OS X. The max limit was unlimited, so it should be possible to increase the current limit from 256 to at least 400 but it seems it's not possible... why?
- 0
laurent demaret ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
acemtp wrote:
The problem is that we have a function 'setrlimit' that doesn't work on Mac OS X. The max limit was unlimited, so it should be possible to increase the current limit from 256 to at least 400 but it seems it's not possible... why?I don't know but what about "not to put useless burden" on the OS ?
Ok, you are the boss ..
Did you read the man page ?
If I use Bwana (thanks to être for the link
http://www.bruji.com/bwana/) and type man:setrlimit in Safari's navigation field I can read
" Only the super-user may raise the maximum limits. Other users may only
alter rlim_cur within the range from 0 to rlim_max or (irreversibly)
lower rlim_max."
Same if you type man setrlimit in the Xcode help menu for Man Pages or in the terminal..
So, maybe, you were not superuser when you tried to put machine at hard work for your purposes ...
Or maybe any other reason given in the man page. - 0
acemtp ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
Yes I read the man page...
There's 2 limits:
struct rlimit {
rlim_t rlim_cur; /* Soft limit */
rlim_t rlim_max; /* Hard limit (ceiling for rlim_cur) */
};The "max" limit on mac os X is "unlimited" and to change it, you need to be root. I don't want to change the "max" limit since it's already "unlimited".
What I want to change is the "cur" limit, that is 256 on mac. And for changing "cur", there's no need to be root. And if you run the test program i gave in the first post, you'll see that there's no error and getrlimit shows that the "cur" limit changed correctly.
The problem is that fopen() standard C function didn't take into account this limit change.
- 0
hansz ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
Try this:
#include <stdio.h>
#include <stdlib.h>
#include <sys/resource.h>
int
main(void)
{
struct rlimit rlp;
FILE *fp[10000];
int i, r;
r = getrlimit(RLIMIT_NOFILE, &rlp);
if (r == -1) {
perror("getrlimit()");
exit(1);
}
printf("before %d %d ", (int) rlp.rlim_cur, (int) rlp.rlim_max);
rlp.rlim_cur = 10000;
r = setrlimit(RLIMIT_NOFILE, &rlp);
if (r == -1) {
perror("setrlimit()");
exit(1);
}
r = getrlimit(RLIMIT_NOFILE, &rlp);
if (r == -1) {
perror("setrlimit()");
exit(1);
}
printf("after %d %d ", (int) rlp.rlim_cur, (int) rlp.rlim_max);
for (i = 0; i < 10000; i++) {
fp = fopen("a.out", "r");
if (fp == NULL) {
perror("fopen()");
fprintf(stderr, "i == %d ", i);
exit(1);
}
}
exit(0);
}You will notice that I
o Added <stdlib.h> to use exit().
o Added a check for an error return on all system calls
o Use perror() if a system call failsWhen I run your program either as "me" or as root I see (correctly)
$ cc -Wall -o rlimit rlimit.c
$ touch a.out && ./rlimit
before 256 -1
after 10000 -1
fopen(): Too many open files
i == 9997So
a.out is opened 9997 times, which is what you expect as standard in (0), out (1) and error (2) are open by default.As to why you want to have that many files open, having lots of network connections open in a server is not uncommon.
Regards,
Hans - 0
acemtp ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
hansz,
I bet that if you execute "ulimit -n" in a terminal, you'll have more than 256 (perhaps unlimited). That explains why it works for you. But I tested your program, and of course, it doesn't exactly the same thing as mine, failed after "i==253".
It's clear that on older mac os x, the limit was higher but if you install 10.6.4 without upgrading, you have only 256.
- 0
hansz ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
{quote}
I bet that if you execute "ulimit -n" in a terminal, you'll have more than 256 (perhaps unlimited).
{quote}Lucky we didn't enter in a wager 😉
$ ulimit -n
256Maybe there is a difference between 10.5 and 10.6. I run 10.5.8. What do these two say on your system?
$ sysctl kern.maxfiles
kern.maxfiles: 12288
$ sysctl kern.maxfilesperproc
kern.maxfilesperproc: 10240Regards,
Hans - 0
acemtp ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
Wow, incredible. I don't understand at all. So it's really a new bug introduced with 10.6?
I'm with latest mac os x 10.6.4.
I have the same values for kernel:
sysctl kern.maxfiles
kern.maxfiles: 12288
sysctl kern.maxfilesperproc
kern.maxfilesperproc: 10240 - 0
hansz ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
{quote}
So it's really a new bug introduced with 10.6?
{quote}Highly unlikely, I guess half the (Mac) world would keel over if such an obvious bug was introduced. Are you
sure
setrlimit(2) does not return -1 on your system?How much memory do you have in your system, I have 4GiB. That could influence the number of files that can be open.
My 10.4 PowerBook gives the same numbers as 10.5, but does not want to open more than 253 files, both as a normal user and as root. It has a (mere) 512MiB.
An other way to find the cause is to look through the kernel source, see [http://www.opensource.apple.com/release/mac-os-x-1064/xnu-1504.7.4]. I'll give it a try this afternoon as I have a long and boring train ride ahead...
Regards,
Hans - 0
etresoft ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
hansz wrote:
As to why you want to have that many files open, having lots of network connections open in a server is not uncommon.But a good server would fork enough processes to handle the load instead of trying to do it all in one app.
There is a good reason for this limit. The select() function, by default, is limited to this value. So, not only is our hapless original poster not using select(), the program is going to require admin authorization each and every time it runs.
There is a reason why increasing this limit is so hard - you aren't supposed to do it!
- 0
acemtp ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
I have the latest MacBook Pro 15inches. It contains 4GiB of RAM.
I'm sure the setrlimit doesn't return -1, I just compiled and executed your version of the test program. And since getrlimit displays correctly the new limit (10000) without error too, we can imagine that the new limit is set.
- 0
etresoft ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
There is no reason to have more than 256 files open. Whatever you are doing, you are doing it wrong.
The setrlimit() function only applies to new processes. It does not affect the current process. You can either run "ulimit -n 10000" in the shell and then start your program or split your program into two segments:
#include <stdio.h>
#include <stdlib.h>
#include <sys/resource.h>
#include <unistd.h>
int main(void)
{
struct rlimit rlp;
int r;
r = getrlimit(RLIMIT_NOFILE, &rlp);
if (r == -1)
{
perror("getrlimit()");
exit(1);
}
printf("before %d %d ", (int) rlp.rlim_cur, (int) rlp.rlim_max);
rlp.rlim_cur = 10000;
/* I'm curious how anyone ever expected it to work or got it to work
without setting the max, because it sure seems to be required. */
rlp.rlim_max = 10000;
r = setrlimit(RLIMIT_NOFILE, &rlp);
if (r == -1)
{
perror("setrlimit()");
exit(1);
}
r = getrlimit(RLIMIT_NOFILE, &rlp);
if (r == -1)
{
perror("setrlimit()");
exit(1);
}
printf("after %d %d ", (int) rlp.rlim_cur, (int) rlp.rlim_max);
return execl("/tmp/f", "/tmp/f", NULL);
}And the source to the "f" program is:
#include <stdio.h>
int main(void)
{
FILE *fp[10000];
int i;
for(i = 0; i < 10000; i++)
{
fp = fopen("a.out", "r");
if (fp == NULL)
{
perror("fopen()");
fprintf(stderr, "i == %d ", i);
return 1;
}
}
return 0;
}
[jdaniel@Pele:584] /tmp $ a.out
before 256 256
setrlimit(): Operation not permitted
[jdaniel@Pele:585] /tmp $ sudo a.out
before 256 256
after 10000 10000
fopen(): Too many open files
i == 9997As you can see, it is convoluted and a real hassle. It isn't a bug in MacOS X 10.6 or any other version.
So, I repeat my question yet again. Why do you want to open more than 256 files? It is, of course, a rhetorical question. Don't do it. If you feel you need more than 256 open files, review your architecture and correct it.
- 0
xnav ·
Imported from the Apple Support Forums via the Wayback Machine 🔗
FYI, if I run on my 10.6.4
Path:~/xcodeprojects/wall/build/debug$touch a.out
Path:~/xcodeprojects/wall/build/debug$./wall
before 256 -1
after 10000 -1
fopen(): Too many open files
i == 253If I run under Xcode
run
[Switching to process 815]
Running…
before 9472 -1
after 10000 -1
fopen(): Too many open files
i == 9469
Debugger stopped.
Program exited with status value:1.(gdb)Seems to validate etresoft.