Massive memory leak in 3.01 OS!!!
Massive memory leak in 3.01 OS!!!
I'm running 3.01 as a DAQ and web server. The data is added to a database every 5 seconds. I ran 48 hour test and noticed the OS memory allocated was growing all the time but the memory used by individual programs was not. Ran mpatrol to verify my application wasn't leaking.
I wrote a simple program that repeatedly opens a file, outputs about 20 bytes of text and closes the file. In about 20 minutes the file grows by about a megabyte, the OS reports 4 additional megabytes used. Closing the application doesn't release the memory. mpatrol reports no leaks in the test app.
This is installed on a flash IDE drive, frugal install.
I had heard there was a problem with the Union FS, where it was not releasing memory even whent eh file system was synced. But I heard this was fixed in 3.0
Is this a new problem? Is this the same problem but still not fixed?
I included the simple test app below.
#include <stdlib.h>
#include <iostream>
using namespace std;
// to test memoroy leak in OS
// open a file, insert text, close the file
// do this as many times as arg1 to program
//
//
int main(int argc, char** argv)
{
// get number of iterations
if(argc != 2) {
cout << "Error, must pass in number off iterations to run\n";
cout << "\n example: test 20000\n";
return -1;
}
int iterations = atoi(argv[1]);
cout << "\nStarting test with " << iterations << " iterations\n\n";
// first truncate file
FILE *fp = fopen("foo", "w");
fclose(fp);
while(iterations--)
{
fp = fopen("foo", "a+");
fputs("Where is this dang memory leak????", fp);
fclose(fp);
usleep(10000);
cout << "+";
cout.flush();
}
cout << "\nTest complete\n";
return (EXIT_SUCCESS);
}
I wrote a simple program that repeatedly opens a file, outputs about 20 bytes of text and closes the file. In about 20 minutes the file grows by about a megabyte, the OS reports 4 additional megabytes used. Closing the application doesn't release the memory. mpatrol reports no leaks in the test app.
This is installed on a flash IDE drive, frugal install.
I had heard there was a problem with the Union FS, where it was not releasing memory even whent eh file system was synced. But I heard this was fixed in 3.0
Is this a new problem? Is this the same problem but still not fixed?
I included the simple test app below.
#include <stdlib.h>
#include <iostream>
using namespace std;
// to test memoroy leak in OS
// open a file, insert text, close the file
// do this as many times as arg1 to program
//
//
int main(int argc, char** argv)
{
// get number of iterations
if(argc != 2) {
cout << "Error, must pass in number off iterations to run\n";
cout << "\n example: test 20000\n";
return -1;
}
int iterations = atoi(argv[1]);
cout << "\nStarting test with " << iterations << " iterations\n\n";
// first truncate file
FILE *fp = fopen("foo", "w");
fclose(fp);
while(iterations--)
{
fp = fopen("foo", "a+");
fputs("Where is this dang memory leak????", fp);
fclose(fp);
usleep(10000);
cout << "+";
cout.flush();
}
cout << "\nTest complete\n";
return (EXIT_SUCCESS);
}
Re: Massive memory leak in 3.01 OS!!!
Questions in color[red]
ScottD wrote:I'm running 3.01 as a DAQ and web server. The data is added to a database every 5 seconds. I ran 48 hour test and noticed the OS memory allocated was growing [what testing program?]all the time but the memory used by individual programs was not [what testing program?]. Ran mpatrol to verify my application wasn't leaking.
I wrote a simple program that repeatedly opens a file, outputs about 20 bytes of text and closes the file. In about 20 minutes the file grows by about a megabyte, the OS reports 4 additional megabytes used [file grows by 1MB but OS reports the file grows by 4MB?, what testing tools and proceedure used?]. Closing the application doesn't release the memory. [again how are you testing? what are your expectations? that closing a 10MB program will give back 10MB?] mpatrol reports no leaks in the test app.
{rest cut}
-
- Posts: 208
- Joined: Sat 10 Mar 2007, 14:49
A screenshot or three, time sequenced, of what "top" reports might be revealing.
[size=84][i]hangout:[/i] ##b0rked on irc.freenode.net
[i]diversion:[/i] [url]http://alienjeff.net[/url] - visit The Fringe
[i]quote:[/i] "The foundation of authority is based upon the consent of the people." - Thomas Hooker[/size]
[i]diversion:[/i] [url]http://alienjeff.net[/url] - visit The Fringe
[i]quote:[/i] "The foundation of authority is based upon the consent of the people." - Thomas Hooker[/size]
- Dougal
- Posts: 2502
- Joined: Wed 19 Oct 2005, 13:06
- Location: Hell more grotesque than any medieval woodcut
Re: Massive memory leak in 3.01 OS!!!
I don't know exactly what was fixed or not, but note that you can use either unionfs of aufs -- I think it might be aufs that had the "true flushing".ScottD wrote:I had heard there was a problem with the Union FS, where it was not releasing memory even whent eh file system was synced. But I heard this was fixed in 3.0
What's the ugliest part of your body?
Some say your nose
Some say your toes
But I think it's your mind
Some say your nose
Some say your toes
But I think it's your mind
Well I installed Captura so I could get some screen shots but when I started Captura, the memused went up by 3 or 4 meg, and after every screen shot the use would rise by a mb, making it a pretty useless way to track system memory.
So I tried something really simple so you could verify the problem.
1.) Boot puppy, don't start any extra apps.
2.) start up two term windows(rxvt) and run top in one of them. Let it run for a few minutes to verify you memusage is stable. It may move up and down every so often but the low and high point will be the same. Total memused is in the upper left corner.
3.) Write down the memused reported by top.
4.) then copy a large file 5 times(cp foo foo1, cp foo foo2, etc..), something about 1mb in size. Give it 15 seconds to stablize. Note how much memused rises. Then delete the copied files. Note how much memused comes down.
On my machine I loose a couple hundred kb .. every time I do the copy delete routine.
help!!!!
Scott
So I tried something really simple so you could verify the problem.
1.) Boot puppy, don't start any extra apps.
2.) start up two term windows(rxvt) and run top in one of them. Let it run for a few minutes to verify you memusage is stable. It may move up and down every so often but the low and high point will be the same. Total memused is in the upper left corner.
3.) Write down the memused reported by top.
4.) then copy a large file 5 times(cp foo foo1, cp foo foo2, etc..), something about 1mb in size. Give it 15 seconds to stablize. Note how much memused rises. Then delete the copied files. Note how much memused comes down.
On my machine I loose a couple hundred kb .. every time I do the copy delete routine.
help!!!!
Scott
I think you may be getting excited about something that doesn't mean anything. One of the most important jobs of an OS is memory management, and I can't believe Linux is that bad.Try this: take a program that seems to be hoarding memory even though it should be releasing it, and run it until, by your measurements as given above, it should have used up all the memory and caused Puppy to screw up somehow. I'll bet it never happens.
Files have to be copied from disk to a memory buffer, then from the buffer back out to disk. The hardware cannot manage a copy from disk file directly to another disk file. I don't think it can be done even if the source and destination files are on different physical disks.I was also suprised that that the memused goes up when I copy a file? Why would ram be used unless the filesystem is in ram?
It used to be in the old days, that there were two buffers. The source file is writing one buffer while the other buffer is being dumped into the destination file. Then the roles switch. I have written such programs myself.
Here's my top usage line:
Mem: 294040K used, 2301872K free, 0K shrd, 19836K buff, 140368K cached
How much memory am I using? In a very real sense I'm using 294040K, but I think it's being used this way: Mem used minus the buffers and cache.
-----------------------
The free utility concurs as shown by the output below:
Busybox free does not consider the buffers/cache the way free does, as shown by its output
So when the buffers/cache is factored in (out), I'm using 133748K
Below is a pic of what htop thinks and it says I'm using 131MB RAM, but it's hard to read due to contrast.
Puppy's top and free are Busybox utilities and as such are not as full featured as the actual utilities Busybox intends to substitute.
This is easy enough to note simply by the output of free and busybox free - htop and busybox top.
In conclusion: I think Linux caches and doesn't release its cache all that easily, and when using cut down utilities, this will be reported as increase in used memory, which could be interpreted as a leak, when it is merely normal cache/buffer behavior.
Mem: 294040K used, 2301872K free, 0K shrd, 19836K buff, 140368K cached
How much memory am I using? In a very real sense I'm using 294040K, but I think it's being used this way: Mem used minus the buffers and cache.
-----------------------
The free utility concurs as shown by the output below:
Code: Select all
[/] free
total used free shared buffers cached
Mem: 2595912 294100 2301812 0 19836 140516
-/+ buffers/cache: 133748 2462164
Swap: 554200 0 554200
[/]
Code: Select all
[/] busybox free
total used free shared buffers
Mem: 2595912 294100 2301812 0 19836
Swap: 554200 0 554200
Total: 3150112 294100 2856012
[/]
Below is a pic of what htop thinks and it says I'm using 131MB RAM, but it's hard to read due to contrast.
Puppy's top and free are Busybox utilities and as such are not as full featured as the actual utilities Busybox intends to substitute.
This is easy enough to note simply by the output of free and busybox free - htop and busybox top.
In conclusion: I think Linux caches and doesn't release its cache all that easily, and when using cut down utilities, this will be reported as increase in used memory, which could be interpreted as a leak, when it is merely normal cache/buffer behavior.
- Attachments
-
- htop-131-mb.png
- (29.23 KiB) Downloaded 1007 times
According to the Puppy documentation installation on IDE drives do not use RAM for the file system. Frugal or not.
That's why when I shut down I always get the message that the drive is top mounted and therefore doesn't need to be saved.
Scott
That's why when I shut down I always get the message that the drive is top mounted and therefore doesn't need to be saved.
Scott
Everitt wrote:Frugal installs use the RAM just like the live CD'sScottD wrote:I'm installed onto a IDE drive, which I thought would allow puppy to not use ram for the file system ... Though I am using a frugal install.
ScottD's error was not posting the link to the documentation he read.ScottD wrote:According to the Puppy documentation installation on IDE drives do not use RAM for the file system. Frugal or not.
That's why when I shut down I always get the message that the drive is top mounted and therefore doesn't need to be saved.
Scott
Everitt wrote:Frugal installs use the RAM just like the live CD'sScottD wrote:I'm installed onto a IDE drive, which I thought would allow puppy to not use ram for the file system ... Though I am using a frugal install.
Nevertheless, it's patently false to think that a Frugal install loads all of the filesystem into RAM.
It is also false to think that a Frugal install will arbitrarily copy even as much as pup_301.sfs into RAM.
Included with post are two screen shots, one where pup_301.sfs is copied to ram and the other where it is not.
--------------
One screen shot per post?
---------------
- Attachments
-
- 301-not-copied-to-ram.png
- (7.98 KiB) Downloaded 804 times
Last edited by Bruce B on Mon 26 Nov 2007, 16:28, edited 1 time in total.
I guess it wouldn't take two pics, here's the other one. The words to look for are copying to ram ..., although there are other more technical ways to determine if it copied to RAM or is running mounted on the HD.
- Attachments
-
- 301-copied-to-ram.png
- (5.38 KiB) Downloaded 792 times
The configuration is identical. The difference is Puppy's calculation of RAM and the decision it makes based on this calculation.mcewanw wrote:Interesting...
But what were the differences in the configurations used that caused one boot to say it was copying the pup_301.sfs into RAM and the other not?
The one test where pup_301.sfs was NOT copied was 128MB, the other where it was copied was 384MB.
Bruce
That makes sense. Allows a low-mem machine to keep running when otherwise it would run out of RAM. I wonder if older Puppy versions such as 2.16 or 2.17 employ the same basic algorithm or if this is new for the bigger 3.01 image?Bruce B wrote:The one test where pup_301.sfs was NOT copied was 128MB, the other where it was copied was 384MB.
It would be good to know where, in which script (inside the initrd image) the code is that makes the decision. I'd be interested to play with that (someday) in order to see if an ancient very low mem (32MB I think) machine I have would boot Puppy from a frugal install of 2.16 or 2.17 (the latter being the main version I regularly use presently). Maybe it would, but I doubt it. I was going to do a full install to get round the problem (though I'm too busy doing other things at the moment, so it will be a while before I try any such thing anyway).