Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Tue 18 Sep 2018, 10:24
All times are UTC - 4
 Forum index » House Training » HOWTO ( Solutions )
Release cached memory manually (with user input)
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [15 Posts]  
Author Message
nic007


Joined: 13 Nov 2011
Posts: 2578
Location: Cradle of Humankind

PostPosted: Tue 23 Jan 2018, 09:49    Post subject:  Release cached memory manually (with user input)  

This may be useful to those running fully in RAM (and especially those not using swap). I have 2GB of RAM and run without swapfile. I've observed that watching HD videos for an extended period on my machine results in an inevitable system crash and was wondering why because VLC was mostly the only application in use. Somehow this must have something to do with system memory control. I noticed that the crash generally occurs (or is likely to occur) when all the "free" memory is cached. The biggest chunk of cached memory is actually still supposed to be available to the system and should not compromise the stability of the system. However, this does not seem to be the case in my experience using Puppy. Since clearing the cached memory in regular intervals, my system has been more stable (haven't had a crash since, touch wood). There are two ways to flush the cached memory manually:
1) Using this command:
Code:
echo 3 > /proc/sys/vm/drop_caches

I run this in intervals of 30 minutes (using Pschedule) when watching very high bitrate HD videos.
2) the second method is to simply unmount the drive you were reading/writing from, memory will be flushed. This method can obviously only be used if the drive is not in use.
Back to top
View user's profile Send private message 
musher0

Joined: 04 Jan 2009
Posts: 12677
Location: Gatineau (Qc), Canada

PostPosted: Tue 23 Jan 2018, 11:18    Post subject:  

Hi nic007.

Good reminder, this. Thanks.

I use the < sync > command before so that any data still in transit is sent to the
destination before I start "vacuuming" the RAM, like this:
Code:
sync;echo 3 > /proc/sys/vm/drop_caches;wait
Also with < wait > at the end, to make sure my "vacuuming" is over before bash
processes another command.

BFN.

_________________
musher0
~~~~~~~~~~
Siempre será canción nueva... (V. Jara, Manifiesto)
Back to top
View user's profile Send private message 
nic007


Joined: 13 Nov 2011
Posts: 2578
Location: Cradle of Humankind

PostPosted: Tue 23 Jan 2018, 18:47    Post subject:  

Hi, yes aware of the sync command but as far as I know the system runs that regularly anyway. I mean, every second script has sync in it. Also, the cached memory is not flushed completely so wonder if sync really is necessary. I have previously tried sync && ...... to run as "one command" but it doesn't work that way when using Pschedule. I'll try the way you suggested, thanks.
Back to top
View user's profile Send private message 
greengeek


Joined: 20 Jul 2010
Posts: 5161
Location: Republic of Novo Zelande

PostPosted: Tue 13 Feb 2018, 04:52    Post subject:  

Hi Nic - two questions:

- how can I work out the current state of cache? Is there some way to know if it is choked?

- What form of system crash were you experiencing? Did you experience the personal storage icon turning red?
Back to top
View user's profile Send private message 
B.K. Johnson

Joined: 12 Oct 2009
Posts: 665

PostPosted: Tue 13 Feb 2018, 10:43    Post subject:  

Hello nic007
Quote:
Code:
echo 3 > /proc/sys/vm/drop_caches
I want to use this, but...

    I don't understand what it does [I have my newbie dunce cap on Laughing ]
    Despite not getting an error when it's run, how do I know it is working: cache used before and after
    My Firefox cache wasn't emptied when run

_________________
B.K. Johnson
tahrpup-6.0.5 PAE (upgraded from 6.0 =>6.0.2=>6.0.3=>6.0.5 via quickpet/PPM=Not installed); slacko-5.7 occasionally. Frugal install, pupsave file, multi OS flashdrive, FAT32 , SYSLINUX boot, CPU-Dual E2140, 4GB RAM

Back to top
View user's profile Send private message 
nic007


Joined: 13 Nov 2011
Posts: 2578
Location: Cradle of Humankind

PostPosted: Tue 13 Feb 2018, 19:36    Post subject:  

greengeek wrote:
Hi Nic - two questions:

- how can I work out the current state of cache? Is there some way to know if it is choked?

- What form of system crash were you experiencing? Did you experience the personal storage icon turning red?


Hi, greengeek. This explains it well: https://www.linuxatemyram.com/ Use the free command in terminal to check the status. The important thing is that the total free memory as reported, could actually be less than the actual free memory available. Evenso, in my experience, the indication of very low total free memory alone could be an indication of an imminent crash.
Crash as in freeze up, computer needs to be rebooted. Instability of software and drivers can of course also play a role. My conclusion is that under normal use the system seems to respond adequately and it should not be necessary to flush the memory cache at any stage. Under heavy load for a prolonged period, flushing the cache every now and then could be useful and prevent an imminent crash (it does for me).
I don't have a savefile. The personal storage metre almost always report 1GB free.
Back to top
View user's profile Send private message 
nic007


Joined: 13 Nov 2011
Posts: 2578
Location: Cradle of Humankind

PostPosted: Tue 13 Feb 2018, 19:44    Post subject:  

B.K. Johnson wrote:
Hello nic007
Quote:
Code:
echo 3 > /proc/sys/vm/drop_caches
I want to use this, but...

    I don't understand what it does [I have my newbie dunce cap on Laughing ]
    Despite not getting an error when it's run, how do I know it is working: cache used before and after
    My Firefox cache wasn't emptied when run


Hi, B.K. Johnson. This relates to the system memory cache (not your browser cache). Run the
Code:
free
command in terminal before and after you have run the command to flush the cache and you will see the amount of free memory increase. Also - read my answer to greengeek.
Back to top
View user's profile Send private message 
greengeek


Joined: 20 Jul 2010
Posts: 5161
Location: Republic of Novo Zelande

PostPosted: Wed 14 Feb 2018, 03:01    Post subject:  

nic007 wrote:
I don't have a savefile. The personal storage metre almost always report 1GB free.
Same here, no savefile. My storage meter also never used to suggest any problem either - except I have noticed lately it starts to run low (red) under two specific conditions:

1) When I am using mksquashfs to build large puppy sfs files

2) When i am using mhwaveedit to examine/cut/encode large audio files.

In both cases my system grinds to a very slow state and programs seem to fail with odd errors.

(I have 6GB swap on hard disk and this seems to match the total shown in my storage icon)

I don't know why this has started happening - except that it seems to be noticeable when I have been using Google Chrome - and I am starting to wonder if Chrome mucks up my cache and/or RAM somehow.

Chrome has also started mucking up my LibreOffice suite - causing "bus" errors - whatever they are.

I will try your cachedrop when it next happens and see if anything improves.

cheers!
Back to top
View user's profile Send private message 
nic007


Joined: 13 Nov 2011
Posts: 2578
Location: Cradle of Humankind

PostPosted: Wed 14 Feb 2018, 03:48    Post subject:  

greengeek, correct. My storage space metre drops only when one do operations relating to writing to / which will involve using RAM if you work without a savefile. That metre has nothing to do with the system process of caching memory, I think. PS. I don't have a swapfile, could that large swapfile of yours be part of your problem?
Back to top
View user's profile Send private message 
greengeek


Joined: 20 Jul 2010
Posts: 5161
Location: Republic of Novo Zelande

PostPosted: Wed 14 Feb 2018, 14:10    Post subject:  

nic007 wrote:
That metre has nothing to do with the system process of caching memory I think
This is an area I do not understand well enough. I always thought that if we ran without savefile then the "personal storage" meter became irrelevant - but that is not the case. It is certainly reporting that something is getting clagged up under heavy resource usage.

Clearly that icon is also reflecting something to do with RAM usage and the ability of the system to move data (and I guess code) around the place effectively.

Cache handling must be part of the "code moving" process I would think?

However, after reading some of the internet threads concerning cache dropping it sounds as if:
- The Linux cache controller "looks ahead" to decide what code to move from HDD (or RAM?) to the faster cache and to store it there so it is readily available again later
- RAM control is more dynamic and one of its functions is to move code out of RAM back to disk - especially but not exclusively - to swap partition or swapfile. (so yes - the presence of a large swap partition could potentially be part of my particular problem)

Once the system gets busy I imagine both processes get a real workout. One of the threads I read seemed to suggest that Linux layering processes can trigger bugs with caching (on some Linux kernels) - so I wonder whether Puppy's layering might cause oddball results with both RAM and cache handling.

How does Puppy calculate "free storage" when it has multiple layers in play, and has those layers moving into and out of both cache and swap?? How does it decide when to change the storage icon from green to orange to red? If cache is full of unneeded code does that mean that the rest of the system resources get a harder workout than necessary?

Unless I have a developing hardware problem, then something in my software has been changing lately - which seems odd as my system boots to exactly the same state every time (No savefile and never any changes to the system sfs files). I really do suspect that Google Chrome is misallocating my system resources and somehow cluttering up the way my machine runs.

Can a software program have an impact on what gets cached or prioritised for storage in RAM or swap? I think so.

Maybe dumping cache will help me track down what is really going on
Back to top
View user's profile Send private message 
nic007


Joined: 13 Nov 2011
Posts: 2578
Location: Cradle of Humankind

PostPosted: Wed 14 Feb 2018, 16:35    Post subject:  

Run that cache command in intervals of say 1 hour and see if you get more overall stability. As mentioned, my personal storage indicator seems to drop only when large data writing operations to / (memory being used if no savefile is in use) takes place... but will go up if I delete data that has been written to / . This follows that all operations involving data being written to / will stop if the indicator reaches zero...and software requiring data to be written to / continuously will probably fail at that point. Are you sure it is not the browser cache consuming all your memory?
Back to top
View user's profile Send private message 
greengeek


Joined: 20 Jul 2010
Posts: 5161
Location: Republic of Novo Zelande

PostPosted: Thu 15 Feb 2018, 00:48    Post subject:  

nic007 wrote:
Are you sure it is not the browser cache consuming all your memory?
No, I can't say I am sure of that. I don't really understand all the different caches. However, I don't think the browser can be using all my memory as I never see memory fill up when just browsing - even long sessions on youtube or downloading vids etc. That does not seem to reduce the "personal storage" figure seen by hovering over the icon (and there would have to be enormous browser cache to fill 6GB).

Unless it's something weird like the browser cache has first dibs on free RAM and any other applications are forced out to disk swapping.

I think I need to find out more about that personal storage icon and work out what it really is trying to tell me. I just can't understand how 6GB of swap can fill up just because I am doing a little browsing then manipulating audio files with mhwaveedit. 2GB Ram plus 6GB swap can't be disappearing.

I also need to know more about kernel cache as one of the messages I get sometimes is "unable to fork - cannot allocate memory" which according to a bit of googling seems to be trying to tell me about kernel memory running out. Don't know if that is to do with cache (of some kind) or main RAM, or Puppy's handling of layers.
Back to top
View user's profile Send private message 
TyroBGinner

Joined: 30 Mar 2016
Posts: 228

PostPosted: Thu 15 Feb 2018, 15:34    Post subject:  

Try a crontab directly and see if it works...it is easier as the "pschedule" program is confusing. Enter
Code:
crontab -u root
Then enter
Code:
 */30 * * * * sync && echo 3 > /proc/sys/vm/drop_caches && wait
in the text editor once it opens.
_________________
Win10 and lovin' it!
Back to top
View user's profile Send private message 
bigpup


Joined: 11 Oct 2009
Posts: 11004
Location: Charleston S.C. USA

PostPosted: Thu 15 Feb 2018, 19:32    Post subject:  

Quote:
nic007 wrote:
That metre has nothing to do with the system process of caching memory I think
This is an area I do not understand well enough. I always thought that if we ran without savefile then the "personal storage" meter became irrelevant - but that is not the case. It is certainly reporting that something is getting clagged up under heavy resource usage.

Clearly that icon is also reflecting something to do with RAM usage and the ability of the system to move data (and I guess code) around the place effectively.

If you are running with no save at all.
The personal storage icon in the tray is reading what is being used as the storage while operating Puppy.
Usually that is about half the system RAM as a ram disk.

_________________
I have found, in trying to help people, that the things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected Shocked
Back to top
View user's profile Send private message 
greengeek


Joined: 20 Jul 2010
Posts: 5161
Location: Republic of Novo Zelande

PostPosted: Tue 20 Feb 2018, 04:19    Post subject:  

musher0 wrote:
I use the < sync > command before so that any data still in transit is sent to the destination before I start "vacuuming" the RAM, like this:
Code:
sync;echo 3 > /proc/sys/vm/drop_caches;wait
Also with < wait > at the end, to make sure my "vacuuming" is over before bash
processes another command.

BFN.
Hi Musher - is there any functional difference bwteen your syntax - using semicolon delimiter ";" and Tyrobeginners syntax using "&&" ?
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [15 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » HOWTO ( Solutions )
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0775s ][ Queries: 12 (0.0081s) ][ GZIP on ]