Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Thu 22 Feb 2018, 13:14
All times are UTC - 4
 Forum index » House Training » HOWTO ( Solutions )
zram and zswap for the Dogs
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [5 Posts]  
Author Message
wiak

Joined: 11 Dec 2007
Posts: 421
Location: not Bulgaria

PostPosted: Thu 25 Jan 2018, 00:04    Post subject:  zram and zswap for the Dogs  

zram and zswap for the Dogs

Skip the following long introduction if you already know what zram is and just want to know how to get it working...

Many of us, may have occasionally or often run into problems with web-browsers freezing and becoming unresponsive. The problem occurs most often with large memory hungry browsers such as recent Firefox or google Chrome, and especially, of course, on computers that only have relatively low amounts of RAM, such as 2GB or less. Opening a new browser tab on a heavy web page such as Facebook or Gmail can require over 100MB of RAM use per tab, so it is not surprising that opening five such tabs can cause a 512MB RAM machine to freeze, and so on (you can test/measure this effect by watching free/available memory space with top utility running whilst opening multiple gmail tabs...).

Traditionally, the way to avoid the computer thus crashing due to RAM running out is to use a separate swap partition or swap file, such that the overall 'memory' (called virtual memory) available becomes "RAM + swap space" (active RAM memory pages get swapped in and out of RAM to swap space as and when required). Unfortunately, swap space is generally much slower to read and write than actual RAM (being often on slow hard-drive or flash media), so though the computer no longer crashes once actual RAM is exhausted, it typically starts running very slowly.

zram is a technique which uses a portion of actual RAM as if it is a normal block device (such as a hard-drive), but being in RAM it remains fast. Furthermore it uses compression such that for every 1GB RAM used as zram the storage space it provides is typically 2 to 3 times that amount. zram can thus be configured as swap space in memory, with most of the advantages, but much faster than conventional swap. Disadvantage is that is does use some CPU cycles to continually compress and decompress data and it does use some of the actual RAM; nevertheless, in my own measurements (on a 2GB RAM machine), using zram greatly increases the number of heavy browser tabs that can be opened before virtual memory runs out and browser freezes/crashes and performance remains good even during zram swapping.

Note that zram doesn't even start getting used until swapping becomes required, so there appears to be little or no reason not to use it. On a larger RAM machine, no swapping will generally occur so zram is not so necessary, however, it depends on what kind of processing you are doing. For example, if you are doing large compiling jobs, or processing video files, you may well require large amounts of RAM such that swapping remains likely to occur. Hence, even on my 4GB RAM machine (largest I have), I still enable zram since I find no reason (performance-wise) not to.

Furthermore, if you are using flash-based media, such as SSD, an SD card, or usb-flashstick, using zram for swap does not require continual data writes to the flash device, which helps it last longer. Even modern flash devices can only stand a finite number of write cycles and if, like me, you use ten year old computers (or want your computer to last at least that long) minimising flash-writes is definitely a good thing... Note that there is nothing to stop you also using conventional swap in addition to zram; the zram swap can be prioritized such that it will be used first such that any slower conventional swap is only ever used in particularly critical situations.

wiak (edited 26 Jan 2018)

Quote:
* Introduction from https://www.kernel.org/doc/Documentation/blockdev/zram.txt

The zram module creates RAM based block devices named /dev/zram<id>
(<id> = 0, 1, ...). Pages written to these disks are compressed and stored
in memory itself. These disks allow very fast I/O and compression provides
good amounts of memory savings. Some of the usecases include /tmp storage,
use as swap disks, various caches under /var and maybe many more Smile


Most of the following was tested on a XenialDog 64bit system, so file locations may or may not be different on other distribution variants.

With thanks to backi, dancytron and fredx181 for much of the useful information.

Note that you need to be root user or use 'sudo' to do most of the following:

A. zram

1. Can be installed using:

Code:
apt-get update && apt-get install zram-config


2. On booting, systemd then initialises and starts zram via:

Code:
/lib/systemd/system/zram-config.service


which contains lines:

Code:
[Service]
ExecStart=/usr/bin/init-zram-swapping
ExecStop=/usr/bin/end-zram-swapping
...


By default to created /dev/zram* block filesystems are automatically prepared for use as (compressed) swap filesystems using compression type lzo; one such filesystem per system CPU core. For example, on a dual-core CPU system you will have /dev/zram0 and /dev/zram1 prepared and activated as compressed swap filesystems.

3. You can check you have zram compressed swapspace available using either top, free or zramctrl command. For example:

Code:
free (or free -h)


or, for more detail, use:

Code:
zramctl


Note that the zramctl program (provided by zram-config deb) can also be used for changing swapsize, compression type and so on. For usage help, you should refer to it's online man page or for usage summary:

Code:
zramctl --help


4. If you want to use lz4 compression in preference to that lzo default, edit /usr/bin/init-zram-swapping to contain the extra line shown as "#wiak added" below:

Code:
# initialize the devices
for i in $(seq ${NRDEVICES}); do
  DEVNUMBER=$((i - 1))
  echo lz4 > /sys/block/zram${DEVNUMBER}/comp_algorithm  #wiak added
  echo $mem > /sys/block/zram${DEVNUMBER}/disksize
  mkswap /dev/zram${DEVNUMBER}
  swapon -p 5 /dev/zram${DEVNUMBER}
done


You can then re-activate zram swap either by re-booting your system or re-starting the zram-config.service with command:

Code:
systemctl restart zram-config.service


5. When using the systemd service method of using zram you can disable it simply using command:

Code:
systemctl stop zram-config.service


or by directly executing bash script:

Code:
/usr/bin/end-zram-swapping


6. You can start it up again using either:

Code:
systemctl start zram-config.service


or by directly running bash script:

Code:
/usr/bin/init-zram-swapping


7. Assuming you have disabled zram being used solely for swap, you can use the /dev/zram* block devices like any other block device. For example, you can create a compressed ramdisk (i.e. in RAM memory) with a command such as:

Code:
mkfs -t ext4 /dev/zram1


and use it for anything you like (such as mounting /tmp on it).

Alternatively (or in addition) you can manually create a swap partition on another pre-created zram block device using, for example:

Code:
/usr/bin/end-zram-swapping
modprobe zram num_devices=2
zramctl -r /dev/zram0
zramctl -a lz4 -s 512M /dev/ram0
mkswap /dev/zram0
swapon /dev/zram0


A simpler way of doing the above, however, is simply to make a re-named copy of the script /usr/bin/init-zram-swapping and modify that for the purpose your desire. One example would be to simply modify that script for a different swap size calculation as follows (refer to #wiak added):

Code:
#!/bin/sh

# load dependency modules
NRDEVICES=$(grep -c ^processor /proc/cpuinfo | sed 's/^0$/1/')
if modinfo zram | grep -q ' zram_num_devices:' 2>/dev/null; then
  MODPROBE_ARGS="zram_num_devices=${NRDEVICES}"
elif modinfo zram | grep -q ' num_devices:' 2>/dev/null; then
  MODPROBE_ARGS="num_devices=${NRDEVICES}"
else
  exit 1
fi
modprobe zram $MODPROBE_ARGS

# Calculate memory to use for zram (1/2 of ram)
totalmem=`LC_ALL=C free | grep -e "^Mem:" | sed -e 's/^Mem: *//' -e 's/  *.*//'`
# mem=$(((totalmem / 2 / ${NRDEVICES}) * 1024)) #wiak commented-out
mem=$(((totalmem * 75 / 100 / ${NRDEVICES}) * 1024)) #wiak added

# initialize the devices
for i in $(seq ${NRDEVICES}); do
  DEVNUMBER=$((i - 1))
  echo lz4 > /sys/block/zram${DEVNUMBER}/comp_algorithm  #wiak added
  echo $mem > /sys/block/zram${DEVNUMBER}/disksize
  mkswap /dev/zram${DEVNUMBER}
  swapon -p 5 /dev/zram${DEVNUMBER}
done


8. An alternative to using zram-config (or systemd methods) is to use the debian script, dancytron modded for lz4 compression use, and provided in the following link:

http://www.murga-linux.com/puppy/viewtopic.php?p=980747#980747

More details of that earlier zram discussion can be provided in the pages of that same thread around that post.

You can also find details of setting up zram on systems that still use sysvinit rather than systemd at:

https://wiki.gentoo.org/wiki/Zram

9. Some related zram links provided by backi and others can be found in these earler XenialDog64 pages, including:

https://www.kernel.org/doc/Documentation/blockdev/zram.txt
http://www.murga-linux.com/puppy/viewtopic.php?p=979985#979985
http://www.murga-linux.com/puppy/viewtopic.php?p=980217#980217
http://www.murga-linux.com/puppy/viewtopic.php?p=980733#980733
http://www.murga-linux.com/puppy/viewtopic.php?p=980737#980737
http://www.murga-linux.com/puppy/viewtopic.php?p=980814#980814
https://wiki.debian.org/ZRam
https://starbeamrainbowlabs.com/blog/article.php?article=posts%2F226-Linux-zram.html
https://mxlinux.org/forum/viewtopic.php?f=108&t=42935

Some older zram-related murga-forum threads noted by backi:

http://www.murga-linux.com/puppy/viewtopic.php?p=762989&sid=db08a762fd04493e58ce731587d507c2
http://www.murga-linux.com/puppy/viewtopic.php?p=972381&sid=5715a3cd0fdb2f8e94e7ab257c2a98a8
http://bkhome.org/news/201701/overlayfs-cannot-handle-zram.html

Next I'll write-up some details of the zswap alternative for this thread.

wiak

Last edited by wiak on Thu 25 Jan 2018, 17:13; edited 21 times in total
Back to top
View user's profile Send private message 
wiak

Joined: 11 Dec 2007
Posts: 421
Location: not Bulgaria

PostPosted: Thu 25 Jan 2018, 00:05    Post subject:  

reserved for further info
Back to top
View user's profile Send private message 
wiak

Joined: 11 Dec 2007
Posts: 421
Location: not Bulgaria

PostPosted: Thu 25 Jan 2018, 00:05    Post subject:  

reserved for further info
Back to top
View user's profile Send private message 
backi

Joined: 27 Feb 2011
Posts: 1423
Location: GERMANY

PostPosted: Thu 25 Jan 2018, 07:59    Post subject:  

Hi wiak !
So you have been quite busy lately !You are a real Working-Horse .
Maybe you should mention in your Introduction Zram/Zwap can be used for machines with low Ram....... . Not Everyone is familiar with the Concept of Zram .
Nevertheless great Compilation for Starters ..... Work in Progress .

Greetings Smile
Back to top
View user's profile Send private message 
wiak

Joined: 11 Dec 2007
Posts: 421
Location: not Bulgaria

PostPosted: Thu 25 Jan 2018, 16:48    Post subject:  

Hi backi,

First post edited to first explain zram in some detail.

wiak
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [5 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » HOWTO ( Solutions )
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0467s ][ Queries: 12 (0.0034s) ][ GZIP on ]