Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Fri 24 Oct 2014, 06:37
All times are UTC - 4
 Forum index » House Training » HOWTO ( Solutions )
How to reduce the size of Debian Live image
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [15 Posts]  
Author Message
saintless


Joined: 11 Jun 2011
Posts: 2574
Location: Bulgaria

PostPosted: Fri 06 Dec 2013, 06:45    Post subject:  How to reduce the size of Debian Live image  

This method describes how to slim down debian live squeeze from working image by removing files and applications.
I've done this only once and I can't tell all the steps I had to take to reduce the size at maximum. It took me over a week with different results and upgrading from already finished squash file. But the main concept is this one and with testing and adding advices from other people it could be improved.

Important information : Use only ext partition. vfat can't manage some linux file permissions and will cause errors.

In this example I will use the smallest debian-live-6.0.0-i386-standard.iso from here:
http://live.debian.net/files/oldstable/images/6.0.0/i386/iso-hybrid/

Download and extract the live folder on /mnt/sda1 for example. If you can do it from puppy it will be great.
http://postimg.org/image/7zx9a64jn/
Unsquash filesystem.squashfs.
Code:
unsquashfs -d /mnt/sda1/deb-6 /mnt/sda1/live/filesystem.squashfs

Now you have the content extracted in /mnt/sda1/deb-6 folder.
Add this sources to /mnt/sda1/deb-6/etc/apt/sources.list
Code:
deb http://backports.debian.org/debian-backports squeeze-backports main

Create file /mnt/sda1/deb-6/etc/apt/apt.conf with this content inside:
Code:
APT::Install-Recommends "false"; APT::Install-Suggests "false";

This way apt-get will install only the base packages.
Create folder /mnt/sda1/slim-boot and copy /mnt/sda1/deb-6/boot and /mnt/sda1/deb-6/dev folders + initrd.img and vmlinuz links as shown on the picture:
http://postimg.org/image/4b4w7i8qb/
You can remove one of the kernels as the example on the picture or not. It does not matter on this point. Just be sure to choose which kernel will be booted. It is 686 for me.

Now create squashfs from /mnt/sda1/slim-boot:
Code:
mksquashfs /mnt/sda1/slim-boot /mnt/sda1/live/01-slim-boot.squashfs

You will need this slim-boot squash file only for boot. At the end it will be deleted.
I guess it is possible to include only /dev folder in 01-slim-boot.squashfs and the result will be the same.
Latest test shows this slim-boot squash file can be empty.
Next step is to create live-rw save file. 1Gb is enough. I remember I had some issues if live-rw is placed in sda1. To play safe put it on another partition. sda2 for example.
Code:
dd if=/dev/zero of=/mnt/sda2/live-rw bs=1M count=1000
mkfs.ext2 /mnt/sda2/live-rw


Mount live-rw to /mnt/cdrom for example. I presume the optical drive is empty:
Code:
mount -o loop /mnt/sda2/live-rw /mnt/cdrom


Copy from terminal all files from /mnt/sda1/deb-6 to /mnt/cdrom
Code:
cp -frp /mnt/sda1/deb-6/* /mnt/cdrom

Unmount live-rw:
Code:
umount /media/cdrom


Now you have all the content of filesystem.squashfs inside live-rw file.
It is time to remove filesystem.squashfs from /mnt/sda1/live
We already have in there 01-slim-boot.squashfs + vmlinuz and initrd.img. Now is time to reboot and enter the debian live system.
Use this grub legacy code (for kernel 686 in my example):
Code:
title Debian-live
root=(hd0,0)
kernel /live/vmlinuz boot=live config persistent swapon quickreboot noprompt noautologin
initrd /live/initrd.img
boot


If all went well you will get to debian login prompt. Login as user with password live
Then create root password:
Code:
sudo passwd root

Type and confirm the password.
Code:
sudo login

Login as root and run:
Code:
apt-get update

Install dialog with:
Code:
apt-get install dialog


================================

If you are not going to change the kernel with 3.2.0-0.bpo.4-rt-686-pae don't read this section.
Code:
apt-cache search linux-image-3

Choose the kernel you prefer or thisone:
Code:
apt-get -t squeeze-backports install linux-image-3.2.0-0.bpo.4-rt-pae firmware-linux-free

I had a breaking here because of low ram issue while depmod. If it happen type:
Code:
dpkg --configure -a

It will finish the process.
Code:
cd /boot
ls

Now create initrd.img for the newer kernel:
Code:
mkinitramfs -o initrd-3.2.0-0.bpo.4-rt-pae.img 3.2.0-0.bpo.4-rt-pae


Then copy the vmlinuz and initrd to live folder and rename them to initrd.img and vmlinuz after removing the oldonce.
You might also have to create link from /boot/initrd.img to / if it is broken but it does not matter since we will remove /boot folder at the end.
If everything went well after reboot you will have new kernel. Type to confirm:
Code:
uname -r


Then remove the older kernels with:
Code:
apt-get purge linux-image-2.6.32-5-486
apt-get purge linux-image-2.6.32-5-686

Code:
apt-get autoremove


--======================================

Last edited by saintless on Fri 02 May 2014, 01:29; edited 25 times in total
Back to top
View user's profile Send private message MSN Messenger 
saintless


Joined: 11 Jun 2011
Posts: 2574
Location: Bulgaria

PostPosted: Fri 06 Dec 2013, 06:47    Post subject: Removing packages
Subject description: Removing packages
 

You can see list of installed packages here or in /live folder:
http://live.debian.net/files/oldstable/images/6.0.0/i386/iso-hybrid/debian-live-6.0.0-i386-standard.iso.packages
It is only a list with no description. The better place to choose what to remove is /var/lib/dpkg Check out available or status files. You will have information what is the priority of every package - essential, extra, etc. Be careful not to change anything in dpkg folder.
You can open this available on another computer or make a smaller list with packages to remove, place somewhere and open it from time to time with nano while removing packages.
By the way I prefer to replace nano with e3. If you do this after opening file with e3 the exit is (hold CRL and press one by another K and X).
Start removing packages with:
Code:
apt-get purge package-name
apt-get autoremove

If you try to remove essential package or dependency you will get a message to confirm with:
Code:
Yes, do as I say!

Don't do it! I did it ones and it was a bad idea. I ended up with broken apt-get and had to start all over again.
When you finish and don't think you can remove more it is time to install:
Code:
apt-get install squashfs-tools

And if you like you can do now distro upgrade to the newest packages:
Code:
apt-get dist-upgrade

It will not take much extra space.
Make sure squashfs-tools supports xz compression:
Code:
mksquashfs --h

You should see xz and -Xdict-size options in the output.
Just in case something is installed run:
Code:
apt-get clean

to make sure /var/cache/apt/archives is empty.
Now is time to squash back live-rw
Code:
mksquashfs /live/cow /live/image/live/new.squashfs

You don't need to use maximum compression yet because we have more cleaning to do from GUI.
All your changes are saved now in /mnt/sda1/live/new.squashfs (assuming we use puppy). The path to this place from debian live is different as you see from mksquashfs command: /live/image/live/new.squashfs
You can use safer approach here by mounting live-rw from puppy and copying from command line the content in another folder.
Now before reboot unsquash new.squashfs with this command:
Code:
unsquashfs -d /live/image/new-sfs /live/image/live/new.squashfs

We have the content of new.squashfs in /live/image/new-sfs (or /mnt/sda1/new-sfs from puppy).
Reboot the computer and go back to puppy.

Last edited by saintless on Sat 07 Dec 2013, 05:23; edited 7 times in total
Back to top
View user's profile Send private message MSN Messenger 
saintless


Joined: 11 Jun 2011
Posts: 2574
Location: Bulgaria

PostPosted: Fri 06 Dec 2013, 06:47    Post subject: Last cleaning of files
Subject description: Last cleaning of files
 

The last cleaning we do from GUI.
Open /mnt/sda1/new-sfs folder and start removing this list of files and folders (lets pretend new-sfs is the top of a partition):

/.wh..wh.orph
/.wh..wh.plnk
/.wh..wh.aufs
+ any file starting with .wh here
from folder /etc remove:
fstab, mtab, group- , gshadow- , passwd- ,
open resolv.conf to clean your dns servers and save it. You can also remove it. It will be recreated on boot.
From /etc/udev/rules.d remove:
70-persistent-cd.rules, 70-persistent-net.rules
Check out /etc/network/interfaces If it has some of yours network settins remove them. The file should look like this:
Code:
auto lo
iface lo inet loopback

allow-hotplug eth0
iface eth0 inet dhcp

You can also remove it. It will be recreated on boot.
Check out live, media, mnt, tmp folders. They have to be empty.
From /root remove:
.xsession-errors, .bash_history
Remove everything from /var/cache/apt/archives
From /var/cache/apt/ remove:
pkgcache.bin and rcpkgcache.bin
They will be created again after apt-get update command.
From /var/lib/apt/lists and /var/lib/apt/lists/partial remove the files bigger than 1 Mb. You will see file over 30Mb there. I think you can delete all but the rest is too small to deal with it. The files wil be created again after apt-get update command.

Replase the content of:
/usr/share/doc
/usr/share/man
/usr/share/info
with zerosize files with the same names.
This is great idea from JBV included in a special script for FoxyRoxyLinux.
It can be edited a little to do the same for the files in any folder.
It also searches in all folders for empty .wh files and removes them.
This is part of his script I use for this purpose.
By the way if you have time take a look what can be done with scripts in debian-live system:
http://foxyroxylinux.com/viewtopic.php?f=2&t=108
http://foxyroxylinux.com/viewforum.php?f=9&sid=c4c04c4d51c869734e5e8eede21d6573

Code:
#!/bin/bash
#edited part of 95-clean script created by JBV for foxyroxylinux.com

zerosize() {
  find $* | while read file; do
    echo -n "."
    rm -f $file
    touch $file
  done
}

    zerosize /live/image/02/usr/share/doc -type f -size +1c
    zerosize /live/image/02/usr/share/doc -type l

    zerosize /live/image/02/usr/share/man -type f -size +1c
    zerosize /live/image/02/usr/share/man -type l


    zerosize /live/image/02/usr/share/info -type f -size +1c
    zerosize /live/image/02/usr/share/info -type l

    find /live/image/02 -name '.wh*' -delete

chown -R man:root /live/image/02/usr/share/man

Replace /live/image/02 with the path you use for new-sfs folder.

Now go back to the top and move /boot folder and /vmlinuz and /initrd.img links to another folder. Create squashfs with them in case you need them for upgrading the kernel and removing the older one from the system. initrd.img is already compressed file and you will save at least 15 Mb compressed space as a result. The system will boot without them and missing /boot does not cause any errors as far as the test shows to the moment.
You can load /boot as a squash at any time and any order. It will not break dpkg database and new installed packages.

The last thing is to compress back new-sfs into squash file.
This command gives me best xz compression results to the moment:
Code:
mksquashfs /live/image/new-sfs /live/image/01-filesystem.squashfs -b 1048576 -comp xz -Xbcj x86

Place 01-filesystem.squashfs in /live folder, remove slim-boot.squashfs and reboot.

So, this is all from me to the moment. If something is not clear just ask questions. If I remeber or discover something new it will be added as new post. Happy slimming down live systems Smile

Last edited by saintless on Fri 24 Jan 2014, 13:16; edited 2 times in total
Back to top
View user's profile Send private message MSN Messenger 
sunburnt


Joined: 08 Jun 2005
Posts: 5037
Location: Arizona, U.S.A.

PostPosted: Sun 08 Dec 2013, 03:15    Post subject:  

I looked at the Foxy Roxy site, a NAS unit huh? I thought about making one, but now you can buy a NAS box with no-HDs for about $100 U.S.
So with a pair of 3 GB HDs it comes to about $300 U.S. It`s much smaller and it uses less power with less heat than a PC.

Wow... This code you`ve got looks like it could be wizard scripted pretty easy. There`s decisions to be made (guis) and automating of the process of course.

I need to clarify the sequence of work and start making scripts for each stage. I also need to refresh my knowledge of dpkg, apt-get, and the other pkg. man.s.
My interest is using the pkg. man. to build AppDir pkgs., not to install apps. The O.S. only uses AppDir pkgs, no legacy apps, no sfs, so no union needed.
This simplifies the O.S., and finally gets rid of the multitude of union problems. I`ve thought about this for years now and this looks like the chance to do it.

# What do you think about scripting a wizard for this build setup.?
.
Back to top
View user's profile Send private message 
saintless


Joined: 11 Jun 2011
Posts: 2574
Location: Bulgaria

PostPosted: Sun 08 Dec 2013, 05:16    Post subject:  

sunburnt wrote:
# What do you think about scripting a wizard for this build setup.?
.

It will be nice. By the way JBV made this code much more complicated and polished. It displays messages and gives options to say yes or no. It is only small rough edited part from his script which I use to clean quick before squashing back.
He is doing miracles with FoxyRoxy. It has a script for adding or removing packages in the main squash file and rebuild it again. He solved also the the problem with breaking dpkg database if you load extra squash files in not correct order. He created orphan squash files which can be removed or added to the system while updating dpkg database at the same time. And much much more. I'm inside this process and I still don't get all he does with this distro.

Cheers, Toni

_________________
Farewell, Nooby, you will be missed...
Back to top
View user's profile Send private message MSN Messenger 
sunburnt


Joined: 08 Jun 2005
Posts: 5037
Location: Arizona, U.S.A.

PostPosted: Sun 08 Dec 2013, 14:07    Post subject:  

I`ll look closer at the Foxy site to see what I can get from it, scripts, etc.

If there`s any help you need, guis, scripts, etc., let me know and I`ll focus on it.
Back to top
View user's profile Send private message 
saintless


Joined: 11 Jun 2011
Posts: 2574
Location: Bulgaria

PostPosted: Sun 08 Dec 2013, 15:23    Post subject:  

sunburnt wrote:
If there`s any help you need, guis, scripts, etc., let me know and I`ll focus on it.

Thanks, Sunburnt,

I will ask you for help when my head starts to hurt too much from banging the desk Smile
Today I made my first trial slimming down wheezy. There is hope to get some acceptable size for core and to add more information in this thread.

Cheers, Toni

_________________
Farewell, Nooby, you will be missed...
Back to top
View user's profile Send private message MSN Messenger 
go2null

Joined: 23 Oct 2011
Posts: 11

PostPosted: Tue 10 Dec 2013, 22:14    Post subject: Debian Live Web Builder  

@saintless: Have you tried using the Debian Live Web Builder (Manual)?
Back to top
View user's profile Send private message 
saintless


Joined: 11 Jun 2011
Posts: 2574
Location: Bulgaria

PostPosted: Wed 11 Dec 2013, 01:23    Post subject: Re: Debian Live Web Builder  

go2null wrote:
@saintless: Have you tried using the Debian Live Web Builder (Manual)?

No, but I will try it. Thanks.

BTW I know Sickgut used this method. I'm not sure what exactly was the process after building it but he ended up with 55 Mb squash file without xorg. Light-Debian was made the way described here and the result is the same size + newer bigger kernel, xorg, xfe, web browser.
Back to top
View user's profile Send private message MSN Messenger 
sunburnt


Joined: 08 Jun 2005
Posts: 5037
Location: Arizona, U.S.A.

PostPosted: Wed 11 Dec 2013, 03:21    Post subject:  

Wow... Looked at the Debian Live site and it`s typical of them. What a mass of info.
But as much as there is, it doesn`t clarify much. Takes a Deb. install to make a Live.
There is an online builder, it`s a possibility, but again it`s not real clear how to use it.
Very comprehensive, does distro, arch, media, and a bunch of other stuff too.
Might be a good way to go, but it`s gonna be a large learning curve for sure.

Saintless; I`m thinking we can add to your Squeeze the Deb-Live build setup with apt-get.
Code:
# apt-get install live-build

Then we can all feed info to each other as we go. My slow web won`t allow big iso files.
I`m reading the starting examples and tutorials, and it`s simple to get a Lxde or Xfce iso.
http://live-systems.org/manual/current/html/live-manual/examples.en.html#examples
Quote:
In this tutorial, we will create an image suitable for use as a web browser utility, serving as an introduction to customizing live system images.
783

$ mkdir tutorial2
$ cd tutorial2
$ echo "task-lxde-desktop iceweasel" >> config/package-lists/my.list.chroot

784

Our choice of LXDE for this example reflects our desire to provide a minimal desktop environment, since the focus of the image is the single use we have in mind, the web browser. We could go even further and provide a default configuration for the web browser in config/includes.chroot/etc/iceweasel/profile/, or additional support packages for viewing various kinds of web content, but we leave this as an exercise for the reader.
785

Build the image, again as superuser, keeping a log as in Tutorial 1:
786

# lb build 2>&1 | tee build.log
Back to top
View user's profile Send private message 
saintless


Joined: 11 Jun 2011
Posts: 2574
Location: Bulgaria

PostPosted: Wed 11 Dec 2013, 05:15    Post subject:  

sunburnt wrote:
Saintless; I`m thinking we can add to your Squeeze the Deb-Live build setup with apt-get.
Code:
# apt-get install live-build

Hi, Sunburnt,
included in the wheezy version for now. No problem to add it in squeeze. No problem to remove it if we want later.
Working on wheezy today.

Cheers, Toni
Back to top
View user's profile Send private message MSN Messenger 
go2null

Joined: 23 Oct 2011
Posts: 11

PostPosted: Wed 11 Dec 2013, 10:45    Post subject:  

sunburnt wrote:
Wow... Looked at the Debian Live site and it`s typical of them. What a mass of info.
But as much as there is, it doesn`t clarify much. Takes a Deb. install to make a Live.
There is an online builder, it`s a possibility, but again it`s not real clear how to use it.
Very comprehensive, does distro, arch, media, and a bunch of other stuff too.
Might be a good way to go, but it`s gonna be a large learning curve for sure.


I know Smile That's why it's still on my TODO list.
Back to top
View user's profile Send private message 
emil

Joined: 10 Nov 2009
Posts: 618
Location: Austria

PostPosted: Sun 05 Jan 2014, 04:42    Post subject: Automatization and package handling  

Hi,

saintless wrote (in debin light wheezy thread):
Quote:
The biggest problem for me is to automate the process for every user this way - install package --> clean it the best way to be small and working --> make sfs from it --> rebuild it in the main squashfs as an option --> do not brake dpkg database (most important). If we can do this to be executed with script without need from user confirmation it will be very good solution. Then anyone can create sfs files with one or multiple programs he/she needs.


as saintless mentioned I hope some steps forward are achieved with some sort of automatization and standartisation of the tools used to produce small images and handle packages. At the moment I see 3 main areas:

1) building and stripping the base image
2) use of squashfs (sfs) modules
3) user generated deb packages (conversion of pet to deb)

Currently manual cleaning and modifying is used to prepare a working system. I would call it "hacking the sfs". This approach may be faster and easier for the moment, but in the long term I would prefer a customized buildscript, which should be more maintainable and would reduce future tedious work. The debian live build system allows for extended customisation (e.g. "hooks"). There are very interesting examples of projects which use the live-build tools, and I wonder if anybody here is still using it to get small base image:

example1 (crunchbang-debianlive): http://crunchbang.org/forums/viewtopic.php?id=4753 (build script, interesting because it uses openbox)
example2 (sage debian live): http://sagedebianlive.metelu.net/ - (look under Source Code/Browse files: config.sh, build.sh)
example3 (minimal debian live, example from manual): http://live.debian.net/manual/current/html/live-manual/examples.en.html#831

The minimal example Nr. 3 contains discussion of a minimization tool "stripping.hook" /written by the live-tools author D. Baumann).
There are others or similar:
95-snap.sh (FoxyRoxy) "zerosizes" files to avoid package errors.
Iguleder's package tools have the strippkg and mkpkg scripts.
Last but not least, new2dir from Puppy.

For me it seems that the stripping/splitting of the base image should be automated (to the greatest extend) with the same algorithms as the stripping of further packages or squashfs files.

I wrote this mainly for me as a a reminder to links for a starting point.
kind regards
emil
Back to top
View user's profile Send private message Visit poster's website 
saintless


Joined: 11 Jun 2011
Posts: 2574
Location: Bulgaria

PostPosted: Sun 05 Jan 2014, 06:01    Post subject:  

Hi, Emil,

Thank you for the links. I will check them all when I have some spare time to do it.
JBV created most of the scripts we need in FoxyRoxyLinux but the cleaning process could be improved in order to make the sfs file smaller as we can.
I will not be able to create professional made script but I think I can create a script with simple commands in order to be executed automatic (as example of my point of view what is the best) and then someone maybe will make something better from it.

Cheers, Toni

_________________
Farewell, Nooby, you will be missed...
Back to top
View user's profile Send private message MSN Messenger 
emil

Joined: 10 Nov 2009
Posts: 618
Location: Austria

PostPosted: Sun 05 Jan 2014, 06:26    Post subject:  

Hi Toni,
I think in the spirit of this forum everybody should work on the projects he likes and do it in a way which he feels works best for him. So don't feel obligation to produce such scripts yourself, you already do a lot.
cheers
Back to top
View user's profile Send private message Visit poster's website 
Display posts from previous:   Sort by:   
Page 1 of 1 [15 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » HOWTO ( Solutions )
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1389s ][ Queries: 11 (0.0038s) ][ GZIP on ]