Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Mon 24 Nov 2014, 04:25
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
roar-ng and Subito GNU/Linux 0.9.5 Beta
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 12 [172 Posts]   Goto page: 1, 2, 3, ..., 10, 11, 12 Next
Author Message
Iguleder


Joined: 11 Aug 2009
Posts: 1923
Location: Israel, somewhere in the beautiful desert

PostPosted: Sat 04 Feb 2012, 15:17    Post subject:  roar-ng and Subito GNU/Linux 0.9.5 Beta
Subject description: A fast Woof competitor and a Puppy 6 concept
 

Introduction

roar-ng is a generic distribution building system originally forked from Woof, the Puppy Linux build system.

You can follow its development in GitHub, where I host its code now. If you want to use it, check out its code using Git.

It provides an architecture-independent, flexible and portable infrastructure for the creation of fast, "live" GNU/Linux distributions with support for persistency and dynamic loading of modules. It provides support for the binary package format and repositories of various GNU/Linux distributions.

The development of roar-ng started as a collection of source hacks of Woof and evolved into a complete, independent re-implementation. It provides advanced features not found in Woof, such as parallel downloads, automatic package splitting, simple branding and easy porting to different processor architectures.

Overview

roar-ng, as mentioned earlier, is a build system: it's a tool which receives a list of binary packages, appropriate download locations and the result distribution's details as its input and outputs a ready, bootable image of the distribution.

It consists of 5 main scripts; their names and roles are inherited from Woof:
- 0setup - a script which downloads the package repositories information and converts it into a common format used by roar-ng in order to provide the support for multiple distributions.
- 1download - a script used to download all binary packages specified in the package list.
- 2createpackages - a package processing script used to extract, customize, optimize and split the downloaded binary packages.
- 3builddistro - a script which builds a bootable image of tDistribution Features
---------------------

The distributions built using roar-ng are unique and can compete directly with any popular "live" distribution easily. Their main features are:
- Easy maintenance - low cost, easy development which makes the product live longer and makes it more resilient to the biggest enemies of distributions - loss of developers or community support.
- Portability - the ability to run from any media supported by the kernel.
- Modularity - the ability to load extensions dynamically.
- Persistence - persistent sessions through automatically created save files.
- Extensibility - good quality, generic and well-documented code.
- Simplicity - simple configuration, package management and usage.
- "Native" feel - although the distribution is "live", it does not feel that way; it provides full access to the file system.
he distribution from the processed packages and an operating system skeleton shared by all the distributions built using the build system.
- 4buildpackage - a script which builds binary package within the distribution built using the former script, using build scripts.

With proper input, roar-ng should successfuly output a solid distribution.

History

The development of roar-ng was triggered by personal frustration with Woof's 3builddistro script, which wasn't flexible enough for a Puppy Linux derivative called "Next Puppy" and its successor, "Guy Dog". It started during the development of a Puppy Linux derivative codenamed "Humble Puppy", which was based on the Slackware-based Puppy Linux 5.3.1.

At first, roar-ng consisted of a modified 3builddistro script, which was much simpler and faster than the original, but produced a very similar result and provided minimal benefit. This modified script was soon replaced by a clean re-implementation, which provided much smaller output. In addition, several modifications were made to the Puppy Linux skeleton, which removed many legacy or rarely used components, while extras (such as a new package manager) were provided as packages.

Then, 1download was the second script modified; the most notable addition was parallel download of binary packages, which made it considerably faster. This addition made Woof faster.

Finally, 2createpackages, the most complex script of Woof, had very little room for improvement because of its poor quality, complexity and inefficience. This is the point where roar-ng's development as an independent projected started.

Finally, Builder, an automated package building framework, was merged into roar-ng during a big cycle major changes, which cut the final ties to Woof and Puppy Linux.

It is worth mentioning the roar-ng inherits much from Calf GNU/Linux, a concept distribution developed between 2010 and 2011. It served as a proof-of-concept for the developer's view of what Puppy Linux 6.0 should be.

Usage

roar-ng's configuration consists of 3 files:
- bootrc - boot settings; this file is read by the initial init script.
- distrorc - basic distribution details, such as its name, version, etc'.
- package_list - the list of packages included in the distribution; each package has its own source distribution (e.g the distribution it was taken from).

The configuration files should be prepared before roar-ng's scripts are executed. However, modification of the package list is possible after their execution.

Once the configuration is ready, the scripts should be executed in the following order:
- 0setup
- 1download
- 2createpackages
- 3builddistro

roar-ng's execution does not take a long time, when provided with a fast network connection and fast mirrors, since 1download is the slowest script to execute. 4buildpackage is an optional script useful only in cases where automatic package building is needed.

Distribution Features

The distributions built using roar-ng are unique and can compete directly with any popular "live" distribution easily. Their main features are:
- Easy maintenance - low cost, easy development which makes the product live longer and makes it more resilient to the biggest enemies of distributions - loss of developers or community support.
- Portability - the ability to run from any media supported by the kernel.
- Modularity - the ability to load extensions dynamically.
- Persistence - persistent sessions through automatically created save files.
- Extensibility - good quality, generic and well-documented code.
- Simplicity - simple configuration, package management and usage.
- "Native" feel - although the distribution is "live", it does not feel that way; it provides full access to the file system.

Dependencies

It is worth mentioning that roar-ng has several requirements; some packages must be included in the result distribution in order to make roar-ng's distribution skeleton function.
This is the full list of must-have dependencies of roar-ng and the
distributions built by it. Also, it specifies all special, "system" files
created by the init scripts and the distribution skeleton.

roar-ng Itself
- A POSIX-compliant shell (either Bash or DASH)
- Python (for package list conversion scripts called by 0setup)
- Squashfs tools
- cdrkit or cdrtools (for either mkisofs or genisoimage, respectively)
- GNU Binutils (for "strip")
- cpio
- gzip
- Squashfs tools
- For 4buildpackage: Aufs, in the host's kernel
- Recommended: aria2, for parallel downloads - makes 1download much faster.
- Recommended: AdvanceCOMP, OptiPNG and a recent version of file (for package optimization)

Distribution Kernel
- Squashfs (in mainline, built into kernel)
- Aufs (built into kernel)
- Drivers for all devices the distribution can boot from, built into the kernel
- File system drivers for file systems the distribution can load save files from, built into the kernel

Distribution Packages
- DASH (for many scripts)
- Busybox (with mdev)
- dialog (used for some wizards)
- udev
- dialog (used by many wizards)
- iptables (used by /etc/init.d/firewall)
- hsetroot (used by xinitrc), which pulls in imlib2
- gtkdialog (used by some wizards)
- cwm (the default window manager), which pulls in libbsd
- syslinux (to make the result ISO image bootable)
- librsvg (to generate PNG logo images from the default, SVG one)

Subito GNU/Linux Alpha 1

As mentioned earlier, Subito GNU/Linux is the flagship distribution built by roar-ng and its main use. Here's the beta release, in all its glory.

In its current form, Subito GNU/Linux has only a x86_64 port, so you'll need compatible hardware to use it.

ISO: subito-0.9.5.iso
devx: devx_subito-0.9.0.sfs
MD5: md5sums.txt

This ISO is hybrid, which means you can make a live USB easily, just dd it to the device. Wink

Have fun!
screeny.png
Description  My Subito GNU/Linux 0.9.5 desktop
png

 Download 
Filename  screeny.png 
Filesize  272.18 KB 
Downloaded  1197 Time(s) 
screeny.png
Description  Screeny of Subito GNU/Linux
png

 Download 
Filename  screeny.png 
Filesize  180.3 KB 
Downloaded  1400 Time(s) 

_________________
My homepage

Last edited by Iguleder on Sat 03 Mar 2012, 12:52; edited 10 times in total
Back to top
View user's profile Send private message Visit poster's website MSN Messenger 
ICQ Number 
laurentius77

Joined: 30 Mar 2011
Posts: 61

PostPosted: Sat 04 Feb 2012, 15:43    Post subject:  

I used your tutorials in the past for a no sata/ide modules kernel compiling. In fact they were only tutorials that I could understand and follow and I got a great kernel. I read your posts about Puppy 6 and woof competition. This is the way we should follow, we need revolution not just evolution. Life is almost evolution but sometimes we need something else. I hope your roar-ng solution will succed and maybe we can see our puppy ported on smartphones/tables sometime. Keep going, you are doing a great job.
Now i should test your roar-ng, I know it is great...
Back to top
View user's profile Send private message 
nooby

Joined: 29 Jun 2008
Posts: 10557
Location: SwedenEurope

PostPosted: Sat 04 Feb 2012, 18:16    Post subject:  

iguleder, I am a total noob so I don't get much
but admire what you do.

Hope it is okay if I ask a very naive question.
What your roar-ng allow to build a multi user
Subito then or can it only build single user OS?

No I don't want Puppy to change to be multi user
but it would be cool to have an OS that is 100%
compatible with Puppy repo and that can chose
to be multi user if one wants to but have single
user as the primary set up. The multi can be used
when one get visits from people that only feel
at home with being multi user.

I don't mind if you want Subito to be single user.

I only ask if roar-ng can make a multi user if one tell
it too? Not that you should do it. And if it is not possible
what is lacking for that to get accomplished?

_________________
I use Google Search on Puppy Forum
not an ideal solution though
Back to top
View user's profile Send private message 
pemasu


Joined: 08 Jul 2009
Posts: 5465
Location: Finland

PostPosted: Sat 04 Feb 2012, 18:24    Post subject:  

Posting this from subito. Build by your building system. I just executed the scripts one after another...starting from 0setup.
1download stopped about 10 times...probably the connection timeout exceeded. And the script stopped. I had to launch it again and then it continued. There is vmlinuz and about 150 Mb initrd.gz. Is that intentional. It build devx.sfs though. But the main sfs is inside initrd.gz.

Okay...it booted my laptop. I quickly noticed that the eth0 module for me is atl1c. Then I struggled to manually in console to setup my internet connection. I succeeded as you can see.
It took about 2 hours....to download packages...process them and build this. Getting internet connection took 30 mins...because I didnt remember my numerical addresses and the right syntax.

So....using your semiautomatic build system was success without reading anything than quickly your introduction.
Package processing is fast as is the build process. That big initrd.gz just made me hesitate first.

Err....a little better browser would be good. The netsurf hanged all the time and several times went so long unresponsive that I just had to kill it. I didnt know what browser I would have been able to install.
Back to top
View user's profile Send private message 
starhawk

Joined: 22 Nov 2010
Posts: 3137
Location: Everybody knows this is nowhere...

PostPosted: Sat 04 Feb 2012, 20:05    Post subject:  

Now THIS is very interesting... Igu, do you think you could make this as easy-to-use as sc0ttman's Woofy tool? I know it's a highly technical and delicate procedure to build a Puppy (or anything really) in this manner, but if it could be streamlined to that level...

...well, then again, we'd probably have a whole lot more issues to support Laughing

Still, that would be nice.

_________________

Back to top
View user's profile Send private message 
`f00


Joined: 06 Nov 2008
Posts: 809
Location: the Western Reserve

PostPosted: Sat 04 Feb 2012, 21:24    Post subject:  

Happy Birthday Very Happy to roar-ng and Subito

Cigar/ to the gentleman at the top
Back to top
View user's profile Send private message 
Iguleder


Joined: 11 Aug 2009
Posts: 1923
Location: Israel, somewhere in the beautiful desert

PostPosted: Sun 05 Feb 2012, 03:15    Post subject:  

Thanks for the comments, guys!

nooby - yes, it can be made multi-user very easily. I'm currently working on create_user or whatever it was, it's under /opt/sbin. It creates a new user and does all the setup automatically. And yes, Subito is 100% binary-compatible with Slacko, except the fact it's x86_64. I worked on an i486 flavor before I built the x86_64 port (which became my main focus, since it's more challenging due to the bigger size and requirements) - I might restore this effort once I get the x86_64 flavor stable and usable.

starhawk - I might build some GUI maybe, to make it easier to use. Yad and gtkdialog are included in the distro, of course Wink

pemasu - yes, it is intentional. When the SFS is on a partition -
- You can't boot the distro unless you can mount the partition - which means the distro isn't truly portable.
- You have to locate the SFS, which takes a considerable amount of time.
- The benefit from using a compressed file system is minimal, if any.

When the main SFS is in the initramfs, you read a compressed file system from the system RAM, which means it consumes the compressed size, but it's faster to decompress something you read from RAM than decompress and read (which is the big overhead here) it from a hard drive, flash drive or some optical media.

EDIT: pemasu, try /opt/sbin/load_sfs on the devx Wink

_________________
My homepage
Back to top
View user's profile Send private message Visit poster's website MSN Messenger 
ICQ Number 
01micko


Joined: 11 Oct 2008
Posts: 7841
Location: qld

PostPosted: Sun 05 Feb 2012, 08:19    Post subject:  

Pretty minimal fuss really Smile

For those needing a tool to connect, very light but easy to use commandline tool you can use pns_tool.

BTW, I had to enable nouveau in the specs Smile. Kernel module nouveau was loading but no xorg driver! Wasted a CD Rolling Eyes

I see you have not much hard coded to /root... a very good thing.. what's there now could easy be moved to /etc or wherever appropriate.

_________________
Woof Mailing List | keep the faith Cool |
Back to top
View user's profile Send private message Visit poster's website 
tronkel


Joined: 30 Sep 2005
Posts: 1104
Location: Vienna Austria

PostPosted: Sun 05 Feb 2012, 09:28    Post subject:  

Got the ISO built OK - nice fast build.

No desktop however because of the ATI HD6450 video card.
Tried nomodeset but that made no difference. This video card requires a linux non-free firmware blob as found in Debian distros

Tried to build with the ISO with the VESA driver but this isn't included in the package set - hence an error message.
Does a VESA driver package exist somewhere that I could use?

_________________
Life is too short to spend it in front of a computer
Back to top
View user's profile Send private message Visit poster's website AIM Address 
pemasu


Joined: 08 Jul 2009
Posts: 5465
Location: Finland

PostPosted: Sun 05 Feb 2012, 11:04    Post subject:  

01micko. Thanks. pns-tool worked like charm. Now I have wlan connection.

Iguleder. I did use sfs_load and first I did load opera.sfs. Now I have usable browser. Netsurf really sucks for me. About useless. Then I did load devx.sfs and downloaded mc source. I compiled it and installed it succesfully. Now I have file browser also. The create_save_file errors. Of course I dont know the syntac to use it but that does not inhibit me to try.
create_save_file /mnt/sda6/slacko/savefile.sfs anyway errors with...

cp: cannot create directory `/mnt/tmp.XXXX1nJCLN/usr': No space left on device
cp: cannot create directory `/mnt/tmp.XXXX1nJCLN/var': No space left on device

I get many console pages of those kind. This is fun anyway....

Vifm does not do anything for me...well...trying to kill it takes some time...

But mc from source works ok. At least older 4.6.1, new one cries after some newer dependency which I didnt bother to find out.
Back to top
View user's profile Send private message 
pemasu


Joined: 08 Jul 2009
Posts: 5465
Location: Finland

PostPosted: Sun 05 Feb 2012, 13:26    Post subject:  

I will stop now this test period. I got pets to install with hpm-install...like hpm-install xchat-2.8.8-x86_64.pet

And I used fatdog64 repo:
http://distro.ibiblio.org/pub/linux/distributions/fatdog/pets/500/

I changed to finnish keyboard with setxkbmap fi

Okay...now I will change back to ordinary Puppy. How to create save file would be nice to know.

Where is the configure file for wbar, I tried to edit /etc/opt/default/wbar/wbar.cfg but I couldnt get it to work.
Back to top
View user's profile Send private message 
8-bit


Joined: 03 Apr 2007
Posts: 3393
Location: Oregon

PostPosted: Mon 06 Feb 2012, 05:10    Post subject:  

I also just ran your scripts to try the build. I had step2 in the 0-3 steps stop dead on me trying to get past python. I then ran step 2 again and it continued on through. I ran step 3 and found the iso file in the distro directory.
But did I do something wrong? It is only 8 megs in size.
I went ahead and burnt a cd and tried to boot from it.
I got a kernel panic.
But that, I kind of expected as I was booting it on a dual-core amd processor PC. But.... That PC will NOT run 64bit applications as it was built with a 32bit buss.
Yeah, I know. An almost 64bit PC.
I have as of yet to try it on my laptop that runs Windows 7 64 bit.
But the size of the ISO has me concerned as to if everything got included.

I mean 8 megs is really small for an ISO!

I assume I could build a devx file with dir2sfs using the devx directory that was created.
Back to top
View user's profile Send private message 
pemasu


Joined: 08 Jul 2009
Posts: 5465
Location: Finland

PostPosted: Mon 06 Feb 2012, 07:07    Post subject:  

I suppose you had devx sfs of your used Puppy loaded. It has python included...at least it should, if the used Puppy has been created by Barrys woof and 3builddistro which moves python to devx sfs and the distro specs has python included.

Or just install python to your build, but devx sfs is usually good to be loaded when you build a distro. Strip binary is always needed.

Then you wont get python errors. Iso size was about 156 Mb +/- couple of megs.

With other words. Your build was failure.
Back to top
View user's profile Send private message 
8-bit


Joined: 03 Apr 2007
Posts: 3393
Location: Oregon

PostPosted: Mon 06 Feb 2012, 07:32    Post subject:  

I compiled it in Lupu 520. I had its devx and kernel source loaded when I ran the scripts.
If it used that kernel source SFS that is for lupu, that could have been part of the problem.
I just have never built a SIO before.

The right way to do so, I do not know yet.
I will have to do some more investigation and try it again.
But by never having done such, I have a lot to learn.
Thank you for telling me the size of ISO I should end up with.
Also, it looks like if I had installed the download accelerator it would have helped the download of the files as to speed.
I think the stopping of step 3 messed things up also as in restarting that step, I missed a bunch of files getting made.
Back to top
View user's profile Send private message 
01micko


Joined: 11 Oct 2008
Posts: 7841
Location: qld

PostPosted: Mon 06 Feb 2012, 07:58    Post subject:  

8-bit

Lupu doesn't support xz compression so you have to comment the options to mksquashfs in 3builddistro. I forget the exact line but you'll see a variable set for the options, commenting that will be enough. Be aware that your iso will be bigger.. ~200MB maybe, but it should work ok..
hth

_________________
Woof Mailing List | keep the faith Cool |
Back to top
View user's profile Send private message Visit poster's website 
Display posts from previous:   Sort by:   
Page 1 of 12 [172 Posts]   Goto page: 1, 2, 3, ..., 10, 11, 12 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1286s ][ Queries: 13 (0.0174s) ][ GZIP on ]