roar-ng and Subito GNU/Linux 0.9.5 Beta

Under development: PCMCIA, wireless, etc.
Post Reply
Message
Author
User avatar
pemasu
Posts: 5474
Joined: Wed 08 Jul 2009, 12:26
Location: Finland

#16 Post by pemasu »

Ahh. I have been so accustomed to .xz compression support...long time now...that I had forgot that not all Puppies have it. Good to know. Thanks 01micko.

nooby
Posts: 10369
Joined: Sun 29 Jun 2008, 19:05
Location: SwedenEurope

#17 Post by nooby »

At the moment it's able to produce a concept distro called "Subito", for the x86_64 processor architecture, which consists of Slackware 13.37 and Puppy packages, just like Slacko
Does that mean that wen 8-bit use lucid puppy to bild then it fails but if
one use slacko then it would work?

This is 64 bit is it backward compatible with 32 bit computers?
I use Google Search on Puppy Forum
not an ideal solution though

User avatar
8-bit
Posts: 3406
Joined: Wed 04 Apr 2007, 03:37
Location: Oregon

#18 Post by 8-bit »

pemasu wrote:Ahh. I have been so accustomed to .xz compression support...long time now...that I had forgot that not all Puppies have it. Good to know. Thanks 01micko.
So how do I add, say .xz compression support to a puppy version?
And with the init scripts for some of the packages, it appears it is needed.

01micko,
I just as soon would not want to edit the 3builddistro script to add options to mksquashfs. It seems like the making of SFS files should be a visible option in the first 0setup script.
Or can one use the "bones_GUI" to set all the options one wants?

I really would like to see roar-ing have a setup GUI to do the modifications needed depending on the options one would want.
I am thinking that would keep manual editing and creation of files like "DISTRO_SPECS" from having to be done. A GUI would take care of all the setup and optionally run all the scripts 0-3 to build the ISO.

In other words, it would be a GUI of "Building Puppy for the Complete IDIOT".
You can include me in that category.
Also, one time previously when I tried to build an ISO, I did not know about selecting the packages I wanted included and ended downloading all the files for all the versions of puppy.

If this Idiot is not making himself clear, you know the reason.

User avatar
MinHundHettePerro
Posts: 852
Joined: Thu 05 Feb 2009, 22:22
Location: SE

#19 Post by MinHundHettePerro »

Hiya :)!

Trying to build a slacko-5.3.1 (yes, 32-bit - haven't got a 64-machine, so there wouldn't be any use in going 64-bit here) clone to evaluate this builder.

Basically, I'm (re-)using a modified DISTRO_PKGS_SPECS-slackware-13.37 from slacko-5.3.1, which seem to have a few legacy "exe>dev"s, resulting in 1download not downloading these packages.

Are these (legacy of woof) redirects like "exe>dev" to be avoided in roar-ng - and, perhaps, replaced by "exe,dev".

Sorry, if this was mentioned somewhere and I missed it .......

Cheers :)/ MHHP

EDIT:
2createpackages still hangs at python, whether first or second entry of

Code: Select all

#yes|python|python|exe>dev,dev,doc,nls
yes|python|python|exe,dev,doc,nls
is used. htop says something like

Code: Select all

file -bi -
Continues (at next package after python), though, if 2createpackages is CTRL-C:d and restarted again ...

Code: Select all

python-2.6.6|python|2.6.6|1|BuildingBlock|71410K|slackware/d|python-2.6.6-i486-1.txz||object-oriented interpreted programming language|slackware|13.37|official
EDIT2:
And it stops at dietlibc

Code: Select all

yes|dietlibc||exe,dev,doc,nls
, since processed-packages/dietlibc/ ends up empty - there're only the _DEV and _DOC dirs.

Continues after making a symlink processed-packages/dietlibc/ to processed-packages/dietlibc_DEV.........

EDIT 3:
Sub-packages that are redirected in PKGS_SPECS_TABLE do net get downloaded.
One possible fix for this is:

Code: Select all

# diff support/functions ../roar-ng-001_org/support/functions 
117c117
< 					enabled_sub_packages="$enabled_sub_packages ${sub_package%>*}"
---
> 					enabled_sub_packages="$enabled_sub_packages $sub_package"
124c124
< 		for sub_package in $enabled_sub_packages
---
> 		for sub_package in $sub_packages
# 
There are, apparently, puppy-built packages that dir2pet has split up in PKG, PKG_DEV and PKG_DOC such that e.g. the exe PKG could be empty, save for the pet.specs. This leads to 2createpackages building a package, which is just an empty directory -> 3builddistro stops.
A possible workaround is to make the following changes in 3builddistro, i.e. making sure that the directory is not empty
# diff 3builddistro ../roar-ng-001_org/3builddistro
19,20c19
< # MKSQUASHFS_OPTIONS="-comp xz -Xbcj x86" #MHHP
< MKSQUASHFS_OPTIONS="" <<< yeah, I know ..............
---
> MKSQUASHFS_OPTIONS="-comp xz -Xbcj x86"
61,62c60
< # if [ -d processed-packages/$package ] # MHHP
< if [ -d processed-packages/$package ] && [[ -n `ls -A processed-packages/$package` ]]
---
> if [ -d processed-packages/$package ]
69,70c67
< # if [ -d processed-packages/${package}_DEV ] # MHHP
< if [ -d processed-packages/${package}_DEV ] && [[ -n `ls -A processed-packages/${package}_DEV` ]]
---
> if [ -d processed-packages/${package}_DEV ]
79,80c76
< # if [ -d processed-packages/${package}$suffix ] # MHHP
< if [ -d processed-packages/${package}$suffix ] && [[ -n `ls -A processed-packages/${package}$suffix` ]]
---
> if [ -d processed-packages/${package}$suffix ]
#
And strippkg thinks file-names like e.g. these in slackware-13.37(32-bit) python-2.6.6-i486-1.txz are abominable -> it hangs, with no return

Code: Select all

./usr/doc/python-2.6.6/Misc/TextMate/Python-Dev.tmbundle/Snippets/2 to 3 - Module Deletion (docs).tmSnippet
./usr/doc/python-2.6.6/Misc/TextMate/Python-Dev.tmbundle/info.plist
./usr/doc/python-2.6.6/Misc/TextMate/Python-Dev.tmbundle/Commands/Go to Issue.tmCommand
./usr/doc/python-2.6.6/Misc/TextMate/Python-Dev.tmbundle/Commands/2 to 3 - Module Deletion.tmCommand
I did try to get strippkg to accept these file-names (not sure if it's completely right, and it should probably be worked on by someone knowledgable - that's not merely a happy tinkerer, like myself).
Anyway, I got it going by double-quoting a little in strippkg

Code: Select all

# diff support/strippkg ../roar-ng-001_org/support/strippkg 
51c51
< for file in "$(find "$1" -type f)"
---
> for file in $(find "$1" -type f)
76c76
< 	case "$(file -bi "$file")" in
---
> 	case "$(file -bi $file)" in
# 
Surely could be done better than by my home-brew tinkering, but at least it builds now ... :)

And I really like how you've simplified the building / MHHP
[color=green]Celeron 2.8 GHz, 1 GB, i82845, many ptns, modes 12, 13
Dual Xeon 3.2 GHz, 1 GB, nvidia quadro nvs 285[/color]
Slackos & 214X, ... and Q6xx
[color=darkred]Nämen, vaf....[/color] [color=green]ln -s /dev/null MHHP[/color]

User avatar
MinHundHettePerro
Posts: 852
Joined: Thu 05 Feb 2009, 22:22
Location: SE

#20 Post by MinHundHettePerro »

Rats, it booted, but couldn't log in!!!!!

Code: Select all

# cat skeleton/distro_skeleton/etc/passwd 
root:x:0:0:root:/root:/bin/sh
messagebus:x:503:503:D-Bus User,,,:/tmp:/bin/sh
# 
# cat skeleton/distro_skeleton/etc/shadow 
root::10933:0:99999:7:::
messagebus:!:0:99999:7:::
# 
# cat sandbox3/rootfs-complete/etc/passwd 
sshd:x:33:33:sshd:/:
# 
# cat sandbox3/rootfs-complete/etc/shadow 
sshd:*:9797:0:::::
# 
Excerpt from doinst.sh of slackware-13.37's openssh-5.8p1-i486-1.txz

Code: Select all

# If the sshd user/group/shadow don't exist, add them:

if ! grep -q "^sshd:" etc/passwd ; then
  echo "sshd:x:33:33:sshd:/:" >> etc/passwd
fi

if ! grep -q "^sshd:" etc/group ; then
  echo "sshd::33:sshd" >> etc/group
fi

if ! grep -q "^sshd:" etc/shadow ; then
  echo "sshd:*:9797:0:::::" >> etc/shadow
fi

Well, doinst.sh seems to not have worked against an existing skeleton/distro_skeleton/etc/passwd|shadow ......

Don't know how to fix, will try to rebuild without the disrupting openssh-5.8p1-i486-1.txz.

Would be grateful for input on this ... :)/ MHHP
[color=green]Celeron 2.8 GHz, 1 GB, i82845, many ptns, modes 12, 13
Dual Xeon 3.2 GHz, 1 GB, nvidia quadro nvs 285[/color]
Slackos & 214X, ... and Q6xx
[color=darkred]Nämen, vaf....[/color] [color=green]ln -s /dev/null MHHP[/color]

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#21 Post by Iguleder »

The main reason why it fails hard is the fact it wasn't designed to build Puppy, you're doing something slightly wrong here :lol:

I'm currently working on new splitting and stripping scripts, with emphasis on better performance and the ability to work with files that contain spaces in their paths, which is something that shouldn't appear on a good UNIX-like system.

The overhead of strippkg is very minimal, since it's the optimization itself that takes ages. However, I still want to see where I can improve it.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

gcmartin

#22 Post by gcmartin »

I'm currently working on new splitting and stripping scripts, with emphasis on better performance and the ability to work with files that contain spaces in their paths, which is something that shouldn't appear on a good UNIX-like system.
Don't take this as a negative response because its NOT. But, for what its worth, 98% of the worlds computers, understand, use, and are capable of understanding filenames with spaces. For the past 42 years the industry has been using long filenames (with/without spaces). We should respect that as an obligation to be consistent and not do a "nose up" at this in our industry.

But, as a rule, I agree, that we could encourage the community to use, where necessary, filenames without spaces.

I do understand what you are providing for our use. And I applaud the methodology you are sharing in this community.

Thanks for what you do.

dmabar
Posts: 2
Joined: Thu 09 Feb 2012, 00:50

#23 Post by dmabar »

Greetings everyone.

I am a casual Puppy Linux user who has never built a puplet or distro from scratch using any tools whatsoever apart from some basic remasters for my personal needs, so it is with great delight that I was able to do so without any great hassles on first attempt, as I had been intrigued by the process and this particular tool (funny I chose this over Woof). It was quite easy honestly.

That said, I am posting this from the default distro the builder made, the Subito GNU/Linux 0.0.6, from about a few hours ago and so far everything is breezing along just well... although this 'cwm' window manager is quite confusing for a newbie like me. Nevertheless, I got it working (thanks to 01micko for the 'pns-tool' utility).

Although I am fully aware that this is only a concept distro, like pemasu said, the browser is quite a letdown. So, I downloaded Firefox 10.0 and Adobe Flash Player (don't really use it but since I never tried anything on 64-bit before, I figure I would just try it), resolved some dependencies errors, and -- ta-da! -- got a very nice browsing experience.

All in all, I wish to congratulate Iguleder for this amazing work (as always) and say that this is a very easy tool to use and newbies out there like me should not feel "scared" or anything. I will be using this distro as much as time allows to see its stability and get the feel of it. Liking this minimalistic approach very much.

Sorry for this long post; just registered on the forum. :)

Oh ya, I uploaded my Firefox and Flash PET package for the Subito just in case anyone needs it. Just install using 'hpm-install', run '/opt/firefox/firefox' and you're set to go! (Do remove the Flash Player if you don't use it, thanks. :))

firefox-10.0-flash-11.1.102.55-x86_64-subito.pet
(26MB; MD5SUM: 5e4c20ddd41eaea8080d211035f77d6e)

nooby
Posts: 10369
Joined: Sun 29 Jun 2008, 19:05
Location: SwedenEurope

#24 Post by nooby »

dmabar , Jay that is cool!

So now you and MinHundHettePerro can cooperate to make a
32-bit version so us who have such computers also can test it :)
I use Google Search on Puppy Forum
not an ideal solution though

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#25 Post by Iguleder »

I'm currently working on several bug fixes and improvements:
- The kernel isn't silent as we like in Puppy - it's a bug in the build script I guess; I'm currently trying to locate the problem.
- There's an error when modprobe is called without any kernel modules present, early in the boot sequence
- Updates of Linux, libvorbis and shared-mime-info
- KMS still makes my screen black

I'll do my best to solve these bugs and upload new packages. :wink:
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
01micko
Posts: 8741
Joined: Sat 11 Oct 2008, 13:39
Location: qld
Contact:

#26 Post by 01micko »

Iguleder wrote: - KMS still makes my screen black
Intel eh? I guess that happens in standard slacko or latest slacko beta? They seem to have fixed it for nouveau and radeon in 3.1x kernels, and all my intels work ok too (eee-701SD, compaq-evo-500 w/brookdale, HP on son's lappy, forget chipset atm)
Puppy Linux Blog - contact me for access

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#27 Post by Iguleder »

Nope - the problem is the kernel, framebuffer console support is a module. "reset" needs to run to make it work.

The problem should be gone now. :)
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
MinHundHettePerro
Posts: 852
Joined: Thu 05 Feb 2009, 22:22
Location: SE

#28 Post by MinHundHettePerro »

Subito 32-bit

Slowly getting there ... :)

Cheers :)/ MHHP
[color=green]Celeron 2.8 GHz, 1 GB, i82845, many ptns, modes 12, 13
Dual Xeon 3.2 GHz, 1 GB, nvidia quadro nvs 285[/color]
Slackos & 214X, ... and Q6xx
[color=darkred]Nämen, vaf....[/color] [color=green]ln -s /dev/null MHHP[/color]

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#29 Post by Iguleder »

Just to let you know, guys - I made some very big changes to roar-ng:
- All configuration is now located in a new directory, "conf".
- DISTRO_SPECS is deprecated and replaced with "distrorc", a simpler configuration file.
- The repositories support has changed; now each distro (under the "distro" directory) has a file called "repositories" which lists its repositories.
- The boot sequence is more configurable now, thanks to "bootrc", a new configuration file which allows choice of the file system used with save files and the save file name.
- I rewrote the "buildpkg", "splitpkg" and "strippkg" scripts, which form the automated package building framework. Now they're simpler, shorter, faster and better.
- I merged the "buildpkg" tool with roar-ng, which means any roar-ng distro is now "self-contained" and can be used to produce binary packages automatically and easily.
- hpm (the roar-ng package manager) is now integrated with buildpkg; both work with a new package format called "rxz" (combination of "roar-ng" and "xz"). They are similar to PET packages, but do not contain an annoying directory and pet.specs. Also, pinstall is named post_install.sh instead and puninstall.sh is now post_uninstall.sh. This package format is much smaller than PET and designed to be more simple: packages are not split as in Puppy, since this splitting is handled automatically by roar-ng; if there's any bug in the splitting script, package quality is reduced - no more!
- The package list format is changed; now, entries begin with the distribution name instead of "yes" or "no", which means packages from any distro can be used in one distro. Also, it is no longer a script.
- "subito" is added as a supported distro; its package handling code handles the new "rxz" format.
- Puppy (and PET) support is removed completely.
- From now and on, the devx has a skeleton, which contains package creation tools.
- The skeleton is now more modular and split to three sections: the main file system (e.g the main SFS), the initramfs and the devx.
- I rewrote the functions file, which provides a better, more efficient API for the scripts. It's much simpler to use and faster now.
- Save file creation is handled on the first boot, instead of being handled "interactively"; /opt/sbin/create_save_file is now moved to the initramfs. Once the system is booted and the "home=partition" boot code is specified, the init script attempts to create a save file on the specified partition and continues to boot into a persistent session (unlike Puppy, which has to run "live" at least once for persistency). This makes things easier.

These changes cut the last ties between roar-ng, Woof and Puppy. It's a big step for roar-ng towards a stable, usable version; it simplifies it a lot. Also, they make roar-ng and the distros it builds more modern and simpler (KISS, anyone?).

From now and on, everything related new to Puppy is considered deprecated and unsupported; time to move on, to better package management, modern package formats and cleaner code.

The main reason Puppy support is dropped is its non-standard nature. For example, PET packages already ship in a split form, while roar-ng performs automatic splitting. All the Puppy-specific code is now removed, which means roar-ng lost many LOC.

Regarding Subito (which is the flagship distro built using roar-ng) - I added support for Nouveau (thank you, 01micko!) and fixed some bugs, most notably the kernel (verbosity level and KMS) issues. I'm currently rebuilding it to test the new platform (the new automated package building framework and the package format); so far it seems good.

Viva la revolución!
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
Q5sys
Posts: 1105
Joined: Thu 11 Dec 2008, 19:49
Contact:

#30 Post by Q5sys »

Iguleder wrote: Viva la revolución!
Image
Sorry had to. :P

Love what you've done so far. Im so freaking impressed I cant even put it into words. I wish I had your skill level.

nooby
Posts: 10369
Joined: Sun 29 Jun 2008, 19:05
Location: SwedenEurope

#31 Post by nooby »

May I ask this?
Once the system is booted and the "home=partition" boot code is specified,
the init script attempts to create a save file on the specified partition and
continues to boot into a persistent session
Is ntfs-3g included so it can create these save files even on NTFS?
And can the OS Subito or others created by your program boot frugally?

Isolinux or grub4dos or grub2? or all of them?
I use Google Search on Puppy Forum
not an ideal solution though

gcmartin

#32 Post by gcmartin »


User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#33 Post by Iguleder »

nooby wrote:Is ntfs-3g included so it can create these save files even on NTFS?
No, for two reasons:
- Bad performance of FUSE
- Booting Linux from NTFS is a bad idea

However, you can add the ntfs-3g package if you really need good NTFS support. At the moment, Subito is able to boot from ext2, ext3, ext4, xfs and btrfs; I have tested it only against ext4, though.
nooby wrote:And can the OS Subito or others created by your program boot frugally?
Yeah, any distro built using roar-ng can boot in two modes - either "live" or "frugal", in Puppy terms. Support for "full" installation is not provided because it makes the boot process much more complex and introduces many questions.
nooby wrote:Isolinux or grub4dos or grub2? or all of them?
Subito comes with syslinux, but you can add any boot loader you wish. Boot loader support has nothing to do with the distro itself.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

nooby
Posts: 10369
Joined: Sun 29 Jun 2008, 19:05
Location: SwedenEurope

#34 Post by nooby »

Thanks.
FUSE
http://en.wikipedia.org/wiki/Filesystem_in_Userspace
Filesystem in Userspace (FUSE) is a loadable kernel module for Unix-like computer operating systems that lets non-privileged users create their own file systems without editing kernel code. This is achieved by running file system code in user space while the FUSE module provides only a "bridge" to the actual kernel interfaces.
...

FUSE is particularly useful for writing virtual file systems. Unlike traditional file systems that essentially save data to and retrieve data from disk, virtual filesystems do not actually store data themselves. They act as a view or translation of an existing file system or storage device.
I use Google Search on Puppy Forum
not an ideal solution though

User avatar
8-bit
Posts: 3406
Joined: Wed 04 Apr 2007, 03:37
Location: Oregon

#35 Post by 8-bit »

I just happened to think about my problems trying to create an ISO using roaring.
I had devx and the kernel source SFS files loaded for the version of Puppy (Lupu 520) I was running.
So in making the ISO, do I have to load the kernel SFS for the version I am trying to make?
I have tried twice now to make an ISO by just using the script files in order and both times ended up with an 8 meg ISO.
That is along with the compile packages script stopping with errors on Perl and the previous file another was having problems with.
I am evidently missing some step in that I do not think the file system is getting created successfully.

Post Reply