OpenEmbedded: compile Puppy from source

[BarryK]

This is so interesting, and I am having fun. Posting to my blog, which does not allow reader-comments (I disabled that awhile-back due to security problems). So I thought would create a forum thread and invite feedback.

Woof-CE, or any of the earlier Woofs, or my fork woofQ, can build a Puppy or puppy-like distro from binary packages.
Usually, we get those binary packages from another distro repository, such as Ubuntu, Debian or Slackware.

Some earlier pups, for example Wary and Racy, used binary packages that were compiled from source in T2 (see http://t2-project.org/), however, T2 is not maintained, only one guy does some work on it.

An alternative is OpenEmbedded, which the Yocto project uses. This has 75+ active developers. Despite mainly used for embedded devices, OE can compile anything, even LibreOffice.
So, OE could be used to compile the packages for Woof*, even the Linux kernel.

I have done this, and built a first pre-alpha Quirky Linux, using my woofQ. Yep, have compiled almost everything, including kernel, libreoffice, gimp, kodi, firefox.

What I could do, is upload the binary packages somewhere, and they can then be used by any other Woof*

Here is discussion on woof-CE:
http://murga-linux.com/puppy/viewtopic.php?t=101174
http://murga-linux.com/puppy/viewtopic.php?t=96723

01micko has setup woof-CE to compile source packages. As I understand it, it is not a cross-compiler, and is only at beginner stages.
OE runs on a x86_64 host system and can build for any target, such as x86_64, i686 or armv7.

Also, OE could totally replace Woof*, but that is very much down the track.

Anyway, starting this thread, will post here as it progresses, also on my blog. Here is the blog post reporting on "Pyro64" 0.1:
http://barryk.org/news/?viewDetailed=00537

Earlier blog posts:
http://barryk.org/news/?viewDetailed=00535
http://barryk.org/news/?viewDetailed=00534
http://barryk.org/news/?viewDetailed=00531
http://barryk.org/news/?viewDetailed=00529
http://barryk.org/news/?viewDetailed=00530

[BarryK]

A bit of a timeline. "Pyro" is the name of the next release of OpenEmbedded, due out this month.

When Pyro is released, I plan to take a snapshot of OE from github, and use that as the basis for my own Quirky Pyro64.

I plan to build a alpha release of Quirky Pyro64, for anyone who wants to play with it.

Also, I plan to upload a tarball of OpenEmbedded with my extensions, so anyone can download it, then just run a couple of simple commands, and it will take off, downloading and compiling everything, from kernel to libreoffice.

What someone could do, is experiment with adding more packages, or change the target build to, say i686.

[sc0ttman]

A bit of a timeline. "Pyro" is the name of the next release of OpenEmbedded, due out this month.

When Pyro is released, I plan to take a snapshot of OE from github, and use that as the basis for my own Quirky Pyro64.

I plan to build a alpha release of Quirky Pyro64, for anyone who wants to play with it.

Also, I plan to upload a tarball of OpenEmbedded with my extensions, so anyone can download it, then just run a couple of simple commands, and it will take off, downloading and compiling everything, from kernel to libreoffice.

What someone could do, is experiment with adding more packages, or change the target build to, say i686.

Any chance of getting Alacritty (http://blog.jwilm.io/announcing-alacritty/) or, even better, mlterm-fb working in those Open Embedded builds?


..I've tried and failed numerous times to get mlterm-fb working in any puppy .. not sure why it won't work :/

[sc0ttman]

..deleted double post..

[technosaurus]

A bit of a timeline. "Pyro" is the name of the next release of OpenEmbedded, due out this month.

When Pyro is released, I plan to take a snapshot of OE from github, and use that as the basis for my own Quirky Pyro64.

I plan to build a alpha release of Quirky Pyro64, for anyone who wants to play with it.

Also, I plan to upload a tarball of OpenEmbedded with my extensions, so anyone can download it, then just run a couple of simple commands, and it will take off, downloading and compiling everything, from kernel to libreoffice.

What someone could do, is experiment with adding more packages, or change the target build to, say i686.

Any chance of getting Alacritty (http://blog.jwilm.io/announcing-alacritty/) or, even better, mlterm-fb working in those Open Embedded builds?

..I've tried and failed numerous times to get mlterm-fb working in any puppy .. not sure why it won't work :/

mlterm-fb compiles fine in fatdog64 ... just add --with-gui=fb to the other ./configure options (prefix, libdir,sysconfdir,localstatedir,...) ... although the devs should probably address the cascade of warnings that come up ... it won't build with -Werror set (such as in $CFLAGS) ... then make install mlterm-fb

[mcewanw]

Also, OE could totally replace Woof*, but that is very much down the track.

Since it takes a significant investment in time to learn or maintain any build system the well-supported OpenEmbedded system described here seems to me like a very good way forward indeed for Puppy. I realise OE will have a probably steep learning curve, but appears to be a useful system to learn more generally so having it brought to my attention by this thread, I'm certainly excited by the small-system-building possibilities it offers. I'm thus keen to try it out now too.

Thanks for the info.

William

[BarryK]

Also, OE could totally replace Woof*, but that is very much down the track.

Since it takes a significant investment in time to learn or maintain any build system the well-supported OpenEmbedded system described here seems to me like a very good way forward indeed for Puppy. I realise OE will have a probably steep learning curve, but appears to be a useful system to learn more generally so having it brought to my attention by this thread, I'm certainly excited by the small-system-building possibilities it offers. I'm thus keen to try it out now too.

Thanks for the info.

William

Hi, yes, I mentioned that as a theoretical possibility. In OE (and Yocto) you can set it up with a Board Support Layer (BSP) for a specific hardware target, for example Raspberry Pi, and it will gnerate an SD-card image for the Pi .

I don't know if it will happen for the pups though. it is easy enough to import the binary packages from OE into Woof*.

But, to run just one build script, which compiles everything from source and outputs a live-CD or SD-card file -- the thought is enticing!

[scsijon]

BarryK,

You mentioned problems with Kodi on your blog, it wouldn't happen to be version 17.1 would it? Most 'Linux's' are having various problems with this one, maybe it's worth going back a step for your first build.

If you haven't already, have a look at https://forum.kodi.tv the Kodi community forum is always looking for problem entries (and of course solutions) so everyone knows about them.

regards
scsijon

ps nearly got a t2 puppy packageset built without errors finally, then a new ?racy?6 to build in woof for all!

[BarryK]

BarryK,

You mentioned problems with Kodi on your blog, it wouldn't happen to be version 17.1 would it? Most 'Linux's' are having various problems with this one, maybe it's worth going back a step for your first build.

If you haven't already, have a look at https://forum.kodi.tv the Kodi community forum is always looking for problem entries (and of course solutions) so everyone knows about them.

regards
scsijon

ps nearly got a t2 puppy packageset built without errors finally, then a new ?racy?6 to build in woof for all!

Yes, it is 17.1.

I have good news, now have text.

/usr/share/gconv folder was missing. Fixed.

Tried kodi, there was a couple of peculiar things. Could go back to 17.0

[pakt]

Barry, I can recommend the Synology DS216J 2-Bay NAS. This is a reliable, fast and quiet NAS.

The DiskStation OS is topnotch, with lots of configuration settings. It even has its own 'app' store!

I use this NAS (in 'RAID 1' aka mirroring) for storage and streaming media. My older DS212j 2-bay unit I bought years ago now serves as an archive.

http://www.austin.net.au/synology-ds216j-2-bay-nas.html

[technosaurus]

Barry, I can recommend the Synology DS216J 2-Bay NAS. This is a reliable, fast and quiet NAS.

The DiskStation OS is topnotch, with lots of configuration settings. It even has its own 'app' store!

I use this NAS (in 'RAID 1' aka mirroring) for storage and streaming media. My older DS212j 2-bay unit I bought years ago now serves as an archive.

http://www.austin.net.au/synology-ds216j-2-bay-nas.html

Just in case anyone else decides to do this research, I looked into the most economical method of getting a 1Gbit SATA file server and found the banana pi m1 (in headless mode) to be the most bang for the buck (<$40)... plus you get an extra dual core arm system with a mali GPU for distributed builds. Note that later versions of the banana pi use an inferior usb2->sata controller that severely limits speed. Although the product page states that the m1's drive size is limited to 2Tb, various forums report success with 4Tb.

Edit: for usb3 the lemon pi is a similar price point, but it appears to be poorly documented, which in my experience usually means some of the driver work is still incomplete/experimental.

[BarryK]

Barry, I can recommend the Synology DS216J 2-Bay NAS. This is a reliable, fast and quiet NAS.

The DiskStation OS is topnotch, with lots of configuration settings. It even has its own 'app' store!

I use this NAS (in 'RAID 1' aka mirroring) for storage and streaming media. My older DS212j 2-bay unit I bought years ago now serves as an archive.

http://www.austin.net.au/synology-ds216j-2-bay-nas.html

Just in case anyone else decides to do this research, I looked into the most economical method of getting a 1Gbit SATA file server and found the banana pi m1 (in headless mode) to be the most bang for the buck (<$40)... plus you get an extra dual core arm system with a mali GPU for distributed builds. Note that later versions of the banana pi use an inferior usb2->sata controller that severely limits speed. Although the product page states that the m1's drive size is limited to 2Tb, various forums report success with 4Tb.

Edit: for usb3 the lemon pi is a similar price point, but it appears to be poorly documented, which in my experience usually means some of the driver work is still incomplete/experimental.

Yes, I have wondered about a DIY NAS, maybe using one of the specialized Linux/BSD OS's such as FreeNAS.

[BarryK]

Quoting from my blog:

Yes, confirmed that icu 58.2 is causing firefox and seamonkey to be unstable.

So, trying an experiment. Recompiling in OE with icu 56.1, same version as used internally in firefox and seamonkey.

I also added the patches from seamonkeys's internal icu, mindful that it may be one of those patches that fixes the stability problem.

It will be interesting to see how this turns out.

Ubuntu Xenial Xerus 16.04 only has icu 55.x, and I have compiled seamonkey with it, no problem.

Gradually getting there. Hopefully in a few more days, will be able to upload "oe-qky-src", OE with my customization layer, and a Quirky build, codenamed "Pyro64".

I am using Pyro64 every day right now. Just some details need to be sorted out, like SeaMonkey crashing!

[BarryK]

The way that OE builds packages is very interesting. Each package is built in isolation.

Unlike other systems such as Buildroot and T2, there is no single target rootfs, that each package gets installed into as it is compiled.

We are familiar with running "configure" and it sees what is installed and makes choices what dependencies to use. OE can't do that, you have to specify all deps in the recipe.

They don't have to be hard-coded deps, there can be conditional tests in the recipe.

It does require more work when creating a recipe, to decide what deps are essential and what are conditional.

This isolation does have advantages. I am seeing this, when doing a rebuild, after having changed icu.

[BarryK]

Quirky Pyro64 version 0.2 is uploaded, I have started a new thread:

http://murga-linux.com/puppy/viewtopic.php?t=110541

[BarryK]

OpenEmbedded, with my "meta-quirky" customization layer, is available as a tarball:

http://barryk.org/news/?viewDetailed=00546

To run it, your host OS will need the "devx" installed, and python3 -- in the latter case, install it from the PPM, along with all deps.

Afterward, do this:
# ln -s python3.5 /usr/bin/python3
# ln -s python2.7 /usr/bin/python2

as OE wants those names.

After expanding the tarball, you will find a getting started file.

So, anyone who is game, can give it s go. An interesting exercise, that I have not yet tried, will be to configure it for a different target architecture, such as i686 or armv7.

[slackfan]

Some earlier pups, for example Wary and Racy, used binary packages that were compiled from source in T2 (see http://t2-project.org/), however, T2 is not maintained, only one guy does some work on it.

An alternative is OpenEmbedded, which the Yocto project uses. This has 75+ active developers. Despite mainly used for embedded devices, OE can compile anything, even LibreOffice.
So, OE could be used to compile the packages for Woof*, even the Linux kernel.

I don't know if that is really a criterium.

Today, Nutyx appears as a distribution, a kind of standard one, along about 10 years with only one real developper, the Swiss man Thierry Nuttens (and a second one, "pierre" how was always adding one big but accessory environment package, the all KDE compiling it each parution of a new Nutyx one!) being always compiled from A .. Z: all comes from LFS and LFS was never delivered pre-compiled (excepted you go to, even, Nutyx, to get LFS in pre compiled form!).

All 6 .. 12 month did Thierry in the past about renounce to continue his work, so difficult it is to produce and follow as an one-man-show at it's fast rythm of parution the LinuxFromScratch...

but it is now the probably first distribution able to compile all Standard Linux including, now, a lot of other classic environments, not "only" KDE any more, at home from the user, if he is willing so (for ex. to try to compile exactly for his own processor and other hardware etc.) better as install a ready to use distribution.

the process is not automatic any more (3..4 years ago, Nuttens did announce -to fast- to publish his distribution in the future only in not pre compiled form but prepared to be compile from A..Z in real automatic way . but if you follow http://nutyx.org/en/build-pre, the major difficulty is only to mark each next step from the prepared commando lines about unchanged with you mouse in one of your consoles, hit «enter» and continue on the next line and page if you see no error message at this step). The intervention of human is only needing to avoid a complex gestion of the changing stage of login as "root" or "user" and controll the progression of operations (nutyx has also a script permetting the net installation installation out nutyx itself, of course, but also Puppy as far the version of Puppy uses a compatible bash with the bash of LFS. As some Puppy use LFS I suppose those are able to operate it!), and it contribute to let the PC disponible at compiling time for other "users" (youself working or playing at somewhat different!) as you are and let yourself all the time the "director" of all permissions!

but I am sorry to admit, that it is often nonsens for the most people to compile all Linux themself excepted you really will to publish complete distributions!

it is to much!

you can do nothing more excepted that and I see out the other highly interesting home pages from people like Barry Kauler (his page about small camping! Wonderfull!!! Thank you for the page Barry) Barry himself has certainly more interests as only compile and compile all the time !

ideal would be to compile so, as Nutyx does it, a one bone of Puppy including a remastering routine to produce an ISO or a big kernel with all Puppy in it.

advantage: the extrem stability of the system against errors and changes after that is done equal where (home use, community of friends, associations, school, enterprise)...

I, for example, need today only few app's. leafpad, a small browser (luakit is ideal as extensible and permitting all excepted, perhaps, spell control), didiwiki, not forget didiwiki!, rox-filer and rox-terminal (to copy contents from window to window including in CLI!), flashplayer services, mtpaint and viewnior, mplayer2, to see videos, and nted, as I find a person with culture has to be able to use / offer to his children a way to write in music writing, and I install since a few years calligrasheets / calligrawords / kwave / skanlitle / kate as all those are smaller as to install full LO or OO over Puppy and permit, if so or so preinstalled, to continue with low memory need with more powerfull app's from KDE world if you need them as the minimal KDE/QT4 or 5 is already preinstalled (marble, usw.). I always need merkaartor to cartography my environment. I did constat it with the kde4.sfs derivated from Slax: is it loaded, and you have immediately an extended environment mit konqueror, okular, gwenview etc.

so is a full puppy / quirky for me false today: to much stuff I will never more use (my experience with Abiword is, I am sorry: great difficulties with far east writing where Calligra has no problem! Also are Calligra texts like about all standard KDE in about all languages available in constant form including 2 artificial linguages: Esperanto and Interlingue!)...

note after change: Nuttens requires now to use the script to install a nutyx version. it was not required in the past and better as it was possible to compile themself the first Nutyx installation on the own HD (Nutyx can work with "embedded" Chroot's without separate partitions and respects so better the disk organisation of the user as, for ex. TOPLESS!) being working at other things in a good prealable Linux environment including some Puppy's, not all. But there are divers restrictions perturbing the job if not Nutyx, and Nuttens, in his position as one-man-operator also on the forum, he is about always present and ready to help, can really wish, not to be confronted any more to the unprevisible difficulties resulting from a differenting desk surface to work!

[TeX Dog]

Would like to try an ARM build.

I see there are already OpenEmbedded RaspberryPi writeups. Is there a merge step? do we run parallel creates and merge differences after, also can we run in target mode on say big intel box for ARM or are we stuck compiling ARM on ARM?




https://raspinterest.wordpress.com/2017 ... erry-pi-3/

[BarryK]

slackfan and Tex Dog,
I just now had a quick look at Nutyx.

Yes, a one-man show is fine, if the person is dedicated to it. In the case of T2, the developer has a full-time job, and does not put in the time to fix problems or to bring it up-to-date. So many of the packages are ancient, with old techniques, such as still using "module-init-utils" instead of "kmod", and heaps of bugs.

He did comment recently, that he would love to put more time into it, if he was paid for it. But nobody hits his "donate" button.

On the otherhand, OpenEmbedded has 75+ active developers, and has industry participation.

Another important point is that LFS is not a cross-compile system. It is only for x86[_64]. You have to be running a host x86 Linux OS, and build for a x86 target.

T2 and OE on the otherhand, are cross-compiler systems. You run on a x86_64 host Linux OS, and can build for various targets, including ARM (at least with OE you can, T2 is mostly broken these days).
Which I haven't tried yet, keen to do so.

A build for the Pi is on my to-do list ...soon, hopefully.

[don570]

For developers on the Arm platform there is a Kickstarter project
https://liliputing.com/2016/12/firefly- ... r-139.html

_____________________________________________________