The Problem with Building Apps in 64-bit “Ubuntus

[mikeslr]

Hi All,

Edit: The problem discussed in this post is avoided by using the new version of PaDS RSH (now known as ITSMERSH) has published. See this thread: http://www.murga-linux.com/puppy/viewto ... 922#998922

As you may know, from time to time I build applications for Tahrpup64 and XenialPup64. See, for example, http://www.murga-linux.com/puppy/viewto ... 751#940751

I’ve also built apps for their 32-bit siblings. My technique for 32-bits is pretty straight-forward:

1. Install LazyPuppy’s PaDS plus unrpm-all from this thread: http://www.murga-linux.com/puppy/viewto ... 751#940751
1. Use PPM to download (but not install) the application’s .debs and dependencies.
2. Move the downloaded .debs to a folder bearing the name and description of the application to be built, e.g. MyApp_xen64-1.2.3 identifying it as “MyApp

[DPUP5520]

Why aren't you jus compiling from source? It's just as easy if not easier and a lot of times (if not almost always) the programs in the actual buntu repos are older versions than what are currently available.

[mikeslr]

Hi DPUP5520,

Why aren't you just compiling from source? It's just as easy if not easier and a lot of times (if not almost always) the programs in the actual buntu repos are older versions than what are currently available.

Compiling from source is a skill set I don't have and frankly am intimidated by. It may be easy to compile simple applications, or those which "are tried and true" by merely following in the instructions found with the source package. To a large extent, my impression was that those applications are provided by the Devs who create Puppies and by others who make their end products available.

Beyond that I thought to compile from source you actually had to know in advance where you wanted files to actually end up in the finished application. In other words, you have to know how all the bits and pieces which make up a Linux OS tie together, something I also don't know.

And at what point do you start compiling from source. 56 debs were used to build the xfce4_panel for Xenialpup64. It was my impression that it was each deb (and maybe --not sure-- their dependent libraries) which is the end product of a separate compile. I may be mistaken but I have the impression that during the compile process you can provide the compiler with instructions where each set of libraries is to be located in the finished product. xfce4_panel uses over 100 libraries. I don't know which deb used included which libraries. I'd have to know that before initiating each of the 56 compiles.

And while you are correct in that the Ubuntu/debian repos often do not have the latest versions of application, the latest debs published by an application's creator are compiled to run under Ubuntu/debian OSes and consequently include libraries at those locations where Ubuntu/debian OSes would expect to find them. [On occasion, I've built applications from the debs available via web-browser download from the original publishers; resorting to Ubuntu/debian or other repos only for "missing libs"].

Lastly, compiling from source should be a method of last resort -- for that application you and only a few others would want which isn't otherwise available or needs to be specially constructed to function under your hardware. There's just nothing efficient about many people having to compile from source an application which many people would want especially when compiled binaries are easily available. Compiling from source may well be a relic of a time when internet speed limitations took longer to download the text file instructions (the source) as it takes today to download the deb.

As I said in my original post, I built xfce4_panel for both Slacko64 and Xenialpup64. With Slacko64, It took about 6 minutes to download the 44 txz's and about 2 minutes for PaDS to produce a finished product. Add another 10 minutes to discover that xfce-session (not identified as a dependency, but needed if xfce4-panel was to preserve configurations when Xfce isn't the Window-manager) was necessary, download it and re-run PaDS. How is compiling 44 binaries easier than that?

If you're going to spend the time to build a complex application which many people may want, why not make the finished product available to them. I intend to make both versions available through my FREE account at mediafire. But frankly, for Slacko64 that isn't really necessary. Anyone who wants it can just spend 8 minutes with PPM and PaDS. With XenialPup64 on the other hand, add about 15 minutes of grunt work ONCE YOU KNOW WHY PPM + PaDS DIDN'T PRODUCE A WORKING APPLICATION. Letting people know why was one of the reasons for my post.

mikesLr

[DPUP5520]

All fair points, and I didn't realize nearly quite how large the entire package for the xfce4_panel is, that would be quite a large undertaking.

[Mike Walsh]

@DPUP5520:-

I suspect Mike's more like me; happier manually 'digging around' in the file system than he is doing stuff the 'pure' geek way; via the terminal.

I'm the same; I've been trying for 4 years now, but I just cannot get my head round CLI-stuff. But manually moving files around to where I want them, creating/deleting stuff as and where necessary, making/breaking sym-links, setting up permissions, etc.....I can whizz round in a fraction of the time and achieve what I want to do, far quicker than trying to suss out where I've missed (or added) an extra space, or where I've mis-spelt/mis-typed something slightly wrong in the terminal.

We're not all gifted with the same abilities; I'm more of a 'mechanic' than a brain surgeon.....and I suspect Mike is, too..!

-----------------------------------------------------

@Mikeslr:-

Back on topic. I agree with you about all this blasted 'multi-architecture' stuff. I like the way that Phil has sym-linked lib64 into lib, and /usr/lib64/into /usr/lib in both Tahr and Xenial. Makes things quite a bit easier. Mick doesn't do that in Slacko, since I guess it's not part of the Slackware 'structure'.

I don't consider myself to be much of a 'packager', TBH; that's why I tend to stick with the easier stuff, where I know the capacity for cock-ups is much reduced!

And, too (like yourself), I initially make stuff for myself. If it works as expected, I'm more than happy to share.


Mike.

[mikeslr]

Hi DPUP5520, Hi Mike,

I think Mike has identified something I know, but always forget. Everyone's mind works somewhat differently. What some people find easy, I find difficult.

I have a really poor memory for details. Working with the command line actually involves remembering all the details of a formula and, if necessary, changing them.

My memory is very visually oriented. While in college, over fifty years ago, I wrote a paper on a particular subject in which I referenced information in a book from my college's library. I don't remember what was written, the name of the book, or who wrote it. But I do remember on which floor, at which location, on which self of the stack, and roughly in which part of the book the information was found.

When I come across some code someone posted on the forum which I think may be useful, I'll cut and paste it into geany or LibreWriter and save it to /mnt/home/my_stuff/Notes/xxx_SUBJECT. I won't remember the code. I'll remember where I put it. For me, that works fine for filling an occasional need. It would be torture if I had to constantly stop and reference my notes in order to get something done.

So I write about structures, how I visualized them, and how I manipulated them, picturing as I write, how something was moved from here to there >motion pictures.

When, in my prior post, I wrote "many people having to compile from source" I had visualized something not unlike that of a sweat-shop, filled with people, all doing the same tasks, striving to produce identical items. I must admit, however, that now that I've thought about it, the same picture exists if the "many people" were "manipulating" rather than compiling. My evaluation was biased by my inability to do what others may find easy.

So DPUP5520, you're probably right. If you have a decent memory for details, compiling from source --even if it isn't as efficient as stealing someone else's work -- may be easier and certainly would be more gratifying.

mikesLr

[LateAdopter]

Hello Mike

I'm glad I'm not the only one who doesn't understand how multiarch is meant to work. The documentation tells packagers what to do but doesn't explain what the result is.
From the debian multiarch howto:

In general you can have libraries of more than one architecture installed together and applications from one architecture or another installed as alternatives. Note that it does not enable multiple architecture versions of applications to be installed simultaneously.

I think this means that there is no need to keep 32 and 64 bit libs in a separate directory, its more about resolving dependencies.

I have not found any mention of the backwards symlink solution, but it must originate from debian/ubuntu because dpkg needs to know that it should follow the symlink to the parent directory and not convert the symlink to a directory. With this solution 32bit and 64bit libs end up in the same directory and don't conflict, I think/hope.

The multiarch documentation says the the triplet directories (their jargon) need to be on the path for the libs to be found.

So there seem to be two options that can't coexist:
a) a backwards symlink that puts the libs directly in the /lib directories when you install from a .deb. But you have to manually move the files up a level when making an SFS

b) have real multiarch directories and put them on the path. I don't know whether you can get an SFS to add the multiarch directories to the path, but that would eliminate the need to move the files manually.

It is a one way change but wouldn't cause problems for libs already in the /lib directory except that you might have duplicates.

[Mike Walsh]

Hi, LateAdopter.

Ah, I'm glad somebody else agrees with me!

I guess its summat we're gonna be stuck with from now on.....especially with the general move to 64-bit only computing. (That's why I'm glad Puppy still supports - and produces new versions of - 32-bit OSs. Keeps my 15-yr old Dell Inspiron P4 lappie alive and kicking.....and far more productive than could ever be expected with MyCrudSoft's offerings..!)

It's also why I tend to produce all my packages to sit in /opt, wherever I can get away with it. It's a nice 'catch-all' directory, that neatly sidesteps & avoids all the aggravations with this multi-arch stuff. I know /usr/lib is traditionally where a lot of stuff is supposed to go.....but I've never been much of a traditionalist..!

Nice to see another UK 'Puppian'. We Brits need to stick together!


Mike.

[mikeslr]

Hi LateAdopter,

Thanks for getting this thread back on track. I wasn't aware of the documentation you cited. Here's what I do know from experience. When tracking down Ubuntu libs via https://packages.ubuntu.com/ often you'll get to a point at which you're offered the choice between 32-bit and 64-bit packages (and others) or sometime the package is for "all" architectures. As far as I know, the package for the 32-bit bears the same name as the package for the 64-bit, except that the former will include i386 (or 4 or 586) while the latter will include amd64 in their respective labels. I haven't pursued that far enough to know whether the files within each deb bore distinct names. If they do, both files will happily exist within the same folder. If they don't, the last file installed will overwrite its predecessor.

When a file with the wrong architecture employed, the application will not run and ldd will report that the wrong "ELF class" was attempted to be used. [I would think it rare that you would ever need both the 32 and 64 bit version of a file.] The most likely cause is "human error" --in searching you followed the wrong path; less likely, PPM made a mistake. But sometimes --especially when trying to build using the latest --not a Ubuntu repo-- version of an application, the necessary lib isn't in a Ubuntu repo at all. So, you've expanded your search to pkg.org and others, used what appeared to be the most likely package --albeit built for Opensuse or whatever-- and ended up with an ELF error. At the point at which a package isn't available in the repo of the distro to which your Puppy has 'binary compatibility', the ability to compile just the package you need would really be a useful.

My inability to do that surfaced a couple of days ago. Rather impressed by how well Slacko64 worked, I set out to see whether the applications I run as external files and AppImages could be run under it. Many worked, but a couple required Qt5. Maybe I missed them, but I couldn't find any Qt5 packages for Slackware or its spinoffs.

But, getting back on point, "have real multiarch directories and put them on the path." Emphasis supplied. So, in addition to the 'solutions' I mentioned, that would be another. But as with those I previously mentioned, I don't have the ability to accomplish that either.

mikesLr

p.s. Just read Mike Walsh's post above. That's also been my experience. But I don't know why. When --except for a symlink on the path to the application's binary-- an entire application including the folders containing its libraries-- is located in /opt or /mnt/home/SOMEPLACE 'Ubuntu' Puppies don't have any difficulty finding the libs in, for example, /opt/my-application/usr/lib/x86_64-linux-gnu. So, another possible solution. But this one also involves moving stuff, albeit perhaps easier: the entire structure to /opt with one move. But is, for example, /opt/my-application/etc or /opt/my-application/root a configuration Puppies would know how to handle?

p.p.s. Might not be as simple as that. Just tried this: Still having the original debs available in the folder I used to create the preliminary SFS, I used PaDS to create a new SFS. Then I created a blank folder which included the term OPT in it so that I could distinguish it from the working version. Mounted the new preliminary SFS and copied its files into a folder named opt within a folder named xfce4_panel. Adjacent to opt, I created a folder named root, within that one named Startup. xfce4_Panel has no menu entries, but to run before it xfce4-session and then it must be started. So in /root/Startup I placed a script which would call both from their locations in /opt. dir2sfs the xfce4...opt folder creating a xfce4_panel..opt.sfs. Unloaded my working SFS and loaded the opt.sfs. Restarted X. Nothing happened. Typed xfce4-session in terminal and received the message that a library was missing. So type ldd followed by the full path (opt/etc.) and name of xfce4-session and received the message that libraries I know are in /opt/...etc/usr/lib were missing.

But I may have done something wrong. Still worth exploring by others.

[LateAdopter]

Hello mikeslr

You can edit the /etc/ld.so.conf file to add the x86_64 paths and then run ldconfig each time you load an SFS.
666philb already put the i386 paths there and says that you need to run ldconfig when installing the 32bit libs SFS.
That may be why you have better results with 32bit than 64bit

EDIT
From http://www.debian.org/doc/debian-policy ... dlibs.html
---
8.1.1. ldconfig

Any package installing shared libraries in one of the default library directories of the dynamic linker (which are currently /usr/lib and /lib) or a directory that is listed in /etc/ld.so.conf [60] must use ldconfig to update the shared library system.

The package maintainer scripts must only call ldconfig under these circumstances:

When the postinst script is run with a first argument of configure, the script must call ldconfig, and may optionally invoke ldconfig at other times.

When the postrm script is run with a first argument of remove, the script should call ldconfig.
----

There must be a way to get AUFS to execute ldconfig, but I can't find any instructions for this. You can put ldconfig in rc.local so that it is always run during boot-up, but it will slow boot a bit.

EDIT 2

It looks as though rc.update runs ldconfig anyway, during boot, so maybe adding the paths to ld.so.conf is all that is necessary. However I don't know whether that always happens.

[mikeslr]

Hi LateAdopter,

Very nice info regarding id.so.conf. But perhaps --hopefully-- you've bought into a problem before it exists. Maybe all that would be necessary is to "inform" 'Ubuntu' Puppies that the directories it couldn't find do exist, and where, by editing id.so.conf to specify them. AFAIK, id.so.conf is part of the system, read-in and established on boot-up. So it may not be necessary to re-run it each time an installed pet is opened or an SFS is loaded and/or opened.

We'll see. I can test. Attached is a pet which adds the problem directories to id.so.conf. After re-building xfce4_panel from the original debs again, I'll install the pet and load xfce4-panel into 're-virginized' Xenialpup64; then see if the panel will run, and if ldd continues to report libs 'not found'. I won't structure the panel to run from /opt [which would add a layer of complexity] -- just leave it as the Ubuntu devs structured it to run under Xenial Xerus.

But before I take the time to do that I really want to publish the xfce4_panels that I've had sitting around for over a week. It's taken much longer to provide guidance than to create them.

Edit: Just re-read your post which now has Edit 2. Seems we've developed the same hypothesis, 'though --as to be expected-- you're reasoning provides more details.


mikesLr