How to build a 3MB static Midori with plugins

How to do things, solutions, recipes, tutorials
Post Reply
Message
Author
User avatar
jemimah
Posts: 4307
Joined: Wed 26 Aug 2009, 19:56
Location: Tampa, FL
Contact:

How to build a 3MB static Midori with plugins

#1 Post by jemimah »

In this example I'm compiling webkit 1.3.12 and midori 0.3.2.

This tutorial is not intended for beginners at compiling.

First I patch webkit since it segfaults like crazy otherwise.

Code: Select all

diff -urB webkit-1.3.12/Source/WebCore/platform/text/gtk/TextBreakIteratorGtk.cpp webkit-1.3.12-new/Source/WebCore/platform/text/gtk/TextBreakIteratorGtk.cpp
--- webkit-1.3.12/Source/WebCore/platform/text/gtk/TextBreakIteratorGtk.cpp	2011-01-12 09:40:52.000000000 -0500
+++ webkit-1.3.12-new/Source/WebCore/platform/text/gtk/TextBreakIteratorGtk.cpp2011-03-10 08:12:00.000000000 -0500
@@ -210,8 +210,7 @@
     int charLength = iterator->m_charIterator.getLength();
 
     iterator->m_type = type;
-    if (createdIterator)
-        g_free(iterator->m_logAttrs);
+    //if (createdIterator) g_free(iterator->m_logAttrs);
     iterator->m_logAttrs = g_new0(PangoLogAttr, charLength + 1);
     pango_get_log_attrs(iterator->m_charIterator.getText(), iterator->m_charIterator.getSize(),
                         -1, 0, iterator->m_logAttrs, charLength + 1);
This patch probably introduces a memory leak, so if you have a better patch let me know. I'll probably have something better shortly

Then I set up my environment variables to optimize for size.

Code: Select all

export CFLAGS=" -pipe -combine -Os -ffunction-sections -fdata-sections -momit-leaf-frame-pointer -fomit-frame-pointer -fmerge-all-constants -march=i486 -mtune=i686 -momit-leaf-frame-pointer -fomit-frame-pointer -mpreferred-stack-boundary=2 -fmerge-all-constants"

export CXXFLAGS=" -pipe -combine -Os -ffunction-sections -fdata-sections -momit-leaf-frame-pointer -fomit-frame-pointer -fmerge-all-constants -march=i486 -mtune=i686 -momit-leaf-frame-pointer -fomit-frame-pointer -mpreferred-stack-boundary=2 -fmerge-all-constants" 

export LDFLAGS=" -Wl,-relax,-Os,-s "
Configure webkit.

Code: Select all

./configure --enable-3d-transforms --enable-fast-mobile-scrolling --with-unicode-backend=glib --enable-static --disable-shared --prefix=/usr
Don't get too creative with the options. Webkit tends not to compile if you turn things on that are off by default.

Make clean does not work right. If you need to start over, delete and re-extract the whole source tree.


Now edit GNUMakefile. The configure script probably added an -O2 to the end of the CFLAGS and CXXFLAGS variables. You need to remove that or you won't get a small build.

Now run make and make install. This takes forever even on a fast machine.

Edit the file /usr/lib/pkgconfig/webkit-1.0.pc to look like this:

Code: Select all

prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: WebKit
Description: Web content engine for GTK+
Version: 1.3.12
Requires: glib-2.0 gtk+-2.0 libsoup-2.4
Libs: -L${libdir}
Cflags: -I${includedir}/webkit-1.0
Webkit isn't going to link correctly like this but we need to link by hand for this project anyway.

Now you're ready to configure Midori.

Code: Select all

./configure --prefix=/usr
Run make. It's going to bomb when it gets to the first plugin that needs webkit, but that's ok - it still creates the plugin despite the error message. We are trying to prevent the plugins from linking in webkit because the results will be huge otherwise.

Run make a few more times until you reach this step:

Code: Select all

[ 60/116] cc_link: _build_/default/midori/main_2.o -> _build_/default/midori/midori
Cd to the _build_ directory.

Create /usr/lib/midori if it doesn't exist. If it does exist, delete any plugins that are inside.

Copy the new plugins to /usr/lib/midori.

Code: Select all

cp default/extensions/*.so /usr/lib/midori/

Now we are going to link the plugins in as shared libraries. This will allow us to have only one copy of webkit and we won't need to export the symbols, which saves even more size. Of course now Midori won't be able to start if its plugins are not there.

There might be a better way to accomplish this - but this is the best I could figure out

Code: Select all

/usr/bin/gcc default/midori/main_2.o /usr/lib/midori/libhistory-list.so /usr/lib/midori/libcookie-manager.so /usr/lib/midori/libformhistory.so /usr/lib/midori/libweb-cache.so /usr/lib/midori/libfeed-panel.so /usr/lib/midori/libadblock.so /usr/lib/midori/libaddons.so /usr/lib/midori/libshortcuts.so /usr/lib/midori/libcolorful-tabs.so /usr/lib/midori/libmouse-gestures.so /usr/lib/midori/libstatus-clock.so /usr/lib/midori/libtabs-minimized.so /usr/lib/midori/libcopy-tabs.so /usr/lib/midori/libpage-holder.so /usr/lib/midori/libstatusbar-features.so /usr/lib/midori/libtab-panel.so /usr/lib/midori/libtoolbar-editor.so -o default/midori/midori  -Wl,-O,-Os,--gc-sections,-s -pthread -o default/midori/midori  -Ldefault/midori -L/usr/X11R7/lib -L/usr/X11R6/lib  -lmidori-core  -lm -lunique-1.0 -lgtk-x11-2.0 -lgdk-x11-2.0 -latk-1.0 -lgio-2.0 -lpangoft2-1.0 -lgdk_pixbuf-2.0 -lpangocairo-1.0 -lcairo -lpango-1.0 -lfreetype -lz -lfontconfig -lgobject-2.0 -lgmodule-2.0 -lrt -lglib-2.0 -lsoup-2.4 -lgnutls -lgthread-2.0 -lsqlite3 -lnotify -ldbus-glib-1 -ldbus-1 -lwebkitgtk-1.0 -lpcre -lenchant -lgailutil -ljpeg -lxslt -lxml2 -lXt -lX11 -lstdc++ -lgstbase-0.10 -lgstpbutils-0.10 -lgstapp-0.10 -lgstvideo-0.10 -lgstinterfaces-0.10 -lgstreamer-0.10 -lXss -lsoup-2.4
You will need to adjust this depending on what dependencies you're building with.

Now you should be able to run make install. Check the size of the Midori binary - I get around 8.6MB. You can then upx it to get it under 3MB.

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#2 Post by sc0ttman »

Great post, I love to see exactly how others compile stuff, it's so useful to know, yet seems to be a closely guarded secret most of the time! (Except in debian and ubuntu forums, where people seem to have to work hard to compile anything...)

I think I will try this in Puppy 4, just to see...
(probably won't work, if I do it!)

And I know this may be out of the scope of this thread, but it would be nice to have a similar, simple, single post with:

- how to use .diffs, apply patches
- setup 'configure', common options used, etc
- using CFLAG and CXXFLAG explanations
- using 'make', or 'make install'
- troubleshooting a failed compile attempt

(maybe using a few popular apps as examples...)

I personally would also like to know an easy way to split the package (and its DEV and NLS) easily from the source (or main filesystem if it was installed), once built...
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

User avatar
jemimah
Posts: 4307
Joined: Wed 26 Aug 2009, 19:56
Location: Tampa, FL
Contact:

#3 Post by jemimah »

Amigo wrote a program called src2pkg which can do most of that stuff automagically. It applies any patches, sets up your CFLAGS, configures and compiles many different types of build systems, and makes pets for you (split or not depending how you set it up). The packaging part actually works significantly better than the new2dir script.

The only thing it doesn't do is troubleshoot. :)

If you want to build midori without the voodoo, try a dynamic build. It's a little bigger but there are fewer issues to worry about. To get a dynamic build, just omit the "--disable-shared" flag when configuring webkit. Midori actually builds pretty easily if you stick with the default options.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#4 Post by technosaurus »

@Jem glad you finally got it, I know it was a lot of painstaking work.

@Scott when I was working on pcompile(GUI), I read lfs hints and studied the code in variousbuild systems (including amigo's). Buildroot has links to a lot of them.

BTW the stuff you want is mostly in my signature... also a sticky in additional pets
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

2lss
Posts: 225
Joined: Sun 20 Sep 2009, 23:54

Re: How to build a 3MB static Midori with plugins

#5 Post by 2lss »

Nice explanation!
jemimah wrote:First I patch webkit since it segfaults like crazy otherwise.
Have you tried this patch on any newer versions of webkit?

For me, anything past 1.3.5 seg faults a lot. But thats on 64 bit Debian. I haven't tried building one for puppy yet.

User avatar
jemimah
Posts: 4307
Joined: Wed 26 Aug 2009, 19:56
Location: Tampa, FL
Contact:

#6 Post by jemimah »

Yes the problem started in 1.3.6. I believe the patch would probably work for anything newer than 1.3.5. I suspect this bug is only a problem if you compile with --with-unicode-backend=glib.

However I think the patch does cause midori to crash during text entry for long posts and such. I think it should be better to check if the pointer is null before freeing it rather than just commenting out the free statement. I will fix the patch and post it once I'm happy with it.

User avatar
jemimah
Posts: 4307
Joined: Wed 26 Aug 2009, 19:56
Location: Tampa, FL
Contact:

#7 Post by jemimah »

Ok this patch should be better.

Code: Select all

diff -urB webkit-1.3.12/Source/WebCore/platform/text/gtk/TextBreakIteratorGtk.cpp webkit-1.3.12-new/Source/WebCore/platform/text/gtk/TextBreakIteratorGtk.cpp
--- webkit-1.3.12/Source/WebCore/platform/text/gtk/TextBreakIteratorGtk.cpp	2011-01-12 09:40:52.000000000 -0500
+++ webkit-1.3.12-new/Source/WebCore/platform/text/gtk/TextBreakIteratorGtk.cpp	2011-03-15 16:24:15.000000000 -0400
@@ -210,8 +210,9 @@
     int charLength = iterator->m_charIterator.getLength();
 
     iterator->m_type = type;
-    if (createdIterator)
-        g_free(iterator->m_logAttrs);
+    if (createdIterator) {
+	if (iterator->m_logAttrs) g_free(iterator->m_logAttrs);
+    }
     iterator->m_logAttrs = g_new0(PangoLogAttr, charLength + 1);
     pango_get_log_attrs(iterator->m_charIterator.getText(), iterator->m_charIterator.getSize(),
                         -1, 0, iterator->m_logAttrs, charLength + 1);
I think these assertion failures from midori are somehow related to this bug. But at least it doesn't crash anymore.

Code: Select all

(midori:380): GLib-CRITICAL **: g_regex_replace_eval: assertion `string != NULL' failed

(midori:380): GLib-CRITICAL **: g_strescape: assertion `source != NULL' failed

(midori:380): GLib-CRITICAL **: g_regex_replace_eval: assertion `string != NULL' failed

(midori:380): GLib-CRITICAL **: g_strescape: assertion `source != NULL' failed

(midori:380): GLib-CRITICAL **: g_regex_replace_eval: assertion `string != NULL' failed

(midori:380): GLib-CRITICAL **: g_strescape: assertion `source != NULL' failed

(midori:380): GLib-CRITICAL **: g_regex_replace_eval: assertion `string != NULL' failed

(midori:380): GLib-CRITICAL **: g_strescape: assertion `source != NULL' failed

(midori:380): GLib-CRITICAL **: g_regex_replace_eval: assertion `string != NULL' failed

(midori:380): GLib-CRITICAL **: g_strescape: assertion `source != NULL' failed

(midori:380): GLib-CRITICAL **: g_regex_replace_eval: assertion `string != NULL' failed

(midori:380): GLib-CRITICAL **: g_strescape: assertion `source != NULL' failed

(midori:380): GLib-CRITICAL **: g_regex_replace_eval: assertion `string != NULL' failed

(midori:380): GLib-CRITICAL **: g_strescape: assertion `source != NULL' failed

(midori:380): GLib-CRITICAL **: g_regex_replace_eval: assertion `string != NULL' failed

(midori:380): GLib-CRITICAL **: g_strescape: assertion `source != NULL' failed

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#8 Post by technosaurus »

I found a potential webkit alternative, but can't find any source code.
http://www.access-company.com/products/ ... sernx.html

Has anyone tried it & can point me to the source code???
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Dougal
Posts: 2502
Joined: Wed 19 Oct 2005, 13:06
Location: Hell more grotesque than any medieval woodcut

#9 Post by Dougal »

technosaurus wrote:I found a potential webkit alternative, but can't find any source code.
http://www.access-company.com/products/ ... sernx.html

Has anyone tried it & can point me to the source code???
According to this:
License: Proprietary
(Not to mention that it doesn't seem to support x86...)
What's the ugliest part of your body?
Some say your nose
Some say your toes
But I think it's your mind

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#10 Post by technosaurus »

update: perhaps a webkitgtk version with google's v8 javascript engine is worth testing
https://gitorious.org/~nayankk/webkit/nayankk-webkit

I also found a tutorial for making ICU smaller if the glib unicode backend causes problems:
http://userguide.icu-project.org/packaging
(btw the data section is the important detail there)
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

Post Reply