Page 3 of 14
Posted: Wed 08 Aug 2007, 23:08
by Springer
JohnM:
Just to clear up a misconception, NoSQL and its relatives (rdb, /rdb, starbase, etc.) ARE real relational databases. That's the idea - each of their application environments provides relational operators that work out-of-the-box on the tables, which happen to be (usually) TSV text files, sometimes with header lines.
IMHO, every well-read Unix hacker should at least have read this paper to see how Unix (the toolset, more than the OS proper) can be leveraged as an app platform, not just for underlying OS support:
The Unix Shell as a Fourth Generation Language.
Stream/Operator databases will never flat outperform "real database engines", but they offer very real advantages, and in today's world, where the penalty for interpreted code is trivial and huge, fast harddisks and RAMdisks are dirt-cheap, the approach makes more sense than ever.
FWIW, here's a (slightly edited) post I made to the NoSQL mailing list a while back, when one of the new guys was wondering how to present NoSQL to his LUG as an alternative to "regular" SQL databases:
Well, Evan Schaffer's paper "The Unix shell as a 4GL" is a great place to start. Mainly, the advantage is not so much the database itself (DBs are just a way to hold and find stuff, after all) - it's the "stream-operator" paradigm that a database like /rdb, Starbase, or NoSQL makes possible.
This approach gives you awesome modularity and leverage, and lets non-DB people write serious DB apps without ever having to know all the arcane crap about how SQL works.
Further, in today's world where Gigabytes of RAM and RAMdisks are everywhere, speed isn't really the concern most people think it is. 4GB is actually a really LARGE database, and I'll stack up a RAMdisk against the latest snazzy FibreChannel SAN array any day! Another nice thing about using a filesystem-based table store is that you don't even need BLOBs: BLOBs are represented as the files they started out as - just drop them into the regular disk-based filesystem! If you like to work, you can keep a pointer to the BLOB's binary file in your tables, but even that's often not necessary if the directory structure and file naming scheme has been thought about very much.
Generally, people who say they need speed in databases need it to try to make up for extremely poor platform and/or application architectures - unfortunately, many of today's web apps fall into that category. Screaming DB speed isn't usually necessary, though - *good* design won't beat on the DB that hard in the first place. (A couple of years ago, I designed a high-performance deeply embedded web application server finely tuned for real-world I/O. Even though it ran on an *8-bit* microcontroller, with only about 3 KILObytes of RAM, it easily outperforms its Pentium-based competitors. That's not BS, it's true - good architecture really does make that much difference, and most modern, popular architectures are just flat *awful* from a performance point-of-view.)
Another reason this is an interesting idea: a number of years ago (literally around the dawn of Linux), RSW (Evan Schaffer's company, which wrote the commercial /rdb database that pretty much originated this whole idea) sold a product called Xrdb - a graphical (X/OpenWindows) interface to stream/operator programming. We bought a copy along with our /rdb license, and even though they kinda ripped us off becasue they never really finished the product, it was heading in a really cool direction - think of Yahoo's new Pipes, but building unix pipelines based on rdb (and other) operators, and you've got the idea. Of course, you could have separate scripts to perform certain data transformations (stored procedures, anyone?) that you could plug into the stream wherever you needed them, for stripping out phone numbers, formatting a report, whatever. Like I said, really cool, and it's about time somebody tried to build something like that again...
Really, I see stream-operator databases like NoSQL as mostly orthogonal to SQL-based RDBMSes, as both are orthogonal to specialized hybrid transactional systems such as Intersystems' M/Cache - they have different strengths, and there are good reasons for choosing each. Sadly, despite great potential, the rdb approach is the one that's still waiting for the world to notice - so go ahead and give a presentation or three...
(BTW, Carlo Strozzi, the author of NoSQL, replyed in enthusiastic agreement. He also recommended the following link on the effectiveness over efficiency approach of using text-based data:
http://aplawrence.com/Opinion/textvsbinary.html)
Posted: Wed 08 Aug 2007, 23:12
by WhoDo
JohnMurga wrote:As far as I am concerned it doesn't compare or compete - If you want a relational database, you use one.
Earlier in this thread I expressed what I thought we needed for Puppy, long and short term. Sqlite3, TkSqlite, your GUI frontend ... all offer a good short term solution for those familiar with SQL or not needing anything more powerful than an address book creator.
OTOH, long term we need an easy-to-use, fully relational database that can compete with 80% or more of what Microsoft Access can do and yet can be used by Window$ refugees who might not be able to use even Access but still need to create medium to large datasets. Something that does what FileMaker Pro does in Window$ and Mac environments. I believe GLOM is the answer for that, but compiling it for Puppy is a major task that is presently beyond my meager capabilities. What it offers is the ability to create SQL tables, forms and reports without knowing a single SQL command.
If someone of your stature and experience were to attempt it, we may stand a chance of success. Komodo stands willing to help. So do I. Maybe we could make this a new Puppy project. True relational database capability that is easy to use is all that stands between many Window$ users and a complete break from the Micro$oft stranglehold. What do you think?
Posted: Fri 10 Aug 2007, 08:58
by DavidBell
WhoDo wrote:OTOH, long term we need an easy-to-use, fully relational database that can compete with 80% or more of what Microsoft Access...
I have never found an easy to use GUI based relational database for any OS. They all seem really easy to use until you try and do something not directly supported in the wizards. But I readily admit to being no expert on the subject and have very limited experience.... maybe Filemaker is the holy grail that I haven't tried. At the moment I use Access97 because it works and seems pretty sable, however trying to do some things in it can require a hell of a lot of hoop jumping. It's not intuitive, but in the end it works.
Higher up in this thread, I bagged OOo Base a bit because for me at least it, uuhhmm ... didn't work (and JRE is too big for basic puppy IMO). So the difference is Access is not that great or incredibly usable, it's just that it works.
So as I see it, sqllite engine is (already?) in puppy, it's the UI that's missing. Something that makes tables, queries, and reports just a little bit simpler is all that's needed. It doesn't have to be a complete solution, just a few integrated tools that make life a bit easier.
I'm not sure which of the packages mentioned higher up best suits this, but I think stability is more important than features with databases, because people with ???K records get spooked by even the hint of a failure. If you are targeting windows refugeees, then Access drivers are good, or something that can be used in both OSs and has data that can be manipulated in Access.
My 2c x 10
DB
Posted: Fri 10 Aug 2007, 09:57
by JohnMurga
Springer wrote:JohnM:
Just to clear up a misconception, NoSQL and its relatives (rdb, /rdb, starbase, etc.) ARE real relational databases. That's the idea - each of their application environments provides relational operators that work out-of-the-box on the tables, which happen to be (usually) TSV text files, sometimes with header lines.
NO MISCONCEPTION my friend
I did read the docs and I am acutely familiar with your argument ...
I even worked on a few systems using this paradigm in the early 90s.
And ... Incidentally I found the following article very useful too :
http://www.troubleshooters.com/lpm/200704/200704.htm
BUT ...
It's a bit like calling my calculator a "Personal Computer" ...
In a way it is
Just like you may be able to "pretend" noSql is a relational database by using the "sorttable/jointable" commands and the like, but at the end of the day it's missing fundamental relational database concepts like integrity constraints (most basically referential integrity) ...
As an example ... Foreign keys, the commonly accepted definition, is that these are integrity constraints enforcing that the value of the attribute set is drawn from a candidate key in another relation. This one of many things that noSql doesn't bother with, because to enforce you have to control ... And that's exactly what it DOESN'T WANT to do
(well, as it happens you can't even define, but anyway). It breaks the relational paradigm, sorry ... The database itself is incapable of representing the relationships, you have to do it with code - The database itself is not relational.
And you can do it ... In code ... And its ugly ...
Furthermore in my tests it was pretty expensive to join data anyway.
To anyone that says noSql is relational I'd say :
- * Referential integrity is explicitly not supported.
* Nulls aren't treated uniformly.
* Views can exist, but only as programs, not as 'virtual tables'.
* Forget about data integrity, "data independence" and integrity constraints.
* You have to relate data in code.
I guess you could call it quasi-relational ...
And you can do neat stuff with it ...
I have nothing against noSql, but if you pitch it as something it isn't it'll fall flat on it's face as anything else would under the circumstances.
Cheers
John de Murga
Posted: Fri 10 Aug 2007, 10:10
by WhoDo
DavidBell wrote:.... maybe Filemaker is the holy grail that I haven't tried.
Oh, it definitely is, and it beats Access 97 hands down, but it isn't available for Linux. I've used it for years and I've had it running under Wine but that's not the same thing.
I don't know how much like FileMaker is GLOM, but it looks interesting. I'd like to see someone with more knowledge than me compile it for Puppy. Then we might have a serious contender. Until then, we'll never know.
Posted: Fri 10 Aug 2007, 10:13
by JohnMurga
WhoDo wrote:JohnMurga wrote:As far as I am concerned it doesn't compare or compete - If you want a relational database, you use one.
Earlier in this thread I expressed what I thought we needed for Puppy, long and short term. Sqlite3, TkSqlite, your GUI frontend ... all offer a good short term solution for those familiar with SQL or not needing anything more powerful than an address book creator.
OTOH, long term we need an easy-to-use, fully relational database that can compete with 80% or more of what Microsoft Access can do and yet can be used by Window$ refugees who might not be able to use even Access but still need to create medium to large datasets. Something that does what FileMaker Pro does in Window$ and Mac environments. I believe GLOM is the answer for that, but compiling it for Puppy is a major task that is presently beyond my meager capabilities. What it offers is the ability to create SQL tables, forms and reports without knowing a single SQL command.
If someone of your stature and experience were to attempt it, we may stand a chance of success. Komodo stands willing to help. So do I. Maybe we could make this a new Puppy project. True relational database capability that is easy to use is all that stands between many Window$ users and a complete break from the Micro$oft stranglehold. What do you think?
Flattery will get you nowhere
GLOM looks pretty amazing, but it's a bit heavy for Puppy (very), but the way Puppy's going you never know ...
My GUI front end is a demo for murgaLua that is constantly evolving ... And more gui like features are on the way. However, it is only ever going to support sqLite and it is a tiny part of a larger project with priorities that come from my main users.
As as far as my involvement with anything other than murgaLua at this point ... The fact remains that I've been homeless for a LONG TIME. I have found a house and will be moving in a couple of weeks, so once I have regular net access and don't have to do things from the hostels I've been staying in you might see a bit of more of me around here.
There are a LOT of things I want to do to Puppy
Cheers
JohnM
Posted: Fri 10 Aug 2007, 10:34
by WhoDo
JohnMurga wrote:Flattery will get you nowhere
Ar, rats!
JohnMurga wrote:GLOM looks pretty amazing, but it's a bit heavy for Puppy (very), but the way Puppy's going you never know ...
Yep. That PostgreSQL backend is the killer. Something similar for sqLite would be nice though .... (
hint, hint, hint)
JohnMurga wrote:My GUI front end is a demo for murgaLua that is constantly evolving ... And more gui like features are on the way. However, it is only ever going to support sqLite and it is a tiny part of a larger project with priorities that come from my main users.
Well it wouldn't take much to bring up to a similar standard to TksqLite, would it? At this point I'm for anything that offers an excuse for murgaLua to find its way into Puppy. That would mean we can have the u beaut Control Panel that dvw86 put together some time back:
http://www.murga-linux.com/puppy/viewto ... 53&t=15931
JohnMurga wrote:As as far as my involvement with anything other than murgaLua at this point ... The fact remains that I've been homeless for a LONG TIME. I have found a house and will be moving in a couple of weeks, so once I have regular net access and don't have to do things from the hostels I've been staying in you might see a bit of more of me around here.
There are a LOT of things I want to do to Puppy
Ooohh! Good news! I'm really glad you've found a place to call home. Are you sure I can't sweet talk you into writing a complete GUI frontend for sqLite that a newbie could use?
Posted: Fri 10 Aug 2007, 11:10
by JohnMurga
WhoDo wrote:JohnMurga wrote:GLOM looks pretty amazing, but it's a bit heavy for Puppy (very), but the way Puppy's going you never know ...
Yep. That PostgreSQL backend is the killer. Something similar for sqLite would be nice though .... (
hint, hint, hint)
And Python, otherwise you "could" try to replace the database backend.
WhoDo wrote:JohnMurga wrote:My GUI front end is a demo for murgaLua that is constantly evolving ... And more gui like features are on the way. However, it is only ever going to support sqLite and it is a tiny part of a larger project with priorities that come from my main users.
Well it wouldn't take much to bring up to a similar standard to TksqLite, would it? At this point I'm for anything that offers an excuse for murgaLua to find its way into Puppy. That would mean we can have the u beaut Control Panel that dvw86 put together some time back:
http://www.murga-linux.com/puppy/viewto ... 53&t=15931
It has surprised me that murgaLua is a core part of DSL but completely unused in Puppy, but it doesn't bother me - Once I hit my 1.0 functionality I'll concentrate on apps, and them maybe someone here will be interested.
As for TksqLite ... I am the wrong person as I have been biased against anything TCL based since before Puppy (as I had to some professional work with it some time back and didn't enjoy it).
WhoDo wrote:JohnMurga wrote:As as far as my involvement with anything other than murgaLua at this point ... The fact remains that I've been homeless for a LONG TIME. I have found a house and will be moving in a couple of weeks, so once I have regular net access and don't have to do things from the hostels I've been staying in you might see a bit of more of me around here.
There are a LOT of things I want to do to Puppy
Ooohh! Good news! I'm really glad you've found a place to call home. Are you sure I can't sweet talk you into writing a complete GUI frontend for sqLite that a newbie could use?
That's something that will probably happen at some point, just don't expect it too soon as it is a small priority in a big world
Cheers
JohnM
Posted: Fri 10 Aug 2007, 11:33
by WhoDo
JohnMurga wrote:It has surprised me that murgaLua is a core part of DSL but completely unused in Puppy, but it doesn't bother me - Once I hit my 1.0 functionality I'll concentrate on apps, and them maybe someone here will be interested.
I already am. How far away is that 1.0 functionality? We have guys here that are itching to get their teeth into programming applications for Puppy, but the breadth of development environments available seems to put off all but the most hardy.
Once I saw what dvw86 was able to quickly achieve with murgaLua, I was sold. It is only a matter of enthusing the "market". If what was done was designed to be compatible with IceWM, then I could introduce it through EZpup without impacting what happens in the mainstream version.
Frankly, I wish I knew more of programming ... especially in murgaLua. I'd have a lash at it myself. Unfortunately, my limited past experience with C and Basic barely qualifies me to write Hello World these days.
Keep plugging away John. Anything good is worth waiting for... just ask my wife!
Posted: Sat 11 Aug 2007, 01:21
by BarryK
At this point I'm for anything that offers an excuse for murgaLua to find its way into Puppy.
MurgaLua is in Puppy, but as John says, it's not actually being used for anything ...yet. Looking at it from a fairly tough admin point of view, it would have to ultimately justify its inclusion, like some irresistable application that needs it.
one application
Posted: Sat 11 Aug 2007, 13:10
by raffy
If we are looking at educating kids in cutting-edge programming, we don't even have to wait for version 1.0.
An example application or two that one can easily find in the Puppy menu and possibly modify at will should get things started. We can even call it "kiddie programming (KP)". (Then I can get my hands on it when no one is looking.
)
How about a "create your Puppy" game? (create your unique Puppy from different colors and shapes)
Posted: Sun 09 Sep 2007, 07:25
by andrei
The fonts in TkSqlite are way too small, very hard to read. I solved this problem by editing the configuration file:
/usr/lib/tile0.7.8/fonts.tcl
Now it looks much better.
Posted: Sun 09 Sep 2007, 17:40
by Flash
How did you edit it? Inquiring minds want to know.
Posted: Sun 09 Sep 2007, 20:44
by andrei
See the attached file. Changes are in the x11 section.
Notice that I use the Courier New font which I downloaded from the Internet. I understand that it is legal to use Courier New because it was made available by Microsoft at some point. But I am not a lawyer.
Working SQLiteDBMS
Posted: Sat 08 Dec 2007, 15:27
by Igo AtM
Have just uploaded SQLiteDBMS it is a puppy hacked version and was originally put together as a low resource solution for sqlite3 access and motionMon or litemon with the sqlite db back end. I had seen this topic previously so I am posting a link to those that wish to try it. It is far from a final package but is working and I would appreciate feedback as I am currently re-writing the client. I felt it may also contribute to the ongoing discussion here link is
http://www.murga-linux.com/puppy/viewto ... 180#159180
Yet another interesting sqlite db management option I am uploading is a Gecko based sqlite manager which is very good at the pure db access level the problem with it is it requires seamonkey with the embedded sqlite engine so version 1.1 and above but it adds more fuel to the debate. It is not a pet as it is a native seamonkey extension. You should just be able to upload it unpack it and drag n drop it onto the browser and it should auto install md5 is below
6027224839339bf3d3a34084c11c3d34 sqlite_manager-0.2.8.3.tar.gz
You will need to unpack it as for security can not upload xpi to the forum
Posted: Fri 08 Feb 2008, 05:33
by Lobster
http://www.knoda.org/
it uses kde and python (too big for Puppy)
but may suit some . . .
Anyone use it?
Posted: Wed 26 Mar 2008, 16:57
by Lobster
This is very nice
but a database online?
What happens when they sell to Microsoft, Google, Yahoo, NSA etc?
http://www.blist.com/index.html
Posted: Wed 26 Mar 2008, 18:25
by tlchost
Hopefully there will be an alternative made for Puppy by then. The signup process is simple and you can easily import a .csv file.
Took a quick look at the support forum....there are not many entries, and the project manager appears to be very active answering user questions.
Apparently one can share a database with others on a read-only basis via email.
Posted: Mon 14 Apr 2008, 04:27
by muggins
There hasn't been much activity on this front but, even though I've no use for a database, I thought I'd post this link to various sqlite frontends:
http://www.sqlite.org/cvstrac/wiki?p=ManagementTools
Posted: Mon 14 Apr 2008, 09:22
by nic2109
Another approach to this issue would be to use the XAMPP package (which is huge) and make it an .sfs file. This would add everything to make Puppy into a Web server running sqlite apache perl and php.
You will still need to add a GUI and/or Report writer, which is where Muggins' link will come into its own.