Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 30 Jul 2014, 05:06
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
Puppy Database
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 3 of 9 [134 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7, 8, 9 Next
Author Message
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 7047
Location: Perth, Western Australia

PostPosted: Sun 10 Jun 2007, 11:46    Post subject:  

Komodo, it doesn't have to be made into a single PET package. When Pup 2.17 is released, the TkSqlite PET package will be in the selection list and the information on dependencies is also there, so all those dependent packages will automatically get installed as well.
Or even if someone downloaded 'tksqlite-xxx.pet', the dependency info is in Puppy and also they will get auto-installed.
Back to top
View user's profile Send private message Visit poster's website 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 7047
Location: Perth, Western Australia

PostPosted: Mon 11 Jun 2007, 10:16    Post subject:  

How about this? Hop is a language for creating gui applications viewable in a web browser. It uses sqlite as a backend, so is effectively a tool for creating databases that can be served over the web. Hop is also intended as a replacement for the traditional gui widget tools (like GTK, Qt) for local applications.

Main page:
http://hop.inria.fr/#home-notepad=np:0

Technical details:
http://hop.inria.fr/usr/local/share/hop/weblets/home/articles/hop-lang/article.html

...it's not a gui database-creator-manager thingy though, which is what we have been discussing.
Back to top
View user's profile Send private message Visit poster's website 
IvanF

Joined: 12 Dec 2005
Posts: 15
Location: W. Australia

PostPosted: Sat 23 Jun 2007, 03:11    Post subject: PupBase
Subject description: NoSQL Database
 

I can across NoSQL the other day.

Home page
http://www.scriptaworks.com/cgi-bin/wiki.cgi/NoSQL

A Great Tutorial
http://www.troubleshooters.com/lpm/200704/200704.htm

Another older tutorial
http://www2.linuxjournal.com/article/3294

NoSQL looks like it is built for Puppy. Somone with better unix skills than me may be able to create PupBase with it.

regards Ivan F
Back to top
View user's profile Send private message 
raffy

Joined: 25 May 2005
Posts: 4759
Location: Manila

PostPosted: Sat 23 Jun 2007, 09:42    Post subject: interesting  

Both are very interesting, especially for database use in the future.
Back to top
View user's profile Send private message 
JohnMurga
Site Admin


Joined: 04 May 2005
Posts: 586
Location: Far to the east

PostPosted: Thu 19 Jul 2007, 12:12    Post subject:  

Dunno if it'll be of interest, but I knocked up a quick tool for personal use ...

http://www.murga-projects.com/forum/showthread.php?tid=118



Cheers
JohnM
sqLiteMngr.PNG
 Description   
 Filesize   11.57 KB
 Viewed   1644 Time(s)

sqLiteMngr.PNG

Back to top
View user's profile Send private message Visit poster's website 
Springer


Joined: 22 Aug 2006
Posts: 52
Location: Austin, TX

PostPosted: Tue 07 Aug 2007, 00:55    Post subject:  

I second IvanF's recommendation to consider NoSQL as a potential solution. It's worth a look because of the way it fundamentally changes the way databases are built to leverage the power of common Unix text processing tools and scripting. Tables are regular ASCII files SQL is replaced by shell commnads that perform operations on the data through regular unix pipes, tees, etc. This lets you use *anything* to perform special manipulations: For example, there is one implementation (Starbase) of a stream/operator database (the technical name for "/rdb-like" databases) of this idea widely used by astronomers that includes special astronomical data operators. Speed is not the problem you expect, especially with RAMdisks (and mawk)!

To really understand this, you need to read the Wolf and Schaeffer paper on "The Unix Shell as a 4GL" (http://www.rsw.com/lib/4gl.pdf). For more info, check out the Jorgensen, Schaeffer, and Manis book on /rdb.

I designed a state-of-the-art configuration management system based on this technology for a Fortune 20 company in 1993, and the idea has only become more practical and powerful with time. The power of this idea isn't apparent until you use it - the leverage is awesome, sometimes allowing several lines of script code to replace hundreds of lines of SQL or other alternatives. It might be a good idea to plumb this concept into tcl, but I haven't really thought much about how to best do that...

In addition, I think there could be some interesting synergies between NoSQL or a similar stream/operator database and ROX, which embraces the Unix file metaphor. Imagine a set of operators in ROX that could be strung together to build apps.... RSW tried this in the early 1990's with a product they never finished called /rdbX (or /Xrdb) - it was a graphical pipeline builder that ran under OpenLook that basically built scripts to perform database operations. The nice thing about this approach is that scripting expertise is considerably more common than database expertise, which might make it a doubly good option for a distro like Puppy.

Oh, and BTW, Carlo Strozzi, the author of NoSQL, also has written CSA, the CGI Services Architecture to allow stream/operator or other command-line processors to be exposed as CGI services. (see http://www.scriptaworks.com/cgi-bin/wiki.cgi/CSA/CSA)
Back to top
View user's profile Send private message 
WhoDo


Joined: 11 Jul 2006
Posts: 4441
Location: Lake Macquarie NSW Australia

PostPosted: Tue 07 Aug 2007, 02:36    Post subject:  

Springer wrote:
I second IvanF's recommendation to consider NoSQL as a potential solution. It's worth a look because of the way it fundamentally changes the way databases are built to leverage the power of common Unix text processing tools and scripting.

Wow! That does sound interesting. Wouldn't it be a real coup for Puppy if it included this capacity by default? Imagine the capability it would give this minimal distro! Please, Barry, someone, anyone, consider this option.... Cool

_________________
Actions speak louder than words ... and they usually work when words don't!
SIP:whodo@proxy01.sipphone.com; whodo@realsip.com
Back to top
View user's profile Send private message 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 7047
Location: Perth, Western Australia

PostPosted: Tue 07 Aug 2007, 21:20    Post subject:  

I've downloaded nosql, I'll check it out.

Yes, I agree, Unix/Linux shell scripts can be very powerful, and I have had occasion to rewrite a shell script in C and it was really awkward to do... I didn't bother to complete it.
Back to top
View user's profile Send private message Visit poster's website 
JohnMurga
Site Admin


Joined: 04 May 2005
Posts: 586
Location: Far to the east

PostPosted: Wed 08 Aug 2007, 08:23    Post subject:  

Springer wrote:
I designed a state-of-the-art configuration management system based on this technology for a Fortune 20 company in 1993, and the idea has only become more practical and powerful with time. The power of this idea isn't apparent until you use it - the leverage is awesome, sometimes allowing several lines of script code to replace hundreds of lines of SQL or other alternatives. It might be a good idea to plumb this concept into tcl, but I haven't really thought much about how to best do that...

It's is funny, I messed around with noSql and it too reminded me of a system I wrote around 1993 (on SCO Xenix), however, it is a fundamentally different paradigm ...
As far as I am concerned it doesn't compare or compete - If you want a relational database, you use one.

If you want an infinitely flexible meta-data system ... Then ... Err ...

Maybe that's the reason that as much as noSql has been around for A LONG time I haven't yet seen benchmarks comparing it to anything else ... I have seen a lot of claims and jingo-like statements, but nothing else.

Either way I would have thought this whole thing is a no brainer as even many unix tools and languages are turning to sqLite to store their structured data. So having some cross over might be nice ...

Springer wrote:
Oh, and BTW, Carlo Strozzi, the author of NoSQL, also has written CSA, the CGI Services Architecture to allow stream/operator or other command-line processors to be exposed as CGI services. (see http://www.scriptaworks.com/cgi-bin/wiki.cgi/CSA/CSA)

I would have thought it's obvious why that's a bad idea (a bit like CGI itself). Wink

Execution out of a sandbox, forking for web-requests, etc, etc ...

Cheers
JohnM
Back to top
View user's profile Send private message Visit poster's website 
Lobster
Official Crustacean


Joined: 04 May 2005
Posts: 15117
Location: Paradox Realm

PostPosted: Wed 08 Aug 2007, 08:42    Post subject:  

20k lo-tech flat file console database
http://www.murga-linux.com/puppy/viewtopic.php?search_id=960504053&t=1247

I think that is it here . . .
http://www.ajwells.uklinux.net/idexdb.html

m m m we might have room for two databases . . .

Cool

_________________
Puppy WIKI

Last edited by Lobster on Fri 10 Apr 2009, 07:30; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website 
Springer


Joined: 22 Aug 2006
Posts: 52
Location: Austin, TX

PostPosted: Wed 08 Aug 2007, 19:08    Post subject:  

JohnM:

Just to clear up a misconception, NoSQL and its relatives (rdb, /rdb, starbase, etc.) ARE real relational databases. That's the idea - each of their application environments provides relational operators that work out-of-the-box on the tables, which happen to be (usually) TSV text files, sometimes with header lines.

IMHO, every well-read Unix hacker should at least have read this paper to see how Unix (the toolset, more than the OS proper) can be leveraged as an app platform, not just for underlying OS support: The Unix Shell as a Fourth Generation Language.

Stream/Operator databases will never flat outperform "real database engines", but they offer very real advantages, and in today's world, where the penalty for interpreted code is trivial and huge, fast harddisks and RAMdisks are dirt-cheap, the approach makes more sense than ever.

FWIW, here's a (slightly edited) post I made to the NoSQL mailing list a while back, when one of the new guys was wondering how to present NoSQL to his LUG as an alternative to "regular" SQL databases:
Quote:
Well, Evan Schaffer's paper "The Unix shell as a 4GL" is a great place to start. Mainly, the advantage is not so much the database itself (DBs are just a way to hold and find stuff, after all) - it's the "stream-operator" paradigm that a database like /rdb, Starbase, or NoSQL makes possible.

This approach gives you awesome modularity and leverage, and lets non-DB people write serious DB apps without ever having to know all the arcane crap about how SQL works.

Further, in today's world where Gigabytes of RAM and RAMdisks are everywhere, speed isn't really the concern most people think it is. 4GB is actually a really LARGE database, and I'll stack up a RAMdisk against the latest snazzy FibreChannel SAN array any day! Another nice thing about using a filesystem-based table store is that you don't even need BLOBs: BLOBs are represented as the files they started out as - just drop them into the regular disk-based filesystem! If you like to work, you can keep a pointer to the BLOB's binary file in your tables, but even that's often not necessary if the directory structure and file naming scheme has been thought about very much.

Generally, people who say they need speed in databases need it to try to make up for extremely poor platform and/or application architectures - unfortunately, many of today's web apps fall into that category. Screaming DB speed isn't usually necessary, though - *good* design won't beat on the DB that hard in the first place. (A couple of years ago, I designed a high-performance deeply embedded web application server finely tuned for real-world I/O. Even though it ran on an *8-bit* microcontroller, with only about 3 KILObytes of RAM, it easily outperforms its Pentium-based competitors. That's not BS, it's true - good architecture really does make that much difference, and most modern, popular architectures are just flat *awful* from a performance point-of-view.)

Another reason this is an interesting idea: a number of years ago (literally around the dawn of Linux), RSW (Evan Schaffer's company, which wrote the commercial /rdb database that pretty much originated this whole idea) sold a product called Xrdb - a graphical (X/OpenWindows) interface to stream/operator programming. We bought a copy along with our /rdb license, and even though they kinda ripped us off becasue they never really finished the product, it was heading in a really cool direction - think of Yahoo's new Pipes, but building unix pipelines based on rdb (and other) operators, and you've got the idea. Of course, you could have separate scripts to perform certain data transformations (stored procedures, anyone?) that you could plug into the stream wherever you needed them, for stripping out phone numbers, formatting a report, whatever. Like I said, really cool, and it's about time somebody tried to build something like that again...

Really, I see stream-operator databases like NoSQL as mostly orthogonal to SQL-based RDBMSes, as both are orthogonal to specialized hybrid transactional systems such as Intersystems' M/Cache - they have different strengths, and there are good reasons for choosing each. Sadly, despite great potential, the rdb approach is the one that's still waiting for the world to notice - so go ahead and give a presentation or three...

(BTW, Carlo Strozzi, the author of NoSQL, replyed in enthusiastic agreement. He also recommended the following link on the effectiveness over efficiency approach of using text-based data: http://aplawrence.com/Opinion/textvsbinary.html)
Back to top
View user's profile Send private message 
WhoDo


Joined: 11 Jul 2006
Posts: 4441
Location: Lake Macquarie NSW Australia

PostPosted: Wed 08 Aug 2007, 19:12    Post subject:  

JohnMurga wrote:
As far as I am concerned it doesn't compare or compete - If you want a relational database, you use one.

Earlier in this thread I expressed what I thought we needed for Puppy, long and short term. Sqlite3, TkSqlite, your GUI frontend ... all offer a good short term solution for those familiar with SQL or not needing anything more powerful than an address book creator.

OTOH, long term we need an easy-to-use, fully relational database that can compete with 80% or more of what Microsoft Access can do and yet can be used by Window$ refugees who might not be able to use even Access but still need to create medium to large datasets. Something that does what FileMaker Pro does in Window$ and Mac environments. I believe GLOM is the answer for that, but compiling it for Puppy is a major task that is presently beyond my meager capabilities. What it offers is the ability to create SQL tables, forms and reports without knowing a single SQL command.

If someone of your stature and experience were to attempt it, we may stand a chance of success. Komodo stands willing to help. So do I. Maybe we could make this a new Puppy project. True relational database capability that is easy to use is all that stands between many Window$ users and a complete break from the Micro$oft stranglehold. What do you think?

_________________
Actions speak louder than words ... and they usually work when words don't!
SIP:whodo@proxy01.sipphone.com; whodo@realsip.com
Back to top
View user's profile Send private message 
DavidBell

Joined: 24 Nov 2006
Posts: 132

PostPosted: Fri 10 Aug 2007, 04:58    Post subject:  

WhoDo wrote:
OTOH, long term we need an easy-to-use, fully relational database that can compete with 80% or more of what Microsoft Access...


I have never found an easy to use GUI based relational database for any OS. They all seem really easy to use until you try and do something not directly supported in the wizards. But I readily admit to being no expert on the subject and have very limited experience.... maybe Filemaker is the holy grail that I haven't tried. At the moment I use Access97 because it works and seems pretty sable, however trying to do some things in it can require a hell of a lot of hoop jumping. It's not intuitive, but in the end it works.

Higher up in this thread, I bagged OOo Base a bit because for me at least it, uuhhmm ... didn't work (and JRE is too big for basic puppy IMO). So the difference is Access is not that great or incredibly usable, it's just that it works.

So as I see it, sqllite engine is (already?) in puppy, it's the UI that's missing. Something that makes tables, queries, and reports just a little bit simpler is all that's needed. It doesn't have to be a complete solution, just a few integrated tools that make life a bit easier.

I'm not sure which of the packages mentioned higher up best suits this, but I think stability is more important than features with databases, because people with ???K records get spooked by even the hint of a failure. If you are targeting windows refugeees, then Access drivers are good, or something that can be used in both OSs and has data that can be manipulated in Access.

My 2c x 10

DB
Back to top
View user's profile Send private message 
JohnMurga
Site Admin


Joined: 04 May 2005
Posts: 586
Location: Far to the east

PostPosted: Fri 10 Aug 2007, 05:57    Post subject:  

Springer wrote:
JohnM:

Just to clear up a misconception, NoSQL and its relatives (rdb, /rdb, starbase, etc.) ARE real relational databases. That's the idea - each of their application environments provides relational operators that work out-of-the-box on the tables, which happen to be (usually) TSV text files, sometimes with header lines.


NO MISCONCEPTION my friend Wink

I did read the docs and I am acutely familiar with your argument ...
I even worked on a few systems using this paradigm in the early 90s.

And ... Incidentally I found the following article very useful too :

http://www.troubleshooters.com/lpm/200704/200704.htm

BUT ...

It's a bit like calling my calculator a "Personal Computer" ...

In a way it is Smile

Just like you may be able to "pretend" noSql is a relational database by using the "sorttable/jointable" commands and the like, but at the end of the day it's missing fundamental relational database concepts like integrity constraints (most basically referential integrity) ...

As an example ... Foreign keys, the commonly accepted definition, is that these are integrity constraints enforcing that the value of the attribute set is drawn from a candidate key in another relation. This one of many things that noSql doesn't bother with, because to enforce you have to control ... And that's exactly what it DOESN'T WANT to do (well, as it happens you can't even define, but anyway). It breaks the relational paradigm, sorry ... The database itself is incapable of representing the relationships, you have to do it with code - The database itself is not relational.

And you can do it ... In code ... And its ugly ...

Furthermore in my tests it was pretty expensive to join data anyway.

To anyone that says noSql is relational I'd say :

    * Referential integrity is explicitly not supported.
    * Nulls aren't treated uniformly.
    * Views can exist, but only as programs, not as 'virtual tables'.
    * Forget about data integrity, "data independence" and integrity constraints.
    * You have to relate data in code.

I guess you could call it quasi-relational ... Smile

And you can do neat stuff with it ...

I have nothing against noSql, but if you pitch it as something it isn't it'll fall flat on it's face as anything else would under the circumstances.

Cheers
John de Murga
Back to top
View user's profile Send private message Visit poster's website 
WhoDo


Joined: 11 Jul 2006
Posts: 4441
Location: Lake Macquarie NSW Australia

PostPosted: Fri 10 Aug 2007, 06:10    Post subject:  

DavidBell wrote:
.... maybe Filemaker is the holy grail that I haven't tried.

Oh, it definitely is, and it beats Access 97 hands down, but it isn't available for Linux. I've used it for years and I've had it running under Wine but that's not the same thing.

I don't know how much like FileMaker is GLOM, but it looks interesting. I'd like to see someone with more knowledge than me compile it for Puppy. Then we might have a serious contender. Until then, we'll never know.

_________________
Actions speak louder than words ... and they usually work when words don't!
SIP:whodo@proxy01.sipphone.com; whodo@realsip.com
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 3 of 9 [134 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7, 8, 9 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1190s ][ Queries: 13 (0.0070s) ][ GZIP on ]