Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sat 20 Dec 2014, 21:27
All times are UTC - 4
 Forum index » Off-Topic Area » Programming
filesystem as a database using only shell
Post new topic   Reply to topic View previous topic :: View next topic
Page 2 of 2 [20 Posts]   Goto page: Previous 1, 2
Author Message
tallboy


Joined: 21 Sep 2010
Posts: 454
Location: Oslo, Norway

PostPosted: Tue 14 May 2013, 09:54    Post subject:  

technosaurus, no matter how you succeed with your database, it is always a pleasure to read your code!

tallboy

_________________
True freedom is a live Puppy on a multisession CD/DVD.

Last edited by tallboy on Wed 15 May 2013, 15:17; edited 1 time in total
Back to top
View user's profile Send private message 
jpeps

Joined: 31 May 2008
Posts: 3220

PostPosted: Tue 14 May 2013, 11:41    Post subject:  

amigo wrote:
When you first started the thread I thought you had in mind to use no files at all -that entries would only consist of directories and/or links.

Using flat files to hold data becomes a space problem since every file has a minimum size even if it only contains one character. The block size of the filesystem determines the minimum size requirement -usually at least 4k.


That's just the beginning..think about complexity of memory allocation.

re: planning for ACID compliance. There's the small issue of database integrity. Databases use a complex series of locks, so that the master file maintains integrity while various users are simultaneously reading and writing to it. Issues like that are why they need millions of tests.

For a personal database, there are many ways to create a single file. I have a "MyLibrary" database that creates an XML file which is easily parsed for individual books and their related data.
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4424

PostPosted: Tue 14 May 2013, 16:07    Post subject:  

That was the original plan, but then I read about a couple of filesystems that can store small files in the inode table, which most well planned databases should be. Using the files allows it to maintain a history, but if speed and storage is more important than keeping history it can rewrite the files once they grow large enough that they cannot fit in the inode (git also uses the filesystem and can do a similar operation IIRC)
_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
jpeps

Joined: 31 May 2008
Posts: 3220

PostPosted: Tue 14 May 2013, 16:56    Post subject:  

technosaurus wrote:
That was the original plan, but then I read about a couple of filesystems that can store small files in the inode table, which most well planned databases should be.


Wouldn't that involve a separate inode for every process of every user of every file?
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4424

PostPosted: Mon 01 Dec 2014, 16:40    Post subject:  

jpeps wrote:
Wouldn't that involve a separate inode for every process of every user of every file?
Sorry for the delayed reply, this kind of dropped off my radar, but to answer the question, each file gets an inode in the filesystem, but processes typically would only be opening 1 file descriptor (or so) at a time.

This would also be a good way to implement light weight, parse-free configuration.

the c psuedo code would look like
Code:
int get_value(const char *path, void *buf,size_t len){
  int fd = open(path, O_RDONLY);
  if (fd<0) return fd;
  len=read(fd,buf,len);
  close(fd);
  return len;
}
int set_value(const char *path, void *buf,size_t len){
  int fd = open(path, O_CREAT|O_WRONLY|O_TRUNC);
  if (fd<0) return fd;
  len=write(fd,buf,len);
  close(fd);
  return len;
}

So rather than having a file with lots of VAR=value,
It would be a directory containing files named VAR with value as the contents.
This could be a simple RGB color for a background, a string for a tooltip or even a struct for a desktop icon.

Now all you have to do is run an inotify watch on the directory in case entries are added/changed/removed and the config program can be completely separated. The total size of config data may be a bit larger, but now you don't have to rewrite/parse an entire configuration file for every single change. You don't need to keep global variables around for holding the values (just read'em when you need'em) If you are really clever, with a few code changes you can put the type for each variable in the first byte so that it is simple to automatically build a config gui (for example the enum FILENAME would indicate a type 1 and the gui could read the data as a string for a file selector widget)

Edit: Here are some filesystems capable of storing small files with inode data rather than allocating a whole ??kb block
ext4, btrfs, zfs, reiserfs, gpfs

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 2 of 2 [20 Posts]   Goto page: Previous 1, 2
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Off-Topic Area » Programming
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0536s ][ Queries: 11 (0.0037s) ][ GZIP on ]