Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Tue 06 Dec 2016, 08:32
All times are UTC - 4
 Forum index » House Training » HOWTO ( Solutions )
RAID arrays in (Puppy) Linux
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 3 of 3 [41 Posts]   Goto page: Previous 1, 2, 3
Author Message
tempestuous

Joined: 10 Jun 2005
Posts: 5455
Location: Australia

PostPosted: Fri 25 Oct 2013, 21:49    Post subject:  

toronado wrote:
ROX seems to show empty directories in /mnt/ for partitions that don't exist. So, if I don't mount md0, there is a (empty) directory there anyway. I found many other "dummy" directories there for previously created partitions and what not that no longer exist.

Yes, it seems to be a quirk of the mdadm application that it creates additional "dummy" or "ghost" device nodes, typically of the format "/dev/md_d0" or similar. You can manually stop these dummy arrays as such -
Code:
mdadm --stop /dev/whatever

but as long as these devices/arrays don't interfere with the correct device/array, there's no problem, and you can just ignore them.

toronado wrote:
I just added the commands to assemble and mount the RAID array to my Samba Auto-Start script and it works great.

Sure, that startup script is fine, but generally the correct place to such additional commands is /etc/rc.d/rc.local
I have just updated the instructions in the third post.
Back to top
View user's profile Send private message 
tempestuous

Joined: 10 Jun 2005
Posts: 5455
Location: Australia

PostPosted: Fri 25 Oct 2013, 21:50    Post subject:  

And in the final wash up ... you have Puppy running with a functioning Linux software RAID-0 array.
Bravo.
Back to top
View user's profile Send private message 
toronado

Joined: 04 Sep 2013
Posts: 95

PostPosted: Fri 25 Oct 2013, 23:32    Post subject:  

tempestuous wrote:
toronado wrote:
ROX seems to show empty directories in /mnt/ for partitions that don't exist. So, if I don't mount md0, there is a (empty) directory there anyway. I found many other "dummy" directories there for previously created partitions and what not that no longer exist.

Yes, it seems to be a quirk of the mdadm application that it creates additional "dummy" or "ghost" device nodes, typically of the format "/dev/md_d0" or similar. You can manually stop these dummy arrays as such -
Code:
mdadm --stop /dev/whatever

but as long as these devices/arrays don't interfere with the correct device/array, there's no problem, and you can just ignore them.
This is probably getting off-topic for this thread, but I don't think this issue pertains to mdadm. (It might not have anything to do with ROX either.) For example, well before installing the mdadm pet I noticed empty directories in /mnt/ for partitions such as sda2 sdb2 sdc4 sdd2 etc. and none of these partitions actually existed (or at least they didn't exist at the time I was browsing /mnt/). And while some of these partitions I actually created in the past and subsequently deleted (such as sdc4), others (like sda2, sdb2, sdd2) I don't recall ever creating in the first place. It's an old computer, so maybe it's haunted. Smile

tempestuous wrote:
toronado wrote:
I just added the commands to assemble and mount the RAID array to my Samba Auto-Start script and it works great.

Sure, that startup script is fine, but generally the correct place to such additional commands is /etc/rc.d/rc.local
I have just updated the instructions in the third post.
OK thanks.
Back to top
View user's profile Send private message 
carenrose

Joined: 10 Dec 2012
Posts: 36

PostPosted: Fri 01 Nov 2013, 17:09    Post subject:  

Sorry, I know this thread is old, but I figured this is probably the best place to put this question.

I have succeeded in setting up my RAID (1, btw). I'm now trying - and failing - to install Grub.

From the menu entry gui program, when I get to the step, something like "which disk or whatever do you want to put it in on?" and it suggests "/dev/sda(1?)" I tried both /dev/md0 and /dev/sda (and/or /dev/sda1) and /dev/sdb (and same).
I am not currently on that computer so I don't remember if that step asked for the disk or the partition, but whichever it asked for I put, ok? Very Happy
It said that /dev/md0 "is not a valid Linux" something or other, and it couldn't mount sda or sdb to do its stuff there - which I figured as much.

Anyways, I tried command line. The below is approximation of what it said: (Yes, I know I'm being overly descriptive.)
Code:
# grub
Blah blah something about BIOS this will take a while. ...

grub > find /boot/grub/stage1

Error 15: File not found


Ok, so I copied the files from /usr/sbin/grub (or wherever they are) to md0's /boot/grub that I created ... so it should be present on *both physical drives* right? And grub won't find when I do the above yet again.

What am I missing? Am I just totally lost?
Back to top
View user's profile Send private message 
tempestuous

Joined: 10 Jun 2005
Posts: 5455
Location: Australia

PostPosted: Fri 01 Nov 2013, 21:08    Post subject:  

Ah, you're trying to boot from a RAID array, and that's complicated. Personally, I would avoid this, and install Puppy on a separate (non-RAID) drive - even a small USB flash drive. Then just use the RAID array for your user-data.

But if you're determined to persist, you will need to do some research and experimentation, and then rebuild Puppy's initrd.
Let me explain it in principle: the software RAID array can generally only be understood, and thus accessible, to a running operating system. It's difficult (but not impossible) to access files on the array at bootup.

What you need to do is include all necessary drivers, utilities, and configuration logic into Puppy's initial ramdisk.
In your case, that means rebuilding the initrd image to include the mdadm application, raid1 kernel module, and also modify the initrd startup scripts to assemble and mount the RAID array right at the start of the boot sequence.
I have no experience in this, so cannot help with the fine details.

Once this is all achieved, yes, you can put the grub configuration files onto the array ...
but as I understand it, the Master Boot Record must still be installed onto a single physical drive. I don't believe it's possible to share the MBR on an array - that would effectively be sharing two boot sectors. I can't imagine that any motherboard bios would be able to recognise this.
Back to top
View user's profile Send private message 
toronado

Joined: 04 Sep 2013
Posts: 95

PostPosted: Tue 05 Nov 2013, 01:51    Post subject:  

I've installed a new version of Puppy (PhatSlacko 5.5.02 for its easy Samba config) and I installed mdadm-3.2.5 from the PPM and rebooted but no md0 in /mnt.

I tried:
Code:
# mdadm --assemble /dev/md0
mdadm: /dev/md0 not identified in config file.
So before I go messing things up further, what should I try next?
Back to top
View user's profile Send private message 
tempestuous

Joined: 10 Jun 2005
Posts: 5455
Location: Australia

PostPosted: Tue 05 Nov 2013, 10:00    Post subject:  

So as I understand it, you have installed a new version of Puppy, but you want to access the software RAID array that you previously created?

toronado wrote:
Code:
mdadm: /dev/md0 not identified in config file.

Oops, it sounds like you didn't keep a copy of your configuration file (from your earlier installation) - /etc/mdadm.conf
Back to top
View user's profile Send private message 
toronado

Joined: 04 Sep 2013
Posts: 95

PostPosted: Tue 05 Nov 2013, 10:34    Post subject:  

tempestuous wrote:
So as I understand it, you have installed a new version of Puppy, but you want to access the software RAID array that you previously created?

toronado wrote:
Code:
mdadm: /dev/md0 not identified in config file.

Oops, it sounds like you didn't keep a copy of your configuration file (from your earlier installation) - /etc/mdadm.conf
I haven't erased the previous install (it's on a separate partition). Are you saying all I need to do is copy over the config file?
Back to top
View user's profile Send private message 
tempestuous

Joined: 10 Jun 2005
Posts: 5455
Location: Australia

PostPosted: Tue 05 Nov 2013, 10:58    Post subject:  

I think so, yes.
Back to top
View user's profile Send private message 
toronado

Joined: 04 Sep 2013
Posts: 95

PostPosted: Tue 05 Nov 2013, 23:30    Post subject:  

Thanks, it worked.
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 1701
Location: UK

PostPosted: Mon 18 Jan 2016, 16:12    Post subject: Linux Software RAID  

tempestuous wrote:
3. LINUX SOFTWARE RAID

If your RAID setup is running only under Linux, and there's no need to dual boot into Windows, this is the better RAID solution than bios-RAID.
For motherboards with onboard RAID (Host RAID) as mentioned in the previous section, disable the RAID function in bios. Yes, this sounds a bit strange, but Linux will take over all RAID configuration.
There's a HOWTO here -
https://raid.wiki.kernel.org/index.php/RAID_setup#Mdadm_modes_of_operation
Here's a summary. Let's assume you want RAID1 (mirroring for redundancy)
using /dev/sdb1 and /dev/sdc1 (your drives must be partitioned with GParted, first)
RAID0 support is built directly into the Puppy kernel, but RAID1 support is via the external kernel module "raid1", so we first need to load this module -
Code:
modprobe raid1

Install the mdadm dotpet package attached to this post. Then run this command to configure the RAID array
Code:
mdadm --create --verbose /dev/md0 --level=raid1 --raid-devices=2 /dev/sdb1 /dev/sdc1

You should hear the drives working as the array is initialised. Once this is complete, the new RAID configuration must be saved, with this command -
Code:
mdadm --detail --scan >> /etc/mdadm.conf

(It's worth keeping a copy of this configuration file on a USB dongle, for example, in case your system drive fails in the future, and you need to be able to access the RAID array on a different installation.)
Your new RAID array is /dev/md0. Go ahead and format it with ext3 (alternative filesystems may be considered)
Code:
mkfs.ext3 -v -m .1 -b 4096 -E stride=32,stripe-width=64 /dev/md0

All those extra formatting options I just included are explained in the RAID wiki I mentioned earlier.
Now mount the RAID array -
Code:
mkdir /mnt/md0
mount /dev/md0 /mnt/md0

Now you can browse to /mnt/md0 with ROX, and test it by copying some files to/from.

That's it, but we also need to restore this configuration at each bootup. My mdadm dotpet includes a special mdadm udev rule which, in theory, will auto-detect software-raid arrays at bootup, and restore their configuration ... unfortunately testing under Puppy 5.28 has shown that this udev function fails!!
So you need to add some extra commands to Puppy's startup scripts - open /etc/rc.d/rc.local in Geany, and add these 3 lines -
Code:
modprobe raid1
mdadm --assemble /dev/md0
mount /dev/md0 /mnt/md0

Save.
Now each time you boot Puppy, your RAID array will be ready to use at /mnt/md0


Just to add from a new-user angle, the
Code:
mdadm --create --verbose /dev/md0 --level=raid1 --raid-devices=2 /dev/sdb1 /dev/sdc1

stage took over a hour for me (pair of 45GB USB/HDD). The command when entered 'finished' instantly, but the disks whirred for ages. Found a command that shows the progress of that stage

cat /proc/mdstat

that provides % done and estimated time to completion details.

Another useful command is

mdadm -D /dev/md0

to see the status of the RAID mirror. The standard Linux thing appears to be that the monitor will email root when one of a pair of RAID1 fails, but a Puppy doesn't have standard linux mail I guess you wouldn't find out that one disk had perhaps failed until the second one also failed - which isn't what is desired. Periodic manual checks using the above might indicate that one disk had failed enabling you to make a backup of the /mnt/md0 content before replacing the failed disk.

I've setup a software Raid1 for the purpose of storing one-time-pad keys for encrypted documents http://www.murga-linux.com/puppy/viewtopic.php?p=881999#881999 that I store on googledrive, as the loss of the key is paramount to loss of data. Having encrypted data in the cloud, keys stored on USB, laptop/pc using both to decrypt/work/reencrypt without storing copies unencrypted means that if the laptop is lost/stolen data is still safe (thief can't access personal data without the key). Assuming cloud data is safe/backed up, but unreadable (encrypted) by the host, and two pairs of USB's hold the keys, then all-round that's pretty secure/safe.

On another note, whilst reading around I found the attached youtube report where a guy plugged in a multiple number of usb's into a usb hub and set them up as stripped arrays. The speed of two stripped USB's was significantly quicker than one USB, but then the speed benefit tailed off pretty quickly thereafter for 3, 4 .... 8.
i.png
 Description   
 Filesize   68.1 KB
 Viewed   95 Time(s)

i.png

Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 3 of 3 [41 Posts]   Goto page: Previous 1, 2, 3
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » HOWTO ( Solutions )
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1755s ][ Queries: 13 (0.0347s) ][ GZIP on ]