Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sat 21 Jul 2018, 23:25
All times are UTC - 4
 Forum index » House Training » HOWTO ( Solutions )
Convert any ISO into a hybrid & use dd to place any
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 3 of 3 [44 Posts]   Goto page: Previous 1, 2, 3
Author Message
jamesbond

Joined: 26 Feb 2007
Posts: 3151
Location: The Blue Marble

PostPosted: Sat 17 Oct 2015, 22:15    Post subject:  

Please read this then get the script for your review. It's for fatdog so it's not directly applicable (fatdog isohybrid has 2 partitions instead of one), but the idea is the same.
_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13.
Contributed Fatdog64 packages thread.
Back to top
View user's profile Send private message 
Puppus Dogfellow


Joined: 07 Jan 2013
Posts: 1560
Location: nyc

PostPosted: Tue 20 Oct 2015, 17:23    Post subject:  

is it possible to add files--whether from a decompressed quirky.xz or puppy.iso--to a drive that's already been formatted as two partitions, one a 16mb vfat and the the other the balance of the drive as f2fs? (basically what Barry Kauler's quirky installers do to flashdrives, unless that's changed for the most recent quirky). it'd be nice to be able to keep the data on the large partition and alter files on it and the boot partition as necessary to accommodate different or additional puppies or quirkies (multiboot would be nice, but i'd be happy to just reuse a drive without having to entirely wipe it first). instead of expanding out to an entire partition, can this method be used to force a save folder on that partition? could the conversion to isohybrid be done in a folder and then the contents moved?

thanks for the guidance/patience (i'm not exactly sure how idiotic these questions are, but i feel they may be significantly so).

Very Happy
Back to top
View user's profile Send private message 
amigo

Joined: 02 Apr 2007
Posts: 2641

PostPosted: Wed 21 Oct 2015, 03:34    Post subject:  

Of course you can add, change or delete files from the partitions, they just neeed to be mounted first.

You cant 'hybridize' a folder. The process writes data to the first sector of an iso image(a file).
Back to top
View user's profile Send private message 
Puppus Dogfellow


Joined: 07 Jan 2013
Posts: 1560
Location: nyc

PostPosted: Wed 21 Oct 2015, 09:07    Post subject:  

amigo wrote:
Of course you can add, change or delete files from the partitions, they just neeed to be mounted first.

You cant 'hybridize' a folder. The process writes data to the first sector of an iso image(a file).


thanks for the reply, amigo.

just to be clear, how accurate is the following?

starting with two drives formatted ETP's way or by way of 01micko's f2fs installer or BarrkyK's quirky installer (i apologize if i'm mistakenly clumping these tools together--superficially at least, they seem to do more or less the same thing to the target flashdrive), i use one for a period and decide i want to swap in a different OS. i move all the original 2nd partition files to and archive folder, delete the contents of the boot partition, copy over the files from the second, untouched drive, and drive 1 will boot with the new OS and the archive folder (remnants of personal and system files of the previous installation on the drive) accessible?

if so, that seems extraordinarily convenient.

_____


here's how quirky installs to two partitions:

Code:
16MB partition 1:
boot.msg  ldlinux.sys  vmlinuz
help.msg  logo.16     syslinux.cfg


Code:
27 GB partition 2:
audit  dev   lib       mnt   root   sys  var
bin    etc   lib64     opt   run    tmp
boot   file  man2html  proc  sbin    usr

[this second batch plus any personal files/folders the user creates on the partition would be the ones i'd want to archive.]

how hard-wired (for lack of a more accurate term--accuracy about to plummet further) or unforgiving is the code identifying the drive and tying it to the OS it's running? if this won't work as conjectured, how involved would a manual fix be? this method doesn't require any changes to the menu.list, so it seems like it could be potentially a very newbie friendly installation method. there'd be one fewer thing to mess with, and gparted could allow for very specific formatting of the target drives.
Back to top
View user's profile Send private message 
amigo

Joined: 02 Apr 2007
Posts: 2641

PostPosted: Wed 21 Oct 2015, 13:49    Post subject:  

Whoa, partner. "delete the contents of the boot partition, copy over the files" This doesn't restore the boot sector. The boot sector is not a part of the file system -it precedes the first sectors used by any filesystem. You still need to use 'dd' on the first sector of the drive or image to write any new bootsector.

To use the proper terms and get the whole picture, the first
446 bytes is the executable boot code -the boot sector. The next
64 bytes is the partition table. The next
2 bytes is the 'magic' which signals the end of the MBR. The MBR
is the 512 first bytes of the disc.

A bootable CD or iso image do not have an MBR. There is a significant empty region at the start of the disk -32K IIRC. bootloader code 'knows' to look at this location when booting from a CD device. mkisohybrid takes advantage of the fact that both bootable CD code and MBR can exist on the same image/device without 'colliding'.

filesystems do not usually start in the first (0 zero) sector. They start in the second (1 one) sector.

If you create extended partitions, then the last bits in the partition table will point to the spot where the extended partition data is located -this is also somewhere in the first sector.

bootloaders also can use the space left over in the zero sector.

copy/move operations never touch the first (zero) sector as 'cp' and 'mv' work with filesystems. 'dd' or 'cat' can write to the zero sector. The zero sector refers to whole device -not to a partition ( /dev/hda and not /dev/hda1 ). Technically, it shouldn't matter if any of the partitions are mounted during writing to the zero sector -but good-practise would encourage you to not do so.

Using fdisk also writes to the zero sector, but only to the partition table areas -it does not write to the real bootsector. Hence, any partitioning can be done after dd'ing. Usually, one might dd a complete 512 byte MBR which already has a partition table, then use fdisk to alter the partition table.
Back to top
View user's profile Send private message 
Puppus Dogfellow


Joined: 07 Jan 2013
Posts: 1560
Location: nyc

PostPosted: Thu 22 Oct 2015, 09:28    Post subject:  

amigo wrote:


[...]

To use the proper terms and get the whole picture, the first
446 bytes is the executable boot code -the boot sector. The next
64 bytes is the partition table. The next
2 bytes is the 'magic' which signals the end of the MBR. The MBR
is the 512 first bytes of the disc.


is there any way to copy over the boot sector, partition table, and the 2 bytes of magic from one flash drive to the other without destroying the rest of the data already on the second drive? even if it means the machine misidentifies the drive (is one of these areas where the manufacturers store device-specific info like make, model, etc? how does my machine know a kingston drive from a corsair?), will it boot if you could somehow just transfer over the MBR? is there info in the MBR that's looking for a specific OS to boot? is it tied to some sort of identifying code on the drive housing it? and am i correct in thinking the small 16mb partition just points to needed files on the larger one? assuming you could get the drive to boot and read the first partition, would it then read partition 2 or would it balk because what it finds there wasn't the OS originally burned or dd'ed to the drive?

amigo wrote:

[...]
copy/move operations never touch the first (zero) sector as 'cp' and 'mv' work with filesystems. 'dd' or 'cat' can write to the zero sector. The zero sector refers to whole device -not to a partition ( /dev/hda and not /dev/hda1 ). Technically, it shouldn't matter if any of the partitions are mounted during writing to the zero sector -but good-practise would encourage you to not do so.

Using fdisk also writes to the zero sector, but only to the partition table areas -it does not write to the real bootsector. Hence, any partitioning can be done after dd'ing. Usually, one might dd a complete 512 byte MBR which already has a partition table, then use fdisk to alter the partition table.


perhaps i'm getting ahead of myself, but what programs or commands would i need to copy over the mbr from one drive to the other without touching anything else? if that part works, would a simple transfer of the isohybridized system (boot partition 1, system files in partition 2) face any other obstacles preventing the setup from working as well as if one of the methods described earlier had been used directly on the drive?

thanks again, amigo.
Back to top
View user's profile Send private message 
gcmartin

Joined: 14 Oct 2005
Posts: 6730
Location: Earth

PostPosted: Thu 22 Oct 2015, 11:02    Post subject:  

Hello
Quote:
is there any way to copy over the boot sector, partition table, and the 2 bytes of magic from one flash drive to the other without destroying the rest of the data already on the second drive? even if it means the machine misidentifies the drive (is one of these areas where the manufacturers store device-specific info like make, model, etc?
This is an almost impossible dilemma that would result. Physically, YES, you can copy to a 2nd. Couple reasons why it may not give the results you would want:
    Drive geometry(s) are different. Even sometimes where drives/unit have the same model numbers. And, the PAT tables which are valid on the device you copy from, would be invalid for the drive you are copying to.
    And, other info that would reside identifying useful information, for say, fdisk (and others) would be using information that would not be reliable for the unit.
I am sure that there are some programs, out there, which can interrogate the invalid drive and correct that area; where you would need to use one of them, immediately after a copy operation, to "fix" the information for proper use.

Hope this helps, a little.

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engines or use DogPile
Back to top
View user's profile Send private message 
Puppus Dogfellow


Joined: 07 Jan 2013
Posts: 1560
Location: nyc

PostPosted: Thu 22 Oct 2015, 11:18    Post subject:  

gcmartin wrote:
Hello
Quote:
is there any way to copy over the boot sector, partition table, and the 2 bytes of magic from one flash drive to the other without destroying the rest of the data already on the second drive? even if it means the machine misidentifies the drive (is one of these areas where the manufacturers store device-specific info like make, model, etc?
This is an almost impossible dilemma that would result. Physically, YES, you can copy to a 2nd. Couple reasons why it may not give the results you would want:
    Drive geometry(s) are different. Even sometimes where drives/unit have the same model numbers. And, the PAT tables which are valid on the device you copy from, would be invalid for the drive you are copying to.
    And, other info that would reside identifying useful information, for say, fdisk (and others) would be using information that would not be reliable for the unit.
I am sure that there are some programs, out there, which can interrogate the invalid drive and correct that area; where you would need to use one of them, immediately after a copy operation, to "fix" the information for proper use.

Hope this helps, a little.


thanks, gcmartin, it does help. it also anticipates the follow up, which was going to be along the lines of, if the move is possible, what then may need to be edited or corrected and how involved would that be.
Back to top
View user's profile Send private message 
amigo

Joined: 02 Apr 2007
Posts: 2641

PostPosted: Thu 22 Oct 2015, 12:35    Post subject:  

I usually don't say 'copy' when referring to the MBR and rest of sector zero. I say 'write'. Again, 'cp' and 'mv' only work within filesystems. 'writing' to a full device (not a partition) is done directly by using 'dd' or 'cat', and indirectly by using fdisk or other such tools.

Yes, you can use 'dd' to 'write' any segment of the disk. Using the --skip and --seek options you can tell it to start and stop on any byte you want. Other tools like grub or grub-install also write to the MBR area, but only to the first 446 bytes -the real 'bootsector'. fdisk, parted and such onyl write to the partition table -the 64 bytes after skipping the first 446 bytes. The two bytes of 'magic' are really a part of the partiton table -without them the partition table is not complete.

One thing to realize about disk operations using *filesystem* tools like cp, mv and rm, is that when you 'rm' a file, the data does not get removed at all -only the *entry* for the item gets removed from the filesystem 'journal'. The data from a file remains in place.

Remember that the whole filesystem resides on a partition which is defined in the partition table, which is *not* part of the filesystem. The 'structure' of a filesystem begins with certain characteristic bits -which must be placed at the point where the partition table says the partition begins. In other words, if you change your partition table, the data which resides in the old partitions is not touched. If you change the partition table exactly like it was before, the filesystem will still be intact.

However, if after partitioning you *format* the partition, you destroy the old filesystem journal(or allocation table), but note that the original data contained in your files will *still* be untouched. You could still use a rescue tool to 'read' the files -possibly. But, if you actually start creating or copying new files into the newly-formatted filesystem, you will be overwriting the old data areas, at which point the old data is definitely not recoverable.

If you are hoping to automate something here, then you need to read the info in the original partition table, and then reproduce it so that the filesystem(s) can be found. But that makes no sense -simply use dd to selectively write only what is needed.

vis-a-vis mkisohybrid, I am not exactly sure what all it writes -whether it is a complete MBR, or with an empty bootsector (446bytes) with *only* a partition table. If it is complete, then it will have a standard DOS-type MBR which contains the boot code which does the standard MS-ey thing -look for the first bootable partition and start execution there. The thing which makes the iso *hybrid* is that it has both a partition table and an 'El Toro' boot sector(the boot code for an ISO/CD device beginning after the 32K boundary).

The main utility of the above is that you 'recover' any space left over on your flash drive(or whatever), after writing (using dd) a hybridized ISO image to the drive -which contains a partition table entry for the first partition which skips over the content of the ISO. If the ISO is 50MB, then the first partition must be set to at least that size. Any second (or more) partitions have to be placed after that.

fdisk -l will report the real raw disk size and the size of any partitions. mkisohybrid is used for altering *ISO images*, so it has no way of knowing anything about the size of any drive or partition info. It arbitrarily creates a first partition the same size as the ISO image it is being used on -it doesn't know any more than that, by default. It may have options which let you specify partition size or even multiple ones.
Back to top
View user's profile Send private message 
Puppus Dogfellow


Joined: 07 Jan 2013
Posts: 1560
Location: nyc

PostPosted: Thu 22 Oct 2015, 14:50    Post subject:  

amigo wrote:
If you are hoping to automate something here, then you need to read the info in the original partition table, and then reproduce it so that the filesystem(s) can be found. But that makes no sense -simply use dd to selectively write only what is needed.


i'm hoping to automate or simplify the swapping of puppies or quirkies onto pre-formatted and partitioned flash drives. i'm primarily interested in using f2fs for the main system, but it would be nice to be able to perhaps use some of the drive for other formats. all the tools seem to wipe the drives clean before installing the the OS, so i guess what i'm after is something a little more precise and a little less destructive. here's a scenario. the user has or formats an sd card, flash drive, or ssd in a way that allows the use of not only a primary, f2fs partition (and it's requisite mini-partition, since as it stands, booting from f2fs appears not yet doable), but also (the possibility of) others--ext2/3/4 to be able to boot multiple pups, and/or maybe a swap partition. now, if done correctly in gparted, the drive should have all the necessary mbr bits still in tact after the formatting, but nothing has yet been done to make it into a bootable drive (though the copied directories from the new OS (partition 2 in example above) may be on a/the f2fs partition). how involved would it be to create a correct mbr or change the existing mbr (or parts thereof) on the target drive without touching anything else except perhaps that one mini-partition (if those files also haven't been transferred to the new drive)? even if the user has to manually transfer files over to the f2fs partition and/or the vfat partition, this type of set up would allow fairly easy swaps to other quirkies or puppies--you'd only have to hybridize an iso or img file (is this what quirkies are? can one merely decompress a quirky.xz for the needed files?) somewhere and then move the files over.

how completely off is all this/thanks for your patience.

Smile
Back to top
View user's profile Send private message 
amigo

Joined: 02 Apr 2007
Posts: 2641

PostPosted: Fri 23 Oct 2015, 03:44    Post subject:  

dd if=some.img of=somedrive-or-image bs=1 count=446
writes only the bootsector
dd if=some.img of=some-image-of-partiton-table bs=1 seek=446 count=16
will write only the partition table

You'll need to study the use of dd -especially seek and skip options

The utility of your idea is doubtful. If the person has already partitioned the drive and (worse) already placed files there, then you will have a hard time preserving their files -if you simply dd a hybrid image, then that will blow away their partition table and hence their files.

If you simply want a way to boot various systems from the same drive, then the best way is to include code in the initrd which finds and boot the system -or even better use menu entries for the bootloader since that lets you use various ḱernels as well.
Back to top
View user's profile Send private message 
L18L

Joined: 19 Jun 2010
Posts: 3431
Location: www.eussenheim.de/

PostPosted: Fri 23 Oct 2015, 07:08    Post subject:  

amigo wrote:
If you simply want a way to boot various systems from the same drive, then the best way is to include code in the initrd which finds and boot the system -or even better use menu entries for the bootloader since that lets you use various ḱernels as well.

If you simply want a way to boot various systems from the same drive, then another (simpler) way is rcrsn51's isobooter

Sorry for getting OT
Back to top
View user's profile Send private message 
Puppus Dogfellow


Joined: 07 Jan 2013
Posts: 1560
Location: nyc

PostPosted: Fri 23 Oct 2015, 09:47    Post subject:  

amigo wrote:
dd if=some.img of=somedrive-or-image bs=1 count=446
writes only the bootsector
dd if=some.img of=some-image-of-partiton-table bs=1 seek=446 count=16
will write only the partition table

You'll need to study the use of dd -especially seek and skip options

The utility of your idea is doubtful. If the person has already partitioned the drive and (worse) already placed files there, then you will have a hard time preserving their files -if you simply dd a hybrid image, then that will blow away their partition table and hence their files.

If you simply want a way to boot various systems from the same drive, then the best way is to include code in the initrd which finds and boot the system -or even better use menu entries for the bootloader since that lets you use various ḱernels as well.


it may not be particularly practical or feasible, or maybe even possible, but these obstacles aside, i think there's some utility in being able to drag and drop operating systems in order to install them onto flash drives. i of course realize the skill set it would take to realize this is leagues away from my own. i do appreciate your feedback and guidance, though. what i was hoping to do was bypass dd if at all possible. if the user has a dummy drive set up for just the purpose of getting the files for a hybridized image, then those could be copied to another location for later use and the dummy drive (maybe one of those old 2gb or 4gb dealies that sometimes turn up) reused to get another set of hybrid iso files for yet another drive or future transfer. the end goal would be the fastest way to put a new operating system on a drive, even if that drive has preexisting files on it. i'm pretty new to all this, so i'm not sure how off the wall it is--i just know what i've seen other scripts do. something has to be able to read the drive data and the OS data and output this information--could a file or gui be made (again, i'm not requesting anybody go out of their way for this--i just want to know if it could be done) that, given (either by way of input field or having had the drive and/or OS file dragged onto a script) the correct and necessary info, can automate or simplify the initrd editing necessary for the drive to boot and operate correctly?

again, thanks for your time.

L18L wrote:
L18L
PostPosted: Today, at 07:08 Post subject:

Quote:
amigo wrote:
If you simply want a way to boot various systems from the same drive, then the best way is to include code in the initrd which finds and boot the system -or even better use menu entries for the bootloader since that lets you use various ḱernels as well.


If you simply want a way to boot various systems from the same drive, then another (simpler) way is rcrsn51's isobooter

Sorry for getting OT


thanks for the link, L18L.

also, sorry for hijacking your thread ETP Embarassed


getting back to the original topic, what would be the method for hybridizing an iso file and then creating a bootable f2fs installation of it on a flash drive?

thanks.
Back to top
View user's profile Send private message 
slavvo67

Joined: 12 Oct 2012
Posts: 1538
Location: The other Mr. 305

PostPosted: Fri 23 Oct 2015, 23:31    Post subject:  

The following thread by Bigpup discusses a very good Unetbootin option.

http://murga-linux.com/puppy/viewtopic.php?t=99826

This is an install to USB but Bigpup also references a way to save back to the same USB via the Pupsave file.

This is another USB option but it is certainly not being mentioned to take away from this hybrid option, which worked easily and was excellent.

Best,

Slavvo67
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 3 of 3 [44 Posts]   Goto page: Previous 1, 2, 3
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » HOWTO ( Solutions )
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0938s ][ Queries: 14 (0.0134s) ][ GZIP on ]