Convert any ISO into a hybrid & use dd to place any

[ETP]

An easy way of checking an ISO is to right click it then select properties.

Thanks.

An other easy way I just have found is using command line:

Code: Select all

file *.iso

This is working at least in Fatdog where also "--verbose" works.

Hi L18L,
Nice find. For clarity here is a shot of the expected outputs:

[gcmartin]

As @GoingNuts and I have shown, is there some method that would be needed so that the hybrid can be produced? And, if the problem is always the same for certain PUP's ISOs and a solution is consistent, can a preprocessor method be scripted with the command to convert the ISO into the PET? Maybe this kind of simple preprocessor could detect hybrid and abandon too.

just thoughts...

[Jasper]

Hi ETP,

It seems to me:

For those with a working optical drive, it is easier to use that.

Using a single dedicated stick or card, multiple distros are not possible.

Please comment and correct as necessary.

My regards

[slavvo67]

I am ecstatic with these simple scripts. It's just that save file issue that's hindering me. I guess a workaround would be to remaster an ISO with changes prior to using dd for transfer to USB.

[gcmartin]

Hi @Slavvo67 and @ETP. I think one idea gotten from your posts, combined, is that the hybrid ISO could be used to produce a USB that has the ISO's boot needs in one area, as well as, leaving a writable area for session-save needs. I would imagine that a writable area formatted to a Linux filesystem would allow the running distro to create a save folder/file to the area. One obvious requirement for this multi-partition outcome to occur is that the ISO would need to produce an .IMG file that can be restructure into a new multipartition .IMG file to be written to the USB stick. But, I can see where this may be outside of the scope of this hybrid to USB process.

And, in my current attempts, I tried to use @Smokey01/@CatDude MultiPUP ISO output as input to this process. But, the error I encountered (shown in an earlier post) trying to create the hybrid ISO from a MultiPUP ISO does not allow an OOTB solution.

These are just some ideas.

A comment
Number of writes to a modern USB stick suggest that data integrity remains now, after years and years of use. Some of the early early problems of integrity as use increases, have been resolved. Recent USB Sticks can be written to for years of operation and data integrity remains; just as is found with HDDs.

Hope this info is helpful
Edited: The 1st paragraph.

[slavvo67]

Hi GCMartin:

I don't believe that 's 100% accurate. I think the ISOHybrid format with the dd instruction on page one creates a once only write (unless you use dd back over it) to the USB drive; not allowing for a save file on the same USB. ETP advised the .img format seems to allow for different partitions and, therefore a safe file on the same stick.

Not sure if there's a way to use the "dd" script while carving a second partition for save. That would be the most sensible, as it seems wasteful to use one USB for booting the puppy and another USB just for the save file. I'm not sure if it's possible, however, based on ETP's direction.

For me, the "dd" scripts seem to be the most simplistic way to get your iso on a USB. I've confirmed this process works with Fluxpup, Tahrpup and 3 Headed-Dog so as I believe ETP has stated, it seems to work with all puppies.

Note: I wouldn't use a big USB when using dd, as you'll waste a bunch of space. I would suggest an 8gb stick, maybe even smaller would work.

[goingnuts]

I've confirmed this process works with Fluxpup, Tahrpup and 3 Headed-Dog so as I believe ETP has stated, it seems to work with all puppies.

I do have the earlier quoted problems with isohybrid and fx. puppy-4.1.2-k2.6.25.16-seamonkey.iso
Mounting the iso and copying isolinux.bin out of it - and running

Code: Select all

#strings isolinux.bin | grep ISOLINUX
ISOLINUX 3.63 2008-04-10

indicate that older puppies might not be converted without a remaster of the iso after updating isolinux.bin to a newer version.

[greengeek]

Not sure if there's a way to use the "dd" script while carving a second partition for save. .

I would be interested to know if a second partition is tolerated. I would like to use this method to boot a standard generic puppy (without savefile) then pull a "personalisations" .pet from a second partition and install it to apply necessary personalisations in an effectively readonly manner. (ie the session would have personalisations but not remember todays activities or corruptions...)

[amigo]

OK, let me explain a bit. What the hyrbrid tool does is create a partition table at the beginning of the iso image. CD images are blank for the first 32K (IIRC), so the tool has plenty of room to insert a boot sector. Once done, if the image is burned to a CD, then when the bootloader looks at the drive it sees a normal bootable CD image -since it is a CD drive, the bootloader starts looking at the 32K boundary. However, if the same image is burned to a mass-storage device, then the bootloader will first see the boot sector and partition table.

To repeat, the bootloader will use different criteria to decide if a device is bootable, depending on what type of device it is. The process of creating the hybrid image entails making the image appear to be several things at once. The makebootfat tool does something similar by making an image or RW device appear to be two or three things at once -namely a normal FD floppy drive, a bootable ZIP device, or HD hard drive -and if you don't know it already, a CD image presents itself as a large floppy (2.88K).

Now, since using dd writes directly to the device or image, including writing the boot sector if included in the image. mkisohybrid will create a partition table -it doesn't know the details of the actual drive- so it creates a single partition of a size which will encompass the image.

**After that** you can run fdisk on the image in order to add new partitions or change the size of the first one. You can also do this after dd-ing the image to the drive(USB or whatever). Of course, you'll need to leave the first partition big enough to hold the entire original image, but otherwise you are then free to add as many partitions you like, of whatever size.

As always, the difficulty with implementing a tool which automates this process is in being able to do it safely. What I mean is that you may need/want user input before arbitrarily overwriting a drive -you surely don't want to destroy user data without asking first, right?

[ETP]

Code: Select all

Example 1 - Place 4MB Hybird ISO onto 16GB SD card
(Neither the drive or the iso are mounted)

# dd if=/root/Downloads/sda6-hybrid-boot-rapid-pup-703.iso of=/dev/sde
8192+0 records in
8192+0 records out
4194304 bytes (4.2 MB) copied, 1.12173 s, 3.7 MB/s
#

Remove stick then re-insert.
Hover over sde1 drive icon & now shows as "Filesystem: iso 9660 Size 4MB"
GParted does not recognise it as it is now an ISO and gives a recursive partition error.
I am not aware of any way of making use of the unused space on the card. It is wasted.

Example 2 - Place 245 MB Hybrid ISO using same card still with 4MB iso on it.

# dd if=/root/Downloads/chromebook-pupv1.iso of=/dev/sde
501760+0 records in
501760+0 records out
256901120 bytes (257 MB) copied, 74.5973 s, 3.4 MB/s
#

Remove stick then re-insert.
Hover over sde1 drive icon & now shows as "Filesystem: iso 9660 Size 245MB"
Partview correctly shows size as 245M with 0M free.
Tested bootability on another PC then shut down without saving.

Isohybrid is if you like, a trick that fools a PCs bios into booting a treated bootable CD image (An iso) into
into booting as a CD-Rom when placed on a card or stick. To all intents and purposes it is then behaves in the same
manner as a CD. You cannot create extra partitions on a CD and currently AFAIK there is no way of utilising any
free space on a card or stick as you will destroy the bootability trick in so doing.

A hybrid ISO is very convenient for both users of burners and those people without who can quickly place such an
image on a card or stick.
The ISOHYBRID command has been around for several years & many mainstream distros such as Debian, Ubuntu Arch & Mint in recent
years haved issued their ISOs as hybrids.

The ideal time to create a hybid iso is at the point of production rather than later when problems can arise due to a mismatch
between the versions of syslinux used to create isolinux.bin (the bootloader) and isohybrid which should contain the same MBR code.
goingnuts wrote:

older puppies might not be converted without a remaster of the iso after updating isolinux.bin to a newer version.

I agree with his comment and that may be needed with some later ones post syslinux 3.72

@ amigo
Having written the above I saw your post about using fdisk to make use of any free space.
Could you detail that using chromebook-pupv1.iso as a real world example?

[amigo]

Simply run fdisk on the iso image to view or create a partition table. How big is the image? Round that figure up to a convenient partition size and make the first partition big enough to hold the complete image. Leave it unformatted as it is already 'formatted' as an iso. Create any other partitions you want in the remaining space on the drive. Then format them however you want.

A more geeky solution would have you use dd to write directly to the drive *after* the space occupied by the iso image (using dd's 'seek' option). And you could then associate the data at the point where you dd'ed it to the loop-back device using losetup's 'offset' option. -Better use option above.

[L18L]

Code: Select all

-
 am not aware of any way of making use of the unused space on the card. It is wasted.
-

New UEFI flash drive installetion method

[slavvo67]

Amigo,

It is possible to run fdisk on an uninstalled .iso file? That seems odd. I tried fdisk after dd installing the iso and it didn't work.

There was a better way but I lost the instructions to do it. I think it was creating a small fat32 with a larger f2fs and using grub4dos to boot the fat32 partition. I think it's in this darn forum somewhere but easier to find a needle in a haystack.....

[gcmartin]

Code: Select all

root# fdisk -l /mnt/PC-on-204/PUPs/TahrPUP/tahr64-6.0.3.8.iso

Disk /mnt/PC-on-204/PUPs/TahrPUP/tahr64-6.0.3.8.iso: 234 MB, 234881024 bytes
64 heads, 32 sectors/track, 224 cylinders, total 458752 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x3839bfd0

                                         Device Boot      Start         End      Blocks   Id  System
/mnt/PC-on-204/PUPs/TahrPUP/tahr64-6.0.3.8.iso1   *           0      458751      229376   83  Linux

Not sure if this is what you ask.

[slavvo67]

@GCMartin:

Kind of but adding a partition to an .iso?

[jamesbond]

Please read this then get the script for your review. It's for fatdog so it's not directly applicable (fatdog isohybrid has 2 partitions instead of one), but the idea is the same.

[Puppus Dogfellow]

is it possible to add files--whether from a decompressed quirky.xz or puppy.iso--to a drive that's already been formatted as two partitions, one a 16mb vfat and the the other the balance of the drive as f2fs? (basically what Barry Kauler's quirky installers do to flashdrives, unless that's changed for the most recent quirky). it'd be nice to be able to keep the data on the large partition and alter files on it and the boot partition as necessary to accommodate different or additional puppies or quirkies (multiboot would be nice, but i'd be happy to just reuse a drive without having to entirely wipe it first). instead of expanding out to an entire partition, can this method be used to force a save folder on that partition? could the conversion to isohybrid be done in a folder and then the contents moved?

thanks for the guidance/patience (i'm not exactly sure how idiotic these questions are, but i feel they may be significantly so).

[amigo]

Of course you can add, change or delete files from the partitions, they just neeed to be mounted first.

You cant 'hybridize' a folder. The process writes data to the first sector of an iso image(a file).

[Puppus Dogfellow]

Of course you can add, change or delete files from the partitions, they just neeed to be mounted first.

You cant 'hybridize' a folder. The process writes data to the first sector of an iso image(a file).

thanks for the reply, amigo.

just to be clear, how accurate is the following?

starting with two drives formatted ETP's way or by way of 01micko's f2fs installer or BarrkyK's quirky installer (i apologize if i'm mistakenly clumping these tools together--superficially at least, they seem to do more or less the same thing to the target flashdrive), i use one for a period and decide i want to swap in a different OS. i move all the original 2nd partition files to and archive folder, delete the contents of the boot partition, copy over the files from the second, untouched drive, and drive 1 will boot with the new OS and the archive folder (remnants of personal and system files of the previous installation on the drive) accessible?

if so, that seems extraordinarily convenient.

_____


here's how quirky installs to two partitions:

Code: Select all

16MB partition 1:
boot.msg  ldlinux.sys  vmlinuz
help.msg  logo.16     syslinux.cfg

Code: Select all

27 GB partition 2:
audit  dev   lib       mnt   root   sys  var
bin    etc   lib64     opt   run    tmp
boot   file  man2html  proc  sbin    usr

[this second batch plus any personal files/folders the user creates on the partition would be the ones i'd want to archive.]

how hard-wired (for lack of a more accurate term--accuracy about to plummet further) or unforgiving is the code identifying the drive and tying it to the OS it's running? if this won't work as conjectured, how involved would a manual fix be? this method doesn't require any changes to the menu.list, so it seems like it could be potentially a very newbie friendly installation method. there'd be one fewer thing to mess with, and gparted could allow for very specific formatting of the target drives.

[amigo]

Whoa, partner. "delete the contents of the boot partition, copy over the files" This doesn't restore the boot sector. The boot sector is not a part of the file system -it precedes the first sectors used by any filesystem. You still need to use 'dd' on the first sector of the drive or image to write any new bootsector.

To use the proper terms and get the whole picture, the first
446 bytes is the executable boot code -the boot sector. The next
64 bytes is the partition table. The next
2 bytes is the 'magic' which signals the end of the MBR. The MBR
is the 512 first bytes of the disc.

A bootable CD or iso image do not have an MBR. There is a significant empty region at the start of the disk -32K IIRC. bootloader code 'knows' to look at this location when booting from a CD device. mkisohybrid takes advantage of the fact that both bootable CD code and MBR can exist on the same image/device without 'colliding'.

filesystems do not usually start in the first (0 zero) sector. They start in the second (1 one) sector.

If you create extended partitions, then the last bits in the partition table will point to the spot where the extended partition data is located -this is also somewhere in the first sector.

bootloaders also can use the space left over in the zero sector.

copy/move operations never touch the first (zero) sector as 'cp' and 'mv' work with filesystems. 'dd' or 'cat' can write to the zero sector. The zero sector refers to whole device -not to a partition ( /dev/hda and not /dev/hda1 ). Technically, it shouldn't matter if any of the partitions are mounted during writing to the zero sector -but good-practise would encourage you to not do so.

Using fdisk also writes to the zero sector, but only to the partition table areas -it does not write to the real bootsector. Hence, any partitioning can be done after dd'ing. Usually, one might dd a complete 512 byte MBR which already has a partition table, then use fdisk to alter the partition table.