afo - Aggressive file obliterator

Filemanagers, partitioning tools, etc.
Message
Author
jafadmin
Posts: 1249
Joined: Thu 19 Mar 2009, 15:10

afo - Aggressive file obliterator

#1 Post by jafadmin »

Code change (Mar-23): Massive speed increase. Re-download afo & bcrypt.
Source updated as well...


This utility installs for use via Roxfiler as well as the command line.

It overwrites files/directories with random gibberish then deletes them. Files deleted with this utility won't be recovered using undelete functions, recovery software, or dd/hexedit.

It permanently obliterates your files to the point they are not recoverable.

Installation adds a right-click item in Roxfiler under the "Open With" menu. Use Rox to select files and folders, right-click, select "Open With", then click on "Afo".

You can specify the number of overwrites on the command line. The default is 1 pass, which should suffice. If you wear your tinfoil a little tighter than I do you can specify any number of overwrite passes. That default parameter for rox can be set by editing the /usr/local/apps/Afo/AppRun file and changing the -c1 to your favorite number. i.e.: -c3

This should run on all versions of puppy. If problems, let me know.

-jafadmin
Attachments
afo-source.tar.gz
(4.84 KiB) Downloaded 445 times
Afo.pet
(6.19 KiB) Downloaded 513 times
Last edited by jafadmin on Thu 30 Jun 2016, 13:48, edited 8 times in total.

jafadmin
Posts: 1249
Joined: Thu 19 Mar 2009, 15:10

#2 Post by jafadmin »

One of the problems with these kinds of utilities is it's sometimes hard to verify that they actually function as advertised.

I ran into the problem originally with the BCrypt utility that is included with all installs of Puppy Linux. I was trying to verify that the "-sN" switch, which tells bcrypt how many times to overwrite the original file with random garbage before deleting, was working properly.

So here at the Jafadmin Furious Linux Research Grotto & Pasta Bistro™ I set up a lab to do just that. The results of that lab were downright terrifying.

I used dd, hexedit and gparted to do the experimentation. First I created a 1M ext2 partition on a spare thumb drive with gparted. Next I used dd to overwrite the partition with zeros, then reformatted it. I created a text file filled with "Lorem ipsum" fill text on the partition, then encrypted/compressed it with bcrypt using the -s2 switch which should overwrite it twice with random gibberish before deleting. Bcrypt created the .bfe file and deleted the original.

Next I used dd to create an image file of the 1M partition and opened the image file with Hexedit. I searched for the string "Lorem", and found the whole entire text of the file.
The file simply WAS NOT getting overwritten at all!

I located the source code for bcrypt on sourceforge and proceeded to investigate. I found two disturbing things in the "rwfile.c" file; 1) The original algorithm for overwriting the files would only overwrite the first 25% of the file. And, 2) Once that was fixed it still didn't work.

What the heck was going on here? If I changed the code so that it just didn't unlink (delete) the file after overwriting, it worked just fine. I wound up with an original file that was filled with random gibberish.

Here's the deal: The program was using high level I/O functions to access the file. Specifically the fopen(), fread(), fwrite() family of file I/O functions. Why was this bad? Because all modern Linux OS's optimize their file I/O with buffering and prioritizing algorithms. The OS processes the "unlink()" command first, and discards pending write requests for that file.

The fix? Low level I/O functions. I rewrote the file clobbering function using the open(), read(), write(), and fsync() functions that bypass OS buffering and work directly on the disk.

So what's my point with all of this? After fixing my version of bcrypt, I wrote "afo" so I had a tool that could do secure deletes on a Directory/File level instead of having to wipe whole disks/partitions.

A Word To The Wise: None of these following things got rid of the original file data.
1) Reformatting failed.
2) Reformatting to a different filesystem failed
3) Deleting the partition failed.
4) Deleting the partition table failed

The only thing that worked was overwriting the partition with dd

Code: Select all

dd bs=1M if=/dev/zero of=/dev/"partition descriptor", or
dd bs=1M if=/dev/random of=/dev/"partition descriptor" 
Here is the original bcrypt source with my documented code changes as well as the original source code. It also contains a new compiled bcrypt binary with the bug fixes:
Attachments
bcrypt-1.1.(jafa).tar.gz
fixed bcrypt
(51.5 KiB) Downloaded 478 times
Last edited by jafadmin on Fri 03 Apr 2015, 01:49, edited 4 times in total.

seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

#3 Post by seaside »

Jafadmin,

File deletion utilities are very close to backup utilities. Very rarely does anyone really test them!

Thank you for actually making this test on bcrypt. It's quite revealing to learn that it doesn't behave as anyone thought.

I'm sure your utility Afo works nicely, and the inclusion of the source code for the Afo binary would make it even more comforting.

Cheers, and thanks again,
s

jafadmin
Posts: 1249
Joined: Thu 19 Mar 2009, 15:10

#4 Post by jafadmin »

seaside wrote:
I'm sure your utility Afo works nicely, and the inclusion of the source code for the Afo binary would make it even more comforting.

Cheers, and thanks again,
s
Agreed. Done.

User avatar
Burn_IT
Posts: 3650
Joined: Sat 12 Aug 2006, 19:25
Location: Tamworth UK

#5 Post by Burn_IT »

A Word To The Wise: None of these following things got rid of the original file data.
1) Reformatting failed.
2) Reformatting to a different filesystem failed
3) Deleting the partition failed.
4) Deleting the partition table failed

The only thing that worked was overwriting the partition with dd
That is perfectly correct and to be expected.The only one of those that might actually write to the data portion of the file rather than just the "table entries" is the reformat, and even then only a FULL format and it only marks the cluster as formatted and doesn't remove prior data.

Even with data overwrite on magnetic disks it is possible (though expensive) to retrieve prior data after a few passes. That is why DOD standards say seven overwrites with different random data each time.
It is also very difficult to "clean" SSD type storage media because of the randomising. The only sure way is to delete the whole disk and force full garbage collection.
"Just think of it as leaving early to avoid the rush" - T Pratchett

jafadmin
Posts: 1249
Joined: Thu 19 Mar 2009, 15:10

#6 Post by jafadmin »

Burn_IT wrote: Even with data overwrite on magnetic disks it is possible (though expensive) to retrieve prior data after a few passes. That is why DOD standards say seven overwrites with different random data each time.
It is also very difficult to "clean" SSD type storage media because of the randomising. The only sure way is to delete the whole disk and force full garbage collection.
I have an associate that is a DOJ certified Forensic IT Specialist. These guys have really expensive hardware and software designed to do just exactly what you said.

According to this individual, the assertion that data can be retrieved after a disk has been overwritten, even with just one pass of zeros, is largely just a theory.

There aren't any actual proofs of this. Anywhere. Think about that. Someone would have published by now.

User avatar
Burn_IT
Posts: 3650
Joined: Sat 12 Aug 2006, 19:25
Location: Tamworth UK

#7 Post by Burn_IT »

That is not entirely true. I have some software that will recover data from a disk that has been overwritten with a simple static pattern.
I have done it. It took some time to rebuild the original file, mainly because there were earlier generations there as well.

What I didn't know at the time was that the person who asked me to do it was employed at the MOD and was testing potential suppliers. They were thinking of buying a product I was selling and were doing security checks.
"Just think of it as leaving early to avoid the rush" - T Pratchett

jafadmin
Posts: 1249
Joined: Thu 19 Mar 2009, 15:10

#8 Post by jafadmin »

It would be fascinating see how software by itself would accomplish that.

Without using a different type of read/write head (hardware) on the disk, I have difficulty understanding how software alone could reliably read a different value from a given disk location.

While an abundance of caution is essential for safe computing these days, I think it's incumbent for we technologists to differentiate in our own minds between theory and reality.

User avatar
Burn_IT
Posts: 3650
Joined: Sat 12 Aug 2006, 19:25
Location: Tamworth UK

#9 Post by Burn_IT »

I used http://www.lc-tech.com/pc/filerecovery/

It was some years ago and was a pro version.
"Just think of it as leaving early to avoid the rush" - T Pratchett

jafadmin
Posts: 1249
Joined: Thu 19 Mar 2009, 15:10

#10 Post by jafadmin »

Burn_IT wrote:I used http://www.lc-tech.com/pc/filerecovery/

It was some years ago and was a pro version.
Yeah, that's pretty standard file recovery stuff. You can do the same thing with dd and a hex editor as I pointed out above. That kind of software automates file reassembly though, which is nice.

That type of software assumes the disk hasn't been overwritten, though. Using either of the dd commands above will render the software useless.

Give it a try some time. This is why lab work is fun.

[edit: 3-31-2015] I published this here for purposes of peer review. I fully expect technical people here to try this on their own then test/publish their independent results.

jafadmin
Posts: 1249
Joined: Thu 19 Mar 2009, 15:10

#11 Post by jafadmin »

For those who use the FolderEnc-1.0 utility to encrypt folders with encfs in Rox-Filer, the following can be added to /usr/local/apps/rox-encfs/encrypt.sh to securely delete the originals after encryption.

Replace step #9:

Code: Select all

# 9. remove the original unencrypted files (from TMP_DIR)
rm -rf "${TMP_DIR}" 
with:

Code: Select all

# 9. remove the original unencrypted files (from TMP_DIR) - jafa mod to use afo
if Xdialog --title "Use afo?" --yesno "Use Agressive File Obliterator to delete originals?" 0 0 ; then
	afo "${TMP_DIR}"  # Yes
else
	rm -rf "${TMP_DIR}"  # No
fi
This will produce a dialog offering to use afo to delete originals. Choosing "no" uses regular delete methods.

.

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#12 Post by jamesbond »

jafadmin, thanks. Afo is now built-in in next release Ftdog64, as well as the option to use afo for rox-encfs. I have also patched bcrypt with your changes above, though I just found out out scrypt and it seems/claims to be more secure than bcrypt.
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#13 Post by Sylvander »

Would it be possible to have afo include in the ROX right-click-context-menu...
An item that says something like "Erase selected item using afo".

It isn't very self-evident that the thing to do is use "Open With->afo" to "Erase" an item.
By the time I get around to needing to use it, I'm likely to have forgotten the name of the program that erases stuff, and HOW.
The WHAT&HOW needs to be obvious.

I remember doing this kind of thing years back in the Windows registry.

User avatar
fabrice_035
Posts: 765
Joined: Mon 28 Apr 2014, 17:54
Location: Bretagne / France

#14 Post by fabrice_035 »

An idea, you can edit 'trash' function, in /usr/local/apps/Trash and add GtkDialog GUI to choose secure delete or normal.

Not work when you choose "delete" in rox menu, files or dirs are never sent to the trash, user must move manually (with drag and drop)

Curiously rox menu send to the trash is only offered when you choose *one* file/dir, not visible with multiple select... :?:

User avatar
fabrice_035
Posts: 765
Joined: Mon 28 Apr 2014, 17:54
Location: Bretagne / France

#15 Post by fabrice_035 »

hi,

afo is very good, i wrote a little interface.


Replace "AppRun" in /usr/local/apps/Afo by this script (don't forget make chmod +x after replace)

I think it's less dangerous than send file/dir directly to afo without question :?

This script run without log containing filename or dir to be destroy.

regard.
Attachments
AppRun.gz
(1.72 KiB) Downloaded 339 times
gtk_for_afo.png
(32.02 KiB) Downloaded 446 times

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#16 Post by Sylvander »

Could you explain:
make chmod +x

What should be done, and why?
What is the +x about?

User avatar
fabrice_035
Posts: 765
Joined: Mon 28 Apr 2014, 17:54
Location: Bretagne / France

#17 Post by fabrice_035 »

Make AppRun executable, in console.

Code: Select all

chmod +x /usr/local/apps/Afo/AppRun
Other solution, you can right click on file, select property and select execute.
Attachments
show_execute.png_2016-03-06_220535.png
(25.18 KiB) Downloaded 250 times

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#18 Post by Sylvander »

OK...

Used 2nd method.
It works. :D

jafadmin
Posts: 1249
Joined: Thu 19 Mar 2009, 15:10

#19 Post by jafadmin »

@fabrice_035 ,

That app is too slick for words. 8)

april

#20 Post by april »

@jafadmin
Wow that was close
d/l instal and try but silly me ,try on a directory thinking it overwrites deleted data only.

Fortunately it did not operate on the directory and my data was safe.

Perhaps you could make that clear? It deletes and obliterates real live files , not deleted files .

Very handy though and appreciate the knowledge and effort that went in and the knowledge it imparts to me as well .

Post Reply