(old)Puppy Linux Discussion Forum

Update, Saturday, Dec. 20, 2014
Please see second message on page 2 for csvfix's new companion, CSVpad-1.2.
Although an ok program, dmscsv is not as powerful. Thanks.
~~~~~~
Update, Friday, Dec. 19, 2014
Please see on top of page 2 for csvfix-1.60. Thanks.
~~~~~~
Update, Sat., Jan. 12, 2013.

Hello.

A year almost to the day, here is csvfix 1.50. The pet archive for this version is
basically structured the same, except that csvfix and the docs have been updated to
the latest version, v. 1.50.

You can see what has changed here: http://code.google.com/p/csvfix/wiki/ChangeLog.
In particular, the new command may come in handy in
some situations.

The complete list of csvfix commands is quoted 5-6 messages below.

Please download this 1.50 version of csvfix from here : http://limelinx.com/chc90.
(EDIT: now a dead link. Please see at top of page 2 for a valid link to a newer version.)

Enjoy!

musher0
//////////////////////////
Hello, everyone.

I just thought I'd share with you that there's a new multi-platorm csv processor out
there: CSVfix 1.20. It was released under a MIT licence late last year.

I've compiled it from source and here it is. gposil already presented a DBF type
database in a personal base thread. As well, there are a few SQL incarnations on
this board, so this good csv processor is just the thing to round out Puppy's
databasing capacity.

I've started using it, toying with a copy of the icewm "menu" present on all
Puppies-- to make it into a real csv file, among other things. I have very little
experience of this program yet, but it seems to be doing what you want it to do.

About the attached pet: in order to save space, the executable has been upx'd (still
works great; upx'ing does not affect performance), and the docs have been
archived in 7z format. You'll have to unpack these html docs to read them. Once
you've unpacked them, you'll find a convenient link to the Introduction in ~/my-
documents/Docs-Linux.

Finally, also included is a simple script I made base on "less" to help you view *.txt
and *.csv files. The script in ~/my-applications/bin, and you might want to link it in
~/.config/rox.sourceforge.net/OpenWith/.text_plain and
~/.config/rox.sourceforge.net/OpenWith/.text_csv

That way, you can view or follow what's happening to your csv file as csvfix
processes it, almost in real time, so to speak.

Although csvfix is not a "window"-type application, the pet contains a loader (edit)
which reminds you what command to type to launch csvfixin the terminal. As you
can realize, it might difficult to make a GUI for csvfix,given that it has so many
functions and parameters. That said, used in conjunction with less (above) and
dmcsv (see a couple of posts down), you get a very powerful tool to process csv
(aka flat-file) databases.

Finally, reading the doc is a must if you want to make the most out of this program.

Hopefully all this will be of use to you.

Enjoy!

Hello again.

The following will speak volumes, to whoever got headaches trying to
successfully process a csv database, about the usefulness of this
program. It also describes what csvfix command to use to solve a
particular problem. From the docs.

TWYL.

musher0

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Solutions to Common Problems
Introduction ››

Here are some of the common problems that you may experience when
dealing with CSV data, and brief suggestions as to how CSVfix can help
you with them:

"The fields in the data are in the wrong order"
* Reorder them using the order command.

"The records in the data are in the wrong order"
* Sort them using the sort command. Remove duplicates with the unique
command.

"Some fields are missing from my data"
* If you simply need to add empty fields to keep your application happy,
use the pad command. If you need to enrich your data with values from
another file, use the join command. If you need to add a fixed string to
the data, use the put command.

"My CSV data isn't comma-separated!"
* Use the -sep and -rsep flags to tell CSVfix what separator to use.

"I need to read tab-separated data"
* Use the read_dsv command with the -s '\t' flag to convert tab-separated
data to CSV.

"I need to convert XML to CSV"
* The from_xml command probably does what you need.

"I need to perform calculations on some of the fields in my CSV data"
* The eval command provides arithmetic and other functions you can
apply to your CSV data.

"I need to extract data from the middle of a field"
* Use the split_fixed and split_char commands to break fields up into sub
fields. If you need to build a field from values in other fields, use the
merge command.

"I need to remove some lines from a CSV file before processing it further"
* The find and remove commands allow you to filter your CSV files using
regular expressions, value ranges and field lengths. You can also create
exclusion lists and use the join command with the -inv flag to exclude
selected rows.

"I need to merge the data from two CSV files"
* Use the order command to get the fields in the same order, and then
the unique command to merge them, discarding duplicates.

"I want to add record numbers to my CSV data"
* The sequence command provides flexible record numbering.

"I want to split my data into different files"
* The file_split command can split CSV data streams depending on field
values.

"My data is full of duplicate values"
* Remove duplicates using the unique command.

"The data I get given is always full of invalid values"
* Validate your data using the validate command. Remove bad values
with the remove command.

"I need to convert CSV data into XML"
* The to_xml command does exactly this.

"I need to convert CSV data to this weird format"
* Use write_fixed, write_dsv and the printf and template commands to
format your output.

"I want to extract data from a SQL database as CSV"
* In the Windows version of CSVfix, the odbc_get command allows you to
extract data from SQL databases.

"I need to import data into a database, but the data is in a weird format"
* Use CSVfix to convert the data to CSV using the read_fixed, read_dsv,
and/ot read_multi commands, and then convert the CSV to SQL INSERT
statements using sql_insert.

"I need special date formatting"
* Use the date_format and date_iso comnands to read and re-format
dates.

Copyright © 2011 Neil Butterworth

~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Thanks @Musher0

I seem to remember few years ago a similar Windows Program for creating and processing CSVs. This was very useful for prepping data for DB entry.

Does CSVFIX has a front-ent gui for processing CSVs or is it a CLI only tool.

Thanks in advance

Update, Saturday, Dec. 20, 2014
Please see second message on page 2 for csvfix's new companion, CSVpad-1.2.
Although an ok program, dmscsv is not as powerful. Thanks.
~~~~~~~~~~~~~~~~~

Hi, gcmartin.

No csvfix is only CLI. But I "intuited" your question, as I came back to
upload dmcsv.

dmcsv is a simple csv viewer -- don't expect CSVed for Linux, here --, but
still it will help you view the results of your processing with csvfix -- if
you're unsure in your mind of what the process will do. It is less abstract
of course than viewing your modified csv file with less -- which is fine, too.

In any case, enjoy! And have a great day!

P.S. Here is a very short description of dmcsv, as given by the author:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
DMcsvEditor is simple csv/tab files editor
darhmedia.hu [where you'll find the archive of the file]
darhmedia@gmail.com
goo.gl/77BAE
*** this version developing on Trisquel GNU/Linux ***
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

And a screenshot.

Here is an easy way to integrate the two.

And at the same it will give the opportunity to explain some csvfix commands.

Let's start with the csvfix commands.

For example, let's say you wish to number a list of clients, putting the entry number as the first field.

You would thus use the command with its parameters:

-n 1 -> start at number one
-i 1 -> with an interval of 1
-p 000 -> pad for up to 999 entries (you want the sequencing column to be three digits wide)
-ibl -> remove blank entries if any
-sep , -> specifies that you wish to have the comma as field separator
-smq -> use smart quotes in output file
-o [name of output file] -> put here a filename you invent to avoid confusion -- without the brackets, of course.
And finally the filename of the file you wish to process (your input file).

Here's the full command line:
Now you'll notice I've added an extra segment to this line starting with "&&". That's for loading dmcsv.

It means : once csvfix has finished processing, load the newly created file in dmcsv so the operator (you!) can see the result.

You'll notice that the filename I ask dmcsv to load is the same as the one vis-a-vis the "-o" parameter for csvfix.

Et voila. Integration of two csv programs through bash!

A variant would be to see the newly created file with less instead of dmcsv, like so:
Have a good one!

If it's of any help, the latest version of less is here:

http://murga-linux.com/puppy/viewtopic. ... 25&t=75161

TWYL.

Hi.

Here is the complete list of commands for csvfix, v. 1.50. Best regards.

musher0

Where did you find dmcsv at?

Distracting a slight bit from the original subject, I recently ram across another light-weight database called 'recutils'. It's a CLI database which lends itself well to scripting or front-ending with some dialog program. Also, it uses flat plain-text files so the databse can easily be worked with using grep, awk, sed or any text editor you like.
recutils is in the ubuntu repos so you can look for it there.

My search for dmcsv came up empty.

Hello, amigo.

Thanks for enquiring. These are the search results I got with ask:
http://www.ask.com/web?q=dmcsv%20csv&o=101451&l=dis
I had to insist with the search parameters : "dmcsv csv".

The Linux version is apparently here :
http://linux.softpedia.com/get/Text-Edi ... 6283.shtml

However, that may be an error, since the latest available Linux version listed on the author's site appears to be v. 0.6... Please see : http://code.google.com/p/dmcsveditor/do ... bel:Latest

(Linux version 0.6 of dmcsv is provided above in this thread.)

dmcsv obviously exists for Windows :
http://www.softpedia.com/get/Programmin ... itor.shtml

and even as a portable app:
http://www.portablefreeware.com/?id=2183

Best regards.

musher0

To download recutils : http://archive.ubuntu.com/ubuntu/pool/u ... 1_i386.deb

I'll be testing it shortly. Thanks again for the tip, amigo.

musher0

Eeeww! dmcsv=Lazarus/Delphi

Still, since csvfix is all CLI, it would also be possible to make a frontend using gtkdialog or other such animal.

amigo wrote:Eeeww! dmcsv=Lazarus/Delphi

Still, since csvfix is all CLI, it would also be possible to make a frontend using gtkdialog or other such animal.

Are you suggesting or volunteering ?
The Linux community would need something like this!

musher0

Oh, I'm definitely suggesting! At least until I have a need or an itch to satisfy somebody else's need. Come on, there's lots of gkdialog programmers around here -even though I rather see a regular gtk GUI or something done with something beside gtkdialog... Far be it from me to implement something which others could do instead... I have my hands full already. But you can (nearly) always count on me for suggestions and wry criticism!

Sort is easy with CSVpad, just go on the choosen colum an d sort A to Z, or Z to A, as you like.
Import the csv external data in Gnumeric, Well done !

Space free, because solved above

Good afternoon, csv table lovers!

I've uploaded version 1.60 of csvfix (released circa mid-2014) at
http://www.datafilehost.com/d/672b6210

I compiled it from source on Slacko-6.0b. Since this powerful csv file processor
is a console app, there should be no compatibility problems with other Puppies.

Please note that the main site for csvfix is now: http://neilb.bitbucket.org/csvfix/

The docs are included in the *.pet package, but they are also available online
at http://neilb.bitbucket.org/csvfix/manua ... svfix.html

Enjoy.

musher0

Hello!

Just a note to say that I have made CSVpad-1.2, a much more powerful
replacement for dmcsv, available through:
http://murga-linux.com/puppy/viewtopic. ... ost#815999

BFN.

musher0
~~~~~~~~~~
Late thanks and apologies to pelo...

CSVpad-1.2_Linux32.pet: 1734 K
dmcsv-0.6.pet: 1013 K

musher0 wrote:Good afternoon, csv table lovers!
I compiled it from source on Slacko-6.0b. Since this powerful csv file processor
is a console app, there should be no compatibility problems with other Puppies.

The link is dead but I compiled CSVfix 1.60 in Tahr and it works fine there, but unfortunately not in Wary (GlibC errors) despite being a console app.

If I try to compile it in Wary I get issues with g++ errors. Don't suppose anyone's managed to compile this on an in-between version of Puppy so it'll work in Wary?

Musher0, please check your link, s'il vous plait..