fields.awk database

[Pete]

@cabbie0

Thanks Ian, that's a handy modification.

[Pete]

Several posts back, I included a one liner to convert a fields.awk DB to html.
It was limited in that in only converted to straight html with no regards for links.

One of my uses of fields.awk is to enter bits of info into a database so I can find them again.
It's a simple database with only 3 fields, Name URL and comments.
Here is an excerpt:

Code: Select all

@@@Name::URL::Comments::
Google::http://www.google.com::search engine::
Puppy::http://www.murga-linux.com/puppy/index.php::Puppy forum index::
Puppy::http://www.murga-linux.com/puppy/viewtopic.php?p=909228::fields.awk thread::
Puppy::http://www.murga-linux.com/puppy/index.php?f=63::Puppt additional pets::
Fields.awk::https://www.youtube.com/embed/HC8Az9W8FV4?rel=0&autoplay=0::fields.awk video::

What I needed was a converter to view this in a web browser so I can click directly on the URLs and go to that web site.
So I came up with a Bash script called fields2html, attached, just remove the fake .zip and don't forget to set permissions.

Now what I do is open fields.awk, select my database described above then do a search/read, saving it to a file, not case sensitive then just pressing enter at the search criteria so that fields.awk will include everything.

Now I run that file generated by fields.awk (search1.txt in /root/.DB) thru fields2html:

Code: Select all

fields2html /root/.DB/search1.txt

Result attached as a screenshot.
Note also that by clicking on a link, it opens in a new tab leaving the original tab still containing the list so that one can open multiple web sites instead of having to click on BACK all time.

Some notes/gotchas:

1) Don't parse the fields.awk DB directly to fields2html as the :: causes the links to become malformed, rather use the file created by fields.awk from a search/read.
Of course if you don't have any URLs in you database, then go for it.

2) The script also gives the user the option to open the newly created html file in the defaultbrowser.
If you are not running Puppy or prefer to open another web browser, modify the last line of the script accordingly.

3) The script creates a html file with the same name (plus .html extension) in the same directory as the input file.

Don't forget to remove the fake .zip in the attached script below.

Lastly, I decided to call the script fields2html and not text2html (which it technically is) as some of you may already have a script/pgm called that.

[greengeek]

So I came up with a Bash script called fields2html,...
Now what I do is open fields.awk, select my database described above then do a search/read, saving it to a file, not case sensitive then just pressing enter at the search criteria so that fields.awk will include everything.

Now I run that file generated by fields.awk (search1.txt in /root/.DB) thru fields2html:

Code: Select all

fields2html /root/.DB/search1.txt

Wow, thats a handy function! Given that ian says fields.awk can be cli driven I wonder if it would be possible to "pipe" the output of fields.awk direct to fields2html without manual manipulation of the database search?

I can't code to save my life but something like this? -

Code: Select all

fields.awk [path_to_datafile] readall "*" /root/searchresult.txt | fields2html /root/searchresult.txt > links_out.html

or:

Code: Select all

fields.awk [path_to_datafile] readall "*" | fields2html > links_out.html

The only thing missing from my example is accuracy and correct syntax

.

[musher0]

@Pete: that is indeed a handy addition!

[Pete]

@musher0
Thanks

@greengeek

Is this what you meant?

Version 2 of fields2html attached, remove fake .zip
Just delete the previous version and replace with this one.

One can now convert a fields.awk database directly without using fields,awk or any manipulation.

Code: Select all

fields2html /path/to/database.txt

Of course you can also use the search file created by fields.awk if you wish.

[cabbie0]

@ greengeek, @ Pete
You're welcome for the mod.You're right, it's better that way.

Pete, Regarding your fields2html script:
Again, good work.

I was confused at first why you couldn't simply click on the links from directly within your terminal, until I realized your default is different from mine (in Debian, or now Devuan which is systemd free). I get the impression you have a default terminal (aside from xterm) of Urxvt, with the option of adding LXterminal. And I see from testing some link-containing data within them that they do not have that option. LXterminal comes closest in that the links will highlight (underlined) when the mouse is passed over them, but right clicking only gives the options to Copy URL, which you would then have to re-paste into your browser.

However, on my current system the default terminal is Xfce4-terminal which allows right clicking highlighted links for an "Open Link" option. Unfortunately that terminal is tied to the Xfce window manager and so isn't entirely stand-alone. But, there is another one called sakura which is standalone and has the same ability. Is sakura available from your Puppy Linux repository? One source for the tarball is here:
https://launchpad.net/sakura

If so, you can then run fields.awk from within sakura, and any link-containing search outputs displayed in the terminal can be launched directly in your browser with a right-click.

Alternatively, so long as fields2html could be made to create an output file automatically ( i.e. _without_ user-interaction regarding opening browser as it is in the download version), you could add one line at the end of the read_from function, something like:
fields2html ~/Desktop/$SAVEAS.txt > ~/Desktop/$SAVEAS.html

Presumably you'd end up with BOTH $SAVEAS.txt AND $SAVEAS.html in ~/Desktop at one go.

(But I have not tried this....)

[Pete]

Hi cabbie0

Yes I use LXTerminal by default, but as you say one can copy the URL but not open a browser directly by clicking it.
Will investigate Sakura.

I like your idea of adding the line fields2html ~/Desktop/$SAVEAS.txt > ~/Desktop/$SAVEAS.html

I don't think it will be a problem having a txt and html version of the same file in ~/Desktop.
The way I will go is make two versions of fields2html.

1) one with no stdout output to be used as above

2) A "stand alone" version (like it is now) with option to open a browser with the newly created html page as I'm finding fields2html quite useful as a general purpose text to html converter.

[Pete]

Hi all

For the modification that cabbie0 suggested, where fields.awk calls fields2html directly, two questions:

1) Where should this version of fields2html reside?
/root /usr/sbin or /usr/bin ?
My "vote" is for /usr/sbin, your suggestions are welcome.

2) Any preferences for the name of the new version of fields2html?
Something different to fields2html so as to allow two versions.

Or perhaps I should make just one version of fields2html but if the -q (quiet) option is passed to it, it suppresses output to stdout and no option to open newly created html file straight away.

Your thoughts?

[greengeek]

Hi Pete, could you just clarify the difference between the two versions please? I'm not sure I followed exactly - but i think from what you said version 1 creates an html file and requires manual opening of the browser and version 2 opens the info direct in a browser is that correct?

Maybe v2 could have a slightly different name like fields2browser?

[Pete]

Hi Pete, could you just clarify the difference between the two versions please?

No worries, greengeek.

At first I thought of making two version of fields2html, one that fields.awk calls automatically and another one could use to convert manually.
The reason for the two versions was that fields2html also sends text to stdout in the form of a question asking you if you want to open the html file.

This would cause problems when called directly from fields.awk

However, I then thought of another, better way, have only one version but with the option of telling it to keep quiet (no text to stdout).

So if you call it manually and want the option to open the html file after creation, call it as before:

Code: Select all

fields2html /path/to/database.txt

and when fields.awk calls it:

Code: Select all

fields2html /path/to/searchfile.txt -q

The "-q" at the end is what tells fields2html not to send any text to stdout.

Hope it makes more sense now.

So now when you do a search in fields.awk, it creates a file in both txt and html automatically.

[Pete]

Latest version of fields2html attached (remove fake .zip).
Copy to /usr/sbin and set permissions.
Delete any previous versions you may have.

By placing it in /usr/sbin, it is accessible from anywhere when you call it when using it "manually" and fields.awk will also find it.
It's actually proven to be a handy general purpose text to html converter.

The latest version accepts a -q parameter (must be the second parameter) after the input text file.
This -q option suppresses any output to stdout and will just quietly convert your text file to html without asking you if you want to then open it.

Attached is also the latest modified version of fields.awk (again remove fake .zip) where it automatically calls fields2html to generate a txt and html version of your searches.
Fields.awk places these files in ~/Desktop directory.

[cabbie0]

Hi,

Congratulations Pete, that html conversion script seems to work fine in the fields.awk mod.
I like the approach of keeping the mods simple by calling the an outside script (such as fields2html or addcol) from within fields.awk, in order to keep the database script itself as true to the original as possible while still adding functionality.
This also seems in keeping with the basic Unix philosophy.

I have a question regarding your suggested placement of the fields2html script.
You recommend /usr/sbin but a regular user shouldn't be able to activate a script in /usr/sbin. Are you running fields.awk as root?
For a regular user I'd place the script in /usr/local/bin. That's what I did and it seemed to work fine.

Cheers

[cabbie0]

Hi,

Regarding my previous post indicating that links in a fields.awk search output should be "clickable" in some terminals but not others:

Note the following differences of behavior in 4 different terminals displaying the same fields.awk search output.
Case 1 = xterm
Case 2 = LXterminal
Case 3 = Xfce4-terminal
Case 4 = sakura

The images were taken with a digital camera to include the mouse-click drop down menus, since I don't see any way to include that with normal screen capture. Sorry for poor quality resulting from size reductions.

Hope this helps.

[mcewanw]

Hi,

Regarding my previous post indicating that links in a fields.awk search output should be "clickable" in some terminals but not others:

Hello Ian,

Just trying out your fields.awk database. Very nice to play with and a very useful and easy to customise commandline script.

I am using it in XenialDog, which is a Ubuntu 16.04 Xenial Xerus fully compatible system which has been special crafted by Fredx181 to be very "puppy-like" in use and size but fully apt/dpkg/synaptics capable:

http://www.murga-linux.com/puppy/viewtopic.php?t=106696

XenialDog doesn't have gawk installed by default, but your script seems to work fine so far simply by creating simlink gawk pointing to existing /usr/bin/awk (which turns out to itself actually be a symlink that ends up at /usr/bin/mawk...).

Main news is that XenialDog uses Roxterm by default and, in that terminal, right-clicking on url links in Roxterm allows either direct opening in browser or in copying to pasteboard. Great!
(EDIT: That's a facility provided by the terminal itself of course, rather than internally by fields.awk; any url seen in a Roxterm window, manually typed or created by any program output will offer the click-to-open functionality).

William

EDIT: of course, if using mawk instead of gawk proves problematic, I can always install gawk via apt or synaptics in XenialDog, but so far no problems anyway.

[greengeek]

One can now convert a fields.awk database directly without using fields,awk or any manipulation.

Code: Select all

fields2html /path/to/database.txt

Hi Pete, I just gave this a try and it gave me the correct output (and asked me if i wanted the browser opened) but the urls do not appear as underlined. Do you think that might be something to do with the way my browser is configured? (see pic)

Also - after the browser displayed the output I couldnt tell what to do next with the terminal output so just did a ctrl C and the terminal closed - but so did both instances of Firefox that I had open at the time. (Just thought I'd mention that in case there is a way to only close the PID of the browser window that was started by the terminal rather than the original I already had open)

[mcewanw]

Note: you can open underlined url links displayed in lxterminal by holding Ctrl key whilst left-clicking on the link if you have xdg-open on your system. To get xdg-open functionality in XenialDog, I first had to install the small xdg-utils package using:

Code: Select all

apt-get install xdg-utils

Also, you can apparently open links in urxvt if you first undertake the setup instructions as given here (I haven't tried this) Again seems to rely on xdg-open:

https://wiki.archlinux.org/index.php/Rx ... kable_URLs

EDIT: Apparently there is a way to enable clipboard support in xterm and clickable url links too (albeit complex triple click...), but again I've not tried this. Find the info in following:

https://lukas.zapletalovi.com/2013/07/h ... xterm.html

William

EDIT2: I've since installed sakura version 3.3.4 on XenialDog and it also now has an Open Link option on right-click. I didn't try it prior to installing xdg-utils, but I expect it needs the xdg-open facility provided by that too. EDIT3: It doesn't seem to need xdg-utils afterall (I tried it without).

[Pete]

Hi,

Congratulations Pete, that html conversion script seems to work fine in the fields.awk mod.
I like the approach of keeping the mods simple by calling the an outside script (such as fields2html or addcol) from within fields.awk, in order to keep the database script itself as true to the original as possible while still adding functionality.
This also seems in keeping with the basic Unix philosophy.

I have a question regarding your suggested placement of the fields2html script.
You recommend /usr/sbin but a regular user shouldn't be able to activate a script in /usr/sbin. Are you running fields.awk as root?
For a regular user I'd place the script in /usr/local/bin. That's what I did and it seemed to work fine.

Cheers

Hi cabbie0

Glad it's working well in other distros too.
As regards /usr/sbin versus /usr/local/bin, most people using Puppy run as root by default, however if you prefer /usr/local/bin and it works then it's fine too.

BTW as regards my mods, you can easily find them by searching for <pete_mod_start> which always precedes them.

[Pete]

Hi Pete, I just gave this a try and it gave me the correct output (and asked me if i wanted the browser opened) but the urls do not appear as underlined. Do you think that might be something to do with the way my browser is configured?

No, your URLs don't have // after the http: part.

They should be:

Code: Select all

http://www.somewebsite.com
or
https://www.anothersite.com

Screenshot below shows your DB with corrected URLs which are clickable.

[Pete]

Hi,

Regarding my previous post indicating that links in a fields.awk search output should be "clickable" in some terminals but not others:

Nice list, will download all the text and pics and make a pdf document and post here.
It will come in handy as a quick reference of clickable links in different terminals.

@mcewanw

Great info too, will include it in the pdf as well.

[greengeek]

No, your URLs don't have // after the http: part.

Oh yes. Sorry. Corrected now but I get quite a different display format. Any idea what I'm doing differently to you? It's fine this way but just wondered why the difference.