VoxClick project - control mouseclicks by voice

A home for all kinds of Puppy related projects
Post Reply
Message
Author
User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

VoxClick project - control mouseclicks by voice

#1 Post by greengeek »

UPDATED Oct 27 2013
(minor update. Addition of troubleshooting/solution info in case of window resize/move freeze)

Wouldn't it be nice to avoid the mechanical difficulty of handling clicks and doubleclicks on your mouse? What if you could TELL the computer when to do a click, doubleclick, rightclick etc?

You would position the mouse exactly where you want it then just give the computer a spoken command eg: "RIGHTCLICK"

The voxclick004.pet 22MByte download is available here:
http://www.mediafire.com/download/5ybrn ... ick004.pet
(If you have already installed the audiotest .pet below, please uninstall it first or at least move any previous voxclick related scripts out of the /root/Startup directory)

The new .pet installs voxclick into the /root/Startup directory so after installing the new .pet just restartx (menu,shutdown,restart x server) and it will be active and ready to receive your voice commands. No voice training is required. Also, a voxclick004 howto document gets loaded into /root.

The following commands are recognised:
CLICK
RIGHTCLICK
CENTERCLICK
DOUBLECLICK
CLICKHOLD
RELEASE

I have also attached a voxclick howto document to this post.

If you are not ready to try the complete .pet yet and just want to test your audio I have also have uploaded a .pet which allows the user to test the quality of voice recognition on their system - without any risk of actual commands being activated. This is a very easy test and requires no voice training etc - just install the pet and read the "how to start" document which you will find in /root.

voxclick_and_sphinx_audiotest.pet 22MByte download here:
http://www.mediafire.com/download/xn5bb ... iotest.pet
Within a few minutes this audiotest will let you see if this project will work for you.

Troubleshooting :
1) In most cases any problems are caused by mic and capture volume being TOO LOUD rather than too quiet. On some machines you may have to turn these settings down so far that you would think the system could not hear the mic - but sometimes that is the right setting...
(and you may have to turn OFF the +20db mic boost)

2) Some puppies exhibit a symptom where window dragging or window resizing does not allow the cursor to "release" so you can't let go of the window corner or whatever. This can be cured by doing the following:
Use geany to open file /root/.jwm/jwmrc-personal as text (note the dot - .jwm is hidden). Part way down the file you will find two parameters called "move mode" and "resize mode". It is necessary to change each of these from 'outline' to 'opaque'.

3) When pocketsphinx first runs and starts listening it seems to be in a mode where it wants to auto adjust it's sensitivity (or at least that is what I interpret...) and it might get the first word incorrect before it "locks in" on the correct settings. I find it useful to begin with a loudly spoken sentence that is not in the special vocab list. Something like "the rain in spain falls mainly on the plain". That seems to wake it up and from there on it recognises the normal command words spoken at normal volume. Might depend on hardware; I don't know, but worth a try anyway if you are having problems.
.
Attachments
voxclick004_howto.txt.falsegz.gz
Remove the false gz suffixes. It's just a txt file
(4.45 KiB) Downloaded 409 times
Last edited by greengeek on Sun 27 Oct 2013, 17:58, edited 6 times in total.

User avatar
Ted Dog
Posts: 3965
Joined: Wed 14 Sep 2005, 02:35
Location: Heart of Texas

#2 Post by Ted Dog »

nice dove-tail into this thread.. see my talk on oneClick and the need to move this function to the earliest in the boot process we can possibly use voice control

http://www.murga-linux.com/puppy/viewto ... 237#731237

User avatar
Ted Dog
Posts: 3965
Joined: Wed 14 Sep 2005, 02:35
Location: Heart of Texas

#3 Post by Ted Dog »

Do you think it could hear this and tie it as a mouse or keyboard click?

http://www.petco.com/product/9698/Petco ... icker.aspx

looking for cheap ways to add OneClick, if vox can identify the sound as a trained word, we solved a big hurtle



Come on controlling puppy linux with an actual dog clicker, how ironic is that! :lol:

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#4 Post by greengeek »

Ted Dog wrote:Do you think it could hear this and tie it as a mouse or keyboard click?
http://www.petco.com/product/9698/Petco ... icker.aspx:
Possibly. There might be some way to train pocketsphinx to react to such a sound. The likely problem is that it would respond to any strong sound though... You would probably have to use such a system in a very quiet room I think.

User avatar
Ted Dog
Posts: 3965
Joined: Wed 14 Sep 2005, 02:35
Location: Heart of Texas

#5 Post by Ted Dog »

The dog clicker has a unique sound pattern (click/clunk), was in town and went to the pet store and got one. Also what is a good 32bit puppy base for VoxClick playing. Been in ARM & 64bit world for a while.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#6 Post by greengeek »

Ted Dog wrote:Also what is a good 32bit puppy base for VoxClick playing. Been in ARM & 64bit world for a while.
I've been using mostly Upup Raring 3992 from Pemasu, but also doing testing in a live session of Lucid 528 freshly loaded from CD just to make sure things work. Keef was using Precise 5.7.1 I think - for some similar sphinx testing and I think that went ok too (apart from the fact that espeak needed a libportaudio lib installed also).

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#7 Post by greengeek »

Updated first post to now include the complete .pet ready to go.

Tested so far on Upup Raring 3992 and also Lupu 528. Works great on my hardware... Hope it works for others too...

User avatar
Ted Dog
Posts: 3965
Joined: Wed 14 Sep 2005, 02:35
Location: Heart of Texas

#8 Post by Ted Dog »

Any one have Spanish language skills reading this thread and would like to help, please respond.

Centro de Rehabilitación Infantil Teletón is building a regional rehab center near to me, much closer then the University on the opposite side of town, but this effort has not jumped into other languages. Do not worry, most staff will be bi-lingual but those would could benefit may be mostly Spanish native speakers, and we have about a year til they open. Just looking ahead.. :P

gcmartin

#9 Post by gcmartin »

@Ted Dog. On your question of a 32bit PUP, Slacko is a great selection. If you want a very stable selection, you may choose PhatSlack as well. These are a slackware kind of PUP but understands Deb packages as well.

@Greengeek. Is there any reason this project's offering would/would-not run on a 64bit PUP, say the other slackware kind of PUP, Lighthouse64?

Just wondering.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#10 Post by greengeek »

gcmartin wrote:@Greengeek. Is there any reason this project's offering would/would-not run on a 64bit PUP, say the other slackware kind of PUP, Lighthouse64?
Hi GC, I am sorry but I don't know the answer to that. I do not have access to a 64bit machine and have not yet run LH64. My guess is that technosaurus would be the best authority on that as he built the pocketsphinx pet this is based on and would probably be in a good position to know if it will be usable on 64bit.

I have tested this pet on 32bit hardware using Upup3992 and Lupu 528 but both of those are more compatible with ubuntu packages so I cannot say for sure if it runs ok under Slackware based pups. I am keen for any feedback others can offer. (I will try to get some testing done on Slacko myself too...)

EDIT :I have tried a few more Pups and these are the results:

Works perfectly on:
Slacko 5.3
Switchpup_Precise561_Radar_firstrun_0.0.9
Switchpup_Raring3992_radar_firstrun_0.0.9

Will work perfectly on the following pups with one small modification:
Slacko 5.6-4G-NON-PAE
Slacko 5.3.3t (THINSlacko)
Obedient Pup v1.0
Precise 5.6.1
(The modification is a small change to /root/.jwm/jwmrc-personal - it is necessary to change the "move mode" and "drag mode" from 'outline' to 'opaque' to avoid a non-release problem when using voxclick to move or resize a window)

These pups did not work due to GLIBC version missing lib /lib/libc.so.6:
Solid pup (based on wary 5.11)
Wary 5.2.2
.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#11 Post by greengeek »

Updated first post with note on overcoming minor issue with window moving/resizing.

User avatar
Ted Dog
Posts: 3965
Joined: Wed 14 Sep 2005, 02:35
Location: Heart of Texas

#12 Post by Ted Dog »

gcmartin wrote:@Ted Dog. On your question of a 32bit PUP, Slacko is a great selection. If you want a very stable selection, you may choose PhatSlack as well. These are a slackware kind of PUP but understands Deb packages as well.

@Greengeek. Is there any reason this project's offering would/would-not run on a 64bit PUP, say the other slackware kind of PUP, Lighthouse64?

Just wondering.
Well this needs to be looked into, but JamesBond has a new method to run 64bit FD630 with a 32bit ISO file of most any puppylinux version. He most likely have a help page to explain that new functionality with full release, it's on his personal blog site.

gcmartin

#13 Post by gcmartin »

@Greengeek and @Ted Dog

1st
Seems, both, of 01Micko's ISO work with VoxClick. Users should NOT feel restricted to non-PAE. This thread may want to acknowledge that find, as well.
2nd - Ted Dog wrote:[... JamesBond has a new method to run 64bit FD630 with a 32bit ISO ...
Understood.

The only issue, there, is of the VM kind of component that supports the FD630's operation(s). This could mask the "real" hardware under control of FD630 away from the running ISO trying to "hear" from hardware, he cannot see. Thus, I will try to get Techno to provide some idea on native operation in FD & LightHouse64

Just something to be aware of. And, the steps to make it operate fully should/will need documenting if it should be made to work.

Just some ideas.

User avatar
Ted Dog
Posts: 3965
Joined: Wed 14 Sep 2005, 02:35
Location: Heart of Texas

#14 Post by Ted Dog »

They run side by side not VM, better compartmentalization so 64bit and 32bit programs stay on their side of bit-ness. 8)
Some what like co-linux did years ago, but done differently.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#15 Post by greengeek »

Updated first post with v004 of voxclick. Very minor code change and addition of troubleshooting/solution info for those who may have experienced a window freeze or similar during window move or resize.

Post Reply