voxpup - a voice controlled puppy

For talk and support relating specifically to Puppy derivatives
Message
Author
User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

voxpup - a voice controlled puppy

#1 Post by greengeek »

Would you like to control puppy by voice command only? Voxpup is my attempt to assemble a puppy that responds to your voice and allows you to start programs, open menus, browse, type (slowly) and basically tell puppy what to do, all without using the mouse.

Download here:

http://www.meownplanet.net/puppylinuxst ... pup001.iso
(thanks russoodle for the hosting!!)
or here: http://www.mediafire.com/download/8umzu ... pup001.iso
voxpup001.iso md5 2b07e99ed3f1c99e0b2656c605e0e98d
(NOTE :This iso is modified so that it ignores savefiles during booting. This is my safety feature to avoid complications for those trialling different pups)

This alternative iso uses standard puppy behaviour and will look for previous savefiles:
http://www.mediafire.com/download/w4cjo ... efiles.iso
md5=1dbda82b6a6d2c0531abe5b703b52083


Voxpup will run out-of-the-box on some machines, but will need microphone and/or capture settings to be adjusted on others.

Handy Hint ; Read the voxmain005commands.txt file that you can find in /root. This will show you what commands to use to control voxpup.

Voice decoding accuracy will depend on various adjustable factors:

- microphone and/or capture volume settings are the most critical factor in making the voice recognition work well. Generally the mic/capture volumes need to be set much lower than you would expect (however voxpup001 sets the volume low by itself, so it may work ok without tinkering...)
- microphone type. Expensive mics are not required - cheap ones work fine but the wiring config needs to match your soundcard's needs (some want a mono mic and some want stereo), and the shielding has to be good enough to avoid picking up electrical noise and hum from mains power adapters etc.. - try what you've got and see if it works...
- correct microphone port needs to be selected (some machines have two mic selections to choose from in the mixer settings)
- your accent. The voice recognition program is written for the American accent, but seemed to work almost perfectly for my NZ (kiwi) accent. (I have made adjustments / duplications within the 'dictionary' so that users have a better chance of getting recognition integrity).

You will probably get best performance from the voice recognition system if you use headphones so that the speaker output does not feed back into the microphone and confuse the voice detection. (I found it worked very well when I used a cheap multimedia 'computer headset' - the type that has headphones and mic built into one headpiece)

I will be posting some tutorials and troubleshooting info over the next few days, but in many cases the user will be able to get started by just plugging in their headset and following the onscreen instructions that appear after boot/startup has finished.

The user will find a couple of "voxmain" documents in /root, which will assist in understanding what commands are available. The voxmain menu will have other submenus added later.

There will also be other versions of voxpup with greater functionality as I get to grips with better coding.

Other Information
voxpup is built on Pemasu's Upup Raring 3.9.9.2
If you want more word processing functionality than the inbuilt Abiword offers, forum member vicmz has listed some good LibreOffice versions here: http://murga-linux.com/puppy/viewtopic.php?t=85915
(I am currently using v4.03 successfully with voxpup)
The voxpup voice recognition module is based on the "pocketsphinx_continuous" .pet offered by technosaurus here:
http://www.murga-linux.com/puppy/viewto ... 5&start=27

Known bugs:
The "close window" command sometimes closes more than the one window I want it to. (something to do with child/parent processes?). This will be fixed in later releases.
.
Attachments
voxpup screenie.jpg
(47.68 KiB) Downloaded 1401 times
Last edited by greengeek on Sat 08 Mar 2014, 21:51, edited 10 times in total.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#2 Post by greengeek »

Basic HowTo:

Mouse control is (at this stage) very basic - you can tell it to move ten, fifty or one hundred pixels left, right, up or down.

Most positioning is best done with the cursor left, right, up or down commands, the arrow left, right, up or down commands, or by the tab key commands.

Choose carefully when you want to use the enter or return key (relating to CURSOR position) or click/rightclick etc (which relate to MOUSE position).

Spelling/typing is by use of the NATO phonetic alphabet (with some modifications/alternatives). See the command list in /root (speaking the command "VOICE-CONTROL-SHOW-COMMAND-LIST" will get this document displayed up on screen)

When using voxpup it is best to issue the commands in a normal speaking voice - just speak at normal volume and speed. No need to shout or stress each word.
Also - if you want to have a conversation with others in the room and want to put voxpup to sleep for a minute just issue the following command:
"VOICE-CONTROL-MICROPHONE-MUTE-ONE-MINUTE"

Important notes :
- Pocketsphinx (which is the program which decodes the speech) only starts decoding when you reach the end of your command phrase and it hears some silence. If the room is too loud it will keep listening but not start decoding. Stop speaking for a couple of seconds after issuing each command...
- voxpup will often show lots of decoded 'junk words' that seem to have no relevance. Don't panic about this - it only happens when your command (or background conversation) falls outside of it's special command list. As long as it is correctly acting upon the commands in the official list that is what we are looking for.

At this stage voxpup is not intended to be capable of handling full vocabulary dictation (such a module may follow later...). If you need dictation functions I recommend looking at ETP's Obedient pup and it's online ability to access the Google dictation services.
.
Last edited by greengeek on Fri 22 Nov 2013, 18:02, edited 3 times in total.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#3 Post by greengeek »

Troubleshooting

**** If you are having problems getting things to work, I recommend doing your troubleshooting in a quiet room with no background noise. This will help provide stable results ****

1) After booting you should see a terminal showing the output of the voice decoder. The normal startup prompt is:
READY....

This means that the program is ready for you to begin speaking.

2) If the startup prompt shows:
READY...
Listening...

This means that it can already detect incoming sound through the microphone and is collecting the voice data. If the program says "Listening..." when you are not yet talking then this is a sign that the microphone/capture levels are set too loud and the mic is picking up background noise, electrical hum, fan noise etc etc.

3) If the "Listening..." prompt shows up only while you are talking that is a good sign. Once you stop talking it should then decode the command that you were speaking.

4) If the "Listening.." prompt does NOT show up while you are talking then that indicates that your mic is unplugged, broken, or that the mixer settings are set for too low mic/capture volume, or that the capture setting is selecting the wrong source (eg mix or line instead of mic), or that the wrong mic is selected in the mixer (eg some PCs allow switching between front or rear mics, or mic1, mic2 or inbuilt).

The actual mixer controls available will vary from machine to machine so if you are having problems please post in this thread and we will try to help.
Last edited by greengeek on Sat 23 Nov 2013, 19:49, edited 1 time in total.

User avatar
ally
Posts: 1957
Joined: Sat 19 May 2012, 19:29
Location: lincoln, uk
Contact:

#4 Post by ally »


User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#5 Post by greengeek »

Thanks Ally!

User avatar
edoc
Posts: 4729
Joined: Sun 07 Aug 2005, 20:16
Location: Southeast Georgia, USA
Contact:

#6 Post by edoc »

This could be really useful for mobile operation of Amateur Radio equipment.

It may need to interface with fldigi flrig etc. to take advantage of their existing interfaces to different equipment.

I will link this at linuxham@yahoogroups.com so that they are aware of it.

Thanks for building and sharing this!
[b]Thanks! David[/b]
[i]Home page: [/i][url]http://nevils-station.com[/url]
[i]Don't google[/i] [b]Search![/b] [url]http://duckduckgo.com[/url]
TahrPup64 & Lighthouse64-b602 & JL64-603

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#7 Post by RSH »

Hi.

This sounds pretty cool.

Would it be possible to modify this for DE users?

Would you like to provide the packages used for this to be downloaded as well?

RSH
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#8 Post by greengeek »

RSH wrote:Would it be possible to modify this for DE users?
I do know how to make a special dictionary file for specific commands in English, but I don't know if the same method will work for the German language.

If you could provide me with a DE translation of the following commands I will make a small DE trial dictionary and see if it works:

CENTERCLICK
CLICK
CLICKHOLD
DOUBLECLICK
RELEASE
RIGHTCLICK

Maybe we can teach it some German!

Basically this method would be trying to 'fool' the voice recognition into learning some new, previously unknown (but actually German) words. It might work so I will give it a try. (Unfortunately I do not speak German so even if I teach it German words I might be doing so in a hideous Americo-Kiwi-Germanic accent that destroys the ears...)

The voice control program used in voxpup is called 'pocketsphinx_continuous' which was offered as a .pet by technosaurus here:
http://www.murga-linux.com/puppy/viewto ... 5&start=27

As far as totally translating the vocal recognition capability of pocketsphinx to a different language is concerned - I do not know if that will be possible. I do know that they are working on extending the language capability but I do not know how far they have got.

see here for a comment about pocketsphinx language modules:
http://www.speech.cs.cmu.edu/tools/lmtool.html
(NOTE: when pocketsphinx tutorials talk about 'language modules' they really mean 'grammar modules' or 'dictionary files' - they don't really refer to the ability to speak other languages).

When I was building my English trial dictionaries I found a LOT of confusing information on the internet (most of it referring to older methods of getting pocketsphinx set up). In the end I had better success doing it my own way. (maybe I just didn't understand the other ways...)

There is quite a bit of info regarding attempts to get German modules running, but maybe these methods are confusing dead ends, I don't know. Anyway, here are some DE links:
http://cmusphinx.sourceforge.net/wiki/s ... usticmodel
http://www.voxforge.org/de/forums
http://www.voxforge.org/home/forums/oth ... no-hits/13

Approx 1/3 of the way down this page:
http://cmusphinx.sourceforge.net/wiki/faq
I did find the following quote which suggests that German should be possible:
Attachments
voxpupGerman.jpg
(100.35 KiB) Downloaded 1121 times

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#9 Post by greengeek »

Hi RSH - I have tried using google translate to get a German translation of the mouse commands I listed above and I have made a .pet based on these commands. It seems to work well, but obviously my accent will not be authentic so I really need a German tester to confirm.

Do these translations seem ok as a starting point?

CENTERCLICK - zentrum klick
CLICK - klicken
CLICKHOLD - klicken Sie auf hold
DOUBLECLICK - klicken Sie doppelt
RELEASE - Freisetzung
RIGHTCLICK - Rechtsklick

Would you have time to test the .pet for me?

gg

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#10 Post by RSH »

Would you have time to test the .pet for me?
Hi greengeek.

Thanks for all the information in above posting.

Of course I will have some time to do some testings.

But first, let's check, if I do get the your commands used in the right direction:

CENTERCLICK --> clicking the centered/middle button of the mouse
CLICK --> clicking the left button of the mouse
CLICKHOLD --> clicking the left button of the mouse and hold it pressed down
DOUBLECLICK --> double-clicking the left button of the mouse
RELEASE --> (left) button of the mouse not longer pressed down
RIGHTCLICK --> clicking the right button of the mouse

If the above was explained (how I do understand this) the right way, I would use German commands as listed below.

Usually we would translate such EN text into full sentences, e.g.:

CENTERCLICK --> drücken Sie die mittlere Maustaste
CLICK --> drücken Sie die linke Maustaste
CLICKHOLD --> klicken und halten Sie die linke Maustatste gedrückt
DOUBLECLICK --> klicken Sie die linke Maustatste doppelt
RELEASE --> lassen Sie die linke Maustaste los (or: linke Maustaste loslassen)
RIGHTCLICK --> klicken Sie die rechte Maustaste

But we would need some kind of short and unique commands, right?

CENTERCLICK --> Maus-Mittelklick
CLICK --> Maus-Linksklick
CLICKHOLD --> Maus-Linksklick-halten
DOUBLECLICK --> Maus-Doppelklick
RELEASE --> Mausklick-freigeben
RIGHTCLICK --> Maus-Rechtsklick

or even

CENTERCLICK --> Mittelklick
CLICK --> Linksklick
CLICKHOLD --> Links-Dauerklick
DOUBLECLICK --> Doppelklick
RELEASE --> Dauerklick-Ende (or: Freigeben, or: Loslassen)
RIGHTCLICK --> Rechtsklick

Just one question: what .pet are you referring to?
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#11 Post by greengeek »

**** I don't think I made it clear that this DE test with RSH is only using a small portion of the voxpup commands - it is only testing mouse click functions. If this test works then maybe it can be expanded ****

*** Ignore these tests if you do not use DE *****
RSH wrote:But first, let's check, if I do get the your commands used in the right direction:

CENTERCLICK --> clicking the centered/middle button of the mouse
CLICK --> clicking the left button of the mouse
CLICKHOLD --> clicking the left button of the mouse and hold it pressed down
DOUBLECLICK --> double-clicking the left button of the mouse
RELEASE --> (left) button of the mouse not longer pressed down
RIGHTCLICK --> clicking the right button of the mouse
Yes, that is the correct understanding.
Just one question: what .pet are you referring to?
The pet I mentioned is one I was testing last night to see if I could "trick" the voice control into recognising German words. It does work ok, but I used my 'google translate' words which are not quite the same as the phrases you suggested. If you want to try my first attempt at this .pet it is available here:

http://www.mediafire.com/download/u9s0v ... k004DE.pet
Or, if you want an sfs (which I think you normally prefer...) try this:
http://www.mediafire.com/download/rwkko ... .9.9.2.sfs (It was made using Upup 3.9.9.2 so I have no idea if it will suit your system - I tried it on ThinSlacko and it worked fine...)

Best to use the pet or sfs with another puppy, not voxpup - otherwise it will conflict with the programs already running in voxpup. I recommend trying in a fresh install or Live session without savefile. Load the pet then restart x.
(Should be straightforward but let me know if problems...)

NOTE: on some systems a mouse freeze might occur but this is simply fixed by a change to the /root/.jwmrcpersonal file to change 'window move' and 'resize' modes to 'opaque' (This was the original reason why I released voxpup as a complete OS rather than just a pet or sfs... The full voxpup is already configured to avoid this)

Usually we would translate such EN text into full sentences, e.g.:

CENTERCLICK --> drücken Sie die mittlere Maustaste
CLICK --> drücken Sie die linke Maustaste
CLICKHOLD --> klicken und halten Sie die linke Maustatste gedrückt
DOUBLECLICK --> klicken Sie die linke Maustatste doppelt
RELEASE --> lassen Sie die linke Maustaste los (or: linke Maustaste loslassen)
RIGHTCLICK --> klicken Sie die rechte Maustaste
I assume that these sentences would be too long to be used all day by the user? You would get lip fatigue if I use these sentences in the program yes? :-)
But we would need some kind of short and unique commands, right?

CENTERCLICK --> Maus-Mittelklick
CLICK --> Maus-Linksklick
CLICKHOLD --> Maus-Linksklick-halten
DOUBLECLICK --> Maus-Doppelklick
RELEASE --> Mausklick-freigeben
RIGHTCLICK --> Maus-Rechtsklick

or even

CENTERCLICK --> Mittelklick
CLICK --> Linksklick
CLICKHOLD --> Links-Dauerklick
DOUBLECLICK --> Doppelklick
RELEASE --> Dauerklick-Ende (or: Freigeben, or: Loslassen)
RIGHTCLICK --> Rechtsklick
Yes, those look like good options - thanks. If you find that my test pet works I can then change the keywords to match your suggested words.

The pocketsphinx software has an interesting characteristic - it is more likely to give an incorrect decode if the command is too short - so there is a benefit in using longer phrases. However there has to be a balance between long phrases that are good for decoding, but short enough phrases that the user can actually be productive.
Last edited by greengeek on Tue 03 Dec 2013, 23:23, edited 1 time in total.

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#12 Post by RSH »

If you want to try my first attempt at this .pet it is available here:

http://www.mediafire.com/download/u9s0v ... k004DE.pet
Or, if you want an sfs (which I think you normally prefer...) try this:
http://www.mediafire.com/download/rwkko ... .9.9.2.sfs (It was made using Upup 3.9.9.2 so I have no idea if it will suit your system - I tried it on ThinSlacko and it worked fine...)
Thanks for the Links.

Yes, I do prefer SFS Modules for the use of programs.

Though, in fact of that I want to have DE menu entries existing for each application, I do manually edit each sort of a package and rebuild everything to a full featured SFS P.L.U.S. SFS Module (those with the 'LP2_' Prefix).

Ok, downloaded.

Will return after doing some testings - at first I have to set up my computer...

RSH
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#13 Post by greengeek »

Just for interest - here are some links regarding the current state of voice controlled computing:

http://info.legalsolutions.thomsonreute ... icle4.aspx
(commentary on overall options/platforms).

http://theubi.com/
(interesting device).

http://www.e-speaking.com/free_pc_voice_control.htm
(Windows based. Free for 30 days, then only $14 to purchase).
.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#14 Post by greengeek »

samstake wrote: I try e speaking what do you think which one is more better for vice control..?
I have not tried espeak - I just included the link as an example of other voice control programs that are available.

I think voxpup is better because it is free :-)

Voxpup could also have it's functionality extended by the modification of the control script (voxmain) or the addition of other subscripts. I don't know if espeak allows the user to do that.

Plus epeak runs on Windows - voxpup uses puppy linux 8)

User avatar
puppyluvr
Posts: 3470
Joined: Sun 06 Jan 2008, 23:14
Location: Chickasha Oklahoma
Contact:

#15 Post by puppyluvr »

:D
Hello,
IDK about ESpeaking above, but espeak is linux and I use it on my puppies.
Close the Windows, and open your eyes, to a whole new world
I am Lead Dog of the
Puppy Linux Users Group on Facebook
Join us!

Puppy since 2.15CE...

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#16 Post by greengeek »

puppyluvr wrote::D
Hello,
IDK about ESpeaking above, but espeak is linux and I use it on my puppies.
Yes, I think the E-speaking mentioned in that link is different to espeak (which I use in voxpup for user notifications).

Do you use alternative voices in espeak? I haven't figured out het how to get a better sounding voice.

User avatar
puppyluvr
Posts: 3470
Joined: Sun 06 Jan 2008, 23:14
Location: Chickasha Oklahoma
Contact:

#17 Post by puppyluvr »

:D Hello,
Espeak has a lot of voices available thru the script you call it with.
I like

Code: Select all

espeak -v en-sc "message"
English with a Scottish accent.. Sort of.
espeak -help explains how to control pitch/timbre/etc.
Close the Windows, and open your eyes, to a whole new world
I am Lead Dog of the
Puppy Linux Users Group on Facebook
Join us!

Puppy since 2.15CE...

ndujoe1
Posts: 851
Joined: Mon 05 Dec 2005, 01:06

Microsoft VX2000 webcam

#18 Post by ndujoe1 »

I have this Microsoft VX2000 webcam which has a microphone built in. VOX pup appears only to reference the motherboard sound card and bypasses this USB webcam sound.

It there a way I can direct VOX pup to use the microphone on this VX2000 as the source for the spoken voice.?

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

Re: Microsoft VX2000 webcam

#19 Post by greengeek »

ndujoe1 wrote:It there a way I can direct VOX pup to use the microphone on this VX2000 as the source for the spoken voice.?
First - a caution - I have found it best to use a microphone that is placed close to the mouth as this increases the "voice volume" relative to the "room volume". I have on occasions successfully used an inbuilt microphone - but I had to turn the capture volume up a bit higher and this is problematic if there is any background noise in the room as it can fool the voice detection module into thinking that you are speaking continuously (so it never begins the decoding phase...). So, for that reason I strongly recommend a headset mic. Having said that, lets try to work out why your webcam mic is not working yet:

Detection / configuration of recording sources can be tricky on any pup, but lets have a go anyway...

1) What is the output if you type:
# arecord -l
(that is a "minus ell" not "minus one")

2) Could you type:
# retrovol
and post an image of the mic config section similar to my screenshot below?
Attachments
retrovol_mic_choice.jpg
(68.98 KiB) Downloaded 261 times

ndujoe1
Posts: 851
Joined: Mon 05 Dec 2005, 01:06

#20 Post by ndujoe1 »

I solved the sound problem by getting a headset that hooks up to the motherboard sound card.

The VOX Puppy's voice control is amazing.

Ran into an unexpected snag. When I save the personal file which is placed on the sda1 drive as my other puppy flavors, VOX Puppy does not see it on boot up. Is it because my partition in sda1 is ext4, My other puppys see their personal files fine.

VOX puppy does reference my Linux swap partition on sda3 OK.

Post Reply