Audio - looping recorder needed [FILLED]

[greengeek]

I would like to record a microphone input 24 hours per day (ie continuously) - with the recording separated into something like 3 minute segments - and I want it to limit the saved files to one storage device, or one partition, and overwrite the oldest files as the device/partition becomes full.

***EDIT : Nov 20 - first version of .pet released here***

I plan to use something like a 32GB usb stick, or 500GB external HDD to record the mic output for security purposes, then playback the files at a later date if something happens. Hopefully this will trap the voices of the offenders.

Any ideas?

cheers!

EDIT : I was thinking that if there was no existing recording program that included the 'overwrite old files' function, it might be possible for me to use a second program to simply start erasing old files (maybe anything 48 hours old??) and set it to delete one file every 3 minutes, so that there was always enough free space for the recording program to continue working...

[mikeb]

arecord is your friend for this.....

a scripted loop would do..... as you say erase older ones or overwrite...but you probably want the time stamps.

you can pipe arecord wav output to lame to make on the fly mp3 to save space...I do that in simple recorder and the overhead is reasonably low.


mike

[Flash]

As for the mp3 settings, 32 kbps mono is perfectly adequate. I listen to lots of audio books, which I convert from CD to mp3 so I can put them on my mp3 player. When I first started converting, I did some experimenting to see how small I could make the mp3 files. I found that I couldn't tell any difference between 32 kbps mono and the original wav file from the CD.

[ETP]

Hi greengeek,

This will do the job. (Both audio & video if you wish)
http://www.maketecheasier.com/setup-mot ... am-ubuntu/

http://www.lavrsen.dk/foswiki/bin/view/ ... oRecording

http://www.lavrsen.dk/foswiki/bin/view/Motion/WebHome

[greengeek]

as you say erase older ones or overwrite...but you probably want the time stamps

So do you think it would be best to find a way to write each file segment with a filename that contains the timestamp? Or I suppose just an incrementing number like rec001, rec002, rec003 would be fine as I could refer to the file creation time/date to pin down actual times couldnt I?

As for the mp3 settings, 32 kbps mono is perfectly adequate

Yes, my mic quality will be the limiting factor. I wont need CD quality and I may even experiment with 8000mono if I need to save on overall storage size.

I started with:

Code: Select all

arecord -vv -fdat recording.wav

Works well but large file size

Then I tried this syntax that I found somewhere:

Code: Select all

arecord -f cd -t raw | lame -x -r - out.mp3

but all I got was terrible hissing so I need to revisit that one and see what I did wrong.

But then I stumbled across a really clever alternative in a Puppy thread about "Sox" which it calls the swiss army knife of linux audio. Sox allows all sorts of controls over sound detection threshold and recording format etc. http://www.murga-linux.com/puppy/viewto ... 166#483166
The linux manual is here: http://linux.die.net/man/1/sox
There are two .pet versions on the puppy thread. I used 'neurino's sox-14.2.0-with-deps.pet (seems compatible with my Slacko5.6 testbed)
rather than 'chrome307's package sox-14.4.0-i686.pet (Don570 left a comment that the 14.4 version wouldnt work for him).

This is the basic syntax I started with:

Code: Select all

sox -t alsa default recording.mp3

Then I found this nifty syntax that allows recording only when the mic output level is at a certain threshold so that it doesnt waste storage when no sound is being heard. The program keeps running but stops writing to disk until voices are heard again, then it starts writing to the disk file again. Seems to work great.

Code: Select all

sox -t alsa default recording.mp3 silence 1 0.1 1% -1 0.1 1%

(See this site for other clever sox syntax options)

I reckon I could use this sox syntax and call it as you've suggested from a loop script that records a 3 minute segment (or a specific file size), then checks for free space, kicks off a new 3 minute recording segment if there is adequate disk space, but if no adequate disk space, deletes the oldest file, then starts recording again.

[greengeek]

This will do the job. (Both audio & video if you wish)
http://www.maketecheasier.com/setup-mot ... am-ubuntu/

Thanks ETP. I had forgotten about "motion". (and I only trialled it a few weeks ago!). Watchdog posted some info here which worked well on my slacko 5.6

I had not thought about stripping out the audio from the webcam so that is a good option.

I do still plan to have the mic script running as a separate backup method in case the webcam is destroyed or breaks. I figured I could hide the microphone easily so that it could not be targeted.

[mikeb]

Oh good...sounds ideal

Just fyi the hissing is when the endidness does not match...from my simple recorder as example.

arecord -D plughw:$CARD,0 -f cd | lame -b $BITRATE - "$NAME" &

mike

[greengeek]

Just fyi the hissing is when the endidness does not match...from my simple recorder as example.

Code: Select all

arecord -D plughw:$CARD,0 -f cd | lame -b $BITRATE - "$NAME" &

Hmmmm, how do I tame my little indians then? Can you help me redo my code to get rid of the hissing?
I had this:

Code: Select all

arecord -f cd -t raw | lame -x -r - out.mp3

and got the hiss.
I'm guessing "plughw:$CARD" is specifying my device right? I didn't seem to need that when I was recording to wav, so do I really need to change that half of my code? Or is it just the 'piped' side i need to adjust? Is "$NAME" ok with the format "out.mp3 as I originally had?
Whats the -b parameter? That doesn't show up when I look at 'arecord --help'

I found this one online and it seems to work but I don't understand why. I'm struggling to understand the structure of the piped portion:

Code: Select all

arecord -f cd -t raw | lame - -r -o out.mp3

(I don't see the '-o' parameter defined when I enter 'arecord --help' either, so I feel I'm working with too little information. Seems to work tho')

EDIT : I can add a 'duration' parameter (-d) as follows:

Code: Select all

arecord -f cd -d 10 -t raw | lame - -r -o outfile.mp3

This is supposed to create a 10 second file, but actually produces a 7 second file. To get 10 seconds of recording I seem to need to set -d to 13, not 10.

In theory, to get my intended 3 minute recording I should be able to use:

Code: Select all

arecord -f cd -d 180 -t raw | lame - -r -o outfile.mp3

EDIT 2: Yes, that works, and creates a file of 2814kB
.

[mikeb]

-b is to set the bitrate...those $BITRATE are variables...I just copied and pasted.

dump the -t raw parameter to match basically.

if you use plughw rather than hw it uses dmix I believe rather than direct to the card...basically allows sharing and in this case it was for a sound meter...also applies to outputs....without the 'plug' only one sound app can run. In the case of one sound cord you don't need it at all.

The -o parameter...hmmm...perhaps for compatability

mike

[greengeek]

Ok, I have decided to stick with arecord for the moment and try to make progress with the rest of the script. I found a way to create the recorded segments and to create the filename as a timestamp. Here is my trial:

Code: Select all

#!/bin/bash
NAME=`date +%Y-%m-%d_%H-%M-%S`
arecord -f cd -d 10 -t raw | lame - -r -o $NAME.mp3
NAME=`date +%Y-%m-%d_%H-%M-%S`
arecord -f cd -d 5 -t raw | lame - -r -o $NAME.mp3
NAME=`date +%Y-%m-%d_%H-%M-%S`
arecord -f cd -d 5 -t raw | lame - -r -o $NAME.mp3

What this does is:
1) Record for 10 seconds to a file that is called "Year-Month-Day_Hour-Minute-Second.mp3"
2) Records to a second file for 5 seconds (labelled with a new timestamp)
3) Records to third file for 5 seconds (labelled with a new timestamp)

To set a record duration of 3 minutes I just replace the -d value with 180.

This is just to prove to myself that I can see the new files being created. The next step is to use something like a "while" loop to allow this process to occur indefinitely. Then the next step after that will be to add a mechanism for limiting the overall storage size to a preset limit/location and "looping" back and overwriting the old files.

This is going to take me a bit of trial and error so anyone please feel free to jump in and teach me how to do this

This method creates a file size of approx 1MB per minute of audio, so my rough calculation suggests that an 8GB usb stick would hold maybe one weeks worth of recording.

Or maybe I could open an HDD directory of a specific size and just limit the script to that dedicated directory?

EDIT : I'm currently referring back to this thread to pick up some tips on structuring loops.

[mikeb]

http://linux.die.net/man/1/lame

well perhaps look at reducing the bitrate and going mono to keep the space needs down.

while [ 1 ]; do

done

keep the spaces for infinite loop.... otherwise you might use a for loop since you will have a predictable space usage when running.
Or you could do a df with every while test assuming you only loop periodically so theres little overhead doing so.

mike

[Barkin]

To set a record duration of 3 minutes I just replace the -d value with 180.

This is just to prove to myself that I can see the new files being created. The next step is to use something like a "while" loop to allow this process to occur indefinitely.

Unless SoX compresses to mp3 on-the-fly , compression takes at least 1/100th the duration of the track after it has been recorded. So just looping script which records 3 minute chunks then compresses the file to mp3, will miss a couple of seconds of recording each time it loops when it is busy doing the mp3 compression ,
[ some sort of buffer may be required for continuous recording ].

[greengeek]

Heres my first trial of the loop:

Code: Select all

#!/bin/sh
#Trial looping script based on simargls offered example here:
#http://www.murga-linux.com/puppy/viewtopic.php?p=691153#691153
#Note Karl Godts caution to add sleep cycles or else the cpu goes into overdrive and heats up
while true; do
NAME=`date +%Y-%m-%d_%H-%M-%S`
arecord -f cd -d 10 -t raw | lame - -r -o $NAME.mp3
sleep 0.2
done

Seems to be basically working so far.

Aah, you posted while I was trialling.

keep the spaces for infinite loop

I dont understand - what do you mean by the spaces? Do you mean the recorded silences when no-one is speaking?

[mikeb]

That's ok you used 'true' .... those [ ] versions have to have a space between them.

Not sure if the heating thing applies since your arecord will stop the loop each time until its completed.

mike

[greengeek]

... miss a couple of seconds of recording each time it loops when it is busy doing the mp3 compression , [ some sort of buffer may be required for continuous recording ].

Good point, that is something i will have to watch out for. I guess if the system gets busy processing other stuff too that might slow matters down even further.

[mikeb]

well in piping arecord stops and that last bit of data is still to be processed before the loop can move on...another reason to leave out the sleep.

As for how long...one to suck and see...might be a second or 2 or perhaps only a fraction of one.

If the chunks are long using low bitrate then you might lose one second every half an hour for example.

mike

[Barkin]

... If the chunks are long using low bitrate then you might lose one second every half an hour for example.

Audacity, (using LAME encoder), takes 5 seconds to compress 100 seconds of 22050Hz monaural audio into an mp3 file with a bit-rate of 64kbps , so that's compression-time of 5% of the recording-time.

If the sound-recording-program compresses to mp3 on-the-fly then that ~5% delay can be avoided.

[greengeek]

Audacity, (using LAME encoder), takes 5 seconds to compress 100 seconds of 22050Hz monaural audio into an mp3 file with a bit-rate of 64kbps , so that's compression-time of 5% of the recording-time.

I'm struggling to understand the format of the syntax regarding format and compression, but I found this example that seems to cut my storage requirements in half, and doesn't seem to impact too much in terms of data loss during compression:

Code: Select all

arecord -f S16_LE -c1 -r22050 -d 10 -t raw | lame -r -s 22.05 -m m -b 64 - Output.mp3

This gives me a loop recorder as follows:

Code: Select all

#!/bin/sh
#Trial looping script based on simargls offered example here:
#http://www.murga-linux.com/puppy/viewtopic.php?p=691153#691153
#Note Karl Godts caution to add sleep cycles or else the cpu goes into overdrive and heats up
while true; do
NAME=`date +%Y-%m-%d_%H-%M-%S`
#arecord -f cd -d 10 -t raw | lame - -r -o $NAME.mp3
arecord -f S16_LE -c1 -r22050 -d 10 -t raw | lame -r -s 22.05 -m m -b 64 - $NAME.mp3
sleep 0.2
done

Seems ok so far. Just got to work on the total storage control and overdubbing of old files next.
(As well as backtracking over the lame man info regarding how to specify format, compression and rate so I can understand and control it better...

[greengeek]

I'm modifying the script to write to somewhere else than /root or the directory wherever else the script is stored, and I want to restrict that new directory to a specific size - however, the threads I have found seem to suggest that I need to use the "-o loop" type of approach which sounds a bit complex for my meagre understanding.

One quote I found regarding directory size limitation is: "I don't think Unix has a built-in way to do this. What people usually do is write a script that runs "du -s <directory>" to get the total size, and alert the administrator if it exceeds the limit."

I guess what this is suggesting is that I could use a different method, such as detecting the current size of the directory and modifying the script behaviour accordingly, so that as it fills up i either switch to a different directory or run a command to start deleting older files.

Any ideas or comments?

[mikeb]

piping is on the fly

drop the sleep 0.2

use df ...easy to get space left

long files ..eg say 10 minutes means tiny tiny lost bits in comparison

Yes getting smaller... perhaps try your 32 kbps suggestion since you already have mono.

mike