Bat detector

Hi,

Edwin reported this morning that the code to prevent the counter resetting to 0 during autorecord has worked. Some more testing will be done and after that I will probably update the code and hex.

regards
Cor
 
I discovered this issue Cor described when I wanted to have some recordings comparing the FG element I recently bought and the SPU element that I have been using up to this point.

Some times the recording seemed to stop during the night but the last one was file number 1 from the early morning when I pressed the stop button and the recorder indicated #400+ where I only had 80 recordings or so.
Cor made a nice log function that showed suddenly overwriting of file 1.


Anyway, for all you curious about my experiments with the FG elements, here are some findings.

I altered a few values in the opamp circuit. The FG element needs quite a bit more gain to produce the same voltgage levels. This is where a low noise design proves to be more important.
(R2 4k7, R4 68k, C1 1nF, C3 3.3nF)

The images below are a recording of the same sounds recorded in my garden.
The top one is the SPU, the bottom the FG

I used the same simple amplifier construction that reduces the low frequencies. People may think that this could cause issues but I had no problems and it gives a much better live experience. If you also amplify the lows quite a bit turning and pressing the controls and walking around become nasty noises. If you use this a as recorder outdoors the whole night without any man made noises in the area you will be fine with an unfiltered amplifier. Specially near roads with car traffic filtering the low sounds is very pleasant. I also did try other filtering but this simple circuit seems te so better than the others I tried.


SPU element

fg1.jpg


FG element

spu1.jpg

One can see there is a bit of difference between high and low frequencies. But there is not a huge difference. The big benefit of the stronger output of the SPU can probably be seen in the noise of the FG. Since we only have low amplification with the SPU and much more gain on the FG the noise also seems to become a little stronger on the FG recording.

Since these are two different devices, I guess I should swap the mics and amplifier circuits to be sure there is no hardware difference, but I used the one with FG element with an SPU before and had very similar results as with the other SPU equipped unit.

As far as I can see we do not gain a lot with the FG element, of we should build an extremely low noise amplifier and stay away from low frequency noise sources.

Edwin
 
Hi,

To get rid of the issues with possible overwrites of files-numbers etc I have changed the code for filenames. This is still a feature that needs some testing but my initial tests look fine.

The files will be saved as a hexencoded datetime using 8 characters, this allows us to always have unique filenames. These filenames however would not look very interesting inside the detector as you cannot see the datetime itself but only an 8byte hexcode. When you want to replay them it would be difficult to find a specific file back. So I changed the code that displays the filename to convert a unique-hexcoded date of the file into a readable timestamp. Normal filenames will be shown as they are still.

If anybody has been using our system and has created a workflow (eg. in audacity) that is based on the current naming style of the files (BXXX_XXX) this change might introduce some problems. But for the long run this is a sustainable setup. Files can be saved and if you move them to a PC (or other permanent storage) you can automatically change the filenames into something readable for storage. This option will only be available when saving in WAV files, WAV files have allready the samplerate encoded inside the file. For RAW files we could use the same naming setup but without any clear information on the samplerate this would not be very helpfull.

Which makes me come to the next question: anybody in favour of keeping RAW recordings ? There is no difference between RAW and WAV in terms of the recorded data so I am planning to remove this option unless some people have very clear demands to keep it available. Than RAW will be still using the old number convention when saving a filename.

regards
Cor

regards
Cor
 
Hi,

For those interested, I have just uploaded (https://github.com/CorBer/teensy_batdetector/tree/master/pre_compiled_hex ) a hex version of the new update (firmwaredev_20200925). In this update (I will push the source out later today) recordings of WAV files will use a hexadecimal encoded filename. The 8 hexbytes of the filename are thus unique for each recording and represent the number of seconds passed since 1 jan 1970 (unix time) this solves issues with wrong filenumbers during recording.
Recordings in RAW are still possible and will be using the old filenumbering system.

kind regards
Cor
PS. Source code can be found on github under /src/update20200925
 
Last edited:
Hi all,

The previous code was working ... but not completely as planned. You could record data but not play them on the device (at least WAV formatted). This has been repaired in https://github.com/CorBer/teensy_batdetector/blob/master/pre_compiled_hex/firmwaredev_20201006.hex and the sourcefile is also updated (https://github.com/CorBer/teensy_batdetector/tree/master/src/update20200925).

As the bat-season is more or less ending in my area I think development will for some time be at a lower pace (or in full hibernation). I still encourage anybody with requests/problems etc to forward them in this forum.

regards
Cor
 
Hi Ivano, Are you still reading this tread? I guess so.

I have been playing around with your schematic, it actually looks like something I tried before. And I mean the active filtering in te second opamp stage.
I never had any luck working with them, that is why I stuck to one simple stage doing te amplifying and applying just some simple filtering.
The active filtering always seemed to start oscillating or give some bad side effects. Yes I know the op-amp I use is different and I also use lower value resistors since these should produce less noise (or I was badly informed). Or maybe those have something to do with my bad results.

I just wondered how you were doing and if there was any progress made already.

I was playing around with an ALC4040 USB adapter myself because that is quite affordable and allows a sample rate of 192kHz enough for the pips here in the garden.

Using a non inverting single stage op-amp with simple filtering like in the Teensybat detector I got these results.

busypips.jpg

You can see although I am filtering out low frequencies, there still is a bit more noise on 23kHz. We also did see similar effects on the Teensybat detector. Some people did blame the electronics in the Teensybat detector but it seems it comes down to the SPU microphone element.

Kind regards,

Edwin
 
Hi Edwin

I started to work on the preamp but I always go slowly to to other project running that must be completed.
However the first test seems to be good even if the signal level is quite low. I used the OPA1642 with a preampli and a second stage with a low pass filter.
I hope to have better results this week.
Ivano
 
Hi all

I am a complete newby to Teensy's (although experienced in electronics in general and bat detector design and implementation). I have just got a Teensy bat detector kit and am preparing to start assembly, but would like to clarify a couple of points before I start. Apologies if this is covered elsewhere or I have just missed something in the documents. The description says that the detector digitises at 384ksps 16bit and uses the audio board for the digitisation - however the specs for the SGTL5000 only go up to 96ksps sample rates - is the audio board capable of being overdriven that much? does it have fast enough sample and hold for example? or am I missing something?
Also looking at the code there appears to be an option for digitising directly in the Teensy 3.6 using Pin A2 but this then appears to use only 12-bit rather than 16-bit conversion 9although the 3.6 should theoretically be capable of up 460+ksps at 16bit. Is this just an experimental bit of code, or do I need to modify the kit to use the teensy instead of the audio shield? In which case does it need the audio shield at all?
mildly puzzled, but looking forward to getting the teensy detector built and working,
Justin
 
Hi Justin,

We are gathering samples much faster that the SGTL chip was intended for but it works. I guess the code describes who found the way to take quicker samples or maybe Cor can reply on that. It works just fine so do not worry about that.

Yes we experimented with analog input directly on the Teensy. This seems to be more noisy, specially when writing to the SD card. Every time the SD card writes it gives a small hickup in the signal. Also the extra microphone gain on the SGTL mic input is quite convenient and for these reasons we kept the design as we started out with the teensy audioboard.
The analog input part is still in the code and should not be hard to activate it if you want to experiment with it.
I guess you still need the audioboard for the headphones output.
We also experimented with line in on the audioboard since people indicated there should be less noise but it did not seem to make a big noticeable difference, we only missed the extra microphone amplification that is only configurable on the SGTL mic input.

You are most welcome to try and improve anything on this detector. If you found anything nice please share it.

The kit, code and building documentation does not need any modification and it will work right away.

Edwin
 
Hi Edwin

many thanks for your prompt clarifications. I will get stuck in and build using the pre-compiled code to start with, but will then probably make some minor adjustments to the auto-record functions as my preference when recording is to record continuously into 4 minutee long wav files. It means more work on the analysis but ensures that very quiet passes, such as from BLEs or barbastelles, do not get missed due to being below the triggering threshold. I might also look at encoding the date and time into the filename since file creation/modification times can get altered when copying files - or possibly go to the lengths of incorporating GUANO type metadata in the file header.

Cheers

Justin
 
Hi,

As edwin stated the code can run the SGTL5000 sampling far faster than what it was designed to do. This is due to code that allows us to increase the samplerate
by:
Frank Boesing
* Audio sample rate code - function setI2SFreq
* Copyright (c) 2016, Frank Bösing, f.boesing@gmx.de


regards
Cor
 
I guess I could also write something about the filenames and recording duration.

We ran into an issue using autorecord. Upon creating a new autorecord file we there was a check for the next number. Somehow that sometimes got messed up and we recorder file number one over and over again. It was easier to create a long filename with datecode as filename so one would never get double filenames which were overwriting older files.

The used libraries however only support 8.3 filenames. It seems not difficult to store a longer filename but the detector can not read the filenames afterwards. Wile a computer can probably handle the files the detector can not list them if you want to use the play function.

It would be nice if the code was changed into longer filenames and the play function would still work.
The trick applied now is creating the filename as a hex filename containing the datecode, upon using the play function the detector displays the hex codes as a normal readable date/time for the user. The filenames do not look like dates for the average user when viewed on an computer but you could calculate the date it you want to.

About the recording time... It is easy to change that in the code I guess. Since I don't gather the weaker recordings I usually only record pass-by which are just a few seconds long. I have my detector set to max duration of one minute but you should be able to set the max duration to 300 seconds.
The recording on autorecord starts if a minimum signal strength and minimum frequency setting is met. Frequency is quite clear I guess, the "low signal strength" setting is a bit hard to interpret. The value of 01 is the level where Time Expansion usually triggers the on e recording there show as very weak signals. I guess if you only want to use it is a counter this would be fine. If you want signals that you want to use to determine species or wat to publish I like a setting of around 10 which should be 1o times the TE trigger level if I am not mistaken. The highest setting is 20 maybe nice is the crickets get more noisy around you.

Anyway, before we did not check for the disappearing signal and recorded for an x number of seconds after te trigger. Now the detector checks if the trigger values are still present. If it still present it will keep recording up to the max duration setting.

Edwin
 
I guess I could also write something about the filenames and recording duration.
.....
It would be nice if the code was changed into longer filenames and the play function would still work.
The trick applied now is creating the filename as a hex filename containing the datecode, upon using the play function the detector displays the hex codes as a normal readable date/time for the user. The filenames do not look like dates for the average user when viewed on an computer but you could calculate the date it you want to.

....

Edwin
On any of the file-systems that allow us to read from an SDcard the viewers/explorers and whatever all will show the recorded-file date-time. So its not necessary to calculate the datetime I think. And if there is enough demand I am sure someone can write a converter that renames files with the 8byte HEX-code to a readable format. The 8byte-hex is the seconds since 1jan1970 counter Linux/Unix systems use.

regards
Cor
 
Nice work Adrian !!

I will see if it runs OK using Jupyter. Ive started programming Python about 3 years ago and shortly thereafter discovered Jupyter as the most interesting way to use Python code. Just program and see the results on screen without the need to even call python ;))

cheers
Cor
 
On any of the file-systems that allow us to read from an SDcard the viewers/explorers and whatever all will show the recorded-file date-time. So its not necessary to calculate the datetime I think. And if there is enough demand I am sure someone can write a converter that renames files with the 8byte HEX-code to a readable format. The 8byte-hex is the seconds since 1jan1970 counter Linux/Unix systems use.

regards
Cor

I have written a simple Python script that converts the HEX filename to a readable timestamp, for example it will rename 5FB128E7.wav to 20201115_131103.wav. Either a single file can be converted, or a whole directory - just follow the messagebox instructions that pop up when running the code.

The simplest way to run the code is to install Thonny (https://thonny.org/) if you don't already have Python installed. I have tested it and it works under Windows 10 and Ubuntu, without having to install any dependencies.

Here's the Python script:
Code:
from tkinter import filedialog, messagebox
import tkinter
import os
import datetime
import glob

path = os.getcwd()
root = tkinter.Tk()
root.withdraw()

response = messagebox.askyesnocancel("Rename", "Rename file(s)?\n\nYes to rename a single file.\n\nNo to rename all wav files in directory.")
if response == True:
    filename =  filedialog.askopenfilename(initialdir = path,title = "Select wav file",filetypes = (("wav files","*.wav"),("all files","*.*")))
    base=os.path.basename(filename) #get base name and extension
    name = os.path.splitext(base) #split
    my_file = (name[0]) # get just base name
    if len(my_file) == 8:
        result = int(my_file, 16) # convert hex to dec
        timestamp = datetime.datetime.fromtimestamp(result).strftime('%Y%m%d_%H%M%S') #convert to timestamp
        os.rename(filename,timestamp + ".wav")
        print("Renamed " + filename +  " to " + timestamp + ".wav")
elif response == False: # rename directory
    directory = tkinter.filedialog.askdirectory(initialdir = path)
    for filename in glob.glob(os.path.join(directory, '*.wav')):
        base=os.path.basename(filename) #get base name and extension
        name = os.path.splitext(base) #split
        my_file = (name[0]) # get just base name
        if len(my_file) == 8:
            result = int(my_file, 16) # convert hex to dec
            timestamp = datetime.datetime.fromtimestamp(result).strftime('%Y%m%d_%H%M%S')
            os.rename(my_file + ".wav",timestamp + ".wav")
            print("Renamed " + filename + " to " + timestamp + ".wav")
elif response is None:
    pass

Have fun,

Adrian
 
Nice work Adrian !!

I will see if it runs OK using Jupyter. Ive started programming Python about 3 years ago and shortly thereafter discovered Jupyter as the most interesting way to use Python code. Just program and see the results on screen without the need to even call python ;))

cheers
Cor

Thanks for your response - I noticed that I had missed one line of code, so deleted my post - but you got there too quickly and already replied. :)

I added a line to be able to select a directory if choosing that option, otherwise it automatically defaulted to the current working directory without giving the user the option of changing it. The re-posted code above has the modification.

Regards,

Adrian
 
Last edited:
I have written a simple Python script that converts the HEX filename to a readable timestamp, for example it will rename 5FB128E7.wav to 20201115_131103.wav. Either a single file can be converted, or a whole directory - just follow the messagebox instructions that pop up when running the code.

<Snip>

My filenames just come as a letter B and an ordinal sequence number, so I added a simple mod to the program to append an industry standard GUANO metadata chunk to the end of the .wav file which includes the time of the recording.

Regards

Justin
 
Hi Justin.

would you be so kind to share the code that you use to create the GUANO standard metadata?
I might be nice for other user to have.

For the hardware part also found something interesting to share.
Here https://peerj.com/articles/9955/ is a study on microphones. Not just in the audible spectrum but also for the ultrasonic range.

The study seem te be mainly about the signal to noise properties.
As far as I can see some of the results are digitally equalized so extra amplification needed is not "real" according to added noise of the amplification but it gives a good idea about the capabilities of the microphones.

Edwin
 
Hi Justin,

Nice work ! Until today I have not even heard of the GUANO standard ;) For those (like me) that want to know more go https://guano-md.org/

Have you tested your additions with recorded files ? If so, did both of the infochunks work as planned. I never had a real good look at the options of the INFO chunk of WAV files but was thinking only one block could be used (since the intial block reports the no of bytes that will follow). But if this works fine, I dont mind adding it. Which software supports the usage of this info ? When adding the wav-header for the teensy-batdetector I had to add the infochunk for instance at the beginning of the WAV file to make it useable with AUDACITY.

regards
Cor
 
The used libraries however only support 8.3 filenames.

This may change soon
Bat-detector uses SD.h and uSDFS.h. The latter always could handle long filenames, which was one reason I ported it Teensy. The stock SD library will in next TD release be based on SdFAT-beta (or V2) which will also support long filenames.
It will be interesting to see a coexistence of uSDFS and SdFat-beta.
 
Hi

For those that compile themselves in the TD(teensyduino) environment, when this leads to problems please notify in this forum as I am not using TD. Everything I develop (be it for a teensy or another MCU) is done in PlatformIO under VisualCode, the libraries in PIO are not allways the same and do not necessarily follow TD.

regards
Cor
 
Hi Justin,

Nice work ! Until today I have not even heard of the GUANO standard ;) For those (like me) that want to know more go https://guano-md.org/

Have you tested your additions with recorded files ? If so, did both of the infochunks work as planned. I never had a real good look at the options of the INFO chunk of WAV files but was thinking only one block could be used (since the intial block reports the no of bytes that will follow). But if this works fine, I dont mind adding it. Which software supports the usage of this info ? When adding the wav-header for the teensy-batdetector I had to add the infochunk for instance at the beginning of the WAV file to make it useable with AUDACITY.

regards
Cor

Hi

I hope that the Info chunk still works - it certainly does in Audacity and I hopefully didnt change anything that affects that. Admittedly I did not change the size of the overall file in the master header - that may need tidying up. Wav files can have have any number of chunks as long as each has its identifier and size and they can be either at the beginning or the end of the wav file. The guano chunks can contain a lot of information, and you can register your own header so that you can define additional fields that are relevant for your device, as Wildlife Acoustics have a WA code and have item headers WA|...: Wildlife acoustics used to use their proprietary wamd chunk but now have switched to the Guano chunk which is also used by Pettersons, BatRecorder, Anabat and others. It is not recognised by Audacity (which has limited support for metadata anyway) but is used by Kaleidoscope which is my other goto program for analysis (the free version).
Audacity is much better for analysing longer recordings (my default is to record continuously into 4/5 minute long files) and for examining call structures in more detail, but Kaleidoscope is quicker when classifying large numbers (1000's) of short files produced by static recorders with triggered recording that are deployed for days at a time. Tagging these files puts the ID tag (and AutoID if you have the pro paid for version) in the Guano chunk and will add those details to my Guano chunk quite happily.
My BatRecordingManager software (available for free from www.echolocation.org.uk) also makes use of the Guano metadata. It stores metadata for each recording session, and recording and identified bats in a local database to make report generation easier and also makes it easier to locate specific recordings, or recordings of specific bats. It can also store screen shots taken during analysis of intersting calls or sequences. The Help file describes all the features in more detail.

regards

Justin
 
Hello Edwin,

i have build your Bat Detector last week together, everything going well! 1 have bought the SA9227+PCM5102A DAC board and the TDA 2030 amplifier but please can you tell me the wiring. The picture in your Ultrasonic test sound source speaker test shows not all details.

Kind regards

Jürgen
 
I did simply connect one of the chinch output connectors (GND and signal lines) to the audio input lines of the TDA2030 amplifier board. And ofource the output of the amplifier to the ribbon tweeter.

And I also connected power GND and +5V of the USB connector on the DAC board to the power supply of the TDA2030 board.

5V power supply on the TDA2030 will only result in a maximum power output of 0,5 watt.


I did update the document yesterday. I am using a smaller solution now (works just as well as the SA/PCM board you have there). I also added a step up regulator, with a higher voltage on the TDA2030 board you can get lager power although for testing 0.5Watt is good enough.

Edwin
 
Back
Top