Audio Amplitude

[CrazyM]

This is a very rough first pass attempt at reading amplitude data from a wav file. I'm hoping to get the signal cleaned up so that I can detect subtle differences in amplitude. Anybody really knowledgeable about the Bank class in regards to audio processing, or using fast fourier transform to smooth data?

 

[Brutile]

I haven't really done much with audio, but I'll try to spend some time looking into it. I take it this is for your lip syncing mechanism?

[CrazyM]

Yes I'm looking into options for porting it to Leadwerks.

[Brutile]

Are you able to send me a code snippet of what you have? It'd be a lot quicker than trying to learn it from scratch.

[CrazyM]

What I have now is ultra hacky, I asked Josh if he could clarify a few details about the Bank class, which I believe is where audio data is loaded into memory before it's passed to OpenAL. Assuming this is true, this is good planning on his part since it appears windowing the data once it's been passed to OpenAL is a no go. My background in OpenAL only goes back a few days so it's quite possible I'm wrong.

 

So rather that post my hacky mess of code, I'll explain what I'm doing as I understand it.

 

I'm creating a Source and loading a clip into it, I believe Sound::Load probably does some detection and parsing of the sample rate, channels etc, then places the raw data into the Bank class, accessible via sound->data. I don't know what "raw data" actually means here since to me that means the entire wav file with its header, format chunk, and PCM data, but I looked for the header and can't find it. Although OpenAL provides a function for retrieving the index of the sample currently being played, I am unable find Leadwerks API that passes this through. So my first best attempt at an alternative was to convert elapsed play time to percentage complete, and use that to track the approximate current index. With (roughly) the current index in hand, I iterate the bank and pop bytes off the stack.

 

My current issues are that it's not clear to me what format the bank data is in, which means I'm guessing at how best to parse it. I'm also still struggling to fully understand FFT windowing, which I believe solves the excessive noise I'm currently seeing in the signal. I can distinguish sound from silence, but not one amplitude from another.

[Ma-Shell]

Although OpenAL provides a function for retrieving the index of the sample currently being played, I am unable find Leadwerks API that passes this through.

 

Which OpenAL-function are you talking about?

If Leadwerks doesn't provide it, you can directly call it. See the following example:

Sound* snd = Sound::Load("Sound\\Footsteps\\concrete1.wav");
Source * s = Source::Create();
s->SetSound(snd);
s->SetLoopMode(true);
s->Play();

if (alIsSource(((Leadwerks::OpenALChannel*)s->channel)->source) == AL_TRUE)
 printf("Source TRUE\n\n");
if (alIsBuffer(((Leadwerks::OpenALSound*)s->sound)->buffer) == AL_TRUE)
 printf("Buffer TRUE\n\n");

alSourceStop(((Leadwerks::OpenALChannel*)s->channel)->source);

 

As you can see, you can get the OpenAL-source and OpenAL-buffer as demonstrated. If you need any other OpenAL-ids, I would be glad to show you, how you can retrieve them.

[Ma-Shell]

I assume, what you want is the following (where s is the source):

 

ALint i;
alGetSourcei(((Leadwerks::OpenALChannel*)s->channel)->source, AL_SAMPLE_OFFSET, &i);
printf("%i\n", i);

[CrazyM]

Wow Ma-Shell that's cool, for some reason I hadn't considered accessing the underlying OpenAL directly. You're right, [alGetSourcei] is what I was referring to. I've been playing with a stand-alone copy of the OpenAL SDK for a few days to get my feet wet and find my bearings. I'm going to run these new ideas through my test environment and see what kind of accuracy I get on my second pass.

 

Thanks for your help!

[CrazyM]

The length of the bank data is twice that of the OpenAL buffer, is this because the bank is storing the data in two channels even though my audio clip is mono?