FFT window on OpenAL buffer (for spectrum data)

[CrazyM]

I'm trying to tap the OpenAL raw audio data to see if it can be processed with an FFT window to build a live stream of the spectrum or amplitude of a playing audio file. This is quite simple in FMOD since it ships with a getSpectrum function. However, we are evaluating the feasibility of bringing our automated lip sync solution, originally built for Unity on top of FMOD, to Leadwerks, and after speaking with Josh, including FMOD as a built in option for Leadwerks users isn't an option at this time. Personally, we would love to see an asset store for Leadwerks as we think Unity has proven it's a good source of income for the host company, encourages developer involvement, and makes it easier for new developers. It is popular enough to convince Unreal Engine to get on board and open their own asset store.

 

Josh informed me that the Bank class pointer references the raw data but I'm struggling to determine if the data in the bank->buf could be processed through an FFT window.

 

Are there any OpenAL pro's here that know if this can be done, and might be will to share some expertise?

 

In case anyone is interested or curious, here are some of our promotional videos for our solution on Unity.

 

Thanks

[Josh]

FYI, we have a system for distributing content built into Steam called the Workshop:

http://steamcommunity.com/workshop/browse/?appid=251810

[Guppy]

FYI, we have a system for distributing content built into Steam called the Workshop:

http://steamcommunity.com/workshop/browse/?appid=251810

 

Yes but we cannot monetise it can we?

[CrazyM]

Yes, I've played with the workshop, it's great, but I was also under the impression there is no option there for monetization.

[CrazyM]

So the bank->buf contains four bytes of data in a char array format, and I can convert this back to an int, but how do I read the next four bytes?

[Josh]

The Workshop is not monetizable, yet.

 

The buf is a memory address. The bank class has another member that indicates the size.

 

The bank class also has a lot of built-in functions for reading memory, like this:

unsigned char c;
for (int i=0; i<sound->data->GetSize(); i++)
{
c = sound->data->PeekByte(i);
}

[YouGroove]

Esenthel have it's own store also.

I agree a monetized asset store is missing , i doubt this will be possible in Steam as only products owners can sell DLC.

The other way would be like the Sci Fi DLC, make some agreement with Josh than let him put on sale your product as DLC on the workshop, but that would represent too much time and effort to manage if they were several people wanting to sell stuff as DLC.

Let's give time to LE3 to maturate first, the store can wait.

[CrazyM]

Thanks Josh, I'm working with the data now.

 

To anyone interested, I found a really great article on processing audio with an FFT window. I think this might work, though I may have to read the article 3-4 more times.

 

Part I

http://www.edn.com/electronics-news/4383713/Windowing-Functions-Improve-FFT-Results-Part-I

 

Part II

http://www.edn.com/electronics-news/4386852/Windowing-Functions-Improve-FFT-Results-Part-II

 

If I can build a getspectrum function, then we can try convince Josh to include it in the API. Having this ability brings a lot of cool possibilities, lip sync, music-based games, etc.