Emitsound() & Source issues (bug?)

[EvilTurtleProductions]

I've been having issues with the audio in our game Disinfection lately. Sounds cut off or don't play at all. I've also been seeing some crashes with an OpenAL AL_INVALID_VALUE error in the log.

I've been trying to figure out why, here's some of my observations:

I'm not sure if these are the right values to check but this is how I've been keeping track of the channels and sources available/in use:

System::Print("MAX available sound channels: " + std::to_string(listener->sounddriver->GetMaxChannels()));
System::Print("Sound channels in use: " + std::to_string(listener->sounddriver->sources.size()));

I also checked 'usedchannels' in the Sounddriver object which gave similar results as the size of the sources list.

I noticed when using sounds with EmitSound() that the size of 'sources' keeps increasing every time I call EmitSound(), if the sounds overlap.
For example: I use 6 different stereo sounds for the player footsteps and the size of 'sources' keeps going up, eventually reaching very high values like 400+. I noticed that crashes often happen at this point too.
At first I thought this was an issue with stereo sounds because when I accidentally set the footsteps sound to the (mono) flashlight sound, which is much shorter, the issue was gone. I then tried using the footstep sounds in mono but the issue was back, because the sounds overlap again.
Another time when I noticed this when the monster entity had a bug where it would do a certain action in quick succession, also producing the same sound in quick succession causing sounds to overlap and the list to increase in size rapidly.

I then started noticing the same issue with other sounds in the game too. For example there's a Fusion Reactor in the game which produces various sounds (a mono looping hum sound and mono start/stop sounds). However this Entity is using a Source for the looping sound and a Source for the start/stop sounds. Again when sounds overlap the 'sources' list keeps growing.

The bigger 'sources' grow the more things start cutting off or not playing at all. I'm not entirely sure the OpenAL crashes are directly linked to it, but they did start happening recently too. I've started using EmitSound() and Source a lot more, since I implemented volume settings in the game. I was simply using Sound->Play(); for a lot of sounds, but it doesn't have volume control so I switched to EmitSound() mostly and that's when this issue started becoming apparent.
The volume settings are saved as System property values and fetched in various parts of the code where needed.

Any ideas on what I could be doing wrong or if this is a known bug or something?

[Josh]

Can you post the code for your player?

[EvilTurtleProductions]

1 hour ago, Josh said:

Can you post the code for your player?

I'll cut it up to the bits related to sound, it's a bit much otherwise. Most of the code is very similar to the standard FPSplayer LUA script anyways, albeit C++

Here's where the sounds are loaded in and Sources are setup:

//sounds
	sounds[0] = Sound::Load("Sound/Player/flashlight_02_on.wav");

	sounds[1] = nullptr; //TODO use a sound here, suit noises loop?

	sounds[2] = Sound::Load("Sound/Footsteps/Astronaut/step1.wav");
	sounds[3] = Sound::Load("Sound/Footsteps/Astronaut/step2.wav");
	sounds[4] = Sound::Load("Sound/Footsteps/Astronaut/step3.wav");
	sounds[5] = Sound::Load("Sound/Footsteps/Astronaut/step4.wav");
	sounds[6] = Sound::Load("Sound/Footsteps/Astronaut/step5.wav");
	sounds[7] = Sound::Load("Sound/Footsteps/Astronaut/step6.wav");

	sounds[8] = Sound::Load("Sound/Player/deathfx1.wav");
	sounds[9] = Sound::Load("Sound/Player/deathfx1.wav"); //TODO more death sounds
	sounds[10] = Sound::Load("Sound/Player/deathfx1.wav"); //TODO more death sounds
	sounds[11] = Sound::Load("Sound/Player/deathfx1.wav"); //TODO more death sounds

	sounds[12] = Sound::Load("Sound/Player/breathing_normal.wav");
	sounds[13] = Sound::Load("Sound/Player/breathing_panting.wav");
	sounds[14] = Sound::Load("Sound/Player/breathing_cooldown.wav");
	sounds[15] = Sound::Load("Sound/Player/breathing_asphyx.wav");

	sounds[16] = Sound::Load("Sound/Player/grunt1.wav");

	//sources
	Sound* airhisssnd = Sound::Load("Sound/Player/airhiss.wav");
	airhiss = Source::Create();
	airhiss->SetSound(airhisssnd);
	airhisssnd->Release();
	airhiss->SetLoopMode(true);
	airhiss->SetVolume(0.5f * vol_fx);
	airhiss->Play();

	breathingA = Source::Create();
	breathingA->SetSound(sounds[12]);
	breathingA->SetLoopMode(true);
	breathingA->SetVolume(1.f * vol_fx);
	breathingA->Play();
	breathingB = Source::Create();
	breathingB->SetSound(sounds[13]);
	breathingB->SetLoopMode(true);
	breathingB->SetVolume(0.f * vol_fx);
	breathingB->Play();
	breathingmode = BREATHING_CALM;
	breathingblend = 1.f;

	//tutorial
	tutorialVO = Source::Create();
	tutorialVO->SetVolume(vol_vo);
	tutorialVO->SetLoopMode(false);
	tutorialmode = false;

The footstep sounds in UpdatePhysics call, 'vol_fx' is the volume setting value for sound effects:

//footstep sounds
if (inputx != 0.f || inputz != 0.f)
{
  Vec3 speed = this->entity->GetVelocity();
  steptiming += speed.Length();

  if (steptiming > 150.f)
  {
    steptiming = 0.f;
    stepsoundind++;
    if (stepsoundind > 7)
    stepsoundind = 2;

    this->entity->EmitSound(sounds[stepsoundind], 10.f, vol_fx);
  }
}

 

[Josh]

I don't see anything out of the ordinary there, but it sounds like you are creating an endless series of sound sources somehow. If you upload the project I can run it in the debugger and see what is happening.

[reepblue]

Most of my sound issues went away when I implemented FMOD and I have not looked back.

Leadwerks + FMOD Studio - reepblue's Blog - Ultra Engine Community - Game Engine for VR

[EvilTurtleProductions]

As a test I commented out every single line with EmitSound() and Source's Play() methods so not a single sound would play.

As expected the amount of sources (sources.size()) stays at 21 (I didn't disable the Source creation code, also 21 seems a bit much?).

If I uncomment the EmitSound() line in the above footstep sounds code then again the sources start at 21, but as soon as I start walking around it increments with each footstep i.e. the behavior as described in my post.

[EvilTurtleProductions]

On 5/14/2022 at 1:57 PM, Josh said:

I don't see anything out of the ordinary there, but it sounds like you are creating an endless series of sound sources somehow. If you upload the project I can run it in the debugger and see what is happening.

Okay another observation: just playing one sound with EmitSound() doesn't increment the sources list size. So it seems playing different (overlapping) sounds on the same entity breaks things? This seems to coincide with what I observed on the reactor core entity in the game too.

EDIT: no, sorry I'm being an idiot, I accidentally set it to the flashlight sound, which doesn't have the issue.

[Josh]

I think your code has an error, but in any case wouldn't footsteps just be played with Sound::Play()? You probably don't want spatialization with that:
https://www.leadwerks.com/learn?page=API-Reference_Object_Asset_Sound_Play

[EvilTurtleProductions]

4 hours ago, Josh said:

I think your code has an error, but in any case wouldn't footsteps just be played with Sound::Play()? You probably don't want spatialization with that:
https://www.leadwerks.com/learn?page=API-Reference_Object_Asset_Sound_Play

That's the thing though, there's quite a few non-spatial sounds in the game (such as the footsteps) but Sound::Play() has no control over volume etc. Unless there's some way to route sounds to dedicated buses that can have their volume set I'm stuck with Source and EmitSound(), because I want players to be able to set the volume of things like sound effects, voices and music.

Also I discovered just now Sound::Play() also has the same issue now with the sources list increasing, I'm very confused haha
Gotta delve some more I guess.

[Josh]

You can create a source object yourself, set the sound, and play it whenever you want. This is what EmitSound() does under the hood:
https://www.leadwerks.com/learn?page=API-Reference_Object_Source

[reepblue]

1 hour ago, Josh said:

You can create a source object yourself, set the sound, and play it whenever you want. This is what EmitSound() does under the hood:
https://www.leadwerks.com/learn?page=API-Reference_Object_Source

Only downside to this is you're responsible for releasing your sound sources unlike using entity->EmitSound which will auto relesse for you.

[EvilTurtleProductions]

I'm well aware of Source, it's what I mostly use for sounds that play often/constantly i.e. the breathing sounds in the code I posted earlier.
I release any created Sources or Sounds that haven't been released yet in the Detach() method and this seems to work for the most part.

The issues is that sounds/Sources are not being released properly when using EmitSound() (and possibly Source too), causing the amount Sources to build up which causes sounds to cut off or not play. I'm revising bits of code to use Source as much as possible and this has helped a little. EmitSound() is a lot more convenient though...
From what I've gathered mono sounds aren't generally causing this issue, though as mentioned before overlapping sounds on the same entity can cause problems. Stereo sounds seem to be the biggest culprit, such as the footsteps I've been experimenting with.
I'm now using a Source for the the stereo footsteps as well as player hurt sounds (also stereo) and I don't see the Sources list increment bug now, which surprised me. I'm going to experiment with this further. Perhaps EmitSound() is just buggy when it comes to stereo sounds and should be avoided for those?

Another interesting observation: when playing a stereo sound using either EmitSound() or a Source() while the Listener is outside of the sound range the sound doesn't play at all. Seems there's still some spatialization going on despite it not being used?
Is there actually a specific way to properly play stereo sounds, with volume control, in LeadWerks?
It's not an huge issue as long as you make sure the Listener is always in the sound's range, but it all seems a bit odd

[Josh]

That's a feature. No reason to play a sound if no one hears it. 

Like I said, I am happy to examine your project, but I can't really diagnose anything without the files.

[EvilTurtleProductions]

54 minutes ago, Josh said:

That's a feature. No reason to play a sound if no one hears it. 

Like I said, I am happy to examine your project, but I can't really diagnose anything without the files.

I'll PM you a link with the project, thanks

[Josh]

I am running the game now. Sound sources stay steady at 31, with a max of 255. How do I produce an error?

In actor_infsource::Detach() you are deleting a lot of entities, and I think the world delete function already deleted them, so they are invalid pointers. (This is a problem Ultra solves by using shared pointers.)

You probably don't need to worry about manually deleting all those things. They'll get deleted by the world deletion anyways.

[EvilTurtleProductions]

5 hours ago, Josh said:

I am running the game now. Sound sources stay steady at 31, with a max of 255. How do I produce an error?

In actor_infsource::Detach() you are deleting a lot of entities, and I think the world delete function already deleted them, so they are invalid pointers. (This is a problem Ultra solves by using shared pointers.)

You probably don't need to worry about manually deleting all those things. They'll get deleted by the world deletion anyways.

Right, I forgot I switched to Sources, which mostly solved the problems. Mostly though, still have sounds cutting out every now and again and it does get worse after playing a few levels, see below too.

To replicate my issue:
In actor_player.cpp there's this bit:

footsteps->SetSound(sounds[stepsoundind]);
footsteps->SetPosition(this->entity->GetPosition());
footsteps->Play();

Comment that out and add:

 this->entity->EmitSound(sounds[stepsoundind], 10.f, vol_fx);

Sources should now increment as you walk around and eventually cause sound issues.

As for the Releasing of things:
The reason I release stuff in all Detach() calls is because I never switch worlds, so world delete never gets called during gameplay. Level generation is all procedural.
If you go out the door, walk upstairs and to the left there should be two screens and a couple of big 'ol buttons:

If you press the ACCEPT button (green, highlighted in the image) it should load up a level, which increases the Sources list because a couple are created in various bits of level code.
When you press ACCEPT again you end the mission. ACCEPT again to get a new selection of missions and ACCEPT yet again to do a mission. You should see things stack up and sounds should start cutting out when you walk through the level.

[Josh]

On 5/23/2022 at 8:02 PM, EvilTurtleProductions said:

To replicate my issue:
In actor_player.cpp there's this bit:

footsteps->SetSound(sounds[stepsoundind]);
footsteps->SetPosition(this->entity->GetPosition());
footsteps->Play();

Comment that out and add:

 this->entity->EmitSound(sounds[stepsoundind], 10.f, vol_fx);

Sources should now increment as you walk around and eventually cause sound issues.

Okay, I am able to create this problem. Now I will see if I can make it happen with a simpler test program.

[Josh]

In a simple application you can see the error does not occur. When you press the space bar the sound sources are staying at a low number:

#include "Leadwerks.h"

using namespace Leadwerks;

int main(int argc, const char* argv[])
{
    Leadwerks::Window* window = Leadwerks::Window::Create();
    Context* context = Context::Create(window);
    World* world = World::Create();
    Camera* camera = Camera::Create();
    camera->SetRotation(35, 0, 0);
    camera->Move(0, 0, -4);
    Light* light = DirectionalLight::Create();
    light->SetRotation(35, 35, 0);

    //Create a model
    Model* model = Model::Box();

    auto sound = Sound::Load("Sound/Footsteps/concrete1.wav");

    while (true)
    {
        if (window->KeyHit(Key::Space))
        {
            model->EmitSound(sound);
            auto driver = SoundDriver::GetCurrent();
            int n = driver->sources.size();
            System::Print(n);
        }

        if (window->Closed() || window->KeyDown(Key::Escape)) return false;

        model->SetPosition(Math::Sin(Leadwerks::Time::GetCurrent() / 10.0), 0, 0);

        Leadwerks::Time::Update();
        world->Update();
        world->Render();

        context->SetBlendMode(Blend::Alpha);
        context->DrawText(model->GetPosition().ToString(), 2, 2);

        context->Sync();
    }
    return 0;
}

I will try adding some looping noises and other things and see if I can create any problems.

[Josh]

Adding several looping sounds to the same model did not produce any error:

#include "Leadwerks.h"

using namespace Leadwerks;

int main(int argc, const char* argv[])
{
    Leadwerks::Window* window = Leadwerks::Window::Create();
    Context* context = Context::Create(window);
    World* world = World::Create();
    Camera* camera = Camera::Create();
    camera->SetRotation(35, 0, 0);
    camera->Move(0, 0, -4);
    Light* light = DirectionalLight::Create();
    light->SetRotation(35, 35, 0);

    //Create a model
    Model* model = Model::Box();

    auto sound = Sound::Load("Sound/Footsteps/concrete1.wav");

    model->EmitSound(sound, 10, 1, 1, true);
    model->EmitSound(sound, 10, 1, 1, true);
    model->EmitSound(sound, 10, 1, 1, true);
    model->EmitSound(sound, 10, 1, 1, true);
    model->EmitSound(sound, 10, 1, 1, true);

    while (true)
    {
        if (window->KeyHit(Key::Space))
        {
            model->EmitSound(sound);
            auto driver = SoundDriver::GetCurrent();
            int n = driver->sources.size();
            System::Print(n);
        }

        if (window->Closed() || window->KeyDown(Key::Escape)) return false;

        model->SetPosition(Math::Sin(Leadwerks::Time::GetCurrent() / 10.0), 0, 0);

        Leadwerks::Time::Update();
        world->Update();
        world->Render();

        context->SetBlendMode(Blend::Alpha);
        context->DrawText(model->GetPosition().ToString(), 2, 2);

        context->Sync();
    }
    return 0;
}

 

[Josh]

Going back to your game, we can see the sources attached to the entity are not being cleared:

this->entity->EmitSound(sounds[stepsoundind]);// , 10.f, vol_fx);
System::Print(String(listener->sounddriver->sources.size()));
System::Print(String(entity->activesources.size()));

The question is why?...

[Josh]

In your game, it looks like the attached sound sources are always staying in the "Playing" state. Very curious...

this->entity->EmitSound(sounds[stepsoundind]);// , 10.f, vol_fx);
System::Print(String(listener->sounddriver->sources.size()));
System::Print(String(entity->activesources.size()));
int n = 0;
for (auto source : entity->activesources)
{
	if (source->GetState() == Source::Playing) ++n;
}
System::Print(n);
System::Print("");

 

[Josh]

I am happy to report I have found the cause of your sound problem.

There is a logic bug I found in the sound code. The reason it has never been uncovered is because it is triggered by a rare combination of things that usually don't happen. When a stereo sound is played with a range greater than zero, the logical bug is triggered, and playing sound sources fail to be released when they are finished.

This is a bug in the engine, not in your game, but it can be easily avoid by just converting your 3D sounds to mono. I used Goldwave Audio to resave these in mono format:

Astronaut.zip

Here is your modified code to load the mono versions of the sounds:

	sounds[2] = Sound::Load("Sound/Footsteps/Astronaut/step1_mono.wav");
	sounds[3] = Sound::Load("Sound/Footsteps/Astronaut/step2_mono.wav");
	sounds[4] = Sound::Load("Sound/Footsteps/Astronaut/step3_mono.wav");
	sounds[5] = Sound::Load("Sound/Footsteps/Astronaut/step4_mono.wav");
	sounds[6] = Sound::Load("Sound/Footsteps/Astronaut/step5_mono.wav");
	sounds[7] = Sound::Load("Sound/Footsteps/Astronaut/step6_mono.wav");

When you run the game with that, the number of sound sources will stay at a stable level. Alternatively, you can just play the sound without a source or entity or anything,. This is probably actually a better option, since you don't really need sound spatilization for something that is always coming from the player themself.

All sounds that are emitted by an entity or played by a sound source, that have a range greater than zero, should be mono. Sound spatialization doesn't work on stereo sounds anyways.

[Josh]

I believe this resolves your problem. Please tell me if there are any other issues with this.

[EvilTurtleProductions]

22 minutes ago, Josh said:

I am happy to report I have found the cause of your sound problem.

There is a logic bug I found in the sound code. The reason it has never been uncovered is because it is triggered by a rare combination of things that usually don't happen. When a stereo sound is played with a range greater than zero, the logical bug is triggered, and playing sound sources fail to be released when they are finished.

This is a bug in the engine, not in your game, but it can be easily avoid by just converting your 3D sounds to mono. I used Goldwave Audio to resave these in mono format:

Astronaut.zip 307.24 kB · 0 downloads

Here is your modified code to load the mono versions of the sounds:

	sounds[2] = Sound::Load("Sound/Footsteps/Astronaut/step1_mono.wav");
	sounds[3] = Sound::Load("Sound/Footsteps/Astronaut/step2_mono.wav");
	sounds[4] = Sound::Load("Sound/Footsteps/Astronaut/step3_mono.wav");
	sounds[5] = Sound::Load("Sound/Footsteps/Astronaut/step4_mono.wav");
	sounds[6] = Sound::Load("Sound/Footsteps/Astronaut/step5_mono.wav");
	sounds[7] = Sound::Load("Sound/Footsteps/Astronaut/step6_mono.wav");

When you run the game with that, the number of sound sources will stay at a stable level. Alternatively, you can just play the sound without a source or entity or anything,. This is probably actually a better option, since you don't really need sound spatilization for something that is always coming from the player themself.

All sounds that are emitted by an entity or played by a sound source, that have a range greater than zero, should be mono. Sound spatialization doesn't work on stereo sounds anyways.

I see! I had an vague hunch it was something to do with stereo sounds and spatialization, thank you so much for diving into this!
I guess my only option is to use mono sounds with clever Source placement. using Sound->Play() is not an option because it has no volume controls. Unless there's another way to control volume for Sound?

You mentioned a "range greater than zero". Would setting the Source range to zero help, or would that rather exacerbate the problem?

If you're still planning to do that last LeadWerks update, maybe then...? heehee

[Josh]

You can set the range to zero, and the error will not occur. If the sound is mono, this will probably result in it not being audible, since it is considered to be out of the range of the listener.

If your sound is stereo, you would not want sound spatialization, so there's no need to place any sources.

The source object does have volume controls.