Performance problem with NPC(AI) and collisions

[Christian Clavet]

Hi, I would like to report a problem that I've seen with the NPC that are making the "update" loop much longer when they "awake".

Description of the problem:
On a terrain that have trees(vegetation layer) with collision enabled, when the player (std FPS controller) get near a NPC(crawler) the update loop can get as high as 7000ms for a few second and come back as normal. Disabling the collision option on the vegetation layer, seemed to fix it. But I "feel" that it does that for every type of collisions (models / brushes). The more you put on the map, the more time it will take on the update loop.

Reproductability: 100%
How to reproduce:
1. Create map with the FPS template.
2. Create a terrain.
3. Place some (3+) Crawlers NPC on the terrain.
4. Create lots of tree (density of 2.2 or less), and enable the collision option on the layer.
5. Start the game and move the FPS player near one of the NPC.
6. Notice the "lag" and stop when the NPC emit the detect sound. Also notice the update loop time increasing a lot!

My impressions about this:
I feel like the navigation system is doing some kind of data copy of collision models from all the surrounding with no filtering of geometry when the AI find the play "in range". An investigation would be welcome...

My current system that was used for the testing:
CPU Intel I7 3770K, 16Gb ram, 512GB SSD + 1TBHDD, NVIDIA GTX1080 at 3840x2160 desktop resolution.
Windows 10 Home (version 1703)

Edited July 3, 2017 by Christian Clavet
Cannot be ray cast, more of a data copy...

[Christian Clavet]

Hi, Jen!

Quote

You're better off making your own AI system.

You mean rewriting your own version of monster.lua or rewriting the whole AI system (Navigation AI C++ side)? Your "AI system" term is confusing me a little here. I'll assume your meaning the LUA script that is used for AI.

This is the part that activate in the LUA code of monster.lua when the player as been seen (in range). I don't see anything there that could do what I've experienced. For my current needs this script is Ok.

I don't see any big loop in there if the NPC switch from IDLE to CHASE, and this the current symptom I have. (update loop take more than 7000ms to update then goes back to normal after a few seconds)

Quote

 

function Script:UpdatePhysics()
    if self.enabled==false then return end

    local t = Time:GetCurrent()
    self.entity:SetInput(self.entity:GetRotation().y,0)
    
    if self.sound.idle then
        if t-self.lastidlesoundtime>0 then
            self.lastidlesoundtime=t+20000*Math:Random(0.75,1.25)
            self.entity:EmitSound(self.sound.idle,20)
        end
    end
    
    if self.mode=="idle" then
        if t-self.lastupdatetargettime>250 then
            self.lastupdatetargettime=t
            self.target = self:ChooseTarget()
            if self.target then
                self:SetMode("chase")
            end
        end
    elseif self.mode=="roam" then
        if self.entity:GetDistance(self.target)<1 then
            self:SetMode("idle")
        end
    elseif self.mode=="chase" then
        if self.target.health<=0 then
            self:SetMode("idle")
            return
        end
        if self:TargetInRange() then
            self:SetMode("attack")
        elseif self:DistanceToTarget()<self.attackrange*2 then
            self.followingtarget=false
            self.entity:Stop()
            self:DirectMoveToTarget()
        else
            if self.followingtarget==false then
                if self.entity:Follow(self.target.entity,self.speed,self.maxaccel) then
                    self:SetMode("idle")
                end
            end
        end
    elseif self.mode=="attack" then
        if self.attackbegan~=nil then
            if t-self.attackbegan>self.attackdelay then
                if self.target.entity:GetDistance(self.entity)<1.5 then
                    self.attackbegan=nil
                    self.target:Hurt(self.damage)
                end
            end
        end
        local pos = self.entity:GetPosition()
        local targetpos = self.target.entity:GetPosition()
        local dx=targetpos.x-pos.x
        local dz=targetpos.z-pos.z
        if self.entity:GetCharacterControllerAngle()>90.0 then
            self.entity:AlignToVector(-dx,0,-dz)
        else
            self.entity:AlignToVector(dx,0,dz) 
        end
    end
end

 

What I've seen, is that the update loop (not the render) will take more than 7000ms to refresh once the NPC see the player (goes from idle to chase mode), after that it move and update refresh goes back to normal.

It could be something to do with recast/detour, that copies the world collision models data or something that initialize lots of data at once then release. (What I'm currently thinking the problem could be, but only Josh could tell.) 

With only a terrain and nothing else, I was able to make it work correctly by disabling the collisions on the tree models of the vegetation, so it seem directly related with collision models.

[Josh]

On 7/3/2017 at 9:30 AM, jen said:

The example templates aren't really optimized for production use. Those are only meant testing/learning purposes. You're better off making your own AI system. I have done that and I can say with confidence my own custom system is far more efficient than the example.

Again, the AI script that come default with Leadwerks is only meant for testing/learning purposes and not for production use.

I do not agree with this at all.

[Christian Clavet]

Quote

The "SetInput" command kills FPS.

I'm sorry Jen. The problem I currently have does not really affect the render loop. This "SetInput" command is not causing the problem actually. It affect the update loop at a very specific moment (switch to chase mode). The results I have is a constant render time, but a sudden bump in update loop time, because of something that is happening from the AI with the collision models.

I posted a bug report for this specific issue and would like that we stay on the subject. Thanks.

[Josh]

Collidable vegetation set at every 2 meters should probably not be done.  The character controller is going to be remade with a swept design but right now requires multiple sub-steps which is probably slower.  Vegetation collision can be rather slow since it involves dynamically generated meshes (see chapter 4 of Game Engine Gems).  If the tree has a high-polygon physics shape, this will be very slow, so hopefully you are using something simple for the physics shape.

[Newton Dynamics]

17 hours ago, Josh said:

Collidable vegetation set at every 2 meters should probably not be done.  The character controller is going to be remade with a swept design but right now requires multiple sub-steps which is probably slower.  Vegetation collision can be rather slow since it involves dynamically generated meshes (see chapter 4 of Game Engine Gems).  If the tree has a high-polygon physics shape, this will be very slow, so hopefully you are using something simple for the physics shape.

Hi,

As I explained on another thread, I believe I can provide a solution for this problem that eliminate the problem with player moving around objects like threes or other players is a way that is independent for the complexity.   

but the first thing is to determine why the test take 7000 ms, that's actually seven seconds which is ridiculous, so I am assuming is actually 7 ms, which is also too extreme regarless of the mesh density.

so if it possible for me to get a test demo to try this, two things can happens, 

1-Maybe there is a bug in newton that nee to be fixed

2-the bug is not in newton and we can eleborate for a clever solution to this problem.

 

[Christian Clavet]

Hi, Thanks! There is no issue with the player collision with the vegetation. Nothing is slowing down with the player. There is no issue with the NPC UNLESS they switch to CHASE mode.

With 5 Zombies (Zombie pack) in an map with vegetation. Everything worked fine until they changed their state to CHASE  mode (when they detect the player) then the game felt it stopped and I was seeing 7000ms at the UPDATE time on the debug screen. I waited a couple of seconds and then everything got back to nornal. The zombies and the player were moving normally.

Disabling the collision on the vegetation layer seemed to fix the problem but the level only contained the terrain and vegetation.

Here is a screenshot so you can see the UPDATE time going crazy.

[Josh]

Can you upload the project for me to try?  I suspect maybe something is generating the first time or loading or something like that.

[Christian Clavet]

The map that showing this the most is too big to sent (1Gb+ zipped). I've sent you a message with the link of a new map (new project) that I've enabled the collision on the vegetation and contain only the stock assets. It's the START.MAP.

Thanks!

[Josh]

I might know what is going on...

[Josh]

If you jump off the dock so that the crawlers can actually find a path to you, and you walk towards them, does the delay go away?

[Christian Clavet]

Never tried that yet!

[Josh]

Well, what is happening is they are performing a search to the max extents because they can't find a solution.

Vegetation makes the problem worse because it breaks up the navmesh into a lot of extra triangles.

[Christian Clavet]

Hi Josh! Thanks again for looking at this!

I never tested off the dock. When it first happened, in my old map, the player was getting outside a building (cube box with doors) by opening doors, and the zombies were downhill. On this map, I walked the dock to get to the crawler so I was inside the navmesh perimeter each time. The navmesh was generated with the default values.

By enabling the collisions on the vegetation layer, it could affect the way the navmesh is processed? (BTW the navmesh was generated with the vegetation layer collision off) If you try the map again, and disable the collisions on the vegetation layer, you will see that there is there is no big update time when the crawler will detect the player.

EDIT: For the map I've sent you, I moved the 5 NPC so you don't have to walk to far away... On my test with this map, I was in the middle of the island when the first NPC appear...