PhysX Fails, Newton Sails

[Josh]

This is why Leadwerks uses Newton Dynamics.

 

 

 

[Crazycarpet]

This test seems extremely bias to me... a lot of these problems would be fixed by making some minor adjustments, it looks more like the tester just fed the same parameters to each physics engine and expected the same results, and decided Newtons were the most "correct".

 

You look at almost every AAA game and it's going to use PhysX or Havok.. because they're extremely performant and extremely accurate in almost all situations. With basic rigidbody Newton is great, but past that it struggles in a lot of scenarios... not bashing Newton but to say it's better than PhysX and Bullet is just bias. They're all strong in some areas and weak with others.

 

I also don't think there's anything official about a test where you don't get the code to falsify his results. I can say with absolute certainty that all the issues this guy shows in the video are his own doing (or whoever made the test). How your physics behave is really implementation dependent to a huge degree.

[Josh]

This test seems extremely bias to me... a lot of these problems would be fixed by making some minor adjustments, it looks more like the tester just fed the same parameters to each physics engine and expected the same results, and decided Newtons were the most "correct".

If you want to disprove a test like this, the way you do that is by creating another test that refutes it. Simply saying "this is biased" is not a response.

 

You look at almost every AAA game and it's going to use PhysX or Havok.. because they're extremely performant and extremely accurate in almost all situations.

Both are well-known to be extremely unstable and problematic. I could show you hours and hours of footage but you would probably say that is "biased" too. It is possible that I may have made good decisions that the AAA studios got wrong. You have proof right in front of your eyes that Newton is much more stable and accurate. When did programmers become so small-minded that they ignore their own eyes and assume any decision a big corporation makes is the best? Sorry, but no marketing campaign is good enough to refute the laws of physics. It is well-known that PhysX relies on "magic numbers" and approximations to get the behavior they want, while Newton is actually based on the real equations of Newtonian physics, right down to conservation of angular momentum.

 

I also don't think there's anything official about a test where you don't get the code to falsify his results and he spells "secunds" wrong

English is not Julio's first language. He's from South America, I think.

[Crazycarpet]

If he truely wanted to show that his engine was more "correct" he could release his tests so people could point out the flaws.

 

"When did programmers become so small-minded that they assume any decision a big corporation makes is the best?" - Well, I counter this with: When did programmers become so small-minded that they assume their decision is correct over that of the masses? When did we start taking an falsifiable test as fact, when did we throw out all the other tests that suggest the opposite of what these tests suggest?

 

Does Julio give us any other information? he's profiling performance but he's not telling us whether the simulation is CPU bound or not, it's common knowledge that PhysX is slow when CPU bound the beautiful thing about it is generally it's not it allows for simulation on the GPU as well. I could find you 100 other tests that show Bullet to be the best, and 100 other tests to show that Havok is the best, and another 100 to show that PhysX is the best. But are they? no they're the best in each test because of the implementation.

 

You need to remember big game studios don't just "make a decision" for example Unity started with Havok, then Bullet, then decided PhysX worked the best for their needs. To say "Newton is the best and most accurate" is not only naive, it's wrong.

 

The proof isn't right infront of my eyes, his test (which is bias just like every other test that claims one physics engine trumps the rest) shows that for his implementation of each physics engine to do the same thing, he produced the best results with Newton. Who's to say I wouldn't be capable of doing that with any other physics engine? I know for a fact that soft body physics aren't "so soft that the joints fall apart". That's simply untrue, that happens because his implementation.

[Josh]

"When did programmers become so small-minded that they assume any decision a big corporation makes is the best?" - Well, I counter this with: When did programmers become so small-minded that they assume their decision is correct over that of the masses? When did we start taking an falsifiable test as fact, when did we throw out all the other tests that suggest the opposite of what these tests suggest?

Where are these tests that show PhysX to be more stable than Newton? That's what I'm asking for.

[Crazycarpet]

Where are these tests that show PhysX to be more stable than Newton? That's what I'm asking for.

 

It's fairly hard to find comparisons against Newton I'll give you that, but you have find comparisons between almost every other physics engine except Newton because they're more popular. I'm not taking away from Newton or saying it's not great... it is. I'm saying it's naive to say it's the best because of a demo that was released by Newton Dynamics itself.

 

Am I saying Newton isn't more stable in many scenarios? No... am I saying it's not more stable in ALL scenarios, of course it's not.

What I'm saying is that this test proves nothing in that regard other than the fact that Julio has made a great, competitive and stable physics engine.

 

The only tests I can find between PhysX and Newton were carried out by Newton developers and the source code is not provided, there's nothing scientific about making a statement like that when it's nothing more than hearsay to say it's true. I'm not trying to say either way what the results would be of a more "fair" test, I'm simply saying this isn't it, and if it is provide people with the source code so they can see for themselves this is the case.

 

As you can see in any demo ever made like this one or the next one, or countless others:

(Last video is a soft-body stress test, the chain holds up! )

 

The things Julio's test shows Bullet and PhysX as failing these tests, hundreds of other demos show the opposite. The tests fail in Julio's implementation, not because of the underlying physics engine. I'm no expert but those Keva planks look fairly stable and balanced to me, and they're stacked much higher than in the other demo.

[Josh]

Those tests actually prove my point. Notice the whole stack turns into a box of water particles the moment the ball makes contact with the structure? Physics are disabled until the collision occurs. This is a common trick the other engines use to make videos like this.

 

PhysX and Bullet do not allow stable stacking anywhere close to what Newton does.

 

I asked Julio for the source code to his tests. I think it is available somewhere, but I forget where.

[Crazycarpet]

Those tests actually prove my point. Notice the whole stack turns into a box of water particles the moment the ball makes contact with the structure? Physics are disabled until the collision occurs. This is a common trick the other engines use to make videos like this.

 

I asked Julio for the source code to his tests. I think it is available somewhere, but I forget where.

 

This is simply not true Josh.... Anyone who's ever worked with Bullet or PhysX can attest to this fact. If the physics engine wasn't capable of that it'd be garbage. And how will you argue with the softbody that seems to (and does) work flawlessly in a VERY old version of Bullet, yet in Julio's test with a newer version it somehow fails from a slight tug on the soft-body chain? That's just not correct...

 

I do agree with your point that they may have physics disabled to a point sure, but the second that ball starts moving that stack is active, and if its not... there's still hundreds of examples where no collision occurs and they stay balanced I can find you them if you'd like. You can't spit on all these AAA physics engines with proven track records because of one video that is undeniably biased.

[Josh]

This is simply not true Josh.... Anyone who's ever worked with Bullet or PhysX can attest to this fact if the physics engine wasn't capable of that it'd be garbage. And how will you argue with the softbody that seems to (and does) work flawlessly in a VERY old version of Bullet, yet in Julio's test with a newer version it somehow fails from a slight tug on the soft-body chain? That's just not correct...

Newton does not support soft bodies at this time. There is an early implementation in there, but I don't think it's high priority for him, and I would not consider it finished or usable.

[Crazycarpet]

I know, I'm trying to make a point though that the tests are biased... and that video really does show it, like lol... neither Newton nor PhysX fails in the situations it fails for in that video.

 

One thing I will blindly concede to though is the "high mass ratio" test, at least when it comes to Bullet physics, this situation is handled poorly by Bullet, but on the flip side how often will you have a situation like that? Doesn't make a lot of sense to have in the real time simulation.. I'd like more to see a test of an object of huge mass accelerating into an object of lower mass. This is the behavior you'd see in real time and it'd be an interesting result to see!

 

The video also makes reference to "their own test" this implies that Bullet/Nvidia had a part in programming the implementations for these tests and I'm fairly sure they didn't.

 

This video shows it better:

 

The stack doesn't simply collapse after it's activated! even missing half it's pieces it maintains it's integrity... many of these tests have source code in the description too and you can see that in many cases the physics isn't actually disabled at any point. The "fluidness" you see isn't the structure collapsing because it was just activated, it's the force transferring from the heavy balls through each rigidbody object causing the structure to collapse in a quite physically accurate manner.

 

Like I said, those tests show that Newton is great with rigidbody (and experimental softbody) nothing more... to make judgements about Bullet or PhysX from them is just inaccurate.

[Josh]

Here's the link to the demo. Those are actually Nvidia's own tests:

http://www.newtondynamics.com/downloads/PEEL-master.rar

[Crazycarpet]

Here's the link to the demo. Those are actually Nvidia's own tests:

http://www.newtondynamics.com/downloads/PEEL-master.rar

 

Thanks, will check it out I'm curious now. This has been updated to the newest Newton version right?

[Josh]

Thanks, will check it out I'm curious now. This has been updated to the newest Newton version right?

I think this is still on Newton 3.13. 3.14 is supposedly a lot faster, but not really important for this.

[Josh]

Use the enter and arrow keys in the 3D window to run a demo.

 

Julio even has an email from Nvidia:

Newton seems to be a lot more efficient with compound shapes than PhysX...Newton handles interpenetration much better than PhysX...All of the demos involving constraints where there are long chains or, especially the 'nets', the behavior of Newton looks *much* better.

-John Ratcliff, NVIDIA Principal Engineer

 

https://blogs.nvidia.com/blog/author/jratcliff/

 

From: John Ratcliff [mailto:jratcliffscarab@gmail.com]

Sent: Thursday, June 06, 2013 10:48 PM

To: Julio Jerez; Jerez, Julio

Subject: Re: PEEL tests

 

Tomorrow I will add the Bullet peel plugin to the depot so we can add it to the data set.

On Jun 7, 2013 12:34 AM, "John Ratcliff" <jratcliffscarab@gmail.com> wrote:

Julio,

 

I'm about to head to bed now; but I've been going through some of the tests; which I assume you have done as well.

 

Here are some things I have noticed

 

#1 : Newton seems to be a lot more efficient with compound shapes than PhysX. This is especially noticeable in test #59 and #60

 

#2 : Newton handles interpenetration much better than PhysX.

 

#3 : Newton seems to have a much more aggressive sleep algorithm than PhysX; as can be seen in large box stack tests where PhysX falls over but Newton puts them to sleep fairly quickly. This isn't necessarily a good or bad thing; it's usually just a tuning parameter.

 

#4 : All of the demos involving constraints where there are long chains or, especially the 'nets', the behavior of Newton looks *much* better. The performance is lower, but that's almost for sure simply because the solver-iteration count on the PhysX constraint is too low. I'm sure if we increased the solver-iteration count on the PhysX constraint to make the behavior match more closely to the quality of the behavior in Newton the performance would be much more close.

 

#5 : The ragdoll demo looks better in PhysX than in Newton. This may be just because you haven't fully hooked up all of the constraint properties yet? The PhysX constraints have a lot of spring/softness to them that tend to make ragdolls look a little nicer.

 

#6 : In general Newton seems to be very performance competitive with PhysX until you start hitting extreme situations of massive numbers of dynamic objects all in a giant pile. I'm going to assume this is what you meant by an artificial demo. I agree this isn't necessarily that big of a deal.

 

As we discussed earlier, you have to be careful about drawing too many conclusions too quickly, since many differences could be due to iteration counts or other kinds of tuning parameters.

 

These are just some things that I have observed so far.

 

You are putting a lot of work into getting Newton to work with PEEL; I think we should discuss a plan to present the data when we are finished. Perhaps something on either your website or my blog. Something which is diplomatic and complimentary.

 

In my personal view, Newton is a very impressive high quality physics engine with excellent behavior and performance. The fact that it is even competitive with commercial physics engines that have had dozens of engineers working on them for years is really an incredible testament to your talent and dedication.

 

I think you have done an amazing job, and we should communicate that to the public as well.

 

Great work.

 

John

[Crazycarpet]

Well can't argue with that, PEEL shows via those demos that Newton is truely the best in those demos, on all of my computers which have very different hardware....

 

The Bullet physics results in most tests are disappointing.

However overall you've got to hand it to PhysX in most scenarios it blows everything else out of the water... Even on my computer without a dedicated GPU.

I've got to say I never thought Newton was THAT good of a physics engine, very underrated i must admit. You do tend to pay in most demos in performance for Newtons accuracy though.

 

The results I got from running peel showed that:

- Overall, Newton gives PhysX a run for it's money in accuracy in most cases although there are still quite a few where PhysX surpasses it. (Both great physics engines.) (Will link the tests PhysX wins in a sec.)

- When it comes to accuracy of simple rigidbody collisions Newton always is the most accurate by far.

- Overall, PhysX performs the best.

- Bullet gets blown out of the water when it comes to accuracy but slightly outperforms Newton in most cases. (I'd take accuracy over performance personally, and it's not like the difference is big.)

- I still can't reproduce the failing soft-body chain on any machine, it works for all engines on all my PCs in PEEL.

 

Of course this is a big generalization but overall, this is what I noticed... those demos from Julio are correct, can't argue with that.

 

Another thing I found interesting about this demo is the fact that on my computer with an on-board chip operations like raycasting are EXTREMELY expensive and PhysX is by far the slowest.... but on my computer with a powerful dedicated GPU these same operations are by far the fastest on PhysX.

 

Will definitely be using Newton in my next project.

[macklebee]

Just as an aside here - but Julio is not some random guy that has no experience with any other physics engines other than his own. He was (possibly still is) one of the main physics programmers for Planetside2 multiplayer that uses PhysX. Point being - the guy knows his sh.it.

 

 

http://www.metacritic.com/person/julio-jerez?filter-options=games

 

http://www.mobygames.com/developer/sheet/view/developerId,7571/

[Crazycarpet]

Oh for sure, I wasn't disputing whether or not he knows his sh*t I mean clearly he's extremely talented. I was just saying that I wouldn't take a test like that at face value as I didn't realize what PEEL was at the time, I thought it was a self-made demo.

Certainly not trying to say anything negative about a programmer who's capable of creating such a robust physics engine.

 

Neat video though.

[Josh]

Thanks for the debate. I did need to find that demo. I plan to record some new videos from it and put them on our channel.

[Pierre]

Hi there,

 

I'm the guy who wrote PEEL. These videos (and Julio's PEEL master branch) unfortunately use an old work-in-progress version of PhysX 3.4. You can grab a more recent build here: https://github.com/Pierre-Terdiman/PEEL

 

There is a long history regarding Newton in PEEL. Let's just say there are two sides to every story.

 

There are also incorrect claims about PhysX in this thread, that I would be happy to talk about if people are interested. Most of the stuff I read online about PhysX is wrong

 

Cheers,

 

- Pierre

[Josh]

Hi there,

 

I'm the guy who wrote PEEL. These videos (and Julio's PEEL master branch) unfortunately use an old work-in-progress version of PhysX 3.4. You can grab a more recent build here: https://github.com/Pierre-Terdiman/PEEL

 

There is a long history regarding Newton in PEEL. Let's just say there are two sides to every story.

 

There are also incorrect claims about PhysX in this thread, that I would be happy to talk about if people are interested. Most of the stuff I read online about PhysX is wrong

 

Cheers,

 

- Pierre

Ah cool, thanks for dropping in! You made a really great tool that has obviously sparked a lot of discussion.

[Pierre]

- Overall, Newton gives PhysX a run for it's money in accuracy in most cases although there are still quite a few where PhysX surpasses it. (Both great physics engines.) (Will link the tests PhysX wins in a sec.)

- When it comes to accuracy of simple rigidbody collisions Newton always is the most accurate by far.

- Overall, PhysX performs the best.

 

Note that the performance / accuracy trade-off of each engine can be changed. What you get with each engine's default parameters is not the whole story.

 

All engines use approximate "linear solvers", so both the performance and accuracy will depend on the number of iterations you choose. Then, they don't all use the same algorithms under the hood: you can have Gauss-Seidel, Jacobi, conjugate gradient, etc. These methods have pros and cons, and in particular they don't all converge at the same rate. So using the same number of iterations for all engines is not guaranteed to produce similar results in terms of accuracy.

 

It was a bit of a problem for me in PEEL: what default parameters should I use? Bullet for example uses 10 iterations by default these days, while PhysX uses 4. So if use 4 iterations for everybody "to be fair", I'm artificially decreasing the accuracy of Bullet compared to what a fresh user would get out-of-the-box, using its default parameters. And on the other hand, if I use Bullet's default values, I can be accused of being unfair since some engines will now use more iterations than others. This is a tricky thing.

 

In any case:

 

- you can improve PhysX's accuracy by increasing the number of iterations in the UI. This will make it slower.

 

- you can improve Newton's performance by decreasing the number of iterations in the UI. This will make less accurate.

 

It's all a trade-off depending on what your game/app needs. These things are not black & white. At the end of the day all these engines are fairly mature and quite similar.

 

Now as far as PhysX is concerned, extra iterations there seem cheaper than for other engines. So for that torus scene, for example, you can increase the iterations quite a bit (say from 4 to 32) and get a behavior that become closer to Newton's (while possibly still being cheaper, YMMV depending on which versions you're using). In the latest released PEEL (1.1), I added something in the PhysX UI to configure it either for performance or accuracy. This changes the results quite a bit for some of the scenes, in particular for the ones that Julio cherry-picked in his videos

 

 

Another thing I found interesting about this demo is the fact that on my computer with an on-board chip operations like raycasting are EXTREMELY expensive and PhysX is by far the slowest.... but on my computer with a powerful dedicated GPU these same operations are by far the fastest on PhysX.

 

I would be interested to hear about this. Which scene are you talking about? In my experience PhysX is pretty much always the fastest for "scene queries" (raycasts, overlaps, sweeps). And since these parts run entirely on the CPU, it should make no difference if you have a powerful GPU or not.

[Pierre]

I'd like more to see a test of an object of huge mass accelerating into an object of lower mass. This is the behavior you'd see in real time and it'd be an interesting result to see!

 

Incidentally, I added a test just like that in PEEL 1.1

 

It's called "HighMassRatioCollisions", should be test #16 in the released build.

[Pierre]

And how will you argue with the softbody that seems to (and does) work flawlessly in a VERY old version of Bullet, yet in Julio's test with a newer version it somehow fails from a slight tug on the soft-body chain? That's just not correct...

Is the "soft-body chain" you mention the chain made of connected fixed-joint torus? It's not soft-body, it only looks soft because the engines use linear /approximate solvers that didn't converge to the exact solution. It's supposed to be all rigid in theory.

 

In any case the behavior here will depend a lot on the number of iterations I previously mentioned. It is possible that Julio's video (which might be from 2013 if it's as old as John's email) used a different number of iterations compared to his latest build. Different Bullet versions also use different number of iterations by default (IIRC they moved from 5 to 10), and it will also ultimately depend on the number of iterations I used in the Bullet's setup code, in PEEL.

 

Generally speaking I wouldn't trust a video showing something *failing*, more often than not you can make things work by just tweaking the parameters (as least as far as the PEEL scenes are concerned) or using the proper engine feature.

[Pierre]

PhysX and Bullet do not allow stable stacking anywhere close to what Newton does.

I am not sure what you are referring to here.

 

You can improve the stacking behavior in most engines by just increasing the number of solver iterations.

 

Also, last time I tested it (at the time of the PEEL 1.1 release), Newton 3.14 had an issue with the "sleeping" algorithm (the thing that deactivates objects when they are not moving much). By default these mechanisms are disabled in PEEL, so that I can have a look at the real performance / stability of each engine. That is, nothing ever sleeps, nothing is ever deactivated (so that's not what you'd get in a real game). However the API for controlling this seemed broken in Newton 3.14, i.e. I couldn't deactivate sleeping anymore (compared to Newton 3.13 for example). That gave Newton 3.14 a bit of an unfair advantage in some scenes IIRC, since it was the only engine for which things went to sleep. Box stacks typically go to sleep immediately, so maybe it could have an effect here.

 

In my experience all these engines handle stacking more or less equally well for the same CPU time budget

[Pierre]

it's common knowledge that PhysX is slow when CPU bound

FWIW, most of the "common knowledge" I see online about PhysX.... is wrong

 

But you are right that none of these engines is "the best". There's performance, memory usage, stability, accuracy, customer support, supported platforms, the API, the documentation, the features, the community, the source code quality, the API stability, and dozens of other aspects, each of them with shades of grey anyway. These engines have pros & cons, strengths & weaknesses, etc. Anybody telling you "mine is the best" is either trying to sell you something, or a fanboy of one particular engine.