In designing the new engine, I have found that there are three distinct types of optimization.

Streamlining
This is refinement. You make small changes and try to gain a small amount of performance. Typically, this is done as a last step before releasing code. The process can be ongoing, but suffers from diminishing returns after a while. When you eliminate unnecessary math based on guaranteed assumptions you are streamlining code. For example, a 4x4 matrix multiplication can skip the calculations to fill the right-most column if the matrices are guaranteed to be orthogonal (non-sheared).

Quality Degradation
This is when you downgrade the quality of your results within a certain tolerable level where it won't be noticed much. An example of this is using a low-resolution copy of a model when it is far away from the camera. Quality degradation can be pretty arbitrary, and can mask your true performance, so it's best to keep an option to disable this.

Architectural
By designing algorithms in a way that makes maximum use of hardware and produces the most optimum results, we can greatly increase performance. Architectural optimization produces groundbreaking changes that can be ten or 100 times faster than the old architecture. An example of this is GPU hardware, which produces a massive performance increase over software rendering. We're seeing a lot of these types of improvements in Leadwerks Game Engine 5 because the entire system is being designed to make maximum use of modern graphics hardware.