Package Compression Speed

[SpiderPig]

Just curious, is loading a compressed package slower than loading an uncompressed package?  Somewhere along the line the data has to be un-encrypted and uncompressed doesn't it?

[Josh]

Yes, it would be slower, but I don't think the difference is significant.

[Ma-Shell]

You also need to take into consideration that you need to read less data from the harddrive and IO is usually quite a large factor in loading times, so on a slow harddrive you might even see better loading times with a compressed package.

[gamecreator]

Enough theory!  Let's get real!  ?  I tested this.

I created a 2048x2048 DXT5 texture (with mipmaps) and copied it to create 90 textures, totaling 480MB.  I then zipped them in both storage (uncompressed) mode (480MB) and "best" mode (246MB).  I then wrote a quick program:

#include "Leadwerks.h"
#include "conio.h"

using namespace Leadwerks;

Texture* texture[90];

int main(int argc,const char *argv[])
{
    //Load any zip files in main directory
    Leadwerks::Directory* dir = Leadwerks::FileSystem::LoadDir(".");
    if (dir)
    {
        for (int i=0; i<dir->files.size(); i++)
        {
            std::string file = dir->files[i];
			std::string ext = Leadwerks::String::Lower(Leadwerks::FileSystem::ExtractExt(file));
            if (ext=="zip" || ext=="pak")
            {
                Leadwerks::Package::Load(file);
            }
        }
        delete dir;
    }

	long starttime = Leadwerks::Time::Millisecs();

	for(int i=0; i<90; i++)
	{
		string filename=std::to_string(i)+".tex";
		cout << filename << endl;
		texture[i]=Texture::Load(filename);
	}

	float totaltime = ((float)Leadwerks::Time::Millisecs()-(float)starttime)/1000;

	printf("\nLoad time: %f\n",totaltime);
	getch();
}

I then put in only the uncompressed zip, ran the program, then swapped it with the compressed and ran it again, multiple times.  Here are the results:

uncompressed (SSD)
2.284 seconds
2.257
2.238
2.236
2.232
2.241
2.235
2.231
2.241
2.250

"best" compressed (SSD)
3.748
3.865
3.811
3.871
3.603
3.650
3.802
3.790
3.668
3.748

uncompressed (HD)
2.145
2.150
2.136

"best" compressed (HD)
3.619
3.646
3.661

Couldn't tell you why the non-SSD is slightly faster.  Out of curiosity I also rebooted my computer to run the program clean and uncompressed+HD came in at over 8 seconds but I'm not confident that the computer stopped loading everything in the background (task manager was calm though).

Edit: I just did two more reboot tests.  Obviously when you first run the program, nothing is already in memory so things take longer.  Compressed: 7.254 seconds. Rebooted again and did uncompressed: 9.310 seconds (yes, compressed was faster here).

[SpiderPig]

I'll run the same test tonight and see how it compares ?

 

It makes sense that a small compression will be quicker to read because of less IO operations.

 

Without compression my SSD is 2x faster than the HD.

[gamecreator]

Curious about your results.  Attached is the tex file if you want to compress 90 of the same one.

0.tex