Video playing example

[Niosop]

Here's a minimal example of how to play a video on a texture.

 

First download libtheoraplayer. It's what does all the heavy lifting as far as ogg/theora decoding goes. And if you've ever tried to write your own theora decoder you know it is a fair amount of lifting. You'll need to link against the .lib file and have the .dll present in the same directory as your exe.

 

You'll also need an ogg theora video, change the name/path to whatever you use.

 

Here's the C code:

 

#include "engine.h"
#include <gl/gl.h>
#include "TheoraVideoManager.h"
#include "TheoraVideoFrame.h"

int nextPow2(int x)
{
int y;
for (y=1;y<x;y*=2);
return y;
}

int main( int argn, char* argv[] )
{
       Initialize() ;
       RegisterAbstractPath(".");
       SetAppTitle( "Video Test" ) ;
       Graphics( 800, 600) ;
       AFilter() ;
       TFilter() ;

       TCamera camera;

       TFramework framework=CreateFramework();
       TLayer layer = GetFrameworkLayer(0);
       camera=GetLayerCamera(layer);

	SetSSAO(0);
	SetBloom(0);
	SetHDR(0);
	SetAntialias(1);
	SetGodRays(0);

	PositionEntity(camera,Vec3(0,0,-4));

	TMesh cube = CreateCube(0);
	PositionEntity(cube, Vec3(0, 0, 0));

	TheoraVideoClip *clip;
	TheoraVideoManager *mgr;
	TBuffer currentbuffer;

	mgr=new TheoraVideoManager();
	clip=mgr->createVideoClip("bunny.ogg", TH_RGB, 0, 1);
	clip->setAutoRestart(1);

	float w=clip->getWidth(),h=clip->getHeight();
	float tw=nextPow2(w),th=nextPow2(h);

	TBuffer testbuffer = CreateBuffer(tw,th,BUFFER_COLOR);
	TTexture testtexture = CreateTexture(tw,th,TEXTURE_RGB);
	TMaterial testmaterial = LoadMaterial("abstract::video.mat");
	TextureFilter(testtexture, TEXFILTER_PIXEL);
	SetMaterialTexture(testmaterial, testtexture);

	PaintEntity(cube, testmaterial);

	unsigned long time=GetTickCount();
	unsigned long t=time;

       while( !KeyHit() && !AppTerminate() )
       {
		if( !AppSuspended() ) // We are not in focus!
           {

			TurnEntity(cube, Vec3(0.1*AppSpeed(),0.1*AppSpeed(),0.1*AppSpeed()));
			TheoraVideoFrame* f=clip->getNextFrame();

			if(f) 
			{
				currentbuffer = CurrentBuffer();
				SetBuffer(testbuffer);

				BindTexture(testtexture);
				glTexImage2D(GL_TEXTURE_2D,0,GL_RGB,w-1,h-1,0, GL_RGB,GL_UNSIGNED_BYTE,f->getBuffer());
				clip->popFrame();

				SetBuffer(currentbuffer);
			}

			t=GetTickCount();
			float diff=(t-time)/1000.0f;
			if (diff > 0.25f)
				diff=0.05f; // prevent spikes (usually happen on app load)
			mgr->update(diff);
			time=t;

			// Update timing and world
			UpdateFramework();

               // Render
               RenderFramework();

			// Send to screen
			Flip(0) ;
		}
       }

       // Done
       return Terminate() ;
}

 

The video.mat file is:

 

blend=0
depthmask=1
depthtest=1
overlay=0
zsort=0
cullface=1
castshadows=1
specular=1.00000000
bumpscale=1.00000000
gloss=0.500000000
shader="abstract::mesh.vert","abstract::mesh_diffuse_fullbright2.frag"
shadowshader="abstract::mesh_shadow.vert",""

 

The mesh_diffuse_fullbright2.frag shader is the same as the mesh_diffuse_fullbright.frag shader but without the bloom:

 

#define LW_DIFFUSE texture0
#define LW_FULLBRIGHT

include "mesh.frag"

 

Your can use any material/shader combo that you want for different effects.

 

Some possible improvements:

 

I'm pretty sure the SetBuffer commands really aren't needed, but something that SetBuffer does IS required. I just don't know all the openGL commands that SetBuffer issues, so not sure which ones are required. Probably just resetting some state or something. It doesn't seem to hurt performance much using SetBuffer so I'm not too worried about it, but it might eat up additional VRAM having it sitting around when it's not needed.

 

libtheoraplayer uses a separate thread to do the video decoding and YUV->RGB conversion so the amount of slowdown you see when playing a video will depend on if you have a multi-core machine. Typically the YUV->RGB conversion process takes longer than the actual video decoding, so for slower or single core machines requesting the data in YUV and using a shader to do the YUV->RGB conversion could result in a significant speedup. I'll take a look at doing this if there's any demand. On my machine I actually see no framerate drop at all when playing a video as opposed to just rendering the spinning cube.

 

I'm using GetTickCount instead of AppTime because it seems to offer higher resolution timing, but I haven't tried AppTime since I did some changes, so it might work just as well now.

 

Need to get lower level access to the sound data buffers that LeadWerks uses so that we can reliably sync the video and audio data. It seems that many libraries use the audio system to set the timing and base the video position off of where the audio system is in the audio buffer. Josh, any recommendations?

 

Big thanks to Josh for pointing out the BindTexture command, it's what made updating the texture at a reasonable speed possible.

 

As always, comments and suggestions are welcome.

[Marleys Ghost]

Excellent and very generous of you to share this I will have a play with that now .. I think I have the libtheoraplayer somewhere on the HDD's .. Thank you.

[Niosop]

If not it's available at http://libtheoraplayer.sourceforge.net/.

[Marleys Ghost]

If not it's available at http://libtheoraplayer.sourceforge.net/.

 

 

Thanks for that Nio, its probably the "quicker" otpion than hunting though the mess that is my storage drive

[Rick]

glTexImage2D(GL_TEXTURE_2D,0,GL_RGB,w-1,h-1,0, GL_RGB,GL_UNSIGNED_BYTE,f->getBuffer());

 

 

That looks to be the only gl command you are calling. What exactly is that doing and was there no LE way of doing it? I'd like to try and get a lua object around this but don't think I can use the gl commands in lua.

[Rick]

Also, where did you get a video to work with?

[Canardia]

You can probably convert any video with the free NCH Video Converter ( http://www.nchsoftware.com/prism/index.html ), and you can download any video from YouTube.

[Niosop]

You can do a search for ogg sample video or convert one, or download the full version of Big Buck Bunny in Ogg Theora format.

 

glTexImage2D(GL_TEXTURE_2D,0,GL_RGB,w-1,h-1,0, GL_RGB,GL_UNSIGNED_BYTE,f->getBuffer());

 

 

That looks to be the only gl command you are calling. What exactly is that doing and was there no LE way of doing it? I'd like to try and get a lua object around this but don't think I can use the gl commands in lua.

 

It takes the decoded RGB from a libtheoraplayer buffer and copies it over to memory referenced by the LE texture. The BindTexture command seems to do the same as glBindTexture, with maybe some additional housekeeping. There's no way to do it via pure LE API commands.

[Pixel Perfect]

I've been blitting animated image sequences to planes for a while but this will make life much easier. Thanks for sharing the code.

[Niosop]

Yeah, I'm thinking that a looping video of a waterfall would make a really good texture for a waterfall.

[TylerH]

I will work on modifying this for you to not use the SetBuffer commands, but it WILL introduce a few more direct OpenGL calls, but speed is the key here, not the comamnd count.

 

@Rick: There is a set of OpenGL scripts for Lua that expose it. I believe Lumooja tested them out before and verified they worked, but I am not 100% sure.

[TylerH]

Ok, I tested loading it in passing TH_YUV to use the default YUV encoding, and convert it to RGB colorspace in the mesh.frag shader.

 

I would like to note that I got 530 FPS using the TH_RGB with no conversion via shader code, and 500 or lower FPS using the conversion.

 

When you think about it, the mesh.frag shader is going to run much slower because of how large the video resolution is, etc. So it becomes a factor that depends on a lot of variables.

 

Using the shader-based approach really seems like overkill, when with only 1 worker thread LibTheoraPlayer seems to load right into RGB pretty darn fast.

[Niosop]

Yup, the only time a shader based approach would make sense is when running on a single core machine where the decoding thread would steal cycles from the engine, then you might want to offload that to the video card. But that's just a guess. Anyone have a single core machine around to test with?

[Shard]

Thank you for uploading this, you are a life (and Project Grade) saver. Is this library usable for commercial games for us to make and sell, in case we ever decide we want to pursue making a commercial version of our game in the future?

[deniz]

Here's a minimal example of how to play a video on a texture.

 

thanks for sharing your code Niosop!

I joined this conversation a little bit late but still I want to ask a question:

why did you define the "testbuffer" buffer?

it seems that you are using it just before calling glTexImage2D() and nowhere else..