Corrupt or missing data in bitstream; continuing...

[Roberto14]

hi all,

 

as title say i have this message when i try to load my ogg fils.

 

I want premise that i was using a third-party lib, https://github.com/R4stl1n/cAudio,

which my ogg files were load without any problem.

( I was using this lib because his feature od 3D spatialization for sounds )

 

Said this, after last update, compiling my project i got some error about ogg header inclusion, and coming to forum i have seen that new ogg support feature were added to engine.

 

At this time i remove every trace of this lib from my project and now compilation will accomplish, but when i try to load a ogg file, i have the message on title writen in console and a program freeze of about 4 ~ 7 sec per file. Of course sound are not loaded and nothing works.

 

Is there a solution for this??

 

PS: I use Nero Wave Editor to convert an audio file to ogg.

 

PS PS:

According to this file:

https://svn.xiph.org/trunk/vorbis/examples/decoder_example.c

 

My problem ocCurs when:

 

int result=ogg_sync_pageout(&oy,&og);
if(result<0){ fprintf(stderr,"Corrupt or missing data in bitstream; continuing...\n");

 

and at https://xiph.org/ogg/doc/libogg/ogg_sync_pageout.html

 

explaination says:

 

Returned value:

-1 returned if stream has not yet captured sync (bytes were skipped).

[Roberto14]

Any news??

[martyj]

Can you send us an ogg file that we can test on?

 

Edit:

--------

 

I just ran roughly the same code on 100 ogg files that were used in my game generated with the linux command "sox" and I haven't hit any issues. I'll need a test file to find the problem.

[Josh]

Here is my current sound loading code. This is much faster since it does not constantly resize the data buffer.

	bool Sound::Reload(const int mode)

{

Reset();

 

//Read the file

Stream* stream = FileSystem::ReadFile(path);

if (!stream) return false;

 

if (String::Lower(FileSystem::ExtractExt(path)) == "ogg")

{

ogg_sync_state oy; /* sync and verify incoming physical bitstream */

ogg_stream_state os; /* take physical pages, weld into a logical

stream of packets */

ogg_page og; /* one Ogg bitstream page. Vorbis packets are inside */

ogg_packet op; /* one raw packet of data for decode */

vorbis_info vi; /* struct that stores all the static vorbis bitstream

settings */

vorbis_comment vc; /* struct that stores all the bitstream user comments */

vorbis_dsp_state vd; /* central working state for the packet->PCM decoder */

vorbis_block vb; /* local working space for packet->PCM decode */

char *buffer;

int bytes;

 

#define LE_OGG_BUFFSIZE 4096

ogg_int16_t convbuffer[LE_OGG_BUFFSIZE]; /* take 8k out of the data segment, not the stack */

int convsize = LE_OGG_BUFFSIZE;

int data_offset = 0; // Bank buffer write offset

 

int total = 0; // Used for file size

 

int eos = 0;

int i;

 

bool streamInit = false;

 

this->data = new Bank();

 

/********** Decode setup ************/

ogg_sync_init(&oy); /* Now we can read pages */

 

while (!stream->EOF())

{

// Start Header Parsing

eos = 0;

 

buffer = ogg_sync_buffer(&oy, LE_OGG_BUFFSIZE);

bytes = StreamRead(stream, buffer, LE_OGG_BUFFSIZE);

 

ogg_sync_wrote(&oy, bytes);

 

/* Get the first page. */

if (ogg_sync_pageout(&oy, &og) != 1)

{

if (bytes < LE_OGG_BUFFSIZE)

{

break; // EOF

}

 

System::Print("Error: Input not a Vorbis bitstream.");

goto fail;

}

 

/* Get the serial number and set up the rest of decode. */

/* serialno first; use it to set up a logical stream */

ogg_stream_init(&os, ogg_page_serialno(&og));

 

/* extract the initial header from the first page and verify that the

Ogg bitstream is in fact Vorbis data */

 

/* I handle the initial header first instead of just having the code

read all three Vorbis headers at once because reading the initial

header is an easy way to identify a Vorbis bitstream and it's

useful to see that functionality seperated out. */

 

vorbis_info_init(&vi);

vorbis_comment_init(&vc);

if (ogg_stream_pagein(&os, &og) < 0) /* error; stream version mismatch perhaps */

{

System::Print("Error: Error reading first page of Ogg bitstream data.");

goto fail;

}

 

if (ogg_stream_packetout(&os, &op) != 1) /* no page? must not be vorbis */

{

System::Print("Error: Error reading initial header packet.");

goto fail;

}

 

if (vorbis_synthesis_headerin(&vi, &vc, &op) < 0) /* error case; not a vorbis header */

{

System::Print("Error: Ogg bitstream does not contain Vorbis audio data.");

goto fail;

}

 

/* At this point, we're sure we're Vorbis. We've set up the logical

(Ogg) bitstream decoder. Get the comment and codebook headers and

set up the Vorbis decoder */

 

/* The next two packets in order are the comment and codebook headers.

They're likely large and may span multiple pages. Thus we read

and submit data until we get our two packets, watching that no

pages are missing. If a page is missing, error out; losing a

header page is the only place where missing data is fatal. */

 

i = 0;

while (i < 2)

{

while (i < 2)

{

int result = ogg_sync_pageout(&oy, &og);

if (result == 0)

{

break; /* Need more data */

}

 

/* Don't complain about missing or corrupt data yet. We'll catch it at the packet output phase */

if (result == 1)

{

/* we can ignore any errors here as they'll also become apparent at packetout */

ogg_stream_pagein(&os, &og);

while (i < 2)

{

result = ogg_stream_packetout(&os, &op);

if (result == 0)

{

break;

}

 

if (result < 0)

{

/* Uh oh; data at some point was corrupted or missing!

We can't tolerate that in a header. Die. */

System::Print("Error: Corrupt secondary header.");

goto fail;

}

 

result = vorbis_synthesis_headerin(&vi, &vc, &op);

if (result<0)

{

System::Print("Error: Corrupt secondary header.");

goto fail;

}

 

i++;

}

}

}

 

/* no harm in not checking before adding more */

buffer = ogg_sync_buffer(&oy, LE_OGG_BUFFSIZE);

bytes = StreamRead(stream, buffer, LE_OGG_BUFFSIZE);

if (bytes == 0 && i < 2)

{

System::Print("Error: End of file before finding all Vorbis headers!");

goto fail;

}

ogg_sync_wrote(&oy, bytes);

}

// End Headers

 

// @ Josh, what do we do about variable bitrate? This code will fail if there is variable Ogg Vorbis streams.

if (streamInit)

{

if (this->channels != vi.channels || this->frequency != vi.rate)

{

System::Print("Error: Variable bitrate and channels are not supported.");

goto fail;

}

}

else

{

this->channels = vi.channels;

this->frequency = vi.rate;

this->bitrate = 16; // 16 bit stream as defined in PCM parsing below

streamInit = true;

 

switch (this->channels)

{

case 1:

{

this->format = Mono16;

break;

}

case 2:

{

this->format = Stereo16;

break;

}

default:

{

System::Print("Error: Unknown channel.");

goto fail;

}

}

}

 

convsize = LE_OGG_BUFFSIZE / vi.channels;

/* OK, got and parsed all three headers. Initialize the Vorbis packet->PCM decoder. */

if (vorbis_synthesis_init(&vd, &vi) == 0) /* central decode state */

{

/* local state for most of the decode so multiple block decodes can proceed in parallel.

We could init multiple vorbis_block structures for vd here */

vorbis_block_init(&vd, &vb);

 

std::list<Bank*> chunks;

 

/* The rest is just a straight decode loop until end of stream */

while (!eos)

{

while (!eos)

{

int result = ogg_sync_pageout(&oy, &og);

if (result == 0)

{

break; /* need more data */

}

 

if (result < 0)

{

System::Print("Error: Corrupt or missing data in bitstream.");

continue;

}

 

ogg_stream_pagein(&os, &og);

while (1)

{

result = ogg_stream_packetout(&os, &op);

 

if (result == 0)

{

break; /* need more data */

}

 

if (result < 0) /* missing or corrupt data at this page position. No reason to complain */

{

continue;

}

 

/* we have a packet. Decode it */

float **pcm;

int samples;

 

if (vorbis_synthesis(&vb, &op) == 0) /* test for success! */

{

vorbis_synthesis_blockin(&vd, &vb);

}

 

/*

**pcm is a multichannel float vector. In stereo, for

example, pcm[0] is left, and pcm[1] is right. samples is

the size of each channel. Convert the float values

(-1.<=range<=1.) to whatever PCM format and write it out */

 

while ((samples = vorbis_synthesis_pcmout(&vd, &pcm)) > 0)

{

int j;

int clipflag = 0;

int bout = (samples < convsize ? samples : convsize);

 

/* convert floats to 16 bit signed ints (host order) and

interleave */

for (i = 0; i < vi.channels; i++)

{

ogg_int16_t *ptr = convbuffer + i;

float *mono = pcm;

for (j = 0; j < bout; j++)

{

#if 1

int val = floor(mono[j] * 32767.f + .5f);

#else /* optional dither */

int val = mono[j] * 32767.f + drand48() - 0.5f;

#endif

/* Guard against clipping */

val = max(min(val, 32767), -32768);

 

*ptr = val;

ptr += vi.channels;

}

}

 

// Resize must be terrible for performance. Except I don't see a way around this as there is no way to get the length of the ogg file.

int size = 2 * vi.channels * bout;

//this->data->Resize(total + size);

//std::memcpy(this->data->buf + total, convbuffer, size);

Bank* chunk = Bank::Create(size);

std::memcpy(chunk->buf, convbuffer, size);

chunks.push_back(chunk);

total += size;

 

vorbis_synthesis_read(&vd, bout); /* tell libvorbis how many samples we actually consumed */

}

}

 

if (ogg_page_eos(&og))

{

eos = 1;

}

}

 

if (!eos)

{

buffer = ogg_sync_buffer(&oy, LE_OGG_BUFFSIZE);

bytes = StreamRead(stream, buffer, LE_OGG_BUFFSIZE);

ogg_sync_wrote(&oy, bytes);

if (bytes == 0)eos = 1;

}

}

 

this->data->Resize(total);

int offset = 0;

for (auto it = chunks.begin(); it != chunks.end(); it++)

{

std::memcpy(this->data->buf + offset, (*it)->buf, (*it)->size);

offset += (*it)->size;

(*it)->Release();

}

chunks.clear();

 

/* ogg_page and ogg_packet structs always point to storage in

libvorbis. They're never freed or manipulated directly */

 

vorbis_block_clear(&vb);

vorbis_dsp_clear(&vd);

}

else

{

Print("Error: Corrupt header during playback initialization.");

}

/* clean up this logical bitstream; before exit we see if we're

followed by another [chained] */

 

ogg_stream_clear(&os);

vorbis_comment_clear(&vc);

vorbis_info_clear(&vi); /* must be called last */

}

/* OK, clean up the framer */

ogg_sync_clear(&oy);

 

this->length = this->data->GetSize() / (float)((2 * this->frequency * this->channels));

 

delete stream;

return true;

 

fail:

if (this->data)

{

delete stream;

delete this->data;

this->data = NULL;

}

}

else

{

int chunkid;

short format_tag, block_align, bits_per_sample; //our 16 values

unsigned int chunksize, endpos, format_length, sample_rate, avg_bytes_sec, data_size, i, size; //our 32 bit values

int datasize;

 

/*

stream->ReadByte();

stream->ReadByte();

stream->ReadByte();

 

//Version

switch (stream->ReadInt())

{

case 1:

break;

default:

delete stream;

return false;

}

 

bitrate = stream->ReadInt();

channels = stream->ReadInt();

frequency = stream->ReadInt();

datasize = stream->ReadInt();

 

data = CreateBank(datasize);

stream->ReadBytes(data->buf,datasize);

 

length = ((float)datasize) / (float)(frequency * channels);

*/

 

if (stream->ReadInt() != 1179011410)//"RIFF" tag

{

System::Print("Error: Unrecognized tag.");

delete stream;

return false;

}

 

size = stream->ReadUInt();

 

if (stream->ReadInt() != 1163280727)//"WAVE" tag

{

System::Print("Error: Unrecognized tag.");

delete stream;

return false;

}

 

//Read subchunks

while (!stream->EOF())

{

//System::Print(stream->GetPos());

//System::Print(stream->GetSize());

chunkid = stream->ReadInt();

switch (chunkid)

{

case 544501094://"fmt " chunk

chunksize = stream->ReadUInt();

endpos = stream->GetPos() + chunksize;

format_tag = stream->ReadShort();

channels = stream->ReadShort();

//sample_rate = stream->ReadUInt();

frequency = stream->ReadUInt();

avg_bytes_sec = stream->ReadUInt();

block_align = stream->ReadShort();

bitrate = stream->ReadShort();

stream->Seek(endpos);

 

//Choose sample format

switch (bitrate)

{

case 8:

switch (channels)

{

case 1:

this->format = Mono8;

break;

case 2:

this->format = Stereo8;

break;

default:

System::Print("Error: Unsupported number of channels " + String(channels) + ".");

delete stream;

return false;

}

break;

case 16:

switch (channels)

{

case 1:

this->format = Mono16;

break;

case 2:

this->format = Stereo16;

break;

default:

System::Print("Error: Unsupported number of channels " + String(channels) + ".");

delete stream;

return false;

}

break;

default:

System::Print("Error: Unsupported bit rate " + String(bitrate) + ".");

delete stream;

return false;

}

 

break;

case 1635017060://"data" chunk

chunksize = stream->ReadUInt();

//endpos = stream->GetPos()+chunksize;

data = Bank::Create(chunksize);

stream->Read(data->buf, chunksize);

//stream->Seek(endpos);

length = chunksize / (float)(frequency * (bitrate / 8) * channels);

break;

case 1952670054://"fact" chunk

default://unknown chunk

chunksize = stream->ReadUInt();

int oldpos = stream->GetPos();

int pos = oldpos + chunksize;

if (chunksize)

{

pos = Math::Min(pos, stream->GetSize());

stream->Seek(pos);

}

break;

}

}

 

delete stream;

if (!data)

{

System::Print("Error: No data chunk found.");

return false;

}

return true;

}

/*

fread(&format_length, sizeof(DWORD),1,fp);

fread(&format_tag, sizeof(short), 1, fp); //check mmreg.h (i think?) for other

// possible format tags like ADPCM

fread(&channels, sizeof(short),1,fp); //1 mono, 2 stereo

fread(&sample_rate, sizeof(DWORD), 1, fp); //like 44100, 22050, etc...

fread(&avg_bytes_sec, sizeof(short), 1, fp); //probably won't need this

fread(&block_align, sizeof(short), 1, fp); //probably won't need this

fread(&bits_per_sample, sizeof(short), 1, fp); //8 bit or 16 bit file?

fread(id, sizeof(BYTE), 4, fp); //read in 'data'

fread(&data_size, sizeof(DWORD), 1, fp); //how many bytes of sound data we have

*/

}

[Roberto14]

Here i have attached a file that i should use in a my game.

 

Do your pojects load this file??

 

final.rar

[martyj]

@Roberto14 I won't be able to check more until I get home tonight. Then I can try it out with Leadwerks. Your file appears fine. My Linux decoding appears to work just fine.

 

I'll update you guys tonight with what I find.

[macklebee]

That ogg file provided fails to load properly in LE and results in the error: "corrupt or missing data in bitstream."

 

But opening the file in audacity and just saving again as ogg and it loads just fine in LE.

final_audacityOGG.zip

Interesting to note is that the resaved ogg file saved as a smaller file.

 

https://sourceforge.net/projects/audacity/

[martyj]

@macklebee Thanks for the insight

 

I wonder if Nero is adding meta data between the different Ogg streams outside of the vorbis data, thus the program fails to load the file trying to parse meta information as Ogg headers.

[macklebee]

Dunno, but is this Nero software spamware? Downloaded it and just got tons of ads popping up to purchase different Nero software. In any case, I used the Nero Wave Editor to convert a song into WAV and OGG and they both failed to load into LE. I used Audacity to just resave these WAV and OGG files and they both work fine in LE. Audacity is free and a great tool without pop up ads. Now excuse me as I disinfect my computer of this horrible Nero software.

[martyj]

@macklebee I think they use to give it out with any optical drive that you purchased. I didn't dare install it though .

[Roberto14]

Ok i found a solution:

This is a simple solution for load and play a ogg sound using OpenAL..

The function LoadOGG do the work and get all information you need for every operation: size, format, frequency;

then store all the raw audio data in a vector, then loaded in a buffer ready to be played.

 

Ogg_Load_Play.rar

 

Source:

https://www.gamedev.net/resources/_/technical/game-programming/introduction-to-ogg-vorbis-r2031

 

In this way every ogg file should be loaded without any problem and from every source.

The proof is that now my "final.ogg" load and play..

 

( I would you take a moment on this Link if you are interesting in something more complex )

 

PS: if u want use proper this files in a LE project, Audiotest function must be called before any LE setup, pratically at main function start, this because LE use a proper Openal context not compaticle with this simple solution.

 

PPS:

That ogg file provided fails to load properly in LE and results in the error: "corrupt or missing data in bitstream."

 

But opening the file in audacity and just saving again as ogg and it loads just fine in LE.

final_audacityOGG.zip

Interesting to note is that the resaved ogg file saved as a smaller file.

 

https://sourceforge.net/projects/audacity/

According to http://wiki.audacityteam.org/wiki/OGG ogg file has usually a variable bit rate, but:

"When exporting to OGG in Audacity, you choose a quality setting from 0 to 10. This tells the encoder a very approximate average number of bits to use for the encoding" ( as listened next in the same page )

and then says

"Audacity's default OGG export quality setting is -q5, implying a bit rate of approximately 160 kbps."

From this you understand that LE has a limitation because it's fixed sound bitrate property, that doesn't allow a range.. for that audacity exported files, that has a fixed bitrate, are loaded without any problem, which does not happen for file with variable bitrates( most common files for what i have seen around ).

[martyj]

From this you understand that LE has a limitation because it's fixed sound bitrate property, that doesn't allow a range.. for that audacity exported files, that has a fixed bitrate, are loaded without any problem, which does not happen for file with variable bitrates( most common files for what i have seen around ).

 

The problem with variable bitrate in leadwerks is it adds a lot of extra complexity. Since leadwerks uses WAV files it doesn't support a changing bitrate through the stream. When you play audio with OpenAL you specifiy the bitrate + size of your data.

 

To variable bitrate you'd have to change this with OpenAL at random spots in the song.

 

Either that or we'd have to resample the audio data which adds another step of complexity.

 

I use GoldWave or Sox to convert all my files and it appears to work just fine for me. I'd assume audacity would work perfect as well, just as long as you disable variable bitrate.

[Josh]

Looks like this function can read ogg file data from memory:

https://xiph.org/vorbis/doc/vorbisfile/ov_open_callbacks.html

 

Callback structure:

https://xiph.org/vorbis/doc/vorbisfile/ov_callbacks.html

[Josh]

The Vorbis bitrate does not make any sense to me. I am expecting a value of 8 or 16 and instead it is 192000. If I divide this by 8 and then by 1000 I get 24, but there is no 24-bit OpenAL sound format.

 

This is for the file Glowsphere.ogg.

 

I was able to load final.ogg, but again I am forcing the bitrate to 16.

[martyj]

@Josh you mean Bits Per Sample instead of Bitrate?

 

Bitrate is like 44100 which is the number of samples/second for audio.

 

 

Leadwerks.bitrate = Bits Per Sample

Leadwerks.frequency = Bitrate.

[Josh]

I just set the Leadwerks bitrate variable to 16 and it worked.

[Josh]

Update is up on beta branch, Windows/Lua only. This should be able to load more OGG files successfully.

[Roberto14]

Gook work Josh, i think we can consider LE able to load n play every common ogg file.

 

Actually LE audio sector lacks of some features as sound effect( reverb, chorus, echo, distortion, flanger, an so on ) but on this we should go forward a step at a time. ( btw i'm not a very expert on this kind of things )

 

I'm glad of you all, really.

[Kaustubh]

Hi Roberto,

I am developing the lib for playing the ogg file on Mobile Device but I am facing the Issue for Error Code -132 in ov_open_callbacks.

After checking the contents of the file I found that Ogg file didn't contained the 'Vorbis' i.e Bitstream does not contain any Vorbis data.

could you help with the solution for the problem or point in the right direction.

The resource link provided on https://www.gamedev.net/resources/_/technical/game-programming/introduction-to-ogg-vorbis-r2031 doesn't works.

On 11/01/2017 at 1:46 PM, Roberto14 said:

This is a simple solution for load and play a ogg sound using OpenAL..

The function LoadOGG do the work and get all information you need for every operation: size, format, frequency;

then store all the raw audio data in a vector, then loaded in a buffer ready to be played.

 

Ogg_Load_Play.rar

 

Source:

https://www.gamedev.net/resources/_/technical/game-programming/introduction-to-ogg-vorbis-r2031

 

In this way every ogg file should be loaded without any problem and from every source.

The proof is that now my "final.ogg" load and play..

 

( I would you take a moment on this Link if you are interesting in something more complex )

Regards

Kaustubh