simple fwrite error, or something deeper?

[mbozza]

I get this wierd output, both to file and screen. At certain points, I think at 32 or 36 bytes in, the data reads in ok but writing out seems write ahead or skip bytes, writing the *member variable oddly, with part of the next variable, *member_size added on.

I feel that maybe I am seeing some kind of struct alignment error, or perhaps I'm reading the value of the data[0].member twice and not resetting the pointer for that variable between-times, or maybe it's that I'm not accessing struct members exactly as I should. I just don't know.

Move the second "test print" to screen down the code, past another fread and it will output differently.

btw, if you run this you will probably get a "pointer p differs in levels of indirection" warning, but if this section of code is taken out it still writes to file erroneously.


#include <stdio.h>
#include <stdlib.h>


struct data_block{

char *header;
char *name;
char *name_size;
char *member;
char *member_size;
char *stream_id;
char *mo_data;

};

int main(void)
{
FILE *fpin, *fpout;

struct data_block data[1];
char *p, *start;
int i;


fpin = fopen( &quot;somfile.in&quot;, &quot;rb&quot; );
fpout = fopen( &quot;somefile.out&quot;, &quot;wb&quot; );


data[0].header = (char *)calloc( 24, sizeof( char ) );
data[0].name = (char *)calloc( 4, sizeof( char ) );
data[0].name_size = (char *)calloc( 4, sizeof( char ) );
data[0].member = (char *)calloc( 8, sizeof( char ) );
data[0].member_size = (char *)calloc( 4, sizeof( char ) );
data[0].stream_id = (char *)calloc( 4, sizeof( char ) );
data[0].mo_data = (char *)calloc( 4, sizeof( char ) );

// file read and screen print test
fread((&data[0].header), sizeof(char), 24, fpin);
fread((&data[0].name), sizeof(char), 4, fpin);
fread(&(data[0].name_size), sizeof(char), 4, fpin);
fread(&(data[0].member), sizeof(char), 8, fpin);

//test print 01
p = start = &data[0].member;
for(i=0; i < 8; i++)
{
printf(&quot;%c&quot;, *p++);
}
printf(&quot;\n&quot;

//test print 02 **try moving this down**
p = start;
for(i=0; i < 8; i++)
{
printf(&quot;%c&quot;, *p++);
}
printf(&quot;\n&quot;

fread(&(data[0].member_size), sizeof(char), 4, fpin);

fread(&data[0].stream_id, sizeof(char), 4, fpin);
fread(&data[0].mo_data, sizeof(char), 4, fpin);


// file write
//fwrite(&data[0].header, sizeof(char), 24, fpout); //don't write this because the fwrite(&data[0].name, sizeof(char), 4, fpout); //error's further on
fwrite(&data[0].name_size, sizeof(char), 4, fpout);
fwrite(&(data[0].member), sizeof(char), 8, fpout);
fwrite(&data[0].member_size, sizeof(char), 4, fpout);
fwrite(&data[0].stream_id, sizeof(char), 4, fpout);
fwrite(&data[0].mo_data, sizeof(char), 4 , fpout);


// close files
//fflush(fpout);
fclose(fpin);
fclose(fpout);

} //end main

 

[Salem]

Probably because you're taking the address of a pointer

All these

 fread((&data[0].header), sizeof(char), 24, fpin);

should be

 fread( data[0].header, sizeof(char), 24, fpin);

> btw, if you run this you will probably get a &quot;pointer p differs in levels of indirection&quot; warning,
This should have told you that you were making a mistake

p = start = &data[0].member;
should be
p = start = data[0].member;


> data[0].header = (char *)calloc( 24, sizeof( char ) );
Unless you're compiling this with a C++ compiler, the cast is not necessary (and may mask an error in the worst case)

The worst case is as follows:
If you don't include stdlib.h, then malloc will be implicitly declared as returning int. int's and pointers can have different sizes so there is potential loss of information there.
If you don't cast, the compiler will warn you about int to pointer conversion. With the cast, this important warning will be suppressed.

--

http://www.catb.org/~esr/faqs/smart-questions.html

 

[mbozza]

Yes, I am compiling with Visual C++.

The reason I was taking the address of the pointer was because most of the code used to be in functions, passed a pointer to struct. I moved my code back up into main to see if it would cure the problem and kinda forgot to change it. The code now works in main, using your suggestion. However, I need this stuff in functions


void write_block(FILE *fp, struct data_block *db)
{
/** this will write **/
fwrite(&(db->member), sizeof(char), 8, fpout);
fwrite(&(db->member_size, sizeof(char), 4, fpout);

/** whereas this won't **/
fwrite((db->member), sizeof(char), 8, fpout);
fwrite((db->member_size), sizeof(char), 4, fpout);

}

...probably as expected. But the write-ahead cum skip thing has re-emerged in the function, using the same code, except adjusted to work with a pointer to struct.

Is there anything tricky about working with pointers to structs? This one problem is holding me back

 

[Salem]

All things being equal,

 fread( data[0].header, sizeof(char), 24, fpin);

in main, and

 fread( db->header, sizeof(char), 24, fpin);

in a function should be identical

If you're still stuck, then you would need to post a complete (cut-down to save space) program which demonstrates the problem.


> Yes, I am compiling with Visual C++.
But are your source files named prog.c or prog.cpp ?
The distinction tells the compiler which language you're programming in

--

http://www.catb.org/~esr/faqs/smart-questions.html

 

[mbozza]

Thank you for your advice. I have solved the problem of the 'skipped' bytes.
My source file is a C file, e.g. foo.c

Here is my code till now (bare essentials). My problem now is how to allocate enough memory dynamically in order to read in fairly large files (3-5Mb). Although I know the sizes of the smaller data units, each data block has a 'biggie', anywhere from a few hundred bytes up to 40,000 or so. And I only find out the size of these blocks at read time, hence me trying to allocate inside the function.

i.e.
// get stream size
int myStream = *((int*) db->stream_size);
// allocate
db->stream_size = (char *)calloc( myStream, sizeof( char ) );
// read
fread(db->stream_data, sizeof(char), myStream, fp);

...this is rejected by the compiler.

But even if I forget the above and allocate memory statically in main, say 30000 bytes for every *stream_data member, it still bugs-out straight away when I run it. I don't want allocate statically anyway for reasons of .exe file size.

So, I don't know whether I'm not allowed to allocate in a function, to a pointer to struct, or whether I'm trying to allocate too much to something which exists only as an address on the stack.

Maybe I should just go with an array of pointers?

// listing...
#include <stdio.h>
#include <stdlib.h>
#include <io.h>


#define SIZE 300 // No of data blocks

void init_db(FILE *, struct data_block *);
void prnt_block(FILE *, struct data_block*);



struct data_block{

char *header;
char *name;
char *name_size;
char *member;
char *stream_size;
char *stream_data;

};


main(void)
{

FILE *fpin, *fpout;
struct data_block data[300];
int i;

// open files
fpin = fopen( &quot;C:\\somefile.in&quot;, &quot;rb&quot; );
fpout = fopen( &quot;C:\\somefile.out&quot;, &quot;wb&quot; );

///////////////////////////

// allocate
for(i = 0; i < SIZE; i++)
{
data.header = (char *)calloc( 24, sizeof( char ) );
data.name = (char *)calloc( 4, sizeof( char ) );
data.name_size = (char *)calloc( 4, sizeof( char ) );
data.member = (char *)calloc( 8, sizeof( char ) );
data.stream_size = (char *)calloc( 4, sizeof( char ) );
//data.stream_data = (char *)calloc( 30000, sizeof( char ) );
}


// init data block
for(i = 0; i < SIZE; i++)
{
init_db(fpin, &data);
}

// print data block
for(i = 0; i < SIZE; i++)
{
prnt_block(fpout, &data);
}


// close files
fclose(fpin);
fclose(fpout);


}//end o' main


void init_db(FILE *fp, struct data_block *db)
{
int myStream;

fread(db->header, sizeof(char), 24, fp);
fread(db->name, sizeof(char), 4, fp);
fread(db->name_size, sizeof(char), 4, fp);
fread(db->member, sizeof(char), 8, fp);
fread(db->stream_size, sizeof(char), 4, fp);

// get stream size
myStream = *((int*) db->stream_size);
// allocate
db->stream_size = (char *)calloc( myStream, sizeof( char ) );
// read
fread(db->stream_data, sizeof(char), myStream, fp);

}

void prnt_block(FILE *fp, struct data_block *db)
{
int myStream;

fwrite(db->header, sizeof(char), 24, fp);
fwrite(db->name, sizeof(char), 4, fp);
fwrite(db->name_size, sizeof(char), 4, fp);
fwrite(db->member, sizeof(char), 8, fp);
fwrite(db->stream_size, sizeof(char), 4, fp);

// get stream size
myStream = *((int*) db->stream_size);
fwrite(db->stream_data, sizeof(char), myStream, fp);

}

thanks, your patience is much appreciated.

 

[Salem]

Why isn't stream_size just an integer?
It would save all this casting...
myStream = *((int*) db->stream_size);

And you allocated the wrong variable in your init_db function
db->stream_size = (char *)calloc( myStream, sizeof( char ) );

struct data_block{
    char *header;
    char *name;
    char *name_size;
    char *member;
    int   stream_size;
    char *stream_data;
};

Which you read with

  fread( &db->stream_size, sizeof(int), 1, fp );

and write with

  write( &db->stream_size, sizeof(int), 1, fp );

And you allocate the space, and read with

  db->stream_data = malloc( db->stream_size );
  fread( db->stream_data, sizeof(char), db->stream_size, fp );

And write with

  fwrite( db->stream_data, sizeof(char), db->stream_size, fp );

--

http://www.catb.org/~esr/faqs/smart-questions.html

 

[mbozza]

>Why isn't stream_size just an integer?
>It would save all this casting...
>myStream = *((int*) db->stream_size);

Good question. Partly because it suited me of old and hasn't been subsequently amended in the

development process. Partly for aesthetic reasons - which shouldn't figure at all in computer

programming.

All the above problems in previous postings are now a thing of the past (you hope - ed) and my

program marches on. This is in no small measure due to your kind advice.

Thanks again.