Outputting data to text file - data missing...

[dbrb2]

Hi,

I'm using a java program to run a simulation. The results of this simulation are a 80 by 80 array of numbers.

I then output these numbers to a text file,using tabs as delimiters, and use a third party program (excel or gnuplot) to give me a surface plot.

However, for some reason the program seems to give up writing to the file a few elements before the end.

To check that it was not a problem with the data itself, I had my program output the data to screen at the same moment it is outputted to the text file. The screen output was all there...


However, when I import the data into excel the last 50 or so
elements are missing...



ANy ideas? My final text file is 120Kb - is there some kind of size limit...?



Cheers,


Ben

 

[timw]

Are you calling the close() method on your file output stream, or whatever. If you don't, whatever is in the buffer, but not yet written to file, will be lost when the program exits.

Tim

 

[dbrb2]

Thanks - I checked that, but it seemed to make no difference.

It seems the problem is not directly with the program - I tried copying the data that was outputed to screen (correctly) into a text file, at which point the same problem appears.

I scrolled to the far side of the notpead screen to see what was up, and for some reason the line lengths are not all the same - even though the data I pasted was nice tab seperated collums.

I've turned off word wrap in notepad, but am now a bit stumped...

 

[timw]

Can't imagine that it's anything to do with the file. I've written megs of text to log files with no loss.

Post your code please...

 

[sedj]

Are you flushing the stream ?

--------------------------------------------------
Free Database Connection Pooling Software

http://www.primrose.org.uk

 

[dbrb2]

Hi,

Theres a fair bit here, and it's not standalone. But I've highlighted the sections that deal with file I/O - they are at the very start and very end respectively...


import java.io.*;
import java.lang.Double;
import java.text.*;

public class simulate_test_2
{


public static void main(String[] args) {


int counter=0;
int X,Y,count;
double theta;
double play;
double current_error;
double temp_x,temp_y;
int index_r,index_c;
double no_solution_counter;
int check =0;

position pos=new position();
DecimalFormat decimalFormat = new DecimalFormat("#.##");
PrintWriter error_file;
PrintWriter invalid_soln_file;


//Take simulation parameters in here, or set to default
if(args.length != 2){
System.out.println("Error: defaulting to play = 0 degrees, triangle side=540mm");
pos.camera_play=0;
pos.side=540;
set_constellation(540,pos);
}
else{
pos.camera_play=Double.parseDouble(args[0]);
pos.side=Integer.parseInt(args[1]);
set_constellation( Integer.parseInt(args[1]),pos);
}

////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////

//Try to open two files for the two sets of data we'll be producing
try {
error_file = new PrintWriter(new FileWriter(new File("error_results.txt")));
}
catch(IOException e) {
System.err.println("Error opening file "
+ ": " + e.getMessage());
return;
}

try {
invalid_soln_file = new PrintWriter(new FileWriter(new File("invalid_solution_results.txt")));
}
catch(IOException e) {
System.err.println("Error opening file "
+ ": " + e.getMessage());
return;
}

////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////

//Set camera play

play=pos.camera_play*(Math.PI/180);

//set array index counters
index_r=0;
index_c=0;

//we want to increment our coordinates in steps of 25, which is whey we need seperate index counters (see above)
for(X=0;X<2000;X+=25){
for(Y=0;Y<2000;Y+=25){





//radius 1:
pos.radii[0][0]=Math.sqrt( ((X-pos.X1)*(X-pos.X1))+((Y-pos.Y1)*(Y-pos.Y1)));
theta=Math.atan(pos.height/pos.radii[0][0]);

pos.radii[0][1]=pos.height/(Math.tan(theta+play));
pos.radii[0][2]=pos.height/(Math.tan(theta-play));

//radius 2:
pos.radii[1][0]=Math.sqrt( ((X-pos.X2)*(X-pos.X2))+((Y-pos.Y2)*(Y-pos.Y2)));
theta=Math.atan(pos.height/pos.radii[1][0]);

pos.radii[1][1]=pos.height/(Math.tan(theta+play));
pos.radii[1][2]=pos.height/(Math.tan(theta-play));

//radius 3:
pos.radii[2][0]=Math.sqrt( ((X-pos.X3)*(X-pos.X3))+((Y-pos.Y3)*(Y-pos.Y3)));
theta=Math.atan(pos.height/pos.radii[2][0]);

pos.radii[2][1]=pos.height/(Math.tan(theta+play));
pos.radii[2][2]=pos.height/(Math.tan(theta-play));

counter=0;
//for everyt single possible combination of radii, find position
for(int i=0;i<3;i++){
for(int j=0;j<3;j++){
for(int k=0;k<3;k++){
calculate.pose(i,j,k,counter,pos);
counter++;
}
}
}

//find error from actual position for each of the 3*3*3=27 possible positions

for(int l=0;l<27;l++){
if(pos.pose[l][0] == -1 && pos.pose[l][1] == -1)
pos.pose[l][2]=-1;
else{
pos.pose[l][2]=Math.sqrt( ((X-pos.pose[l][0])*(X-pos.pose[l][0]))+((Y-pos.pose[l][1])*(Y-pos.pose[l][1])));
}
}


//find maximum error
current_error=0;
no_solution_counter=0;
for(int m=0;m<27;m++){
if(pos.pose[m][2]==-1)
no_solution_counter++;

if(pos.pose[m][2] > current_error)
current_error=pos.pose[m][2];
}
//Enter maximum error, and number of times no solution was found, into upper and lower layers of our 3d array
pos.error[index_r][index_c][0]=current_error;
pos.error[index_r][index_c][1]=no_solution_counter;

//increment row counter
index_r++;
}
//increment collumn counter
index_c++;
//reset row counter
index_r=0;
}

////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////
//output data to file and to screen
//we have an [80][80][2] matrix to store our data in. One lot of data goes in the top layer,
//theother in the bottom.

for(int n=0;n<80;n++){
for(int p=0;p<80;p++){

invalid_soln_file.print(pos.error[n][p][1]+"\t");
//Problem here. But it should be identical to line below, which works...
error_file.print(pos.error[n][p][0]+"\t");
//This outputs to screen perfectly
System.out.print(pos.error[n][p][0]+"\t");
}
//at end of every row, insert a new line
error_file.println("");
System.out.println("");
invalid_soln_file.println("");
}
//close both files
error_file.close();
invalid_soln_file.close();

}
////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////



//this function deals with setting the simulation parameters
public static final void set_constellation(int sides,position pos){
pos.X3=pos.centroid_x-(sides/2);
pos.X2=pos.centroid_x+(sides/2);
pos.X1=pos.centroid_x;

pos.Y3=pos.centroid_y+(sides/(2*(Math.sqrt(3))));
pos.Y2=pos.Y3;
pos.Y1=pos.centroid_y-(sides/(Math.sqrt(3)));

System.out.println("Corner coordinates: ");
System.out.println("1 ("+pos.X1+" , "+pos.Y1+")");
System.out.println("2 ("+pos.X2+" , "+pos.Y2+")");
System.out.println("3 ("+pos.X3+" , "+pos.Y3+")");
System.out.println("");
}
}

 

[dbrb2]

Hmmm.... I've done some investigating...


It does seem to be something to do with maximum permissable line length. Although the screen dumed results normally come out fine,I can replicate the problem I get with the text file if I set the screen buffer width to less than that needed to display my data in a grid...

 

[dbrb2]

Yep - that was the problem...

I replaced the tab delimited data with comma seperated data, and everything is rosy - only reason I can think of is that a comma is shorter than a tab...

However, it would still be nice to find a way around the problem, as itputs limits on the dataset size I can fiddle with....

 

[timw]

Well, can't you put out the data as 80 values to one line, then.

 

[dbrb2]

Yes, I could - you're right.

However, I've just tried increasing the data size, and it seems it was not size after all. I increased my array from 80*80 to 120*120 elements and everything was fine.

For some reason it seems that the use of tab seperated variables was causing grief somewhere along the line - with csv everthing works fine.

Long live the comma!

Cheers,

Ben

 

[timw]

Okay then. All's well that ends well, they say.

 

[stefanwagner]

//at end of every row, insert a new line
error_file.println("");
System.out.println("");
invalid_soln_file.println("");

marginal hint:
use:

    System.out.println ();

for just a newline.

seeking a job as java-programmer in Berlin:

http://home.arcor.de/hirnstrom/bewerbung

 

[dbrb2]

Thanks - it seems to be working now, althoug buzzarely only when I use csv instead of tab seperated variables...

What is the difference though between ....println() and
....println("") ?


Cheers,


Ben

 

[stefanwagner]

println (); is intended to print just a linefeed.
println (""); adds needless noise.

Readers of the code will ask themselve, if there got something lost between the "".

seeking a job as java-programmer in Berlin:

http://home.arcor.de/hirnstrom/bewerbung