Read header row of remote file

[davida37]

Hi,

I am trying to open a remote file to read the header row to establish which columns the file has. The code below works but takes ages with a large file. I have a feeling the whole files contents is being read into memory?

Is it possible to read a header row of a remote file without reading the whole file into memory? the files I want to peek at will have 500K + rows so I need a quick and efficient solution.

any help would be much appreicated.

Thanks
David


Protected Sub btnUpload_Click(ByVal sender As Object, ByVal e As EventArgs) Handles btnUpload.Click

Dim theStream As Stream = File1.PostedFile.InputStream
Dim tmpArray() As String

Using sr As New StreamReader(theStream)
Dim line As String = sr.ReadLine()
For x As Integer = 0 To 0
tmpArray = line.Split(Convert.ToChar(ddlDelimiter.SelectedValue))
line = sr.ReadLine() ' CRITICAL line else you will go in an endless loop
Next x
End Using

'ListBox1.Focus()
'ListBox1.DataSource = tmpArray

For y As Integer = 0 To tmpArray.Length - 1
lblColumnHeaders.Text += tmpArray & vbCrLf
Next
End Sub

 

[jmeckley]

you are reading the entire file with this line

For x As Integer = 0 To 0
   line = sr.ReadLine()
Next x

if all you want is the header
just read the first line and then dispose of the file.

Not sure about VB but C# has the concept of delayed execution which you can utilize with the yield keyword with IEnumerable. If VB harnesses this as well. then the code above should be effective no matter how big the file is.

Jason Meckley
Programmer
Specialty Bakers, Inc.

 

[davida37]

Hi Jason,

I still a bit confussed ..How do you just read the first line and then dispose of the file? could you provide sample code? (vb or c#).

thanks

 

[jmeckley]

something like

string header;
using(var reader = File.OpenText("path to file"))
{
   header = reader.ReadLine();
}
parse the header.
[code]

Jason Meckley
Programmer
Specialty Bakers, Inc.

 

[davida37]

Hi Jason,

This does exactly the same as before. If its a large file is looks like its taking the whole file into memory. Also, the file will be on a remote client machine, so we cant refer to the file with a path. So the only options I can think of are to:

1) Upload the file (using a fileUpload control) to get the file locally on the processing web server, then examine the file locally on the server.
2) Use a client side scripting language (javascript) which executes on the users local pc and posts the header row

Ideally, I would still like to examine the header row of a file on a users/client machine to examine the header row before uploading the file.

thanks

 

[jmeckley]

Ideally, I would still like to examine the header row of a file on a users/client machine to examine the header row before uploading the file.

I won't say it's impossible, but browser try to prevent this, as it requires you to have access of the clients machine. This is how viruses and malware gets around. the server takes control of the client.

If its a large file is looks like its taking the whole file into memory.

yes, because it's transferring the file to the server, and then you are examining the first row.



Jason Meckley
Programmer
Specialty Bakers, Inc.