Reading large (over 8 Mb) ASCII files

[IlyaRabyy]

Colleagues,
As established here in the past, the String size limit is 2 GB, but it's better bet to limit it to 1 Gb.
I have files that all are smaller than that (300+ MB max.) that my Program needs to read, and this is where I hit the roadblock!
Here's the problem:
My.Computer.FileSystem.ReadAllText() function can't read files over 8 Mb in size: system suspends execution with this screen:

It's showing up in the Debug mode. I can just select "Continue execution", but it's not a solution.

I tried to find a function/method (e.g. in StreamReader) that reads a large ASCII file by chunks of the ordered size in bytes, and couldn't find any that would exactly fit for my purpose.
For instance:
StreamReader.Peek() - "Returns the next available character", that is reads just one;
StreamReader.Read(Char[], Int32, Int32) - "Reads a specified maximum of characters from the current stream into a buffer, beginning at the specified index" - closer, but the "buffer" is an array of Chars, not a String.

Are there any other, more easily applicable solution to this problem than that latter?

AHWBGA!

P.S. By my past experience (e.g. in late 1990s-2000s), String size was limited to 16 Mb... but, as stated above, even that breaks the system in VS 2019's VB.NET...

 

[strongm]

>I can just select "Continue execution", but it's not a solution.

But it could be pointing you in the right direction. This is really just a warning in the debugger (it won't appear in a release) that a process is taking longer than expected. You can disable this warning.

[tt]Debug > Windows > Exception Settings > Managed Debugging Assistants[/tt] and uncheck [tt]ContextSwitchDeadlock[/tt]

 

[IlyaRabyy]

You can disable this warning.

T's a relieve!
However... When processing such a long file, the system looks "frozen", and the end User may fall into panic and start calling for tech support... and SDTH on my aching head! :-(

Maybe I shall set the mouse pointer to that blue will?

Regards,

Ilya

 

[SaltyTheFrog]

Is a SQL bulk import into a database and using an option?

 

[IlyaRabyy]

Is [this] a SQL bulk import into a database...?

Nope, it's pure Disk I/O Operation, a String being written (appended) into an ASCII (actually - XML) file on disk.
I lately managed to make the file size's top limit 2^27 (128 Mb), and My.Computer.FileSystem.ReadAllText() can read it w/o throwing error...
Gonna try to raise it to 256 MB...

Regards,

Ilya

P.S. 2023-02-23: with 256 Mb chunks, My.Computer.FileSystem.ReadAllText()/WriteAllText() works. When I raised the limit to 300Mb - it failed.
Just FYI.