exclude erroneous returns / line feeds from count() ?

[admoore]

I am updating legacy php code which imports csv data to a mySQL database. The code includes error checking which compares the number of imported rows to the number of lines found in the import file. The problem is that a small percentage of records includes returns or line feeds enclosed within fields. MySQL will happily load the data; but, when I count() the number of lines in the source, the number is artificially inflated due to the enclosed returns & line-feeds.

At present, the code is simply patched to force the counts into an equal condition.

I am looking for a solution to more accurately count the lines in the import file, ignoring the enclosed returns and line-feeds. Any suggestion is appreciated...

imported file can be large, so also trying to avoid loading file as array...

Thanks in advance...
-Allen M.

 

[Olaf Doschke]

The question is whether you really need to check the import got all lines. The easiest to get the number of CSV records is counting newly imported records in the target table, as delimited text fields are read correctly.

If there is any reason like check constraints rejecting records find out using a staging table that doesn't have such rules and can therefore simply import all rows in any case. Then you have your reference count.

Bye, Olaf.

Olaf Doschke Software Engineering

https://www.doschke.name