I'm sure there is a better solution, but this is a kind of workaround.
So, just handle the files as normal files and not as csv files when you compare them, that seems to work fine.
$Master = Get-Content master.csv
$New = Get-Content new.csv
Set-Content -Path finalnew.csv -Value...