Researching a problem with CSV output in a German locale (they use ; to delimit columns rather than commas), I stumbled across this interesting post by Jeff Atwood and the linked posts are also worth a read.

The solution to the CSV output problem in C# is to use:

CurrentCulture.TextInfo.ListSeparator