character issue

[stasJohn]

I have strange characters appearing in my mysql data wherever a ’ (acute accent) appears. Mysql converts it to ’ (a bunch of chars)

First off, I collect some data from a text field. I pass the information through a couple of functions...
1) trim
2) regexp to replace multiple spaces with a single space
3) strip_tags

Next, I store the string into my database. I don't have magic_quotes on, so I do make a call to addslashes before the query is executed.

Now, when I pull the information back from the db and display it on the screen, it looks fine. Before displaying to the screen, I do run the string through htmlspecialchars. I tried using htmlentities but that made the accents look all garbaled.

Now, if I export my sql data (with phpmyadmin for instance) anywhere those accents are, I get the strange characters.


These characters are coming from clients who cut and paste information from word. They are apostrophes, but for some reason become these slanted apostrophes. Now, would it be okay if I run a regular expression to convert all accents to regular apostrophes? or am I doing this all wrong?

Thanks in advance.

 

[Chacalinc]

hmmm... may be these pages can help you (I hope so):

http://dev.mysql.com/doc/mysql/en/character-sets.html

http://cl2.php.net/htmlspecialchars

http://www.w3.org/TR/REC-html40/charset.html

http://www.w3.org/International/O-HTTP-charset

Cheers.

Chacal, Inc.

 

[stasJohn]

Thanks for the links.

I did discover that if I serve the page as text/html with iso-8859-1, the problem did not occur.

The strange characters would show up for all other combinations.
text/html with utf-8
application/xhtml+xml with iso-8859-1
application/xhtml+xml with utf-8

I did some googling for using mysql (4.0) with utf-8 and application/xhtml+xml but really couldn't find anything. If anyone has encountered this, please share.

For now, I'm just converting the string to utf-8 and then using preg_replace to replace the accent character with a regular apostrophe. Then converting it back to iso.