I18N suggestions?

[OsakaWebbie]

What do you guys consider a standard (i.e. actually used, not deprecated or abandoned) class, script, etc. for doing internationalization? I have a PHP/MySQL application whose interface is currently just English, even though it's designed to handle data based in Japan - pretty shortsighted of me, huh! So now I want to add the option to see the interface in Japanese. There is a lot of text (labels and such) on my pages - I have gradually started adding calls to a function i18n("the text") which currently just sends back the passed text, and now I'm looking for what code I should use to handle the translations, whether from flat files, a MySQL table, or whatever. I don't have to worry about encoding, as I already require a native UTF-8 environment for my app, but I don't want to have to demand too much in the way of libraries and such, because generally it's running on shared hosts with no control over the PHP installation.

I looked around, but it seemed like everything was either dependent on less-than-totally-common libraries (or a really new version of PHP), or obscure and ignored (and therefore I have no idea if it's any good). I'm hoping for something simple, but it does need to handle sprintf-type grammar differences, not just plain strings. So far the one that looks the most straightforward is by flaimo, aptly called i18n, but it seems ignored for the last two years and not used by anyone, so that makes me wonder. I haven't tried using it or anything else yet - I thought I'd first ask you guys what you use or consider standard.

 

[jpadie]

I advise taking a look at wordpress.

The function they use is simply __($text); and for echoing text _e($text)

which back off to core functionality contained in wp-includes/pomo/translations.php and other files withing wp-includes/pomo/

 

[OsakaWebbie]

You are suggesting that I install WordPress and reverse engineer it to find the routines needed for the translation stuff and put it in my code. I don't have WordPress, but I use Joomla a lot, which is probably similar, and I already considered doing that with the Joomla i18n code. But I am so lost with OOP concepts in PHP (I'm fine with procedural code, but...) that I didn't even know how to look for what I needed - in Joomla the command is JText::_($text), but I have no idea what "::_" means, or WordPress's "__" for that matter (and when I don't know something I normally Google for tutorials, but it isn't possible to search for punctuation), and it just gets worse from there. Every time I try to journey into Joomla code I get lost, and I suspect that WordPress would be no different. Yeah, I could eventually figure it out, but I would probably end up spending more time extricating the complex code from WordPress or Joomla than I would writing something from scratch in my old-school coding style.

I assumed that there were standalone scripts floating around that had been written for internationalization, since there generally are for all kinds of other common coding needs. But maybe I'm wrong?

 

[jpadie]

that's why I gave you the file and directory references, so that you could extract the code without installing wordpress and without requiring a great deal of intellectual parsing of the code structure.

as posted the user interface functions are __($text) for getting the translated string into a variable and _e($text) for echoing the translated string. you do not need to address any objects.

the codex article on wp internationalisation is at

http://codex.wordpress.org/I18n_for_WordPress_Developers

 

[OsakaWebbie]

Thanks for that link - it was very helpful. And after more investigation, I discovered that I must have made a mistake early in my research that was forcing me to avoid all the best tools. I had previously run into many references to gettext, but I had checked phpinfo() on one of the servers that run my app and thought it wasn't there. So gettext was what I was primarily thinking of when I wrote the term "less-than-commonly-installed libraries" in my first post. But when I noticed that WordPress uses gettext, I looked again and discovered that I had somehow missed it - it is indeed installed on all the servers I am currently running my app on! So that opens up a whole world of possibilities I didn't have before.

But the code is another story. Pointing me to the pomo directory was a start, but it's still extremely difficult to disentangle desired code from something as complex as WordPress. I finally succeeding in following code trails, searching all the WP files for file includes, adding more files and calls to my code, etc. until I now can call __) without getting an error. It took about four hours of frustration. (In case you are wondering, in addition to the pomo directory, you need the l10n.php and plugin.php files, or at least some portion of them, and of course calls to them.)

But next I thought I'd try the rest of the process by making a POT file and then the MO file from my first call to __(). But I'm at a dead end again - xgettext and msgfmt appear to be Linux command line commands. I'm on shared hosters only - I have no command line access. The WP Developers page you linked mentions other methods of generating a POT file, but they are only available to official developers of WP plugins. Other Google searches did not turn up any more hopeful answers. What now?

 

[jpadie]

I didn't know that the compilation required linux. how curious.

official developers of WP plugins: there is no magic to this. It's just an application. takes a few days to get an account at worst. sometimes less than a day.

re the various code trails you followed - yes, I suspect that you would have had to have il10n. but the plugin file should not have been necessary; and so has me wondering.

anyhow - another thought is, of course, you could have used the entire wordpress framework in a spare directory, set it up as if it were a blog (i.e. wp-config.php) and then in your code just 'included()' the bootstrap and then used the functions as you wish. or even better, use wordpress as the framework for your web app. There is, of course, no requirement for you to use any of the template or other display functionality in wordpress: it's just a framework.

 

[OsakaWebbie]

official developers of WP plugins: there is no magic to this. It's just an application.

Yeah, but it's also likely that the application will still require Linux for something. They are so convinced that everyone is working at a Linux shell prompt that their example of how to run makepot.php was even via the commmand line ("php makepot.php ...")! Also, even if I find another way to make the POT file, there is no mention of another way to make the MO file, merge PO files during later revisions, etc. - everything requires a command at a shell prompt.

...but the plugin file should not have been necessary; and so has me wondering.

Without it, the function apply_filters() was undefined. (And I suppose there might have been others - that's just the first one that produced the error.)

you could have used the entire wordpress framework in a spare directory...

That wouldn't solve the problem - the WP developer page itself is what casually tells the WP plugin developers to run things at the command line. They don't even mention that it is the command line - they assume everyone knows that (there is a subtle reference to bash at one point, and I know what that is, but how many Windows and Mac users do?). They also assume everyone will know the syntax for the xgettext command, which I thought was curious - they carefully spell out syntax for msgfmt, but for xgettext they just say, "After the strings are marked in the source files, a gettext utility called xgettext is used to extract the original strings and to build a template translation POT file." I guess that was left as an exercise for the reader... The syntax is not rocket science but it's not trivial either.


This isn't really a WordPress question anymore - I have read more about gettext, and it really does all the heavy lifting - I probably could just call its functions directly, provided I can figure out how to make those durn-blasted translation files.

The thought occurred to me to wonder whether it might be possible to use the exec() function inside a PHP script to run the various commands, but I keep thinking that surely someone else has already asked this question! But I sure can't find it. I found an occasional question from someone trying to install and use gettext on their own Windows-based webserver (usually with no good answers), but nothing about writing i18n applications for use on Linux-based hosting servers with no customer-level shell access, which I think would be far more common.

 

[jpadie]

via the commmand line ("php makepot.php ...")

that command can be run on any OS with a terminal. windows, mac and *nix. you can also do the equivalent from a web page.

if you are stuck on this then send me the PO files and i will generate the MO files for you. use my handle at apple's consumer email domain.

apply_filters(). true. however this is only required if you are using WP in anger. either create a dummy function or comment out the calls to apply_filters().

That wouldn't solve the problem

no. however it would have saved you time.

... bash at one point, and I know what that is, but how many Windows and Mac users do?

I have bash on my mac. it's what comes up when I run terminal

let me know if I (or we) can help further.

 

[OsakaWebbie]

that command can be run on any OS with a terminal. windows, mac and *nix. you can also do the equivalent from a web page.

Yes, I know - but it has to be a terminal on the server that is running Apache, PHP, and MySQL. And yes I know that a PHP can be run from a browser. My only point in mentioning that bit from the WP Developer page was that it was evidence that the writers of the tutorial must have been assuming that everyone has terminal access to their webserver.

if you are stuck on this then send me the PO files and i will generate the MO files for you.

Thanks for the offer, but I need a solution where I can generate POT files, PO files (with merge functionality for when I have added new labels to the interface), and MO files anytime I need to without bugging you or anyone else for favors all the time. I keep thinking that surely lots of other people are in the same situation as me, and there is a simple solution...

...if you are using WP in anger.

Who me, angry? Seriously, though, I don't know what you mean by that, but it doesn't matter. I'm not using WP, and I'm not angry (yet - I admit that I am starting to get a bit frustrated...). And long term, I expect that I'll move away from using the WP files completely, as the main stuff I need is already provided in the gettext functionality. So this is really just a gettext discussion.


Eventually I would love to have my own testbed for development - set aside an old PC, install LAMP on it, and be able to do all the preparation for just uploading the files to the hoster for the live version. But that will require a good chunk of time, an extra computer, and a place to put it, none of which I have right now...

 

[jpadie]

using something in anger just means using it on a day to day basis, fully properly etc. no other subtext implied!

i suggest rackspace.com (the us site) - buy a vps from them. it costs only 50p per day that you actually use it and is fully configurable. great value. and you can close the server instance to save cash whenever you want to.

Yes, I know - but it has to be a terminal on the server that is running Apache, PHP, and MySQL. And yes I know that a PHP can be run from a browser.

nope. I will write you a wee script that you can run on your computer so that you can upload files and get them converted.

I keep thinking that surely lots of other people are in the same situation as me, and there is a simple solution...

not many people develop fully internationalised sites. it's a big overhead.
slight internationalisation can be achieved just by loading different text files into arrays depending on the language selected. you do not need to use il8n.

 

[OsakaWebbie]

using something in anger just means using it on a day to day basis, fully properly etc. no other subtext implied!

Hmm, I guess I need to study English again after I get done studying Japanese - I've never heard that one before. (Must be one of those differences based on which side of the Atlantic one is from.)


A VPS is overkill for my little testbed purposes, would still require time I don't have, and may not be set up the way I need it to be (simulating the Japan-based hosters I use frequently). If I was doing this full-time, it might make more sense, but I split my time among a variety of things, so I would be constantly needing to remember to turn it on and off. Thanks for the suggestion, though.

not many people develop fully internationalised sites. it's a big overhead.

I doubt that mine will be "fully internationalized" either (oops, there is another one of those Atlantic differences - I spell it with a "z"), but for doing even some of it, I figured that it would be smarter to use tools designed for the purpose than to reinvent the wheel. Except for the shell access snag (and needing to sorta "compile" the translation files, meaning that an administrator won't be able to easily change wording to suit them), the gettext utility looks like a good tool for the job.


If you are writing a "wee script" to run the gettext command line utilities, does that mean that PHP's exec function (or one of the other similar functions) can indeed handle those commands (provided that I grant world-write permission to the related directories beforehand, that is)? I was wondering about that, but I don't have enough experience doing low-level stuff from PHP to know what the system will let me do and what it won't.

 

[jpadie]

i don't develop on windows any more so i cannot be certain that exec calls will work as expected. but in principle they should so long as the shell scripts are in identical form.

however I was going to write a long hand script that did not use shell. but then I found this and thought that there was no point reinventing the wheel ....

http://www.poedit.net/

no need to use WP of course. just set your text domain and use _('') in the normal way. you may find it helpful to create some short hand functions like _e()

 

[OsakaWebbie]

i don't develop on windows any more so i cannot be certain that exec calls will work as expected.

I'm not developing on Windows either - all servers I use are *nix. Only my local machine where I edit the files is Windows. So if you think exec will do what I need on Linux/FreeBSD, etc., then I might do something like that.

however I was going to write a long hand script that did not use shell. but then I found this and thought that there was no point reinventing the wheel ....

http://www.poedit.net/

Oh. I assumed that when you said "script" you meant a PHP script that called the gettext commands (via exec or whatever). I already previously found that page about Poedit. When I first read it, I wasn't desperate yet, so when I saw that it needed a whole C++ GUI toolkit to run on top of, I went on to looking for something simpler (which I was sure would exist). I also thought it didn't do the first step - the job that xgettext does. But now I'm more desperate, and I read the description more carefully, and it does say it can scan source code for translatable strings.


For the next few days I'm up to my ears in something unrelated to all this, but when I get a chance to get back to it, I'll probably try either installing wxWidgets and Poedit, or writing a little PHP script with exec to use the real gettext commands. (The Poedit editing interface is nice, but a PHP script is more portable for when I make changes to my code when I'm not at this computer. Unless wxWidgets can be installed and run from a thumbdrive - that would be cool...)