Well, it all started a long time ago when Europeans came up with an idea of adapting a known character set (that is Latin) to represent sounds of entirely exotic languages (Vietnamese for instance). It seemed a great idea back then — so cosmopolitan and all — but now it turns out it's not that awesome after all.

To cut the long story short we ended up with a modern Vietnamese alphabet containing a meaningful character of "y with dot below" (ỵ) and couple of other characters that reassemble regular ASCII characters, just in the same fashion as, say, umlauts do in western Europe languages. The only difference is that we know about umlauts and we learnt to expect them unlike some weird "y with dot below". It simply leaves us in a place where we cannot treat Vietnamese alphabet any different than we do treat any other and that’s exactly why IE7-ish way of failing to convert a word paỵpal into ASCII is pretty much justifiable — seeing the letter ỵ is what Vietnamese people are expecting to see, not the xn-- weirdo something (it's worth mentioning that IE7 properly converts the letter-diacritic pair into a one Unicode character representing the same glyph thus normalising the URL).

OK, so now what? How are we going to defend against it? And even more importantly — how are we going to defend our users against scams that use this method? Well, one way that's always good and never sufficient, is to increase user awareness. After all, it's pretty straightforward to look for the IDN button at the right to the address bar. But is it really? For your parents, grandma or your artistic-minded girlfriend or boyfriend?

And could we expect registrars to forbid registering certain domain names reassembling trademarked names? Or, if such domains get registered, to feed them into anti-phishing databases? Well, first of all, it's not their business. Their business is to register domains and make their customers happy (i.e. not have their domains submitted to the anti-phishing lists for a start). And secondly, maintenance of such a black-list looks like a huge load of trouble. So, my guess here is that this problem is not going to be fixed on the registrar level in any foreseeable future.

But, what we're left with is a way deeper problem. The problem of maintaining a healthy balance between features (like everybody having their domain names set in type as expected in their language) on one side and security (like people not having to worry about fonts and characters that they have never even suspected exist to impersonate some ASCII characters) on the other. What is the good balance? It seems like there is not much of a consensus in this. Surely there are two opposite ways: Firefox goes for security, when IE7 gives people their well-deserved funny characters everywhere possible.

Which is better? Decide for yourselves.

Could it be done better? How about allowing only those non-ASCII characters that are used in a language that a user’s browser is localised for and converting everything else to ASCII representation? This appears to be quite a reasonable balance — we’ll see if it makes its way to the web browsers at some point or not.