SSH to FreeBSD in UTF-8

Question

I'm trying to ssh to FreeBSD-11.2-Release from my Windows machine using Mintty, but when I set LC_ALL to en_US.UTF-8 and try to enter/paste a non-ASCII character, connection got closed.

I'm using Cygwin, and have also tried Qterminal and Terminator.

Does FreeBSD have a problem with UTF-8 characters at 11.2-Release version? Or is this just Windows-specific (by which I mean macOS or Linux users have no issue with it)?

Note: I don't have any window system installed on the FreeBSD machine.

Answer 1

You claim setting LC_ALL but I am not sure you are doing it correctly. Everything is actually covered in The FreeBSD Handbook: 22.2. Using Localization

I would strongly recommend that you use the "Login Classes Method" (login.conf) rather than the "Shell Startup File Method" (.profile) as it is much more consistent across the system and shells.

When people then say they have modified language related settings they often forget 2 things:

1. When changing either /etc/login.conf (system wide) or ~/.login.conf (user) you need to run cap_mkdb to have any effect!
2. And when testing remember to start a new session!

The first thing is to verify your settings by running the command locale. On a vanilla FreeBSD system it will show you this:

$ locale
LANG=
LC_CTYPE="C"
LC_COLLATE="C"
LC_TIME="C"
LC_NUMERIC="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_ALL=

With these settings, you will have Unicode problems with the most common terminals. I am using PuTTY, but am not able to paste the following test into the terminal:

ÆØÅ жз ઑ ᱵ

This is because the default setting of FreeBSD is not to use unicode. This answer was tested against 11.2 but in the future from 13 and onwards the default will be "C.UTF8" and no change will be needed. On a vanilla install you can, however, make a very quick check by setting your user to use "russian". The reason is that russian is the only language defined out of the box (in /etc/login.conf) and it is using unicode.

pw usermod johndoe -L russian

Now login to a new session using the "johndoe" account (whatever your user is named). You will now be able to paste unicode (from the test above) into the shell. Verify how the settings has been picked up using locale

$ locale
LANG=ru_RU.UTF-8
LC_CTYPE="ru_RU.UTF-8"
LC_COLLATE="ru_RU.UTF-8"
LC_TIME="ru_RU.UTF-8"
LC_NUMERIC="ru_RU.UTF-8"
LC_MONETARY="ru_RU.UTF-8"
LC_MESSAGES="ru_RU.UTF-8"
LC_ALL=

As I now have verified that basic unicode actually works - then I can start setting things to my liking. As I am danish, I will perform the following simple steps:

1. Add language settings

For each language I need I will add a section to /etc/login.conf

danish|Danish Users Accounts:\
        :charset=UTF-8:\
        :lang=da_DK.UTF-8:\
        :tc=default:

2. Update database

Whenever you edit the source login.conf, you need to update the database.

cap_mkdb /etc/login.conf

3. Set user languages

I usually use pw to manage my users. Now I just need to specify the language when I create or modify my users.

pw useradd janedoe -L danish

TIP: My personal preference is to not change the system wide "default" but keep it as default. Then I set the language for system accounts such as "postgres". If you have very specific settings needed for a system account not covered by your language setting, then I would create a separate "language" (login class) for that account in /etc/login.conf. This keeps things nice and tidy.

TIP 2: The above shows how to set class on user basis. If you want a system wide change you can change the "default" class in login.conf.

PuTTY TIP: Your terminal naturally needs to be UTF-8 aware as well. In PuTTY this is in the settings: Window ⮕ Translation. I have mine set to "UTF-8". If you are in the CJK area you might want to check "Treat CJK ambiguous characters as wide". And Cyrillic users usually likes "Caps Lock acts as Cyrillic switch".