What characters are safe in cross-platform file names for Linux, Windows and OS-X

Question

Currently, I use a YYMMDD-NAME+PAGE name for most of my files. NAME has spaces converted to underscores.

I'd like to use the YYYY-MM-DD date format, but I am not sure how to separate it from the name. A - would look strange if the name started with a number. If I use a _, then it conflicts with the underscore representing a space.

What characters are reasonably safe in file names that would work here? I am on Linux, but I might share files with other people (Windows 7, Mac OS X).

Answer 1

Summary:

• Windows: anything except ASCII's control characters and \/:*?"<>|
• Linux, OS-X: anything except null or /

On all platforms it is best to avoid non-printable characters such as the ASCII control-characters.

Windows

In Windows, Windows Explorer does not allow control-characters or \/:*?"<>| You can use spaces. If you use spaces, you will often have to quote the filename when used from the command line (but GUI apps are unaffected so far as I know). Windows filesystem such as NTFS apparently store the encoding with the filename, but UTF-16 is standard.

Some parts of Windows are case-sensitive, other parts are case-insensitive. It is easy to create distinct filenames like "Ab" and "ab" on a Windows NTFS filesystem. These names refer to separate files which contain distinct separate content. However, although the Windows command-prompt will happily list both files using dir, you cannot easily access or manipulate one of them using commands such as type. See below.

Linux, OS-X

In Linux and OS-X only / of the printable ASCII set is prohibited I believe. Some characters (shell metacharacters like *?!) will cause problems in command lines and will require the filename to be appropriately quoted or escaped.

Linux filesystems such as ext2, ext3 are character-set agnostic (I think they just treat it more or less as a byte stream - only nulls and / are prohibited). This means you can store filenames in UTF-8 encoding. I believe it is up to the shell or other application to know what encoding to use to properly convert the filename for display or processing.

Conclusion

So you could probably safely use something like ✣ (if it weren't so hard to type)

---

Case-(in)sensitivity in Windows

C> dir /B
Ab
aB
аB

C> type Ab
b
b

C> type aB
b
b

C> type аB
unicode homograph

Note that we cannot type the contents of the second file, the Windows type command just returns the contents of Ab instead. The third file would be distinct from aB on Linux also.

(Windows 10 NTFS).

Answer 2

While RedGrittyBrick's answer is technically correct, safety isn't the only issue: usability is also important. I think a better question is "what characters are good to use in a filename".

Some potential guidelines:

• [0-9a-zA-Z_] - Alphanumeric characters and the underscore are always fine to use.
• \/:*?"<>| and the null byte are problematic on at least one system, and should always be avoided.
• Spaces are used as argument separators on many systems, so filenames with spaces should be avoided when possible. Other whitespaces (e.g. tabs) even more so.
• Semicolons (;) are used to separate commands on many systems. Semicolons and commas(,) are used to separate command line arguments on (some versions of?) the windows command line.
• []()^ #%&!@:+={}'~ and [`] all have special meanings in many shells, and are annoying to work around, and so should be avoided. They also tend to look horrible in URLs.
• Leading characters to avoid:
  • Many command line programs use the hyphen [-] to indicate special arguments.
  • *nix based systems use a full-stop [.] as a leading character for hidden files and directories.
• Anything not in the ASCII set can cause problems on older or more basic systems (e.g. some embedded systems), and should be used with care.

That basically leaves you with:

[0-9a-zA-Z-._]

that are always safe and not annoying to use (as long as you start the filename with an alpha-numeric) :)

Answer 3

You could:

1. replace current underscores with # (proofreader's symbol for space)
2. underscore to 'section' date from filename (or a second hyphen - easier to type)

Alt-1. initial-caps can replace spaces: YYMMDD-HHMM-FileName.ext or YYMMDD-HHMM_FileName.ext

Minimal characters for clear display, which auto-sorts with padded zeroes for Jan-Sep (& 1st-9th ea mo).

Answer 4

The characters have largely been addressed by other people, though I will point out an additional aspect to consider. First, I'm addressing the choice of YYMMDD, which has two problems.

The first problem with YYMMDD is that it won't work for historic data. It will sort 1997 long after 2035, instead of before. Whether that's an issue may be dependant on how widespread you want to propagate the format.

The other issue with YYMMDD relates to the calendar reliance. While the Gregorian calendar is now the most popular one in the world, not everyone uses it or is aware of the day in its reference. Fortunately, the Gregorian year is generally known and accepted, even by those who use different years, but the month/day nomenclature can be meaningless. To be more portable, a format of YYYYDDD where DDD is the day in the year, is more portable. However, for those of us who use the Gregorian calendar that is difficult because we generally are not aware of the day of the year. The MMDD format still is sortable, even if it means nothing to an individual, who themselves might create a date like 20221442 (Gregorian year and their month and day) or 20220047 (Gregorian February 16, the 47th day of the year), thinking they are complying with your format.

Continuing the theme of how widespread the format is to be used, a consideration needs to be made for the characters available around the world. The short dash '-' is available everywhere (?) because it is the minus sign, used globally. Underscore is more of a problem, even for some who use the Latin alphabet. They can normally get to it one way or another but it is not on every keyboard. In some alphabets, the underscore is a character or character modifier, so makes confusion. In many Persian languages, the underscore would be read as a kasheeda. In many alphabets, what we use an underscore for, they would use an overline: something hard to get on our keyboards. Most of the keyboards for technical people have the simple latin alphabet available on them (sometimes on the side of the key), so they can type the letters. But not always the underscore.

Answer 5

also note that: not only characters can be invalid, but also filenames (combinations of characters) can be invalid.

on windows, these filenames are invalid, because they are legacy device names:

CON PRN AUX NUL
COM0 COM1 COM2 COM3 COM4 COM5 COM6 COM7 COM8 COM9
LPT0 LPT1 LPT2 LPT3 LPT4 LPT5 LPT6 LPT7 LPT8 LPT9

even worse, this limitation is case-insensitive, so for example Con is an invalid filename.

even worse, you cannot use these reserved names as basename, so for example Con.txt is an invalid filename.

see also:

• Get reserved file names in Windows
• Naming Files, Paths, and Namespaces § Naming Conventions
• Peripheral Interchange Program uses these filenames as device names

a more well-known limit on filenames is the size limit of 255 bytes (not 255 unicode-characters).

See also:

• https://www.npmjs.com/package/sanitize-filename
• https://pypi.org/project/sanitize-filename/