32

POSIX compliance is a standard that is been followed by many companies. I have a few question around this area:

  1. do all the file systems need to be POSIX compliant?
  2. are applications also required to be POSIX compliant?
  3. are there any non POSIX filesystems?
2
  • 1
    NOTE that POSIX compliance pertains to the OS, not the file system specifically. Part of it does specify the API for accessing the file system. In that context: 1) it's not required, but it's helpful for programmers, 2) only if they want to work with a POSIX compliant OS, 3) yes. See: stackoverflow.com/questions/1780599/…
    – lurker
    Commented Aug 31, 2013 at 16:32
  • Voting to close as too broad: too many questions in one ;-) Commented Aug 29, 2015 at 20:44

3 Answers 3

33

In the area of "requires POSIX filesystem semantics" what is typically meant is:

  • allows hierarchical file names and resolution (., .., ...)
  • supports at least close-to-open semantics
  • umask/unix permissions, 3 filetimes
  • 8bit byte support
  • supports atomic renames on same filesystem
  • fsync()/dirfsync() durability gurantee/limitation
  • supports multi-user protection (resizing file returns 0 bytes not previous content)
  • rename and delete open files (Windows does not do that)
  • file names supporting all bytes beside '/' and \0

Sometimes it also means symlink/hardlink support as well as file names and 32bit file pointers (minimum). In some cases it is also used to refer specific API features like fcntl() locking, mmap() or truncate() or AIO.

4
  • eckes, could you, please, clarify what do you mean by "supports multi-user protection"?
    – pva
    Commented Jan 5, 2017 at 19:41
  • 1
    @pva yes basically what I have wroten in the braces - I am not sure if there is a better official name for it. That there is no way to gain access to junks from deleted files. The later can happen for FAT on DOS if you resize a file.
    – eckes
    Commented Jan 7, 2017 at 4:30
  • rename and delete open files (Windows does not do that) AFAIK, Windows DOES that, but only on Win 10 RS1+ with NTFS file-system.
    – Dan M.
    Commented Apr 1, 2018 at 13:21
  • Windows does allow deletion and renaming of a opened file when FILE_SHARE_DELETE is given to CreateFile. Commented Dec 28, 2019 at 8:33
8

When I think about POSIX compliance for distributed file systems, I use the general standard that a distributed file system is POSIX compliant if multiple processes running on different nodes see the same behavior as if they were running on the same node using a local file system. This basically has two implications:

  1. If the system has multiple buffer-caches, it needs to ensure cache consistency.
    • Various mechanisms to do so include locks and leases. An example of incorrect behavior in this case would be a writer who writes successfully on one node but then a reader on a different node receives old data.
    • Note however that if the writer/reader are independently racing one another that there is no correct defined behavior because they do not know which operation will occur first. But if they are coordinating with each other via some mechanism like messaging, then it would be incorrect if the writer completes (especially if it issues a sync call), sends a message to the reader which is successfully received by the reader, and then the reader reads and gets stale data.
  2. If data is striped across multiple data servers, reads and writes that span multiple stripes must be atomic.
    • For example, when a reader reads across stripes at the same time as a writer writes across those same stripes, then the reader should either receive all stripes as they were before the write or all stripes as they were after the write. Incorrect behavior would be for the reader to receive some old and some new.
    • Contrary to the above, this behavior must work correctly even when the writer/reader are racing.

Although my examples were reads/writes to a single file, correct behavior also includes write/writes to a single file as well as read/writes and write/writes to the hierarchical namespace via calls such as stat/readdir/mkdir/unlink/etc.

5

Answering your questions in a very objective way:

1. does all the file systems need to be posix compliant? Actually not. In fact POSIX defines some standards for operational systems in general. Good to have, but no really required.

2. are applications also required to be posix compliant? No.

3. are there any non posix filesystems? HDFS (hadoop file system)

0

Not the answer you're looking for? Browse other questions tagged or ask your own question.