0

I have a use case where multiple processes start simultaneously on one machine and all open the same set of NFS files (actually stored in another machine in the LAN) to process their content in different ways. My concern is regarding network efficiency: does NFS have any built-in mechanism to optimize this scenario, ensuring that the file content is transmitted only once over the network, even when accessed by multiple processes? If not, would this mean that in a scenario with 100 such processes, the same file content would be transmitted 100 times, potentially causing significant network overhead?

Improve this question
1
  • Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.
    – Community Bot
    Commented Mar 8 at 17:45

1 Answer 1

Reset to default
1

does NFS have any built-in mechanism to optimize this scenario, ensuring that the file content is transmitted only once over the network, even when accessed by multiple processes?

No, basic NFS does not have any way to group reads from multiple clients into a single network transfer.

If not, would this mean that in a scenario with 100 such processes, the same file content would be transmitted 100 times, potentially causing significant network overhead?

Correct. NFS has some ways like caching to mitigate high numbers of unnecessary transfers, but if a client requests data, then the server needs to send the whole thing to that single client.

Depending on your scenario, you might want to investigate multicast file transfer tools like UFTP or mrsync, which will push a single file only once over the network to multiple clients. Note that the network itself also has to support multicast traffic.

Improve this answer
4
  • 1
    The question is about "multiple processes", your answer is about "multiple clients". It's not clear if the question is about multiple processes on multiple machines, or maybe about multiple processes within a single OS. What if the latter? I use mpv with aggressive readahead; or I use vmtouch and vlc; and I use them for large media files I open via (slowish) NFS. It seems to me that within a single OS the filesystem cache runs reasonably well, also for NFS. I mean if (a part of) a file is in the cache because of one program, it's there for another program.
    – Kamil Maciorowski
    Commented Mar 8 at 22:47
  • multiple processes running within a single OS in one physics machine.
    – Xiaoyong Guo
    Commented Mar 8 at 23:26
  • @XiaoyongGuo On the client side, the NFS mount can be told to cache files locally with FS-Cache (fsc), like: mount nfs-share:/ /mount/point -o fsc, but I don't believe it will by default if unspecified. There are some settings in /etc/cachefilesd.conf as well for things like storage location and space usage
    – Cpt.Whale
    Commented Mar 11 at 15:06
  • Thank you very much. @Cpt.Whale
    – Xiaoyong Guo
    Commented Mar 12 at 6:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .