0

I am using CentOS7 and I have to increase the send buffer size (SO_SNDBUF) on a socket listening on a specific port. The application that has the port opened does not natively support this.

Few things that I've tried which increase the send buffer size:

  1. Doing it on the OS - no can do, as this buffer size change has to be specific to this connection only.
  2. Stunnel - does the job but the client's application does not natively support SSL.
  3. Socat - it can quite possibly be done but I only have 1h of lecture until now and I haven't even scratched the surface just yet.

Edit: A bit of background. We have a client from Asia trying to connect to a port on a specific socket on our Java application over a cross connect. Client is experiencing delays and the Network engineers narrowed the issue down to the TCP buffer size being insufficient (big packets are being exchanged). Is there an option to increase the TCP buffer size on this specific socket only without stunnel or increasing it via the TCP kernel settings?

Edit2: If I haven't touched on something specifically related to the issue, please let me know as I am fairly new to this.

Thanks all!

3
  • With #1, are you saying that the app itself is overriding the OS's global default, or are you saying that you don't want to change the OS's global default?
    – Spiff
    Commented Mar 3, 2021 at 21:14
  • The more I re-read this, the more I think this may be an XY Problem. What are you ultimately trying to accomplish here, and why?
    – Spiff
    Commented Mar 3, 2021 at 21:50
  • @Spiff I have a Java application which has a socket opened listening on a port, on that specific port a client is connecting from Singapore and is seeing some latency, the Network team has narrowed it down that it's due to the send buffer size (the packets are big) but the change can't be on the OS's TCP kernel settings as this will affect all connections, the send buffer size increase needs to be on this socket only. Commented Mar 3, 2021 at 22:07

1 Answer 1

2

You can try applying a loader shim that intercepts the call to socket() and does a setsockopt() to change the buffer size before returning to the caller. Presumably you would apply this to the java interpreter running the application, assuming it doesn't set up any weird intercepts of its own. I tested the following C on a C program and it seemed to work. See man ld.so.

/*
 * capture calls to a routine and replace with your code
 * gcc -Wall -O2 -fpic -shared -ldl -o shim_socket.so shim_socket.c
 * LD_PRELOAD=./shim_socket.so SO_SNDBUF=5000 ./test
 * after call to socket() do SO_SNDBUF on it before return
 */
#define _GNU_SOURCE /* needed to get RTLD_NEXT defined in dlfcn.h */
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <dlfcn.h>
#include <sys/types.h>
#include <sys/socket.h>

int socket(int domain, int type, int protocol){
    static int (*real_socket)(int domain, int type, int protocol) = NULL;
    static int newsize = 0;
    int sock;

    if (!real_socket) {
        real_socket = dlsym(RTLD_NEXT, "socket");
        char *error = dlerror();
        if (error != NULL) {
            fprintf(stderr, "%s\n", error);
            exit(1);
        }
        char *sndbufsize = getenv("SO_SNDBUF");
        if(sndbufsize!=0){
            char *endptr;
            newsize = (int)strtol(sndbufsize,&endptr,0);
            if(*endptr!='\0'){
                fprintf(stderr, "env SO_SNDBUF=was not int\n");
                exit(2);
            }
        }
    }
    sock = real_socket(domain, type, protocol);
    if(newsize){
        int rc = setsockopt(sock, SOL_SOCKET, SO_SNDBUF, &newsize, sizeof(newsize));
        fprintf(stderr, "socket SO_SNDBUF=%d rc=%d\n", newsize, rc);
    }
    return sock;
}

Compile and run this as shown in the comment. It looks in the environment for a SO_SNDBUF=9999 value to apply after all calls to socket(). You may want to restrict it to just some socket calls. The fprintf() output is just for testing and may not even appear with java. Remove as appropriate. If your application runs other programs they may inherit the LD_PRELOAD.

Note, the application may actually go on to call setsockopt() itself, changing the size back to a smaller value. You should check with something like strace -f -e setsockopt -o outputlog. If it does this, you can instead use a shim that intercepts this call instead of socket().

Also, the maximum buffer size is set globally to the value in /proc/sys/net/core/wmem_max. You can change this, as only processes that need more buffer space will use it. Normally, the Linux kernel automatically regulates the amount of buffering needed by a process; setting SO_SNDBUF disables this mechanism.

8
  • How do I restrict it to only one socket though? Commented Mar 4, 2021 at 19:19
  • Unless the application opens hundreds of sockets it should not really be of any consequence. The SO_SNDBUF value must be set before the listen() command. The port is set in the bind() command. You can therefore rewrite the code to intercept in the same way the bind() function instead, comparing the port number and setting the size if it is the one of interest.
    – meuh
    Commented Mar 4, 2021 at 20:06
  • But the application does open hundred of sockets, I am compiling now for the first time, could you re-write your answer for a one socket situation? That's what I was originally asking, perhaps I wasn't explicit enough. Commented Mar 4, 2021 at 20:14
  • It sounds like you want to bind and listen to a single port, accepts hundreds of connections from it, and only change the buffer size for the one coming from asia. That's more difficult. You would need to know the ip address of the remote and compare it with the result of getpeername() or accept(). You should just try increasing the global system max size first, and let the kernel auto-size like it is supposed to, to see if it does actually fix your problem.
    – meuh
    Commented Mar 4, 2021 at 21:26
  • I definitely didn't explain it right in my post. It's only one connection coming in from Asia, I know their source IP, which is connecting on port 13100, on that one socket on port 13100 the SO_SNDBUF needs to be increased. Commented Mar 4, 2021 at 22:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .