2

I am reverse engineering a program that uses a custom auth protocol. This is a basic outline of the process:

  1. Client takes input for username and password, then sends username to the auth server.
  2. Auth server calculates a public key and responds with it.
  3. Client uses that key to calculate a session key, which is sent to the server.

The communication is all done via HTTP. My goal is to locate and analyze the functions responsible for calculating the session key. My most recent approach was to set a breakpoint on ws2_32.recv, begin the login process, then step through the execution from the recv breakpoint to analyze how the public key was being used to calculate the session key.

However, the breakpoint isn't triggered at the expected time. A breakpoint on ws2_32.send is triggered as expected. I'm pretty stumped on this. Is there something I've overlooked? Is there a better approach to locating the functions?

Improve this question
1
  • Is the client a (windows) executable? You could look at what the executable imports in the Import Address Table or run it through a disassembler.
    – Remko
    Commented Jun 24, 2019 at 21:27

1 Answer 1

Reset to default
1

Your program might be using ws2_32.recvfrom instead of ws2_32.recv. Or alternatively, it could be using one of the lower-level Winsock functions, such as ws2_32.WSARecv, ws2_32.WSARecvEx, ws2_32.WSARecvFrom, or ws2_32.WSARecvMsg.

You may want to try one of the following approaches to determine what the program is doing:

  • Use an API monitor such as API Monitor to determine what networking functions are called.
  • Use Process Monitor to capture the network-receive events and double-click on those events to see the callstack.
Improve this answer
2
  • I took a look at the process with API monitor and monitored all functions under "networking". I also set individual breakpoints on recv, recvfrom, WSARecv, WSARecvEx, and WSARecvFrom. Though the auth library calls send, it never seems to call any of the recv functions listed. The only calls to recv are from libcef.dll, and the calls are out of order in relation to the auth library's send calls. I'm watching the recv packets come in simultaneously using network monitor, too. Also, WSARecvMsg isn't recognized by API Monitor nor Immunity Debugger for me.
    – Invalidation
    Commented Dec 17, 2015 at 3:21
  • Sounds like using Process Monitor is the way to go, then.
    – Jason Geffner
    Commented Dec 17, 2015 at 15:22

Not the answer you're looking for? Browse other questions tagged or ask your own question.