How to create HTTP GET request Scapy?

Question

I need to create HTTP GET request and save the data response. I tried to use this:

    syn = IP(dst=URL) / TCP(dport=80, flags='S')
    syn_ack = sr1(syn)
    getStr = 'GET / HTTP/1.1\r\nHost: www.google.com\r\n\r\n'
    request = IP(dst='www.google.com') / TCP(dport=80, sport=syn_ack[TCP].dport,
            seq=syn_ack[TCP].ack, ack=syn_ack[TCP].seq + 1, flags='A') / getStr
    reply = sr1(request)
    print reply.show()

But when I print reply I don't see any data response. In addition, when I checked in 'Wireshark' I got SYN, SYN/ACK but I didn't get an ACK.

Image:

Edit:

I try to do that now:

# Import scapy
from scapy.all import *

# Print info header
print "[*] ACK-GET example -- Thijs 'Thice' Bosschert, 06-06-2011"

# Prepare GET statement
get='GET / HTTP/1.0\n\n'

# Set up target IP
ip=IP(dst="www.google.com")

# Generate random source port number
port=RandNum(1024,65535)

# Create SYN packet
SYN=ip/TCP(sport=port, dport=80, flags="S", seq=42)

# Send SYN and receive SYN,ACK
print "\n[*] Sending SYN packet"
SYNACK=sr1(SYN)

# Create ACK with GET request
ACK=ip/TCP(sport=SYNACK.dport, dport=80, flags="A", seq=SYNACK.ack, ack=SYNACK.seq + 1) / get

# SEND our ACK-GET request
print "\n[*] Sending ACK-GET packet"
reply,error=sr(ACK)

# print reply from server
print "\n[*] Reply from server:"
print reply.show()

print '\n[*] Done!'

but its print me in reply from server;

0000 IP / TCP 192.168.44.130:23181 > 216.58.208.164:http A / Raw ==> IP / TCP 216.58.208.164:http > 192.168.44.130:23181 A / Padding None

And I need Line-based text data: text/html.

Answer 1

You are sending a RST segment in response to the SYN-ACK because your kernel has no knowledge of the SYN you sent via Scapy (see here). This could be solved with an iptable rule:

iptables -A OUTPUT -p tcp --tcp-flags RST RST -s <your ip> -j DROP

Because you are ending the connection with that RST segment, when you send your HTTP request, the endpoint answers with a RST too because connection is not established and so you are using show() on a RST segment with no data, that is why you do not see anything.

Answer 2

You are sending a SYN and correctly receiving a SYN_ACK. At this point, you should generate and send an ACK based on the SYN_ACK that you've received, and THEN finally transmit the HTTP GET request. It seems that you are somewhat confused about the TCP 3-way handshake mechanism. In short, you are not supposed to 'get' an ACK, you are supposed to generate and send this yourself.

Answer 3

After setting the rule in your iptables as has been suggested above, you could do the following :

from scapy.all import *

seq = 12345
sport = 1040
dport = 80

ip_packet = IP(dst='192.168.56.107')
syn_packet = TCP(sport=sport, dport=dport, flags='S', seq=seq)

packet = ip_packet/syn_packet
synack_response = sr1(packet)

next_seq = seq + 1
my_ack = synack_response.seq + 1

ack_packet = TCP(sport=sport, dport=dport, flags='A', seq=next_seq, ack=my_ack)

send(ip_packet/ack_packet)

payload_packet = TCP(sport=sport, dport=dport, flags='A', seq=next_seq, ack=my_ack)
payload = "GET / HTTP/1.0\r\nHOST: 192.168.56.107\r\n\r\n"

reply, error = sr(ip_packet/payload_packet/payload, multi=1, timeout=1)
for r in reply:
    r[0].show2()
    r[1].show2()

Hope this helps. Basically, the first response you get back does not really hold the HTTP response data. I tested the script against an INETSIM simulated HTTP server and in that case (at least) the first packet (after the 3-way TCP handshake) that the server responded with was a series of NULL (0x00) bytes. Hence using multi somehow did the stuff in my case.