In the painting as a whole I see a lot of Dutch flags. In the enlarged part of the painting there is what looks like a Dutch flag with letters on it (East or west India Company?). I also see what looks like a yellow flag with something red on it, and possibly horizontal stripes. A building in the background has what looks like the Amsterdam coat of arms.
I suppose it is a Dutch port and there are Dutch and foreign ships in it. I think that looking up the 17th century flags of Dutch and foreign provinces and seaports would be useful.
I note that the flag of Flanders in France & Belgium was yellow with a black lion on it but those look like eagles and not lions.
Most of the north German ports had red and white flags.
The Holy Roman Emperor and the Tsar of Russia had yellow flags with black 2 headed eagles. And some of their subjects might have flown such flags. Just as a stone slab with a carved 2 headed eagle was excavated from the site of the Hanseatic offices in London. The king of the Romans and future emperor used a yellow coat of arms with a single headed eagle but that ship in the painting is unlikely to have used his banner. A coat of arms yellow with a single headed eagle was sometimes also used for the kingdom of Germany and so some German ships MIGHT possibly might have flown a flag with that coat of arms.
You should look in 17th century flag charts for similar flags. In my experience flags with eagles were rare.
https://www.google.com/search?q=17th+century+flag+s&espv=2&biw=1280&bih=894&tbm=isch&tbo=u&source=univ&sa=X&ved=0ahUKEwjC_aTMwdzRAhXLNSYKHSQXDQAQsAQIIA#tbm=isch&q=17th+century+flag+charts&imgrc=Fz03tP66sSBuEM%3A[1]
There was no law against a shipowner having a personal coat of arms and banner of yellow with a black eagle, so it could have been a private flag.
What type of ship flew the eagle flags and how far was it likely to travel? It looks like a small ship to me. Putting banner like flags on staffs at the end of diagonal booms is an unfamiliar practice to me. Are there specific occasions or types of ships that would be done on?
Here is a site for flag identification questions:
http://www.crwflags.com/fotw/flags/ufe.html[1]
Added 04-02-2018 - I note that the eagle flag looks square, like a banner, while most of the other flags seem rectangular, like flags normally flown by ships, which may be a clue.