127

Is something wrong with StackOverflow's spam filter? Recently there have been huge numbers of spam topics related to football streaming. They have all been spam flagged and deleted but there are more getting posted all the time, I just flagged another 6 or so a few minutes ago. This has been going on for around 3 days now with these topics getting constantly posted over and over again in the tag.

enter image description here

Improve this question
20
  • 3
    The spam is annoying, and they keep coming over and over again... no-life spammers, bah.
    – Tredged
    Commented Aug 11, 2013 at 17:43
  • 118
    "Is there something wrong with out spam filter": nah, you seem to be working fine ;)
    – Gordon
    Commented Aug 11, 2013 at 17:46
  • 95
    The hell do we have a NFL tag for?
    – animuson StaffMod
    Commented Aug 11, 2013 at 17:47
  • 7
    The spam filters (i.e. users flagging) seem to be working fine to me :) I'm not seeing anywhere this number of flags in the mod queue so you guys are knocking them out very quickly indeed.
    – ChrisF Mod
    Commented Aug 11, 2013 at 17:47
  • 2
    @animuson: I was about to ask the very same thing; nominate for burnination, maybe? Or perhaps anyone asking a question that they tag only 'NFL' has their question dropped, and ip blacklisted?
    – David Thomas
    Commented Aug 11, 2013 at 17:47
  • 19
    @ani keeping all the spam in one easy to find location.
    – Flexo
    Commented Aug 11, 2013 at 17:48
  • 2
    @BadWolf - there is some automatic blocking of known spam, but that has to be maintained and can be circumvented.
    – ChrisF Mod
    Commented Aug 11, 2013 at 17:51
  • 24
    Please don't automatically block it. I'm trying to badge whore here.
    – PeeHaa
    Commented Aug 11, 2013 at 17:52
  • 6
    I'm curious what will happen now that they have no NFL tag to play with...
    – animuson StaffMod
    Commented Aug 11, 2013 at 17:55
  • 3
    @animuson Now that tag is gone they are powerless!!!! chefmonsta.com/wp-content/uploads/2012/02/mini_me.gif
    – PeeHaa
    Commented Aug 11, 2013 at 17:57
  • 3
    It's not recent, that guy's been at it for months. They seem to be good at working around the spam filter and keeping up with evolutions.
    – Gilles 'SO- stop being evil'
    Commented Aug 11, 2013 at 18:01
  • 7
    Man, am I missing out on all the flagging fun?!
    – Martijn Pieters
    Commented Aug 11, 2013 at 18:18
  • 9
    Once again, the use of link shorteners to work around URL blacklists for this spam has me questioning their utility here.
    – Brad Larson
    Commented Aug 11, 2013 at 19:02
  • 2
    @psubsee2003 - The tag seemed legit. I was looking at it the other day because of the spam. There were 6 or so questions tagged nfl, several asking about football APIs. I was going to post here recommending we delete the tag, but then remembered this discussion on Sudoku, in which that tag was not deleted.
    – chue x
    Commented Aug 11, 2013 at 19:11
  • 15
    @chue x: It will be deleted automatically in the next day since there are no questions that use it. It's not the same as sudoku, though: no one is writing a program to run the NFL, and you're right to think that it's unnecessary. If there were an official NFL Data API, then nfl-api or the like would be good, but "Getting schedule data for NFL games" tagged:nfl is like "Getting World Sudoku Championship scores" tagged:sudoku -- the meaning of the data you're trying to get is almost certainly irrelevant to the technical issue.
    – jscs
    Commented Aug 11, 2013 at 19:21

3 Answers 3

Reset to default
97

We're trying to get better at keeping this at bay. The problem affects all sites that see any kind of traffic, some more than others. I can't go into very much detail because frankly, there isn't a whole lot of detail that belongs to an agreed implementation at this point.

However ....

A big mistake we have made in the past is not fully utilizing the signal that we get from moderators and the community when destructive actions are taken. We know when moderators destroy or delete accounts, but we don't know why they did it, at least not programmatically. A change working its way through implementation now tracks this by doing something remarkable, we actually ask them to indicate a reason for the action.

When this is implemented, we have a much easier time querying across the network to better visualize these fools as they move from site to site, occasionally changing origin. This makes the data that they inadvertently leave behind much more valuable when it comes to automatically reacting to, and subduing these sorts of spam floods.

It's a bit of a trick, we don't want to prevent anyone from reading our sites, and even poorly written perl bots deserve a fighting chance in life. Still, we can be a little more picky about content we accept, especially when we have a much clearer picture about the recent behavior of the remote host. I'm not going to go into specifics that I don't yet have to offer, but it is a problem we're taking seriously.

Phase one of this (collecting reasons) should be out soon, then we need some time with the data we get.

That's not the only reason we implemented this, playing chat tag with moderators to get context when someone writes in asking why they were removed is sub optimal. But this is going to give us a much clearer picture of what we're dealing with.

Update

The feature that requires a reason for account destruction is being pushed now, and will be live shortly. For transparency sake, here's the interface (click image for full resolution):

How can I kaboom thee? Let me count the ways ...

As you can see, certain reasons are much more interesting than others when it comes to tracking repeat abuse, since the extra signal is now being captured. Given the volume of crap that some sites see, it won't be too long before we have enough data to confirm / discover patterns.

Improve this answer
11
  • 4
    This is awesome! A simple idea, of course, but brilliantly executed. The UI looks well thought out, and everyone loves data. Except spammers.
    – Cody Gray - on strike
    Commented Aug 12, 2013 at 8:03
  • 1
    In a comment above someone wrote "editing spam is a bad idea (resets spam counter)". If the flags are indeed reset upon editing, then care to edit that into Should Spam posts be edited?, to make that official too?
    – Arjan
    Commented Aug 12, 2013 at 9:29
  • @Arjan I'm going to check with Jarrod to see what changes may still be in the pipe for spam flags, there's some recent UI oddities around them as well, so I'll answer that once I'm sure I'm correct.
    – user50049
    Commented Aug 12, 2013 at 9:43
  • 3
    -1 for Comic Sans :P
    – Luke_0
    Commented Aug 12, 2013 at 13:54
  • 6
    To be clear, editing doesn't invalidate spam flags - rolling back an edited post removes the flags added after the edit, @Arjan. I've edited that post to clarify. The best reason not to edit blatantly off-topic spam is simply that it makes it harder to find and remove though.
    – Shog9
    Commented Aug 12, 2013 at 14:33
  • @TimPost Just used this on a spammer. Thanks.
    – mpdonadio
    Commented Aug 12, 2013 at 16:13
  • 7
    @American Luke: That's not even Comic Sans.
    – BoltClock's a Unicorn
    Commented Aug 14, 2013 at 9:53
  • 1
    @BoltClock'saUnicorn Looks like it at this resolution :P
    – Luke_0
    Commented Aug 14, 2013 at 12:06
  • I'd love to be flagged on my site when a moderator destroys an account on another site. If someone looked at stackexchange.com/users/3208723/user2709183?tab=accounts you would see the sort of spam that propagating the destroy action to all sites where a linked account is present and has posted at least once would accelerate the cleanup until the filters can be tuned.
    – bmike
    Commented Aug 23, 2013 at 18:08
  • It's been a little while, I'm curious to see what patterns you found so far...?
    – Taryn East
    Commented Feb 1, 2015 at 1:24
  • Whole lot of explanation for what a placeholder is…
    – bjb568
    Commented May 12, 2015 at 3:01
132
+200

I've decided to attack the problem at the root, shutting down the spam blogs themselves.

The secret here is to run Google searches using queries like site:blogspot.com watch football (the site: operator as used here limits the search to the blogspot.com domain), click on and examine any suspicious results, and paste the URL of each spam blog into the reporting form on the "Spam, phishing, or malware on Blogger" page.

A few months ago, I did just that, and managed to shut down about 30 such spam blogs, mostly about boxing. If you do this en masse, this will likely catch the attention of Blogger staff, making it more likely that they will take legal or other action on the operators of these blogs since they are probably run by just a few people.

You'll often find that these spam blogs contain distinctive phrases like:

These spam blogs may also use distinctive images, such as the one in this Google Search by Image query. Using the above queries, I have identified and reported more than 200 spam blogs.

The posts in these blogs often include a long list of related terms like "live broadcast, live sop cast, live telecast, live web cast, live HD coverage". This kind of highly aggressive SEO is meant to overrun legitimate websites in search results, and is a clear indicator of spam. As an example on how hard they push SEO, I found this on one of the websites (in a spoiler block so as to not be disruptive):

Rugby,Rugby Live,Rugby Live Free,Rugby Live Stream,Rugby Live Free Online,Rugby Live Stream Online,Rugby Live Stream Online Free,Rugby Live Stream Online Here Now,Rugby,Rugby Live,Rugby Live Free,Rugby Live Stream,Rugby Live Free Online,Rugby Live Stream Online,Rugby Live Stream Online Free,Rugby Live Stream Online Here Now, Free Rugby,Free Rugby Live,Free Rugby Live Stream,Free Rugby Live Stream Online,Free Rugby Live Stream,Free Rugby Live Online,Free Rugby Live Online,Free Rugby,Free Rugby Live,Free Rugby Live Stream,Free Rugby Live Stream Online,Free Rugby Live Stream,Free Rugby Live Online,Free Rugby Live Online

Of course, I will flag any posts on Stack Exchange that are obviously spam as well.

If you try the searches I suggested in this answer and in the comments, you'll find that many of these blogs are now gone:

Blogger blog removed

Improve this answer
12
  • 42
    This is awesome. Salting the roots like this helps to make the Internet a better place.
    – user50049
    Commented Aug 12, 2013 at 5:34
  • 18
    @TimPost: google.com/… – more than 40 blogs reported. I could do this all day...
    – bwDraco
    Commented Aug 12, 2013 at 6:37
  • 7
    google.com/… (sorry, the previous link got messed up)
    – bwDraco
    Commented Aug 19, 2013 at 13:20
  • 11
    You are my hero, by the way.
    – Andrew Barber
    Commented Aug 19, 2013 at 13:45
  • 2
    @AndrewBarber show it by some rep! Oh wait, Mat already did that... ;)
    – Shadow Wizard
    Commented Aug 19, 2013 at 14:05
  • 1
    @ShaWizDowArd Hmm... yes; that would be a great thing to use some of the rep I've gained from all the posts I've earned 'Reversals' on ;)
    – Andrew Barber
    Commented Aug 19, 2013 at 14:07
  • 2
    Here's the bounty notice for posterity: "Awesome Fiery Dragon SPAM hunters are awesome." – Mat i.sstatic.net/nqMy2.png
    – bwDraco
    Commented Aug 19, 2013 at 20:21
  • 6
    It would be nice if Google removed the search results at the same time as they destroyed the blogs.
    – user102937
    Commented Aug 19, 2013 at 21:26
  • I came to this answer myself too and came here to post it :-) Thanks.
    – Preet Sangha
    Commented Aug 24, 2013 at 22:50
  • 1
    A moderator at Academia has referred to this post to handle a similar spam issue. meta.academia.stackexchange.com/questions/903/…
    – bwDraco
    Commented Apr 24, 2014 at 14:35
  • 1
    This is a good solution in terms of taking a proactive approach, although some of these search terms show more than 1,000,000 results. If we resort to manual flagging it will take uncountable resources and efforts. Since there is spam on a massive scale,google can themselves bring such spam down by running these queries on a periodical basis or we should bring forward a public forum where these links can be listed for blogs like google to take action about or search engines should evolve to learn the spam behavior.
    – pal4life
    Commented Apr 29, 2014 at 16:05
  • I cannot joke about how many clickbait YT videos and fake sites put in SEO tags like you mentioned. :/
    – Zackary
    Commented Aug 26, 2018 at 0:07
43

The spam is not new. Drupal Answers was hit really hard with it starting in Dec/Jan. Other sites have had it bad, too (Ask Ubuntu was one). The SE team has done a fantastic job with the automatic filters, but the spam evolves rather often, so automatic blocking isn't effective for long.

The proper course of action is to flag the post as spam. Three spam flags will remove it from the front page, six will delete it. Don't edit it, don't downvote it, don't use another flag. Flag as spam and move on.

If a mod sees it before the autodeletion, they can destroy the account, which will take all spam from that account with it.

Improve this answer
19
  • 24
    +1 for adding to not do anything else to it but flag as spam! This actually affects how quickly the content is nuked!
    – Andrew Barber
    Commented Aug 11, 2013 at 22:49
  • 6
    I know editing spam is a bad idea (resets spam counter) but why is downvoting it bad?
    – Richard Tingle
    Commented Aug 11, 2013 at 23:25
  • 7
    @RichardTingle Downvoting can mean that fewer people see the spam, so it ends up with fewer spam flags, so it actually sticks around longer and/or it doesn't end up with enough flags for auto-deletion.
    – mpdonadio
    Commented Aug 11, 2013 at 23:34
  • why not use a custom flag in cases where autodeletion appears insufficient?
    – John Dvorak
    Commented Aug 12, 2013 at 9:17
  • @AndrewBarber - But if the spam is edited out of the post anyway it is less of an issue that it is not acted on quickly. On slower sites it might be half a day before the spam is removed. Surely it is better it does not contain the promotional content for that length of time thus making the spam pointless.
    – Martin Smith
    Commented Aug 12, 2013 at 10:34
  • 4
    @JanDvorak Spam flags have higher priority in the flag queue. Mods may not see the custom flag until they have cleared others out.
    – mpdonadio
    Commented Aug 12, 2013 at 10:39
  • 3
    @MartinSmith When we are dealing with the flags, it means that we need to go through the edit histories to verify that the flags were correct. This causes more work for the mods.
    – mpdonadio
    Commented Aug 12, 2013 at 10:44
  • @MPD On slower sites I can't imagine this will be a huge problem. I doubt they get that many flags anyway. Probably better to remove the spam from the post and create a small amount of additional work for the moderators then leaving it hanging there for a day. If spammers see that it survives for that length of time it provides an obvious incentive to spam more.
    – Martin Smith
    Commented Aug 12, 2013 at 10:50
  • 5
    Or perhaps spam flags that are unhandled for (say) 30 mins should get pushed into a network wide spam queue so they can be dealt with by moderators from other sites. On sites with only a few moderators it is unlikely there will be 24/7 moderator coverage.
    – Martin Smith
    Commented Aug 12, 2013 at 11:19
  • @Richard, as for invalidating flags, that's not fully accurate. See Shog9's comment.
    – Arjan
    Commented Aug 12, 2013 at 14:34
  • 1
    Why can't moderators nuke the account after deletion? (At least that's what it sounds like)
    – Cole Tobin
    Commented Aug 17, 2013 at 15:35
  • @ColeJohnson Moderators can destroy an account on every moment, if its reputation is not too high. If we notice a flag for spam on an account, we just destroy it, letting the associated posts be automatically deleted.
    – avpaderno
    Commented Aug 21, 2013 at 1:17
  • 1
    @kiamlaluno define "too high" for reputation, and why can't you delete accounts with "too high" of a reputation?
    – Cole Tobin
    Commented Aug 21, 2013 at 1:29
  • @ColeJohnson Only accounts with a reputation lower than 500 can be destroyed; the others can be deleted. The difference is that in the first case the posts created by that account are all deleted; in the second case, only the posts with negative score are deleted, while the rest of them is denormalized. Destroying is disabled because users with a reputation of at least 500 have probably positively contributed to the site, and their posts should not be automatically deleted, when the score is not negative.
    – avpaderno
    Commented Aug 21, 2013 at 1:43
  • 1
    @MPD, By that, do you mean we should be upvoting spam?
    – Pacerier
    Commented Apr 25, 2015 at 23:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .