Real Comment Throttle plugin 0.1

Time I got back in several saddles, not the least of them Movable Type plugin writing. I’ve been thinking today about posting like it’s a dead fly on your tongue, and about posting like you didn’t know that weblog software company employees have to always remember the banjo, and their code equivalent, Release Early, Release Often. Of course, to make that work you really need to do both parts, not just the first, my usual course of action, but my intentions are, once again, good. As are commas.

Real Comment Throttle (the name has a wink in it, but an actual ;) would wind up causing me filename trouble) is a plugin to add the throttle that Jacques and I have been using since last January, to limit our exposure to floods of comments.

The builtin MT comment throttle, while well-meaning, is basically useless for anything but the most clueless of spammers and trolls. It blocks based solely on IP address, and any spammer worthy of even that low calling has thousands of anonymous proxies at his disposal, so he can still drop hundreds of comment-turds per hour in your weblog.

RCT simply refuses to accept more than the number of comments you specify, per hour and per day. If you’re sure you’ll never get more than 150 comments in 24 hours, and you don’t want to have to clean up after more than 50 in an hour, it’ll put on the brakes once you’ve got that many, waiting for you to flush out the crap or realize that you’ve actually become insanely popular (my most popular content-free entry did actually manage to trip my throttle with legitimate comments). At least with this plugin version, when you become popular beyond your wildest dreams, you only have to go to the plugin config page to increase your limits, unlike the hack that required you to find the magic numbers in the source.

There’s still plenty to be done to it: it only lets you set one limit, not a different limit for each blog (if your limit is 50 per hour, each blog can have 50 comments per hour, but you can’t set your main blog to 50 and your rarely-commented exercise blog to 5), it only notifies you that it has been triggered in the activity log, instead of emailing you (only once, which is where I’m stuck), so far you can’t return a custom message for a CommentThrottleFilter so even if you’re full of comments for the day it’ll still say “please wait a short while and try again,” it should really send throttled comments to moderation rather than refuse them, and the code is horribly inelegant (patches, or instruction, welcome), but it seems to work for me, and as long as you don’t have any pets that aren’t neutered, I think it’s pretty safe to use.

Requirements: the Storable Perl module, so it can store your settings in PluginData, and MT 3.1. Oh, 3.1 isn’t out yet, not until tomorrow? Well, I did say I was going to release early. Enjoy.

33 Comments

Comment by Jacques Distler #
2004-08-30 23:54:38

Nice work, Phil!

I owe you a beer (or two or three).

 
Comment by Anil #
2004-08-31 00:11:08

Posting a plugin without screenshots? For shame!

Comment by Phil Ringnalda #
2004-08-31 00:28:31

While I’m quite proud of the visual design (I chose to put the labels for the two text fields where you enter the two numbers before the fields themselves!), I think a screenshot might be a bit overboard. Maybe when I add in the select list to choose which blog (err, at which point I ought to strip the select list from the header.tmpl, since right now it looks like it might select which weblog you’re adjusting the throttle for, not which weblog you could jump to, leaving the plugin behind).

 
 
Comment by Gregory Blake #
2004-08-31 10:22:41

So, not related question… How are you doing the threaded comments? Is there a plugin out there that works with 3.0/3.1?

Comment by Phil Ringnalda #
2004-08-31 22:16:38

Still MTThreadedComments. It won’t work with dynamic archives until someone (bleah, that’s probably me, isn’t it?) ports it to PHP, and the patch has gotten a little bit-rot, but it’s still pretty easy to make the hacks by hand, and works just fine with 3.x once you do.

 
 
Comment by Luke Hutteman #
2004-08-31 21:24:22

Unless I misunderstand how this works, I don’t think I like this idea.

If I have a 50 comments/hour throttle and get spam-bombed with say 100 messages, this plugin would block at least half of the spam, but at the price of then blocking all comments to my blog (including legitimate ones). I’d still have a bunch of junk to clean up, but (using MT-BlackList) this won’t take much longer for 100 messages as it would for 50.

The only thing I see that this may be helpful against is malicious spammers that leave random (not easily blacklisted/mass-deleteable) comments using random proxy-servers. Most of us deal with spammers that have something to sell though, which tends to make them much easier blacklisted and wiped.

or am I missing something?

Comment by Phil Ringnalda #
2004-08-31 22:13:50

Nope, I don’t think you are missing anything. If you get 100 comments all linking to one URL (or 50 each linking to two URLs), then MT-Blacklist (or maybe even just MT itself, which will show you all comments from a name, email, IP, or TypeKey identity, though oddly not a URL) will fix you right up.

However, you and I both know a spammer who has at least 50 domains that I’ve seen, and who knows how many more. Spamming through redirects is quite effective, so any free PHP host (do they still exist?) is an infinite number of URLs, unless you are willing to block the root host. For that matter, doesn’t Google follow <meta http-equiv="Refresh" ... />? That makes every free site on earth an infinite number of URLs: two hundred blog comments pointing to a page that refreshes to the spammer’s site (or even just has one link, to her site) might not be as valuable as two hundred direct links from comments, but it’s certainly more valuable than zero links.

But there’s more to being flooded with comments than just spammer psychology, or how many clicks are required to delete the spam. I get email notification of new comments, like most MT users do. If I get 100 comments in an hour, or 1000 in a day, I then download all those emails, over dialup. Eh, could be worse: there are still parts of the world where people pay per minute, or by the byte. Every single one of those comments triggers a rebuild, of the individual entry archive, of every archive where the entry appears (monthly, weekly, daily, category, individual entry RSS, …), and of all index templates. I know several people who have been kicked off their server because of the load caused by comment flooding, and I wouldn’t be surprised if some of the people with corrupted Berkeley DBs can blame it on floods of comments, too. And just because the original crapflooders who wrote FloodMT have gotten bored doesn’t mean there aren’t other morons around, and copies of FloodMT. Like the one I have. My proxy list is a little stale, but with some cleanup and freshening, I could give you a thousand comments with nothing to blacklist or filter by to mass delete them, in just a few minutes.

In the worst case, no common IP, no common email, no common URL, having to delete one by one, how many comments are you willing to delete in one day? How many comment notification emails are you willing to get? I’ve decided that there’s a maximum number for me, a number much higher than the number of real comments I’ve ever gotten in all but one day. I’ve had far more than that, several times, before I started throttling, and I didn’t like it in the least.

If your throttle gets triggered, then real commenters will get a (currently inaccurate) message saying what’s happened, on a Comment Error template page, where their comment is still there, waiting to be resubmitted. If they really want to leave the comment, they can just leave that tab/window open, and wait, or copy it out to an email. If you don’t have a throttle, and they submit it in the midst of a huge flood of spam or crap, judging by the sorts of entries I’ve seen people write in the aftermath, they are likely to just have their comment deleted as the victim says screw it and deletes everything that came in after the start of the flood.

I certainly wouldn’t insist that anyone use it (well, having advocated for including it in the MT core is, kinda, but I wouldn’t object to an easy way to turn it off), but for those of us who know why we want it? We want it. Given time, and a better plugin architecture, I’ll make it better, and less drastic, but I’m not going to do without it.

 
 
Comment by Jacques Distler #
2004-09-01 00:55:00

How quickly they forget …

Crapflooding was a real problem for MT users, bringing many to tears. Getting 5000 comments in the space of an hour will cause gnarly server loads, database corruption, and other evils.

The N comments/hour limit is supposed to contain a crapflood and prevent your system from being rendered unusable.

The M comments/day limit was intended as more of a comment-management measure. Sifting through 5000 comments from hundreds of distinct (anonymous proxy) IP addresses, with random text, random ”author”s, etc, is tedious task, which is hard to automate (unless you just decide to delete all comments from the past 24 hours). Not to mention the notification emails …

It seemed to me that putting an upper bound on the total number of comments/day (smaller than 24 times the limit on the number of comments/hour) was a sensible precaution.

Despite being crapflooded, I’ve never had that throttle tripped.

Personally, I think it would be very, very foolish to run a blog without a comment throttle of some sort (and I don’t mean the useless built-in one that comes with MT). But that’s just me.

 
Comment by Gina #
2005-05-21 20:31:49

Since there wasn’t an MT version compatibility note for this plugin, I downloaded it on chance. Um…I don’t have a ”Configure Active Plugins list” Does this work with 2.661 at all? If not, is there something similar out there? Thanks. I’m just not able to upgrade yet, and I’m desperate to find a solution like this one.

Comment by Phil Ringnalda #
2005-05-21 21:12:24

Requirements: the Storable Perl module, so it can store your settings in PluginData, and MT 3.1.

The hook that it uses to have MT call it, and allow it to refuse a comment, wasn’t in 2.6x, so your only option is to hack at the core. The words ”Jacques” and ”I” in the second paragraph link to descriptions of how we were doing that.

 
 
Comment by Gary LaPointe #
2005-09-15 03:22:43

I want a comment filter to keep track of the last 30 URLs (just the domains) it has had in the the comments/trackback and site URL field. It would also keep track of how many times that URL has appeared in the last 24 hours and when it’s a number like 15 times, it would automatically moderate or delete it. (Of course all these numbers would be adjustable). Of my 61,000 spams in the last 4 months, at least 65-75% of it has the same URL in it that appeared in the 2 hours (generally 20 minutes). Who would be legitimately be posting that many comments/trabacks?

This seems like it would be very little server load also.

I don’t mind moderating, I do mind it getting by. I probably do delete 98% of all moderated mail BUT 90% of that is groups of 20-200 messages at the same time (from the previous 2-8 hours).

 
Trackback by Burningbird #
2004-09-01 10:21:40

Bad Mans Find Good Woman

I’ve been hit severals time recently with comment spams. In fact the frequency of attack has really picked up as WordPress has become more popular.

I don’t use any form of blacklist, but controlling the spam is still pretty trivial. There’s one …

 
Trackback by Amanita.net #
2004-12-16 09:12:15

links for 2004-12-16

LiveJournal Latest Posts Sneak a peek at what people are saying. There are so many posts per minute that even an immediate refresh won’t…

 
Trackback by Musings #
2004-12-17 14:09:12

The Pace of Innovation

Spambot authors take a cue from the crapflooders and make MT users cry.

 
Trackback by <Tag> memes #
2005-01-06 00:17:31

comment spam

Six Apart, Movable Type developers, have just published a weblogger’s guide to combatting comment spam. read the Six Apart Guide to Comment Spam online and/or download it in pdf from Six Apart’s Guide page. the guide points to some useful…

 
2005-01-11 19:57:08

We’re Rollin’

Tech support from my host wrote me back today. I guess I have to take back my premature rant, because this actually went very smoothly. Maybe the incident from this summer was an anomaly. They installed Storable perl module, so…

 
Trackback by Client and Server #
2005-01-19 19:32:23

Gonna be some changes made

The comment spam problem is growing at a geometric pace. Since I upgraded mt-blacklist six months ago, I’ve had more than 7,000 comments moderated or denied. Comment spammers now make up about 75% of the hits on this site. It’s…

 
Trackback by Procrastination #
2005-02-11 06:24:44

Front-end and Back-end Changes

There have been a lot of changes here recently, most of them on the back-end. Most of this work was related to having a bilingual (English and Urdu) blog along with MathML equations. This required valid XHTML 1.1 and serving…

 
2005-02-11 16:58:10

on the move to MT 3

I seem to be taking ”better late than never” to its ultimate extreme. It’s getting close to a year since MT 3.0 was released (I think about nine or ten months now). All this time, I’ve still been on MT…

 
Trackback by Sans Telos #
2005-02-17 08:26:27

Spam

I got out of my New Testament Literature exam today with a mission: update MovableType, spam-proof it per [this article](http://www.sixapart.com/pronet/comment_spam.html ”Six Apart Guide to Comment Spam”), and write a post about it. Commence phase thre…

 
Trackback by The McWetlog #
2005-03-06 07:32:38

Movable Type upgrade, part two — fixing comments

Comments — which on my Movable Type install are only enabled on The Map Room at this point — went blooey a bit. At the outset, I tried enabling TypeKey registration and moderating unregistered comments: TypeKey logins would be posted…

 
Trackback by birdhouse.org #
2005-03-19 17:28:14

Field Notes on Comment Registration

From the perspective of a web host with a dozen customers running MT weblogs, I can confirm what many hosts have reported before: At the server level, massive comment spam blitzriegs are effectively denial-of-service attacks. Every comment submission i…

 
Trackback by skitz.org #
2005-03-29 06:35:32

Comments and Spam

Since deciding to allow comments on the site, I figured I should probably do my best to stem the flow…

 
Trackback by 4 Banalitaten #
2005-04-27 05:13:52

Prima o poi doveva capitare

Da quando uso Movable Type il problema dello spam nei commenti ero riuscito ad arginarlo decentemente. Ieri però l’attacco è stato pesante (mi scuso se nel fare pulizia ho eliminato qualche commento legittimo). Il trucchetto di cambiare nome allo scr…

 
2005-10-24 18:58:24

We’re Rollin’

Tech support from my host wrote me back today. I guess I have to take back my premature rant, because this actually went very smoothly. Maybe the incident from this summer was an anomaly. They installed Storable perl module, so…

 
Comment by Gina #
2005-11-01 20:21:04

Does this work with 3.2? Thanks.

Comment by Phil Ringnalda #
2005-11-01 20:29:39

Near as I can remember from back when I used Movable Type, yes, but I wouldn’t guarantee it right now, and at the moment I don’t have a working MT install to test it on. If you want to install it, though, I’ve got some attack scripts I could use to trigger it for you ;)

Comment by Gina #
2005-11-02 09:31:39

Hm, seems to work, except that when I hit the maximum hourly comments, it just gives the message ”In an effort to curb malicious comment posting by abusive users, I’ve enabled a feature that requires a weblog commenter to wait a short amount of time before being able to post again. Please try to post your comment again in a short while. Thanks for your patience.”.

Didn’t it used to give a different error message, such that it let the user know that the maximum had been reached so don’t keep trying for awhile? LOL. I don’t want to frustrate my commenters if I get spammed.

Comment by Phil Ringnalda #
2005-11-02 09:45:26

Hard to believe I’ve gotten so forgetful that I have to quote myself from just a year ago to remember things, but:

so far you can’t return a custom message for a CommentThrottleFilter so even if you’re full of comments for the day it’ll still say “please wait a short while and try again,”

from up there at the top of the page.

Interesting to see that at the time I saw dropping them on the floor, rather than sending them to moderation, as a bug, since now I’m claiming the opposite.

Comment by Gina #
2005-11-02 11:27:18

Sorry, I’m blind and missed it in the text. Hmm, maybe I can edit the error in MT then to something more…helpful. :) Thanks. Are you going to continue upgrading this valuable (IMO) plugin now that you’ve switched to WP? I’ve tried to love WP, I really have, but I fiddled with it for two months and have yet to produce my blog the way I want, so I downloaded MT again, *shrug*.

Comment by Phil Ringnalda #
2005-11-02 21:17:26

Okay, for you, and for Neil, and for all the good times, we’ll have Fabulous Parting Gifts, and I’ll do a new version for MT 3.2 this weekend.

 
 
 
 
 
 
Comment by Gina #
2005-11-03 18:49:04

Whoo Hoo!!!! Thank you :) :)

 
2006-07-09 21:48:41

[…] Real Comment ThrottleWhat this plugin does is to limit the total number of comments coming into your blog within a certain time period. When spam causes this limit to be breached, the plugin will automatically stop all further comments from coming in. However, while this plugin is reasonably effective in tackling spam spikes, it will also prevent legitimate comments from coming in after the set limit has been breached. Support for MT 3.2 and above does not seem to be available though. […]

 
Name (required)
E-mail (required - never shown publicly)
URI
Your Comment (smaller size | larger size)
You may use <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <del datetime="" cite=""> <dd> <dl> <dt> <em> <i> <ins datetime="" cite=""> <kbd> <li> <ol> <p> <pre> <q cite=""> <samp> <strong> <sub> <sup> <ul> in your comment.