Why not making accounts creation more complicated?

We've seen a dramatic increase of the spam that we get on the forum lately, and I am wondering if it isn't the right moment for increasing the complexity of accounts creation. Would it only be with two captchas, that could pretty easily be implemented : one graphical and why not, as we are on Mathworks, one mathematical (e.g. basic arithmetic with a few levels of parentheses, enough for screwing up most regexp-based parsers)?

Cheers,

Cedric

EDIT - some arguments from my comments below

I don't really care about clicking on multiple threads to see whether they are spam or not. What I do really not like though, is to have to do it multiple times for the same user. I think that any solution which allows a same user account to send multiple posts after one has been closed or flagged as spam by a high enough rep. member, is not adequate.

If I look at the list of members with a rep above even 1000, I don't see anybody who should legitimately not be trusted about using adequately a mechanism which allows to block an account until a high rep member (2k+, 3k+,editor,admin?) decides to unblock or to cancel/delete it. This mechanism could even be limited to accounts with a rep below 25 or 50, because no spammer will ever pass this threshold (see Jon's comment about this point).

EDIT - There could/should also be a list of forbidden domain names for creating TMW accounts, typically those which allow the creation of anonymous/disposable accounts automatically. The rational is that no professional/student should have any valid reason for using such email accounts for creating a TMW account.

[Cedric Wannaz]

.. also, trustees (to define) should be able to put accounts on hold (block until closed or re-opened by an admin/editor).

[Star Strider]

I would also support those at a certain level — perhaps 1500 or 2000 — being able to ‘hide’ (or ‘unhide’) posts such as spam subject to later ‘unhiding’ or deletion by those with appropriate privileges to do so. That would remove the posts from view, and the ability of others with similar privileges to ‘unhide’ potentially legitimate posts before deletion would prevent potential abuse by any hyper-competitive contributors. These would be stored in a ‘hidden posts’ section so they could easily be viewed and reviewed by those with appropriate privileges.

[John D'Errico]

That would help I suppose. Today for example, I caught one spammer soon after their first spam. Deleted it, send the account to the FEX admin person as I always do. They delete those spam accounts in batches after a few days.

This works for many spammers, if they see their spam being deleted immediately after they post, they give up on that account. But this fellow just kept posting one after another about 20 minutes apart. Still watching him. Still deleting after 7 spam "questions" I think.

[Image Analyst]

I don't want to hide them. It's pretty obvious which are spam and I trust the others to just go ahead and delete them immediately. What I don't like are having to delete each one one at at time. Plus I have to go to their account and check Questions, Answers, and Comments to make sure I get all of them. Star should be 2000 this weekend - I'm voting him up to get his help controlling this scourge. Once at 2000, even though he can't delete yet (until 3000), he can at least edit to delete the content. I look for flagged content as a clue to which to delete, so keep doing that until you can delete them yourself. Right now there are only about a dozen people who can delete posts.

[Star Strider]

I’ll do what I can!

I suggested the ‘hide’ option because it would expand the population of those who can help deal with the scourge. The ‘Hidden Questions’ section would get the spam out of sight until those with privileges to do so could deal with them definitively.

Also, I suggest VOTING THIS QUESTION UP both to get it noticed and show support for the concept.

[John D'Errico]

Well, you don't need to click on each of those tabs. On the account page, on the top right, it tells you how many questions, answers, comments the person has posted. When all three numbers are zero, you are done.

[John D'Errico]

Funny though. I just had to delete a spam comment on this very post. So we may have the attention of one of them.

[Image Analyst]

We're in a full onslaught. I've deleted dozens in the last 10 minutes. I think Loreal must be a bot, though I've seen some that have added a Product and tags. Well I have to go to a party now. So it's up to John and the others for the next few hours.

[Image Analyst]

John: I'm pretty sure the comments are not counted and you can't see them unless you click the comments tab.

[Cedric Wannaz]

I've closed/flagged a dozen this afternoon as well.

I don't really care about clicking on multiple threads to see whether they are spam or not. What I do really not like though, is to have to do it multiple times for the same user. I think that any solution which allows a same user account to send multiple posts after one has been closed or flagged as spam by a high enough rep. member, is not adequate.

If I look at the list of members with a rep above even 1000, I don't see anybody who should legitimately not be trusted about using adequately a mechanism which allows to block an account until a high rep member (2k+, 3k+,editor,admin?) decides to unblock the account or to cancel it. This mechanism could even be limited to accounts with a rep below 25 or 50, because no spammer will ever pass this threshold.

[John D'Errico]

True, spammers never manage to get much rep here, as long as we catch them. However, I have seen spammers get their answers accepted. I have seen spammers answer their "friends" questions. So I could see someone getting up to 25 rep or so if they actively worked on it. A moderately large number is probably good.

[John D'Errico]

True. The number of comments are not reported on that page.

[Star Strider]

One reason I want MathWorks to become extremely aggressive — and to allow us to become extremely aggressive — on this issue is that SPAM posts/comments/answers can hide legitimate follow-up comments by the original posters or other legitimate posters that would provide necessary information or alternative solutions to the posted problems.

I do not want to have to open every Comment with a DamnSpam pseudo-Answer or pseudo-Comment to see if someone actually needs clarification or is supplying additional information that I need to provide an appropriately complete Answer+Comments to the original Question.

Allowing this plague of SPAM significantly undermines the purpose and intent of MATLAB Answers and does not improve the reputation of MathWorks for its inability effectively to deal with this problem.

I don’t apologise for my anger in this situation!

[Star Strider]

I just now discovered that Closing a question seems to eliminate it, perhaps ‘hiding’ it. All I know is that while others have flagged them as spam, when I Close a question, it now disappears!

That even seems to have come to the notice of the spammers, who have actually complained about ‘closing questions so quickly’!

Victory?!

[Cedric Wannaz]

Sorry, but no victory ;-) I've always been closing them actually, but it doesn't prevent the same accounts to re-post and re-post again, hence my comment above on having a mechanism for blocking accounts until a higher rep or admin re-opens or deletes them.

[Image Analyst]

It doesn't prevent them from posting again but I did just notice the same thing Star did. I didn't notice any spam in the main listing, but then when I looked at flagged content, it was there. So I think either closing, putting spam in front of the subject line (which per has been doing), or setting a "spam" tag prevents it from showing. At least it hides it from nearly everyone until I can get around to deleting them (and per and star and Cedric shortly). So it's somewhat of a partial victory. By the way I've spent literally hours deleting hundreds of spams from these 43 pieces or garbage who have been wasting my time and keeping me from helping legitimate users. And this list is over just the last 30 hours or so!

http://www.mathworks.com/matlabcentral/answers/contributors/5388126-tukrz
  http://www.mathworks.com/matlabcentral/answers/contributors/5388433-buahsia
http://www.mathworks.com/matlabcentral/answers/contributors/5388431-renhard
http://www.mathworks.com/matlabcentral/answers/contributors/5387392-double-dragon
http://www.mathworks.com/matlabcentral/answers/contributors/5383167-kristian
http://www.mathworks.com/matlabcentral/answers/contributors/5388737-gertarea-grr-la
http://www.mathworks.com/matlabcentral/answers/contributors/5389006-hangingheart3
http://www.mathworks.com/matlabcentral/answers/contributors/5389122-james
23 from http://www.mathworks.com/matlabcentral/answers/contributors/5389285-nicole
13 from http://www.mathworks.com/matlabcentral/answers/contributors/5389150-danial-monark
20 from http://www.mathworks.com/matlabcentral/answers/contributors/5389228-ligheverymatches-sdfdsf
http://www.mathworks.com/matlabcentral/answers/contributors/5389190-kuch-227 
http://www.mathworks.com/matlabcentral/answers/contributors/5388989-mikaela-banes
  http://www.mathworks.com/matlabcentral/answers/contributors/5389374-bgfd
http://www.mathworks.com/matlabcentral/answers/contributors/5389306-arthurking
http://www.mathworks.com/matlabcentral/answers/contributors/5389097-rafin
http://www.mathworks.com/matlabcentral/answers/contributors/5389329-sama-suri 
http://www.mathworks.com/matlabcentral/answers/contributors/5389164-vikas-saluja
5 from http://www.mathworks.com/matlabcentral/answers/contributors/5389252-twilb 
http://www.mathworks.com/matlabcentral/answers/contributors/5389436-hunny
  http://www.mathworks.com/matlabcentral/answers/contributors/5389407-fahim-rana
  http://www.mathworks.com/matlabcentral/answers/contributors/5389362-emanuel44swansonc
http://www.mathworks.com/matlabcentral/answers/contributors/5387072-sunil-narayan
http://www.mathworks.com/matlabcentral/answers/contributors/5388672-songukou-novetas
  http://www.mathworks.com/matlabcentral/answers/contributors/5389514-rahul-shabs
http://www.mathworks.com/matlabcentral/answers/contributors/5383167-kristian
  http://www.mathworks.com/matlabcentral/answers/contributors/5389514-rahul-shabs
  http://www.mathworks.com/matlabcentral/answers/contributors/5388474-ronaldo
  http://www.mathworks.com/matlabcentral/answers/contributors/5389722-samar-gill
  http://www.mathworks.com/matlabcentral/answers/contributors/5389717-jassi
http://www.mathworks.com/matlabcentral/answers/contributors/5389742-arocker
  http://www.mathworks.com/matlabcentral/answers/contributors/5389150-danial-monark
http://www.mathworks.com/matlabcentral/answers/contributors/5389814-shannon-scratchley
  http://www.mathworks.com/matlabcentral/answers/contributors/5387392-double-dragon
  http://www.mathworks.com/matlabcentral/answers/contributors/5389821-loreal
http://www.mathworks.com/matlabcentral/answers/contributors/5389826-ayame
  http://www.mathworks.com/matlabcentral/answers/contributors/5390052-jessica
http://www.mathworks.com/matlabcentral/answers/contributors/5388672-songukou-novetas
12 spam from http://www.mathworks.com/matlabcentral/answers/contributors/5389252-twilb
15 spam from http://www.mathworks.com/matlabcentral/answers/contributors/5390041-willis-chahate
http://www.mathworks.com/matlabcentral/answers/contributors/5389362-emanuel44swansonc
http://www.mathworks.com/matlabcentral/answers/contributors/5384442-navar-rohart
http://www.mathworks.com/matlabcentral/answers/contributors/5389959-lee

[Star Strider]

Cedric — I would agree (since you’re usually correct), but my closing them has resulted in their magically disappearing only in the last two hours or so. (I was away for a while, so it could have been longer.)

I noticed that while Per flagged them as ‘spam’ and changed the tags to ‘spam’, only my closing them made them disappear. (I assume Per could have done so as well, but he likely didn’t realise that ‘Closing’ them was now more than a symbolic act. I’m still several RPs short of 2K, so my status hasn’t changed.)

Closing = ‘hiding’ (or deleting?) spam wasn’t the situation until some time this afternoon (GMT-6), so mayhap you made a definite and positive difference with this thread?

If so, I would recommend you be knighted for services to MATLAB Answers!

[Image Analyst]

I'm voting all you guys up, at least until you reach 3000 points because we need all the help we can get. I hope you all do the same.

[Star Strider]

Agreed!

[per isakson]

Got it! Now I have realized and tested [Close].

[Star Strider]

Yes!

Apparently they have to be flagged as ‘spam’ as well. Cover all the options and we’ll be rid of the scourge!

ADDENDUM: I never realised that zapping spam could be so much fun!

[Image Analyst]

It is a pain in the neck though there is some delight and sense of accomplishment in being able to say "Touche, gotcha! Take that you sleazy spammer!" They waste my time and I'll waste theirs right back.

[Star Strider]

Definitely! It’s almost like Skyrim! (Guilty pleasure.)

I voted up Cedric as much as I could find (honourably, in my opinion), and voted dpb to 1500 in the process, so since for those who are >= 1500:

Delete_spam = flag_as_spam + Close;

We can recruit dpb to the Cause as well. We simply need to inform him of such.

[Cedric Wannaz]

Star will hit 3k next month if he holds the rhythm ;-) and well, give me another year or so and I'll be there too. Here are a few additional remarks..

• As I already expressed in a couple posts last year, I think that it is a big mistake to have maintenance tasks reserved to high rep. members. They have the most valuable experience, yet they spend their time cleaning, formatting, etc.
• I think that there should be a "trustee" status which allows members who are flagged as trustees by high reps, editors, and admins, to perform these tasks.
• As already developed here and here, there would be ways to proceed which still prevent trustees to have all privileges. For example, if trustees can only add/remove white spaces at the beginning of lines when they edit a post, the most they can do is to format the code right, which is what is needed ;-)
• Thinking about it again, the forum should really make the distinction between having privileges and being doomed with the implicit duty of cleaning the forum.
• Another example of what a trustee could do is described in my question: block an account temporarily (e.g. of people with a rep below 50 only, who appear to be spamming), until someone with privileges checks the case and decides to suspend/delete the account or to unblock it. The list that you made with the counts of spam posts per account, Image Analyst, supports this proposal: it is absolutely not normal that spammers can come back nearly 30 times a day and post their crap, when more than 10 members with reps ranging from 1k+ to 17k are present.
• I hate throwing cheap assumptions./accusations like that, but I guess that if we/you go on flagging/closing/deleting the spam, TMW will have little incentive to change anything.

On that note, have a good night! Well, I'll pass Star 2k and then good night!

[Star Strider]

I agree.

W.R.T. your last item, I sort of see TMW’s side of this (although I don’t agree with it) of allowing essentially open access to MATLAB Answers.

I would like to have all accounts attached to individual or institutional MATLAB licenses. Secondary ‘anonymous’ accounts would be allowed for those who prefer not to reveal their identities. (Mine is actually tied to my license. Although I use my pseudonym here, TMW knows who I am, and my primary e-mail address.)

It simply seems incongruous to me that someone without a valid MATLAB license of some sort ( i.e. owning some version of MATLAB or having institutional access to it ) would have any genuine interest in MATLAB Answers.

Thank you, Cedric!

[Image Analyst]

A good proportion of the high rep members are actually Mathworks employees, though you might not know unless you look at their profile, and sometimes not even then. They always have full privileges though ,sometimes, they oddly don't use them. I've seen some Mathworkers flag spam rather than delete it, leaving it for us to delete.

[Cedric Wannaz]

--- TEST -----------------------------------------------------------------

(Not all accounts/emails below are spammers; I am using refs to regular users for testing a new tool.

SPAM_ACCOUNTS

 http://www.mathworks.com/matlabcentral/answers/contributors/5449160-mytah
 5451368-dss-fs-df-s
 5451420-valdez-vega
 http://www.mathworks.com/matlabcentral/answers/contributors/5452193-ricky

SPAM_NAMES

mytah, John Keen,

SPAM_EMAILS

 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail] 
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]  
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]
 [e-mail]

SPAM_PROVIDERS

 guerrillamail.com
 guerrillamail.net
 guerrillamail.org
 guerrillamail.biz
 guerrillamail.de
 guerrillamailblock.com
 sharklasers.com
 spam4.me
 my10minutemail.com
 teleworm.us

SPAM_KEYWORDS

 movie, Throne, Game, Live

SPAM_REPORT

gmail.com, yahoo.com, hotmail.com, outlook.com

[Image Analyst]

What's this test?

[Cedric Wannaz]

Hi Image Analyst,

I am building a tool during my lunch pauses for scanning threads and spotting spam. I will probably post it over the week end. One feature is that it uses a database of accounts/emails/etc which lies in a thread (the thread that I will use for posting the tool). So I am just testing the mechanism using this old thread of mine.

Cheers,

Cedric

[Image Analyst]

OK. Did you see my FindSpam code I posted elsewhere? I don't bother looking for emails since some of those will still exist even after the Mathworks hears about them. They seem to just disable them rather than delete them entirely. Plus once the emails have been used for a few days, they generally don't use them anymore. They seem to have gotten adept at creating new accounts almost instantaneously, so checking old emails seems like a waste of time. I just do a general search for terms and web sites commonly contained in the spam.

[James Tursa]

Do you want the community to stop editing out spam links in the short term while you make tests?

[Image Analyst]

No, you can still do that as far as I'm concerned. There's enough coming in that we can still "harvest" content before I delete them. However, it's more convenient for me to find them if you just set a flag for spam rather than close them. I know they're still there but it's easier to find than if they're closed, where I have to do several pages of "recent activity" and search for the "closed" word. You can go ahead and delete the content before you flag it as spam.

The one thing I'm not sure about is if Cedric's search will find the stuff anymore since it's now "collapsed" in the "old comments". It may not find it since it's (perhaps) not returned from urlread(). If that's the case, he can make it an "Answer".

[Cedric Wannaz]

Hi James and Image Analyst,

Thank you for asking, but I have enough "specimens" to feed my engine during the development phase. It is not going to be overkill though, because I had only a limited amount of time (a few lunch pauses), but it may be useful, in particular to check that accounts known to have spammed are not active anymore.

I saw your script Image Analyst, and I am just extending a bit the approach with a crawler which checks flags, closed threads, known accounts, and analyzes threads' content, so we don't have to open everything to check.

After playing a bit with the first version, I realize that the most interesting, as far as I am concerned, is the tracking of known accounts, so we can check whether TMW deletes them or not.

Cheers,

Cedric

[Image Analyst]

I'm not sure that's that interesting because I know for a fact that they don't delete them . I've checked ones I've sent them back in May and they're still there. I think they just disable them, but I can't even be sure about that - how could I tell? But anyway I'd be interested in seeing your program when it's fleshed out enough to share.

[Cedric Wannaz]

I've also seen these old accounts, reported many times already, which irritates me. If TMW takes spam very seriously (as mentioned below), there is no objective reason for keeping spammers' accounts, .. I am hoping that if we provide a list of these accounts using the official support channels, they will be deleted. But for that we need to keep track of them for a moment.

Answer 1

[Image Analyst]

I don't think they're bots posting. I actually saw one person respond to the spammer saying that the tv/movie web sites were probably illegal and the spammer actually responded saying he didn't think so. And each account only posts 1 to 20 or so posts. So I think that actual real live people are posting , not bots. We've discussed ways before, such as suspending an account if the editors delete more than 2 posts or the account has any posts flagged as spam, but the Mathworks is keeping any possible solutions secret for now. All I know is that they're working on some kind of solution but I don't know what form that will take.

By the way, Steve's blog has a captcha where you have to do math to post a comment. And Google is famous for that, like in their help wanted ads or billboards like this:

[Cedric Wannaz]

To me this looks very much like a bot with someone behind tweaking a little the content occasionally to try to get more outbound clicks. Many of these emails actually came from services which allow automatic emails generation.

I'll use my bots to post your billboard above tomorrow as a question here, and see what I get in return ;-)

[Star Strider]

That’s an experiment that really needs to be done!

[Image Analyst]

I've deleted over a hundred spam since yesterday from over 20 spammer accounts. One of them had edited their account so I doubt that one was a bot but I don't doubt that some of them are bots.

[Image Analyst]

And I've seen some add tags or copy other posts partially. That would be a pretty fancy bot.

[John D'Errico]

In case anyone cares, looks like ... 7427466391, in the form of digits 100:109. An interesting variation might be to find the first point where the smallest possible 10 digit prime occurs. Or the largest, take your pick. (You need to go out a ways just to find the smallest possible 5 digit prime.)

e = hpf('e',[500100,0]);
ed = e.Migits;
ed10 = [ed(ind);ed(ind+1);ed(ind+2);ed(ind+3);ed(ind+4);ed(ind+5);ed(ind+6);ed(ind+7);ed(ind+8);ed(ind+9)]';
n = 500100;
ind = 1:(n-9);

find(isprime(base2dec(char(ed10(1:200,:) + '0'),10)))
ans =
   100
   124
   150
   172
   183

ed(100:109)
ans =
     7     4     2     7     4     6     6     3     9     1

isprime(7427466391)
ans =
     1

[Image Analyst]

So what is that? Is it Google's main site?

[Cedric Wannaz]

[Spoiler alert] =>

[Star Strider]

Interesting!

This was 10 years ago?

[Cedric Wannaz]

With hpf in the code written by Jon above available here.

[John D'Errico]

Just for kicks, I had to look for larger primes. It is surprising that to find the first 500 digit prime in the digits of pi one only need to look as far as digits 35:534 of pi.

Answer 2

[John Kelly]

At MathWorks we are also frustrated with the recent trend of spam being published to MATLAB Answers. It is a top priority of the MATLAB Answers Team to have more efficient spam controls in place very soon.

[Star Strider]

Letting us flag them as spam, then locking them and by that process getting them out of view, was a huge improvement.

[Cedric Wannaz]

I appreciate the effort, but I maintain my point above: it is non-sense that e.g. account "Danial Monark" could come back 30 times over the week end and re-post spam, leaving us with the only option to flag 30 times the crap (and to close it when it was a "question") and leaving Image Analyst (a forum editor with ~17k rep.) with the only option to delete 30 times the posts. And this was just for one spamming account, when dozens have been used over the week end.

I think that forum editors, at least, should have the option to block an account (if only from posting on the forum, and limited to low rep. accounts), until a TMW admin. decides to re-open the account or to delete it.

[Kelly Kearney]

I assume there's no benefit to multiple people flagging content? I've come across several posts that have already been flagged as spam, but not yet removed by someone with power... perhaps there could be some procedure put in place where lower-rep members could combine their points to remove a post.

[Star Strider]

Those who now have the ability to lock Questions (RP >= 1500) and make questions flagged as ‘spam’ disappear, can’t see the flags unless we click on the Questions. Eliminating the extant Tags and replacing them with ‘spam’ helps.

A Question flagged as ‘Spam’ should have a visible signal symbol that we can easily see. Perhaps simply displaying the ‘Flag’ symbol on the Question — probably on the right margin — would be enough. If I knew to look for it there, it would immediately draw my attention.

A red ‘Flag’ symbol would indicate a flagged Question, yellow an Answer or Comment that need to have the spam edited out (RP >= 2000) or the Answer or Comment deleted (RP >= 3000).

[John D'Errico]

By the way, adding a spam tag is quite helpful for me. That way I can search for any recent spam and delete it.