'Copybot' and forum security

Hello

This is another thread to talk about things on the forum itself, particularly spam. Hopefully the moderators and web project manager can join here and allay any fears about technical risks. The last thread, called 'Chat Bot' was partly about how you tell the difference between a genuine user and abuse, and has reached 167 replies, so it was suggested that we start a new thread for each subject. There's also a thread from the last few months called 'Mods Please Make the Spam Stop', which has covered some of this and also covered the times when obvious spam is left on this forum. I don't personally think it's a massive problem, especially compared to some other forums, but it may make people uneasy unless it's dealt with in a clear way.

As I understand it, and  or  or @WebPM can correct me, every interactive site on the web is subject to some abuse, and the forum software the NAS uses (Telligent Community) has some automated ways to detect and moderate this. However, occasionally some advertising for irrelevant products isn't so obvious, and gets through. There are also some other 'borderline' things, where we're not sure if the user is genuine, and interact with them very cautiously. The way this is supposed to work is that we, the forum users, readers and contributors, help detect the probable spam and click on 'Report as abusive' which pops up when you click the 'More' button below any post or comment. The moderators than consider this, and take action such as locking or deleting the thread. There's also a 'report as abusive' button on each user's profile for occasions when it looks like the only purpose of the account is spamming or trolling.

[Sorry I'm being so verbose.]

The story so far

In the past week or two (May 2018), besides a small spam outbreak advertising pills and stuff, we've noticed what we're calling 'Copybot', which starts new threads by copying something someone real asked several months or years ago. This causes some confusion as people might start responding to these forgeries, not realising the question is very old and has probably been answered. There have been requests, mostly on the other two threads mentioned above, that the NAS checks its site security, and suggestions about how the site could better prevent Copybot.

I've actually only counted six Copybot threads so far, as of 7 June 2018. I think three of these have been deleted and three locked by the moderators, although some stuck around for several days. (Edit: since then there have been quiet periods and times of ten copied threads per week, which I've been listing at the bottom of this thread.)

What is Copybot?

Copybot is the name we (I) gave to whatever was behind the occasion when three threads showed up, from two users, that looked a bit suspicious partly because the two posts from the same account seemed to be from different people: one a parent, the other an autistic young person.  Since then we've had a few more, mostly appearing overnight. The threads look like they come from a new user with no avatar image and the standard "NAS3nnnn" name. The posts are usually well-written and relevant to autistic individuals and families - which is hardly surprising, because it's copying most of the text from another post. The title is usually transformed a little so 'How to find a girlfriend' became 'I can't find girlfriend', and other ones include 'please everybody help me' to get extra attention - this transformation is apparently automated, in a way that recognises some English phrases, and chooses a random variation on it. Occasionally the fake title can be taken from the first sentence of the post instead.  Sometimes people respond to the bot posting as it sounds genuine, but unsurprisingly I've not seen the bot reply. This is stealing people's real concerns and questions, which we find a bit creepy.  Sometimes the text that is copied is truncated, either omitting the sign-off, or stopping at a punctuation mark.

[Here is probably a good place to stop reading.  It may be too much information already.]

Several theories have been suggested as to Copybot's motives, such as that Copybot will eventually post malware links or impersonate a genuine user so well that personal information is compromised. However, I think it is simply a side-effect of trying to defeat anti-spam systems. If a bot registers and starts posting spam immediately, it's likely to get picked up by the automated anti-spam. If it registers, waits a bit, posts something apparently sensible, which people reply to and nobody complains is abusive, then it gains 'reputation', and when it does post spam, it's 'cleanlisted' and the spam appears on the site without moderation, and can go unnoticed which is why it seems to wait over two months to replace the copied text with spam. Also, if the copied post is automatically detected or treated as spam, then the anti-spam text-detection software may get a bit confused (technically this is sometimes called 'poisoning' a Bayesian classifier) and so won't be able to detect adverts for pills and so on so accurately.

A web search for "hi guys, i have a question about" and "i have a question, need help" shows that around June 2018 Copybots also started posting to other forums that use other types of forum software, including phpBB, myBB, vBulletin, Vanilla, Invision Community and Discourse. (Only in a technical Plone Discourse forum did I see someone notice that people were responding to bots, although moderators delete some threads.) The earliest Copybot thread I've found on the web is called "a quick question about business or public courses" on the thoroughly infested "Singapore Expats Forum" dated 26 April 2018, where the content was obviously different originally and then replaced with Vietnamese spam (I'm not linking to it for obvious reasons).

How to check, and what to do if there is a Copybot sighting

I've recently been on the forum a lot, and when I see any new post by someone I don't recognise, I check it. First I look at the post, and think about whether the title is written in a matching style to the text; then I look at the first few words and see if they also appear in the 'Related' bar to the right below one of the titles, and if they do, I look at that other post. I also might hover over the user name or avatar of the NASxxxxx poster to get a pop-up that shows how many 'points' they have; or follow the link on that user name to see their profile. So far, for Copybot, there's been nothing written on the profile, and there are 7 or 14 'points'. (An account gets 7 points for each thread started, and 5 for a reply, so 21 might also be suspicious, but we haven't seen a single account as active as that yet.) You can also check the 'Activity' tab of the profile to see if the posts are consistent and genuine.

If still suspicious, I also see if there are distinctive words or phrases and search to see if those have happened before. For example if the phrase 'depersonalization symptoms' appears, that's pretty rare with an unusual spelling, so I can put that into the search bar at the top and press 'Return' - if it shows a previous thread I check that. You can also check using a standard web search engine, by taking half of a well-written sentence (maybe six to ten words or so), putting double quotation marks (") around it and searching - if it only comes up with the latest NAS page, I'd assume it's not Copybot and we have a welcome post from a new user. If it comes up with other, older hits (I've not seen any from outside the NAS site yet, but it's possible), then I compare the two passages to see if they are more or less identical, and if the new post really is a copy.

If it looks genuine to me, I may like the post, or try to add a quick response, hoping other regulars know I check for Copybots. (It probably isn't appropriate to just say 'you're not a bot' politely, and ignore what the real human poster has said.)

If I find it's a copied post, what I do is:

  1. Reply to the post to warn people that 'this is a copy of a thread from...(however long ago)' and use the word 'Copybot' - this helps find current spam threads without linking to them.
  2. Copy in a full link to the original, genuine thread, (a) so the moderators can verify the copying issue; (b) so people interested in the issue can see other people's responses and contribute their own somewhere that is not likely to be deleted.
  3. Ask the moderators to delete or lock the post, and to moderate the user.
  4. Try not to link to the copybot post from other threads, as that may improve the search engine ranking of the page or bot.
  5. Click 'report as abusive' on the post
  6. Click 'report as abusive' on the user

[OK, it really does get dull and technical after this.]

Then it's up to the moderators to lock or delete the post as appropriate. Maybe more abuse reports from different people catches the moderators' attention more. If someone has added a valuable additional reply, I don't see any problem in locking the thread so that reply, and the link to the original thread, is still available. They may want to reassign the post to 'Deleted user' to prevent the spammy user from posting more copies or spam, but

If no obvious action is taken, then I suppose we can communicate with the moderators by mentioning them in this thread, via Direct Message if we've already had a message from them, or the communitymanager@nas.org.uk address. Forum rules are here by the way: community.autism.org.uk/.../rules

Technical countermeasures

If this becomes a bigger problem, something more may need to be done until Copybot gives up. DongFeng5 suggested using a 'hash' of the text of a post to check for duplicates in an automated way, or use the type of software that claims to score plagiarism by students. I think this is something NAS would have to suggest to the software suppliers as a feature request. I know a bit about this subject (I've written hundreds of anti-spam regexes for a job), and a 'fuzzy hash' should be possible and cope with minor text changes. However, Copybot may also copy anything about autism from other sites so as not to be detected  - someone said copied text from an article about baseball had also been used - or possibly use a Markov-chain text from multiple sources to generate random, but vaguely realistic, text. (We have also seen a short post, probably the same or a different bot, keyed to the forum title by NAS38283.)

Copybot seems unable at the moment to set an alias, avatar photo or profile text on Telligent. Therefore requiring a non-default alias in order to post may stop Copybot until its full features are implemented. It has been suggested requiring some kind of name would at least overcome the problem of not being able to tell the difference between 'NASnnnnn' users. If it is possible to require this in the current forum software settings it would seem worth doing. The accessibility problems with screening signups with ReCapctcha are probably prohibitive given many people with communication difficulties, and a maths Captcha probably wouldn't work. The software does have an option for custom fields to be mandatory. On some other forums, a bot sometimes posted spam in Vietnamese about cosmetics and pills and called itself 'amelinda' or 'philomena', so requiring a non-default alias to post may or may not stop Copybot.

StopForumSpam.com seems to be tracking a lot of related spammers, and there should be a free plugin for SFS for Telligent, although it's not listed on the SFS site. See also Project Honeypot, another free anti-spam service which is basically an IP address blocklist. A simple addition would be to use GeoIP to check for forum submissions from particular Asian countries, or if that's not possible could explicitly ban or firewall the main Vietnamese ranges.

Making the site HTTPS, partly to protect anyone from having their site password compromised if using unencrypted wireless, has also been suggested. This was done in June. It had no effect on Copybot. A related consequent suggestion was permitting non-alphanumeric characters in passwords.

[Oh, blimey. I do go on.]


Etc

We can also use this thread to report any new instances of Copybot, although I think adding a comment identifying it as Copybot and reporting it as abuse, as described above, is better.  Perhaps mentioning the NAS number without linking would show a useful pattern in the spam signups.

The weather forecast for today, Thursday 7th June 2018 is: no Copybot sightings. Nothing on Friday either, so we're doing well. In fact I haven't noticed a peep out of it until:

Saturday 16 June.

  • NAS37990, approx 4am - thread locked around 9pm, user still exists, but presumably moderated.
  • NAS37991, approx 10am - locked by Monday afternoon, user still exists, but probably moderated
  • may be worth checking IP addresses for NAS37988 NAS37989 NAS37994 NAS37995 to see if part of pattern

Tuesday 19 June:

  • NAS38026, approx 4pm - (one reply) thread locked on Wednesday, user in moderation (check ...27 and ...28?)

Thursday 21 June:

  • NAS38049, approx 10am. Two threads both titled 'NEED HELP?', copying parts of different threads, 5 minutes apart. Not locked as of 10:40, reported and deleted some time that day.

Friday 29 June:

  • NAS38140, about 7am. First since HTTPS enabled. Thread taken from 'Autistic adults' category and copied there. Title 'Talking' transformed to 'hi guys, i have a question about Talking, please help me'. Reported 7pm. Locked within a day or two.

Thursday 5 July:

  • NAS38186, approx 5am, copied thread to same forum (Miscellaneous and chat) with inappropriate title 'need help?' Reported, deleted within 24 hours.
  • NAS38188, approx 8am, copied two threads to the same respective forums (adults, h&wb), one replaced as 'How to adult wetting ?', other as 'hi guys, need help'. Reported and later deleted or locked..
  • NAS38187, NAS38185 might also need checking out.

Friday 6 July:

  • NAS38195, approx 8am, copied two threads to their respective forums within a minute of each other. The threads chosen were both 3 or 2 months old and have titles a bit similar to Copybot's replacements, 'Help me, please' and 'Hi there'  respectively. Reported, and deleted within 5 hours.
  • NAS38196, approx 10am, spotted by Graham; copied two threads to their respective forums, one very current, both retitled 'need help?' with differing case. Reported and deleted within 5 hours.
  • It does look like copybots come in clusters or avalanches; either the botmasters step up spamming for money, or it's the weather. The fact they mostly occur in the morning might just be when bots in Asia are online, like vulnerable versions of Windows.

Mon 9 July:

  • NAS38230, copied 'what does the diagnosis mean?' to 'A quick question about diagnosis' (recognising key noun?).
  • about a day later, copied '3099/what-s-mild-autism' to 'hi guys, i have a question, please help me !' Both prompted sincere responses, and had head content replaced by mods and locked after 2 days.

Weds 11 July:

  • NAS38261, posted ~9am, copied 1 week old thread 'Are things JUST different?' from autistic adults to misc-and-chat. copied 'Musicals' to 'Hi there' in introduce yourself. Identified by Martian Tom & Nada..
  • NAS38263, copied 'What is Alpha Stim?' in 'Miscellaneous and chat' forum to 'a quick question about Alpha Stim'.
  • The above five posts still unlocked as of 1pm, but locked as of Thurs am.

Thurs 12 July

  • NAS38270, about 3am, copied two threads, one a week and one three months old to same forums. Titles were both 'i have a question, need help?', but one included a space before the comma. Reported, and both threads deleted the same day.
  • NAS38272, about 12noon (shortly after site was back from scheduled maintenance), copied a thread about a year old to the same forum keeping the same title, 'New to all of this and just saying hello', but dropping the first paragraph. Notably the original thread included today's date, 'july 12'. Reported.

Friday 13, copybotageddon

  • NAS38279 spammed two or three replies (with contact details, may have been essay writing spam) to a current thread. Reported and deleted.
  • NAS38280 at 7.30am copied two (?) threads to same forums, parsing the second title into 'a quick question about classified'. First deleted, second OP text replaced but not locked.
  • NAS38283 around 9am did not copy but used a common Copybot title 'Need help?' and minimal text 'My baby has sinusitis. Is there a way to minimize it? Thanks' possibly keyed to 'Health and Wellbeing' forum title. (Does anyone real use OUP's '-ize' suffix?) Reported user and later post. No apparent action, but user may be moderated.
  • NAS38284 at 8am copied two threads to same forums. The first had been copied before. Possibly these were selected for similar 'hello' and 'question' title patterns. Both deleted promptly.
  • Suspect all these, except possibly the first, are part of the same botnet given timing proximity and similar titles. A regular forum member discusses leaving because of copybot etc.
  • After web research, added edit above about 'other types of forum software' and ideas in 'technical countermeasures'.

Saturday 14 July

  • NAS38296 around 0830 copied a thread, retitling it 'A few question about Autism friendly'. Reported and deleted (or moved to abuse queue; included two new replies; ) within an hour or so.
  • NAS38297 around 0830 copied a thread, retitling it 'question about talking therapy ?'. Reported and promptly deleted.
  • Flashback. Three months earlier: NAS37248 posts a thread 'HELLO!!', apparently copying a post from single mother of an autistic young man. This attracted five replies from genuine users including a moderator.
    • Head post text has now been replaced by Vietnamese (or Lao?) text as if from beauty website and three links - this is similar to what has happened on other forums where the original post has been left for over two months. No other activity recorded from NAS37248. Reported and commented (without link to what it copied since that was unknown), and apparently deleted within two hours along with replies.

Sunday 15 July 8am.... coast clear so far.

Parents
  • It looks like today is mercifully free of copybots. For newbies, please be a little cautious responding to posts started by 'NAS(number)' particularly if the title is something like 'question about (Something), please help'. Nothing terrible will happen if you do respond, but the post may be by a bot as described above, so you'll be wasting your time. Also you might like to make sure your own profile includes a name or an avatar picture so we can recognise you as genuine — just go up to the top right of the page and then Profile > Edit Profile and click on your name or picture to change them.

    It is tedious, but I feel it is time to re-raise the subject of copybot spam with NAS staff and volunteers. On Saturday I went looking through pages on the forum from about three months ago, and found what I expected, a single post to which people had replied and which had been replaced with spam in Vietnamese.  Because it had been replaced by the bot I was unable to find what thread it had copied, but it seemed to be from a single mum looking for services for her son. However, I do have a cache of the thread, including responses by Heather-Mod, DragonCat16, NAS37159 (genuine) and California, You can find it in Google cache by searching for "My son is 24 and was diagnosed with Aspergers when he was 6" which shows how the original post text was replaced with spam. It may be that this spamming technique originated in Vietnam (there's also a public database associating the usernames with Vietnamese IP addresses, and the timing of the postings is consistent with Asian PCs), but the bot transforms titles according to English rules. Therefore the botnet may be hired out to gangs elsewhere in future. The intention is what is often called 'Blackhat SEO', getting more incoming links to a spammy site.

    Having found it, I added a note and reported it and the whole thread was deleted, including a response to my comment by Trainspotter. I hope it's OK by Trainspotter to move that conversation here.


    Cassandro said:
    Looks like our earliest example of a copybot (similar to that seen on other sites). Apparently had copied a post from single mother of an autistic young man, and after 2-3 months has returned and replaced the text with something in Vietnamese about beauty treatments, with three links (probably just spam, but could be malware),

    Could the NAS Moderator please remove (or restore?) the OP text, and moderate NAS37248?

    Trainspotter said:
    Looking at this thread confused my computer.  It crashed at the thought of all that incomprehensible (to me) language.

    {quote]Somewhere in the conditions is that we should converse in English.  That being said, this post should be deleted or changed back to the original and then locked.

    {quote]These 'copybots' often have consecutive ids, seemingly launching a concerted attack..

    And is this going to be one thing they do, change their original post to one offering dubious services with links to even more dubious websites?  I have not followed the lynx, I think the cat may roar, bite and scratch.


    Looking at this thread confused my computer.  It crashed at the thought of all that incomprehensible (to me) language.

    Maybe I should have thought about that before bumping the thread, sorry. It's possible your computer loaded a full Unicode typeface that took up a lot of its memory, just in order to show the accented characters. I'll not post samples of the Vietnamese text, but basically it's about beauty services.

    Somewhere in the conditions is that we should converse in English.

    Remember this is a bot. The conditions were written for people who might read and respond according to rules. The bot doesn't understand rules, and doesn't have rights or a need to communicate. So even if it posts something that doesn't break any of the rules that were meant for contributors, it's still abuse.

    I suppose rule 7 covers this activity for human spammers: 'We do not allow users to register an account with us primarily to advertise, or sell, products and services.' A bot is a user as far as the forum software is concerned, but I would say is not a 'user' in terms of the rules or everyday language. You could argue that the botmaster or programmer was human and could obey the rules. However, they're not interested in forum dialogue either and I bet have never even heard of the National Autistic Society.

    That being said, this post should be deleted or changed back to the original and then locked.

    I think being changed back would have been slightly preferable, so that the subsequent discussion (still in Google cache) wasn't lost. (I'm also curious what the original thread was. It's not in the Wayback archive.)

    These 'copybots' often have consecutive ids, seemingly launching a concerted attack.

    True. There's certainly variation over time, and it's accelerated since 5 July. Copybot will probably not want to be predictable. It may be one bot instance (infected Windows PC, or possibly spammer's own equipment) that registers one account to post two threads, or two accounts to post one thread each. On the other hand, since it posts around the morning, it could be that proper users aren't registering much in that time so the user IDs are consecutive by mere coincidence.

    And is this going to be one thing they do, change their original post to one offering dubious services with links to even more dubious websites?

    I believe so. From looking at its activity on other forums, I think copybot waits for at least two months for any discussion to die down so it can furtively swap in the spam text without being noticed. If it posted a new thread that would of course appear at the top of the discussion list.

    I have not followed the lynx, I think the cat may roar, bite and scratch.

    I'd be lion if I said I was sure there was no risk from the little cheetahs.

    More to come still, I'm afraid.

Reply
  • It looks like today is mercifully free of copybots. For newbies, please be a little cautious responding to posts started by 'NAS(number)' particularly if the title is something like 'question about (Something), please help'. Nothing terrible will happen if you do respond, but the post may be by a bot as described above, so you'll be wasting your time. Also you might like to make sure your own profile includes a name or an avatar picture so we can recognise you as genuine — just go up to the top right of the page and then Profile > Edit Profile and click on your name or picture to change them.

    It is tedious, but I feel it is time to re-raise the subject of copybot spam with NAS staff and volunteers. On Saturday I went looking through pages on the forum from about three months ago, and found what I expected, a single post to which people had replied and which had been replaced with spam in Vietnamese.  Because it had been replaced by the bot I was unable to find what thread it had copied, but it seemed to be from a single mum looking for services for her son. However, I do have a cache of the thread, including responses by Heather-Mod, DragonCat16, NAS37159 (genuine) and California, You can find it in Google cache by searching for "My son is 24 and was diagnosed with Aspergers when he was 6" which shows how the original post text was replaced with spam. It may be that this spamming technique originated in Vietnam (there's also a public database associating the usernames with Vietnamese IP addresses, and the timing of the postings is consistent with Asian PCs), but the bot transforms titles according to English rules. Therefore the botnet may be hired out to gangs elsewhere in future. The intention is what is often called 'Blackhat SEO', getting more incoming links to a spammy site.

    Having found it, I added a note and reported it and the whole thread was deleted, including a response to my comment by Trainspotter. I hope it's OK by Trainspotter to move that conversation here.


    Cassandro said:
    Looks like our earliest example of a copybot (similar to that seen on other sites). Apparently had copied a post from single mother of an autistic young man, and after 2-3 months has returned and replaced the text with something in Vietnamese about beauty treatments, with three links (probably just spam, but could be malware),

    Could the NAS Moderator please remove (or restore?) the OP text, and moderate NAS37248?

    Trainspotter said:
    Looking at this thread confused my computer.  It crashed at the thought of all that incomprehensible (to me) language.

    {quote]Somewhere in the conditions is that we should converse in English.  That being said, this post should be deleted or changed back to the original and then locked.

    {quote]These 'copybots' often have consecutive ids, seemingly launching a concerted attack..

    And is this going to be one thing they do, change their original post to one offering dubious services with links to even more dubious websites?  I have not followed the lynx, I think the cat may roar, bite and scratch.


    Looking at this thread confused my computer.  It crashed at the thought of all that incomprehensible (to me) language.

    Maybe I should have thought about that before bumping the thread, sorry. It's possible your computer loaded a full Unicode typeface that took up a lot of its memory, just in order to show the accented characters. I'll not post samples of the Vietnamese text, but basically it's about beauty services.

    Somewhere in the conditions is that we should converse in English.

    Remember this is a bot. The conditions were written for people who might read and respond according to rules. The bot doesn't understand rules, and doesn't have rights or a need to communicate. So even if it posts something that doesn't break any of the rules that were meant for contributors, it's still abuse.

    I suppose rule 7 covers this activity for human spammers: 'We do not allow users to register an account with us primarily to advertise, or sell, products and services.' A bot is a user as far as the forum software is concerned, but I would say is not a 'user' in terms of the rules or everyday language. You could argue that the botmaster or programmer was human and could obey the rules. However, they're not interested in forum dialogue either and I bet have never even heard of the National Autistic Society.

    That being said, this post should be deleted or changed back to the original and then locked.

    I think being changed back would have been slightly preferable, so that the subsequent discussion (still in Google cache) wasn't lost. (I'm also curious what the original thread was. It's not in the Wayback archive.)

    These 'copybots' often have consecutive ids, seemingly launching a concerted attack.

    True. There's certainly variation over time, and it's accelerated since 5 July. Copybot will probably not want to be predictable. It may be one bot instance (infected Windows PC, or possibly spammer's own equipment) that registers one account to post two threads, or two accounts to post one thread each. On the other hand, since it posts around the morning, it could be that proper users aren't registering much in that time so the user IDs are consecutive by mere coincidence.

    And is this going to be one thing they do, change their original post to one offering dubious services with links to even more dubious websites?

    I believe so. From looking at its activity on other forums, I think copybot waits for at least two months for any discussion to die down so it can furtively swap in the spam text without being noticed. If it posted a new thread that would of course appear at the top of the discussion list.

    I have not followed the lynx, I think the cat may roar, bite and scratch.

    I'd be lion if I said I was sure there was no risk from the little cheetahs.

    More to come still, I'm afraid.

Children
  • I'd be lion if I said I was sure there was no risk from the little cheetahs.

    A purrfect answer and one which gives paws for thought.  And yes siamesey about you re-siting my deleted post. I'm feline fine about that.

    Sorry, I am sure there must be more puns but don't want to invoke the off-topic claws ....