The most basic way to cut up text is into characters, and this does carry useful signal. A naïve guess, for example, would be that em dashes (—) signify a comment was generated by a robot. That’s true! Other such typographical marks robots produce more of than humans are semicolons (;), unicode arrows (→), and ellipses (…).

Character frequency analysis also tells us robots generate more syntactically complex comments than humans: they contain more full stops (.), commata (,), parentheses (( and )), and line breaks. Robots also surround literal strings with straight, ascii quotation marks (") more than humans do, e.g. to reference text in user interfaces. In general, robots use a wider variety of symbols than humans, and produce text with a higher symbol density.

There are two groups of symbols that appear more often in human text:

• Colons (:) as separators introducing the next part, as I did in the line leading up to this list.
• Symbols that come from code or parts of code included in comments, e.g. dollar signs ($), less-than and greater-than signs (< and >), backticks (`), equals signs (=), exclamation marks (!), square brackets ([ and ]).

Robots are less likely to include code in comments than humans are, and also less likely to write TODO comments.

The next level up in language abstraction might be words. However, at that point we have to be careful. Due to the way language works (with documents often containing unique words), there’s a risk a word-frequency based classifier learns differences in subject matter rather than style. To avoid subject matter leaking in, we’ll count the frequencies of common, non-subject words only. These are sometimes known as function words.

You know what? Let’s turn it into a fun game!

The list below contains eight groups of words. Some of the groups are more common in human-written comments, and some of the groups are more common in robot-generated comments. For each row, mark it as either “human” or “robot” and see how many you get right!

1. your of my other over since enough they be few towards
2. which how this have about down could out much should
3. must everything before away among the thousand past until their
4. while every both one each none within per nothing once outside
5. so its whether against through back would across up first several
6. therefore yet when off still inside around after from at without
7. more though his can although but except such second either
8. we you who I will because some there these why along our

Done? Then you can check your answers. The first and last two groups (1–2 and 7–8) contain words often used in human-written comments. The middle four groups (3–6) contain words often used in robot-generated comments.

What we find is that robots generate comments that contain more prepositions and directions (through, up, among, within, around, after, at, etc.), whereas humans write comments with more pronouns, qualifications, and story-weaving words (we, you, because, although, except, should, either, some, etc.)

The next step up in abstraction from words might be part-of-speech (pos) tags. A pos tagger replaces words with their grammatical category, which can uncover sentence structures at a higher level than what’s visible through words alone.

I have to admit I’m not very skilled with grammatical categories, so I can’t comment intelligently on this, but some markers of human text include

• personal pronouns (“we”, “us”)
• existentials (“there”, “these”, when they point to something)
• modals (“should”, “can”)
• questioning pronouns (“who”, “what”)
• symbols (abbreviations like “geo” and typographical symbols like the at sign)
• superlatives (“latest”, “most”)

In contrast, robot generated comments contain more

• interjections (“so”, “e.g.”, “yes”)
• 3rd person singular present verbs (“affects”, “is”, “has”)
• past participle verbs (“created”, “copied”, “associated”)
• predeterminers (“all”, “half”, “less than”)
• particles (“up”, “back”, “over”, when they modify the words that come before, as in “start over”)

This hints that maybe the apparent high number of prepositions and directions in robot-generated comments are actually serving the function of particles, rather than prepositions and directions.

That said, this is the weakest discriminator so far, at only 54 % accuracy – barely better than chance.

Instead of looking at the frequencies of single characters, we can look at the frequencies of combinations of two characters next to each other. This isn’t at a higher level of abstraction, but it does tell us more about how robots and humans differ in how they shape their words and how they use punctuation.

The major discovery from this lens is that robots much more frequently end their comments with a full stop. I have advocated for humans to do so for a long time3³ In part because it lets a reader know if a comment is accidentally truncated, but also because it forces better sentence construction on the author of the comment, which usually improves the quality of the comment. but they haven’t caught up, so comments ending in full stops are evidence that a robot generated the comment.

Robots also have some funny quirks around word choices that start to appear at this level of abstraction. These characters are more common in the beginnings and endings of words generated by robots than written by humans:

• Words that begin with “re-” (such as “report”, “recover”, “relocate”).
• Words that begin with “co-” (such as “coverage”, “config”, “copies”).
• Words that begin with “st-” (such as “store”, “stub”, “start”).
• Words that end with “-er” (such as “container”, “user”, “writer”).
• Words that end with “-al” (such as “terminal”, “individual”, “global”).
• Maybe also words that end with “-k” and “-p”.

Some of this I’ve noticed myself (the affinity for “re-” words), and I’m willing to believe it could maybe be a weak signal, at best. I speculate that robots get rlhfd into using punchier words, and those prefixes and suffixes are punchier, maybe.

Once we get into character trigrams, we have to be careful again, because character trigrams are long enough that they start to capture subject differences through parts of words. To reduce that risk, I extracted a list of a few hundred of the most discriminative full words, and added a preprocessing step that removes those that seemed like they leaked subject matter before passing the comments through the trigram analysis.

Looking at the difference through the character trigram lens tells us very little new. This is where we learn that robots are trained to use the word “load” (through the “oad” trigram), as well as words that sound like “surround”, “counter”, and “account” (through the “oun” trigram.)

The reason a model based on character trigrams gets such a high accuracy despite not uncovering anything new is that it captures also the results from virtually all earlier layers. A trigram model gets many of the same signals we discovered through character frequencies, word frequencies, and bigram frequencies. Given a small enough vocabulary with a large enough training data set, it can even detect signals in pos tag frequencies.

If we could learn from bigrams of characters, maybe we can look also at pos tag bigrams. That’s a good idea, because this is where sentence structures start to show through – and why pos tag bigrams give much higher accuracy than plain pos tag frequencies.4⁴ Although it should be noted that pos tagging is expensive, and still not meaningfully better than mere character frequencies.

Common pos bigrams are weighted toward personal pronouns and verbs:

• Personal pronoun + verb or modal, such as “we are”, “they should”, “we may”, “we can”, “we want”, “we deprecate”, “us have”.
• Determiner + verb, such as “this guarantees”, “this is”, “this gives”.
• Personal verb + to, such as “want to”, “have to”, “move to”.
• Existential there + verb, such as “there is”, “these are”.
• Modal + verb, such as “should be”, “can cause”, “will add”.

Robot-generated comments are more abstract through adjectives, and more complex through conjunctions:

• Determiner, possessive, or noun + adjective, such as “an individual X”, “a single X”, “the only X”, “its border-right X”, “its vertical X”, “its already-persisted X”.
• Conjuction + determiner, such as “and a”, “and the”, “or both”.

Although we may recognise some of these patterns from personal experience, pos bigrams are not a very powerful model alone.

We can take it one step further and look at consecutive triples of pos tags to capture even more sentence structures. Accuracy is not much higher than for pos bigrams, but we might recognise many of the human and robot constructions as such.

Here are some examples of human-style writing, which features a lot of verbs and personal pronouns:

• “we may want”, “we can attach”, “they should upload”
• “want to store”, “have to solve”, “want to remove”
• “there is no”, “there is a”, “these are a”
• “since we do”, “after we apply”, “so we check”
• “we are looking”, “I’m guessing”, “we are determining”
• “we check the”, “we do no”, “we have a”
• “will trigger the”, “can have the”, “should cause no”
• “sends data for”, “tells users that”, “causes errors in”

In contrast, robot-generated comments are adjective-laden:

• “only difference is”, “first tab is”, “inner query aggregates”
• “col’s border-right”, “highlight’s vertical”, “layer’s already-persisted”
• “an individual checkbox”, “a single line”, “the only difference”
• “the original uploaded”, “a functioning green”, “the correct quick”
• “this flag is”, “the layout covers”, “this version does”
• “surround option affects”, “reporting service prints”, “state parameter contains”
• “thread reference autocomplete”, “used compression ratio”, “compared year title”

Looking at these examples, it would seem like robots use more big words than humans too, but I haven’t tested that.

If we jam all features at once into the model to try to get them to cancel out their redundancies, here is what remains, in order of most predictive isolated feature to least:

1. Robots use em dashes more than humans.
2. Robots use the connective interjection “so” more than humans.
3. Robots end comments with full stops more than humans.
4. Robots use semicolons more than humans.
5. Robots use parentheses more than humans.
6. Robots use the possessive form “its” more than humans.
7. Robots use adjectives more than humans.
8. Robots use the word “whether” more than humans.

After these top eight, other predictors start to get fuzzy and difficult to interpret in isolation. I suspect much of the power of the classifier does not come from individually strong signals (like the em dash and other typographic quirks), but from combinations of other signals that collectively paint a coherent picture. But those combinations might be different for different documents, and they won’t show up as individual predictors in a list like this.

It would be a cool experiment to have the classifier output the relevant feature combinations when the user hovers over parts of an input to get a better sense of how the classifier sees text, but I haven’t built that.