In the ever-expanding universe of artificial intelligence (AI) content detection, a notable player has emerged from the crowd: Originality.ai. Now is Originality.ai accurate? Check out the results of my twisted test.
This tool has been tasked with a mission: to sniff out AI-generated content from human-written pieces.
No doubt this novel approach has stirred a mix of concern, disdain, apprehension, and resentment among content writers, SEO marketers, and students.
Originality.ai is the self-proclaimed badass sniffer of AI-generated content.
It swings from the rafters of the internet, armed with its 99% detection accuracy. (That’s their claim on its homepage)
Look at it flex those algorithmic muscles on its homepage! The Most Accurate AI & Plagiarism Detector For Serious Content Publishers,” it declares, not shy of a little humble brag.
Above is Originaity.ai’s homepage which mentions their 99% accuracy for AI detection.
In the universe of AI content generation, they’re trying to be the clean-up crew, the content cops, the plagiarism police!
I must say, in a world where new AI platforms are sprouting like digital mushrooms after a data rain, Originality.ai does indeed stand out.
I must say, in a world where new AI platforms are sprouting like digital mushrooms after a data rain, Originality.ai does indeed stand out.
They even have a post on an AI Detection Accuracy Test which you can find here.
The test involved 1,200 data samples consisting of 600 AI-written articles and 600 known human-written articles. The summary of their test is as follows.
AI is by far the most accurate AI detection tool on the market across all NLP models.
It especially outperforms the alternative detection tools on the latest and most challenging NLP models (ChatGPT, GPT-4)
Additionally, it is the only AI detection tool trained to be able to detect Paraphrased content (such as Quillbot).
I’ve personally tested and compared this service to the likes of Undetectable.ai, CopyLeaks, GPTZero, ContentAtScale.ai, and others and I must admit, Originality.ai does live up to the billing that they are tough to fool with AI-written content.
Time and again, while most of the other AI writer detection services give humanized AI-written articles a pass, Originality.ai just keeps flagging the same articles as mostly AI-written like a ruthless Terminator from the future.
It has come to a point wherein I’ve come to believe that Originality.ai was built to be unforgiving just so it can live up to its claim of being the best among AI content detectors.
Time after time, it just keeps flagging humanized AI-written content mercilessly. It seems nothing just can’t seem to pass beyond 75% human score + 25% AI-generated.
How can such an AI content detector be so brutal, remorseless, sympathy less, and inhuman?
Is Originality.ai Accurate? Let’s Put Their “99% Accuracy” Claim To A Test
Originality.ai’s boast of 99% accuracy is like a shiny badge on its chest, gleaming in the binary sunlight.
But one can’t help but chuckle – will it live up to its hype?
If it mercilessly flags humanized AI-written content with consistency, what if I do something different this time?
What if I feed Originality.ai with 100% human-written content?
Will it still flag these as partially AI-written content?
Or …
Will all these 100% human-written articles pass as 100% human-written?
For this experiment, I fed Originality.ai 15 articles that I am 100% sure are human-written.
These articles were all written years ago when AI writing tools were still not in existence. These were all written when dinosaurs still ruled the world.
How do I know they are 100% human-written?
These articles were posted on old websites and domains I own. I wrote some of these articles while my writers wrote the rest.
These articles were all written years ago when AI writing tools were still not in existence. These were all written when dinosaurs still ruled the world.
The Results Of This Twisted Test Might Surprise You
So I pulled out 15 100% original, 100% human-written articles from my sites.
The topics of the articles ranged from babysitting and diets to technology and software.
Article length also varied and ranged between 800 words to 1,100 words.
Article writing dates were from 2008 through 2017. (I told you, age of the dinosaurs in technology-speak.)
The process was simple. I would just copy and paste each article onto Originality.ai and hit scan.
Faster than instant noodles, it then spits out its assessment and provides a score as shown in the sample below:
Below is another example of an article assessment from the test.
I then graded the results based on this scale:
Excellent = 96% and above human-written
Good = 90% to 95% human-written
Fair = 80% to 89% human-written
False Positive = 79% and below
I can provide a screenshot of each of the article test results if I want to bore you and put you to sleep.
But why do that and waste your precious time right?
So here are the results for your serious analysis and pleasure:
As can be seen above, of the 15 100% human-written articles, Originality.ai:
Rated 6 articles with a score of 100% human-written. I rated these Excellent.
Rated 4 articles with a score of 98% to 99% human-written. I rated these Excellent.
Rated 3 articles with a score of 91% to 94% human-written. I rated these Good.
Rated 2 articles with really low scores of 56% and 65% human-written. I rated these as False Positives. In other words … Fail.
Below is a screenshot of the weight loss article that was a False Positive.
Below is the second article, the topic of which is about Diets, which was also a False Positive.
For the two False Positive articles, I know for a fact those articles were definitely 100% human-written because I WROTE THEM when Mr. and Mrs. AI were still in their courting stage and haven’t conceived their AI writing tool lovechild.
Hhhhmmm, makes me wonder now …
Am I really 100% human or am I really 35% to 44% part AI cyborg who was raised by human parents to make me believe I am human?
For now, let’s assume I am human so Originality.ai got that dead wrong.
Is it also possible that Originality.ai hates topics about weight loss and diets because it has a covert mission to make the whole world obese and unhealthy? That’s another topic for discussion.
There were 3 other articles that Originality.ai rated below 95% human-written. I’m awfully curious why that was the case.
At any rate, what the results show is that like any technology …
Originality.ai may not be perfect at what it does so do exercise some discernment and caution when using this service.
At times, it might be like the overzealous cop who gives you a speeding ticket as you travel 50 to 53 miles per hour in a 50 mph zone.
I must say that I STILL think they are one of the best AI writing detection services in the market compared to other services.
It might be best practice for you to use this in combination with other factors or tools to best decide if a post or article is AI-written or not.
However …
I must say that I STILL think they are one of the best AI writing detection services in the market compared to other services.
It’s because Originality.ai is the only one that gets to flag some of my AI-written content as partially written by AI while other services give me a 95% to 100% human-written rating.😉
If you are a huge fan of AI writing tools and want them to pass the human test, all I can say is if you get a pass with Originality.ai, then you get to pass most, if not ALL AI writing detectors out in the market today.
It is important to note that my test consisted of only a small sample of fifteen 100% human-written articles. That definitely may skew the results and may not be representative of results if a test involving a larger sample size of articles were used.
Maybe someone might like to do that someday.
Also, Keep In Mind …
Like any technology, Originality.ai is a work in progress. The world of AI writing and AI writing detection is advancing at an exponential rate, which means we should expect a lot of changes to happen as to the use and functionalities of these tools.
Nevertheless, exercise responsible use and caution when applying these tools to your projects and business.
In case you want to learn more about Originality.ai and what they can do to help you with your blog, business, or whatever, you can learn more about them here.
If you want to learn more about how Originality.ai compares to their AI detection competition, you can learn more about that here.
So what do you think of the results of my test? Let me know.
Meet our resident tech wizard, Steve the AI Guy. Now, before you get any wild ideas, let’s clear up one thing – he’s 100% human! I mean, he’s got the work history to prove it. He spent a decade diving into the deep end of the tech industry doing business intelligence work, splashing around with two of the world’s largest business consulting companies, Deloitte and Ernst & Young. Learn More
