Most AI Output Is Hallucination

Hello!

Welcome to another issue of AI Growth Insights.

In today's issue I want to talk about an important issue with ChatGPT and LLMs in general that's often overlooked.

You see, most of ChatGPT's outputs are plain wrong and can't be trusted.

And if you are using it for critical tasks, you are shooting yourself in the foot.

But that doesn't mean you should go back to using Google and read countless articles just to get to the bottom of things.

By the end of this post, you will know exactly what to do instead.

So without further ado, let's get started.

"Hallucination Is All LLMs Do"

Many people mistakenly use ChatGPT as if it were a new version of Google, searching and trying to get factual information out of it, but this is a big mistake.

And to understand why that's a mistake, you need to know a bit about how Large Language Models (LLMs) are made... (Don't worry, I'll keep it simple)

During the creation of an LLM, a bot analyzes a vast amount of textual content from books and websites to learn patterns about concepts, definitions, ideas, and so on.

However, when you are probing it with questions, it's hard to tell exactly where it's getting its information from.

After all, it has read and processed an insane amount of information during its training stage (almost the entire public internet), and because of that:

• It could be pulling information from outdated sources;

• From incorrect and untrustworthy sources;

• It could be mixing up multiple sources and essentially leading you astray;

• And it could be doing all of the above at the same time.

This means you shouldn't rely on large language models to provide accurate and factual information.

If you do, you're making a big mistake and essentially shooting yourself in the foot.

After all, you can't be certain whether Chat GPT is spouting nonsense that seems true, or if it's providing you with facts grounded in reality.

This doesn't mean you should get rid of ChatGPT, but as I mentioned at the start of this article, there are methods to ensure that it draws from reliable sources.

Or at the very least, sources that you can verify if they are worthy of being trusted...

How To Get Accurate Information From LLMs

Allow me to demonstrate how to get accurate, verifiable information from LLMs with an example.

First, I will show you the wrong way that almost everyone is doing, and then I will show you the right way.

Many people are using Chat GPT for customer research, ICP (Ideal Customer Profile) research, and market research.

So let's take this use case for the sake of this example.

If I were to use ChatGPT (the wrong way) to do market research on my own audience - let's say solopreneurs who want to leverage AI to be more productive and create more content - here's what I would get:

GPT's output was detailed and well-written, one would even be tempted to act on it. But doing so would be a grave mistake, and here's why:

• Generic and unespecific responses;

• Lots of assumptions that could or could not be true;

• Untrustworthy because we can tell what's the source;

• This information is likely outdated at best or plain wrong at worst;

• Almost impossible to pinpoint the inaccuracies or "hallucinations" in the output.

If you make any business decision based on this information, you're essentially taking a shot in the dark because it is probably not grounded in reality.

Worse, it could lead you in the wrong direction.

This doesn't mean you should revert to endless Google searches and reading countless blog posts to find the truth.

Instead, you could gather data that you know is factual, and only have GPT to sift through it and generate insights for you.

You can do this in at least 3 ways:

1. Perplexity AI

If you're not familiar with Perplexity, think of it as a blend of Google Search and ChatGPT.

When you enter a search query, Perplexity searches Google and then summarizes the results using ChatGPT's capabilities.

This gives you the advantage of quick, generated responses with information that's been actually searched for.

Plus, if you need to dig deeper into a topic or verify the source and date to ensure the information is current and reliable, you can just click a button and check it for yourself.

Since Perplexity is doing its search through Google, it also retrieves information from social media, which is an awesome way to get current and up-to-date information.

Even on topics that are new and ChatGPT can't answer on its own because it's way past its knowledge cutoff.

As I mentioned earlier, Perplexity AI uses Google to find answers to your search questions.

This means you can use advanced search commands here too.

For example, if you want to limit your search only to links that have been published after January 1, 2024, just add "after:2024-01-01" to your search...

Or, if you want to see what people on Reddit think about a topic, use "site:reddit.com" in your search.

Check out Google's advanced search parameters for more ways to narrow down your search and get more accurate, helpful results.

2. Chat GPT Plus

If you have the plus subscription for ChatGPT, you can use it in two ways.

First, you can use the code interpreter to analyze data from Excel tables or CSV files. This is great for looking at your own data.

Second, you can upload PDF or doc files and start blasting questions.

Again, if you're inputting the data yourself and you know it's factually correct, you can be confident that the information GPT gives you is not a hallucination.

You can trust its insights and confidently act on it.

3. Grok

Grok has been facing a lot of criticism from the Twitter AI community recently. Many say it's not useful or capable enough.

While I agree with some of the criticism, there's a significant aspect that many are missing. Grok is Twitter search on steroids.

If we consider Perplexity as a mix of Google and ChatGPT, then Grok can be seen as a combination of LLMs and Twitter.

You can use Grok for advanced searches to find insights, and it will provide you with real conversations from people on Twitter discussing your topic.

However, Grok isn't powerful yet, so you should approach its results with a bit of skepticism and verify the sources it provides.

And not to mention it's kinda quirky and sometimes you need to know how to prompt it correctly.

Despite this, Grok is still a valuable source of factual information.

Since it's connected to Twitter, it's particularly useful for marketing, research, and discovering trending topics, as well as understanding how people are reacting to these topics.

In the end, you'll have access to authentic discussions, which is valuable in itself. It offers more insights for you to act upon than any other tool we've discussed before.

P.S: People still have mixed opinions on Grok...

Some say it's worth the price, others say it's far from it.

Personally, I pay for the Premium Plus subscription because I use Twitter a lot.

But in all honesty, I wouldn't pay for it if it was for Grok alone.

When To Trust GPT

With all that said, does it mean you should never trust ChatGPT?

Should you always be suspicious and skeptical about its results? Probably not.

It's important to note that it's only necessary to go to great lenghts to ensure output integrity if you absolutely MUST have precisely accurate information that is grounded in reality.

If you're working on something that requires complete truthfulness, then yes, you should go through the trouble of ensuring your are working with factual information.

If not, no need to bother. Use the output as it is.

PS: Need help implementing AI in your business?

Let's talk about how you can leverage AI.

• AI Automation;

• Prompt Engineering;

• AI for Content Creation & Social Media Growth;

• And much more.