Skip to Main Content

Evaluating Generative AI Tools

Why is it important to evaluate AI tools?

a woman working at a deskNew generative AI tools are becoming available all the time. For some, the advantages and disadvantages of the tool are well-known. Take ChatGPT. That tool is so well-known and has so many users that even those who have never used it know about its potential flaws, including its tendency to make up information that isn’t true or “hallucinate” sources that don’t exist. Anyone who uses ChatGPT or other chatbots to generate ideas and texts generally knows about these potential errors. 

But what about other tools? For example, did you know that image generators often show gender and racial bias when generating images of people? Maybe you’ve seen videos from video generators that have subtle flaws in how they render people's bodies. Or maybe you’re familiar with the way chatbots can “hallucinate” but you didn’t know about the copyright issues related to how these tools were trained.  

Newer tools that are rushed out by companies that want to jump on the generative AI bandwagon may have further flaws or risks.  

When choosing an AI tool, it’s important to be aware of these flaws and risks and take them into consideration. This shouldn’t stop you from using generative AI if it fits in well with a task you are trying to complete, but it will help you become a better and more knowledgeable user of the tool and to more critically evaluate its output.