Why you can’t always rely on AI search results

It is easy to see why students may be tempted to paste their assignment title, or project outline, into Microsoft Copilot or ChatGPT and ask for a list of references. The tool may then offer to summarise the studies in a way that suggests it is a helpful time-saving approach. 

Why is this a problem? 

GenAI tools do not search the internet or academic databases. Instead, they generate text based on patterns in their training data. This means: 

• Students are not learning and using critical search and evaluation skills. 
• References may be fake: GenAI tools still fabricate citations that look real but do not exist. 
• Summaries may be inaccurate: even when summarising real studies, GenAI responses can misrepresent findings or oversimplify complex ideas. 
• Sources may be unverifiable: GenAI tools do not provide links to specific sections of peer-reviewed literature or evidence for its claims. 

This can create a false sense of confidence for students, negatively impact their learning and lead to issues with academic responsibility and conduct. 

How we support students behind the scenes 

Understanding and using GenAI appropriately is integrated throughout the University of Southampton Library Research Skills Framework, ensuring that we empower students to: 

• Differentiate between appropriate and inappropriate uses of GenAI for academic research. 
• Understand the academic integrity implications and potential biases associated with using GenAI tools
• Critically evaluate all GenAI outputs for accuracy 
• Understand that GenAI tools may generate fake references

Your role: signposting the UoS guidance for students on searching 

You can help your students develop the skills to question and verify AI-generated content by encouraging them to look at Library guidance that has been co-created with students on using (and not using) GenAI for academic searching. 

You can also encourage students to question GenAI outputs e.g.  

• Ask “Is this accurate? Where is the evidence? Does the response cite credible sources? Are all of the references genuine?” 
• Think critically about the response: could any voices or perspectives be missing? Remember that any biases in the training data will affect the output. 

Encourage cross-checking with trusted sources, for example checking library subject pages, academic databases, other library resources and authoritative websites (for example, government or NHS pages). 

Consider using GenAI tools to help develop and refine their search strategy, but not to conduct the search. 

Bite-sized task 

In this task you will watch the first 2 short videos from our GenAI and searching resource, and consider how the topics relate to your students. 

Step 1 – learn 

Watch the first two videos (total three minutes) featuring one of our students talking about GenAI and searching: Academic Searching and Generative AI

Step 2 – do 

Use Microsoft Copilot to search for literature on a topic you know well. How do the results compare with what you expect? 

• Are any key journal articles or other resources missing? 
• Are any perspectives missing? 
• Do all of the articles exist and are they correctly referenced? 

Step 3 – reflect 

• How would you summarise the key points from the videos or your own testing? 
• How might your students misunderstand or misuse GenAI tools for academic searching? 
• How could you share the GenAI and searching learning resource with your students? 

Share your thoughts on this week’s GenAI Essentials post in Teams and join in the discussion. 

Further links 

Searching and Generative AI  

Library Research Skills Framework 

© 2025. This work is openly licensed via CC BY-NC-SA