Checking AI results carefully: What that means, what it requires, and how to do it

“Check AI results carefully!” This is a phrase that has now become part of the standard repertoire at many universities. It sounds reasonable, but it does not lead to anything as long as it is unclear what exactly is supposed be checked carefully and how this should actually be done in practice. Anyone calling for a careful check should therefore focus on three questions: Does the source exist? Does it support the statement? And what data and literature might be missing?

The answer to these questions will turn out not to be that simple.

1. Does the source actually exist?

Large language models can generate references that look absolutely bona fide but do not actually exist. The data shows a massive increase in these sorts of hallucinations across all disciplines and publication formats: from student dissertations to studies in renowned peer-reviewed journals (Topaz et al., 2026; Zhao et al., 2026).

References fabricated by AI often go undetected and are cited by others, thereby becoming part of the academic ecosystem and gaining legitimacy. In 2026, for example, education researcher Olaf Zawacki-Richter discovered a study attributed to him that did not exist. It had already been cited by 72 other publications (Thomas, 2026).

When checking a source, the first question to ask is therefore: does the source actually exist?  No fabricated source would have passed this verification stage if someone had opened the link and cross-checked the details against the document or a catalogue entry.

Where the fabricated sources come from

Language models are probability machines, not truth generators. They are based on statistical calculations. If the patterns they have learned are insufficient to provide an answer, the model generates a plausible follow-up nonetheless. Guesses tend to be rewarded by the training and evaluation procedures more than if no answer is given at all, and hallucinations can thus arise.

The fact that fabricated sources end up in academic work is not merely a technical problem, however, but also the result of a poorly conducted search, as most students and lecturers use general-purpose AI for their data and literature research, above all ChatGPT (Bosse et al., 2026). For searches, it would be better to use AI tools that access academic databases and link their results to sources that really exist. This would significantly reduce the risk of references being fabricated.

One method widely used in this regard is Retrieval-Augmented Generation (RAG). In simple terms, the system first searches for relevant documents in a connected database, and a language model then generates a response based on those documents. AI systems like this can be found at universities in the form of AI-supported specialist databases such as Scopus AI, EBSCOhost Natural Language Search or the wiso-net AI assistant. AI-supported academic search engines like Google Scholar Labs or the preprint server alphaXiv also use this system. In addition, there are now more than 50 standalone, academic-specific AI research tools available, such as scienceOS, Elicit or Consensus.

Double-checking requirement

ChatGPT and Claude can be connected to some of these tools and databases via plugins and connectors and be enhanced through research skills and deep-research modes. For academic research, they should only be used in this form (Walther, 2026). However, the same rule applies even then: open all sources and check the bibliographical information.

Lecturers should show their students these search options and make two things clear in the process. First of all, hallucinations are not technical glitches, but the result of poorly conducted research, and secondly, anyone who accepts an AI-generated source without checking it has failed to meet a basic standard of academic practice.

The result of this is that they have an obligation to double-check: students check their sources. Lecturers verify whether the students have checked them. In the case of bibliographies, tools like BibFox or specialised verification skills help to identify hallucinated references and other inaccurate information automatically.

2. Does the source really support the statement?

A study can exist yet still be cited inaccurately by AI. AI systems have a tendency to generalise results or distort information when processing documents repeatedly (Laban et al., 2026; Peters & Chin-Yee, 2025). Therefore, the link between the claim and the evidence also needs to be checked. Lecturers can give their students a source and an AI-generated statement based on it, for example, and have them assign it to one of three categories:

This verification is also important because an increasing number of students only read the AI response and no longer consult the source itself when conducting a search (Walther, 2026).  Furthermore, the study design, measurement instruments and other criteria should also be checked in order to determine the scope and significance of a study. AI systems can help with this by generating evidence matrices that specify the methodology, results, limitations and so on for each study found.

The prompt determines the search space

For this to work, you need good prompts. That is what is lacking, however. For one thing, because even bad prompts deliver results that sound good, and for another, because most students and lecturers lack a sound education in AI in general and specifically lack knowledge with regard to prompting. A large-scale study showed that out of 5,000 researchers in Germany, only 21 per cent were able to complete a prompting task successfully (Chugunova et al., 2026).

Therefore, here is an example: if you are searching for literature on whether the use of AI leads to a loss of skills among students (known as ‘deskilling’), it is not enough to write the prompt: “Does the use of AI lead to a loss of skills among students?”.

It would be better to formulate the prompt like this:

“Search for empirical literature (2023–2026) on deskilling, downskilling and cognitive offloading caused by AI in higher education. In this context, check changes in writing skills, research skills, problem-solving skills, metacognitive skills and critical-thinking skills. Prioritise European studies, particularly those from the DACH region, and supplement them with international evidence. Identify quantitative, qualitative and mixed-methods studies as well as relevant reports, meta-analyses and systematic reviews. Create an evidence matrix detailing study design, sample size, results, strength of evidence and limitations. Consistently distinguish between measured changes in skills, perceived effects and mere attitudes towards AI. Identify research gaps and derive implications for higher education institutions.”

In short: a good prompt clarifies terms, operationalises the research question, specifies inclusion and exclusion criteria, and defines the desired form of the results.

This describes the first two levels of the check. There is, however, a third level, which has not yet been mentioned and which actually precedes the other two.

3. What does the response not show?

Let us assume that all sources specified by the AI exist and that they are cited and analysed accurately. Does this mean that the response holds up scientifically? Not necessarily, as no research system can provide a comprehensive overview of the empirical evidence on a particular topic. The third question is therefore: which relevant sources, counter-findings or perspectives are missing?

All research systems have limitations and that includes those that operate without AI. Traditional search systems like OPACs primarily influence what literature we find. Generative AI goes one step further: it not only selects the sources but also links up and interprets their contents. The result of this often appears to be well structured and so clear that it is very rarely verified. This makes it even more difficult to recognise bias.

Bias like this is inherent in every AI system. What particular bias emerges depends on the system’s dataset and search logic. Depending on the AI tool, magazine articles in English may be overrepresented, for example, while studies, monographs or humanities research in German are found less frequently. What is missing from the underlying datasets or is poorly indexed will therefore rarely appear in the AI response, if at all.

Being familiar with the data sets, understanding search functions, identifying gaps

When checking a source, the question to ask is therefore: has the state of research been researched widely and transparently enough? Finding the answer to this requires not only information literacy but also AI literacy, because anyone wishing to identify gaps has to understand how the response was generated.

• On what data sets is an AI system based?
• Does it only use bibliographical information and abstracts for the search or does it use complete texts as well?
• How does it read, assess and sort the source?

The answers to these questions clarify what a search system is actually capable of finding and what lies beyond its search space. Furthermore, the search space is determined by the question or prompt, the terms used in it and any additional prompt inputs. Therefore, perform the check on two levels.

1) In the same system: Even small changes to the prompt lead to different hits. Therefore, vary search terms and perspectives systematically, document the results and compare them – feel free to use AI to help with this. That way, you can check which prompts deliver the best results and why that is.

2) Across systems: Search systems differ in their databases and the way they operate: library catalogues, specialist databases, academic search engines, preprint servers, repositories and AI tools all find different data and literature. Therefore, search different systems and look specifically for counter-findings – by using an appropriate prompt, for example. That is the only way that a “check” can become a “careful check”.

One simple sentence leads to a formidable skill

If the much-called-for checking of AI results is to be of any use, it must not remain an abstract task but must be translated into concrete action: verifying sources, comparing statements and evidence, researching the states of research, and using different search systems.

What emerges here is that there are qualitative differences between the various levels and points of verification: a source reference can be checked quickly, but checking the evidence is more complex. And visualising and recognising potential bias and gaps in an AI response or in the state of research is arguably the most challenging task, as it requires not only specialist knowledge but also sound AI literacy and comprehensive information literacy.

Careful checks should not just be required of students, however, but must also be modelled and taught by the lecturers. Lecturers must be able to show what is being verified on which level, how that verification works and what its limits are. The problem is that it is precisely these skills that are not yet sufficiently widespread at universities (Walther, 2026). Therefore, you could voice your objection and ask how lecturers are supposed to achieve that if they don’t have enough AI education themselves yet. My answer to that is that you don’t have to wait until you know everything. Keep on developing your teaching together with your students. Formulate criteria, test them and revise them and be very open about your own gaps in knowledge. Because one thing is clear: checking results carefully is a skill that can be learned and taught.

Danny Walther

+ postsBio ⮌

Danny Walther works as an independent lecturer at universities, educational institutions, and research organizations. He leads workshops and webinars on the use of artificial intelligence in academia and higher education. His areas of focus include data and literature research, visualization methods, and the question of how to succeed in studying, teaching, and conducting research in the age of AI. The goal of his work is to promote a practical and scientifically sound approach to AI.
https://ki-recherchen.de/
https://www.linkedin.com/in/ki-recherchen/