So my partner went to a ChatGPT workshop at his university and the facilitator suggested that ChatGPT can be very useful in generating a bibliography for a literature review, thus making it potentially helpful for students and researchers who are at the start of a project. So I started playing with it and asking it to make bibliographies. What it does is rather astonishing but not in a way that would be helpful. It makes up references that do not exist and does not include references that do exist on the research topic. I don’t understand exactly how ChatGPT works but I imagine its web scraping tools are able to extract bibliographic information from the websites of journals, book publishers, universities, and online academic databases, the sites in which academic publications usually correctly appear, but then it bizarrely mixes fake bibliographic data with real authors and real journals and real publishers to produce something that looks highly credible, especially if you are not familiar with the scholarship in a particular field.
Having published an article recently on Māori activist, Tame Iti, I asked ChatGPT to produce a comprehensive academic bibliography on this topic. This is what it produced. As you can see, ChatGPT “knows” that Tame Iti is a prominent Māori activist and cultural figure in New Zealand. It came up with the following 18 references.
| Creating a comprehensive bibliography of academic sources on Tame Iti, a prominent Māori activist and cultural figure in New Zealand, involves numerous articles, books, and resources. Here is a selection of academic sources related to Tame Iti: Belich, J. (2007). The New Zealand Wars and the Victorian Interpretation of Racial Conflict. Auckland University Press. Boast, R. (2018). No Twisting of the Facts: Te Paparahi o te Raki Inquiry, Tame Iti, and the New Zealand Wars. New Zealand Journal of Public and International Law, 16(1), 39-78. Bourassa, G. (2010). Recontextualizing Tame Iti: Anarchism and the National Imaginary in Aotearoa/New Zealand. Social Identities, 16(3), 327-343. Gunson, N. (2006). Ideology and Iconography: The National Museum and Tame Iti. Counterfutures, 7(3), 11-33. Hayward, J. (2004). Opponents of ‘Militant’ Maori Activism: Maori Party Politicians and Tame Iti. Political Science, 56(1), 91-104. Hutchings, J. (2015). From “Savage” to “Negotiator”: Tame Iti and the Performance of Sovereignty. Cultural Dynamics, 27(2), 281-297. Jansen, R. S. (2012). Maori Terrorism? The Politics of Naming and Framing the 2007 Urewera Raids. The Journal of Polynesian Society, 121(2), 161-186. Johnston, M. (2008). Tame Iti: The Legacy of 19th-Century Maori Sovereignty. History Compass, 6(6), 1351-1366. Jones, C., & Jenkins, C. (2013). The Price of Dissent: Tame Iti’s Performance of Resistance. Sites: A Journal of Social Anthropology and Cultural Studies, 10(2), 92-114. Keenan, D. (2016). Return of the Ancestors: Tame Iti and the Urewera Four. In T. Campbell & D. Keenan (Eds.), Protest Politics in New Zealand (pp. 227-240). University of Otago Press. McLeod, M. (2013). The art of Tame Iti: Globalizing dissent. Continuum, 27(2), 231-245. Mulholland, M., & Loveridge, S. (2012). Securing the Nation: Terrorism and Counter-Terrorism in New Zealand. International Journal of Law, Crime and Justice, 40(4), 217-234. Pihama, L., & Ormond, M. (2009). Return of the tīpuna: The reclamation of mana and tikanga within Ngāti Ruanui. AlterNative: An International Journal of Indigenous Peoples, 5(2), 111-123. Ritchie, J. E. (2017). Tame Iti: A Radical, Globalist, or Māori Traditionalist? New Zealand Journal of History, 51(1), 101-117. Spoonley, P., & McCormick, J. (2006). Tame Iti: The Case of a ‘Māori Radical.’ In Globalisation, Cultural Identities and Media Representations (pp. 183-199). Palgrave Macmillan. Te Awekotuku, N. (2010). Warrior Queers: Tame Iti and the Reclamation of Mana Whenua. Intersections: Gender and Sexuality in Asia and the Pacific, 24. Te Awekotuku, N., & Simpson, T. (2006). Tame Iti: Policing the Māori Warrior. In K. Rogers, L. Russell, & M. Nelson (Eds.), An Anthology of New Zealand Poetry in English (pp. 47-69). Oxford University Press. Thomas, N. (2008). Tame Iti and the Politics of Recognition in Aotearoa/New Zealand. Identities: Global Studies in Culture and Power, 15(2), 210-233. This list provides a range of scholarly articles, book chapters, and academic publications that explore various aspects of Tame Iti’s activism, cultural significance, and impact on New Zealand society. |
The book numbered first in this list exists and includes important historical information about the Tūhoe struggle but it does not refer anywhere to Tame Iti, at least not in the edition I have. But the rest of the references, those numbered 2-18, are all fake. They do not exist. My fear that I might have missed crucial publications on the topic dissipated as soon as I started trying to find them. But what is interesting is that they might well be fake, but they are not complete gobbledygook. They are highly credible. I know many of the authors. They are real people. The list includes titles that these authors might well have written, in anthologies and journals they might well have published in. But my own article – a genuine article about Tame Iti published in the International Journal of Cultural Studies in 2019 – did not appear.
So then I simply asked ChatGPT to produce a bibliography of my own publications. As is the case with all academic publications, these are readily available of course on my own website and that of the university where I work, as well as on various academic databases. But rather than pulling a list of references from one of these sites, ChatGPT invented them. All of them. This is what I got.
| As of my last knowledge update in January 2022, Julie Cupples, a prominent scholar in critical human geography, had authored various publications across her career. Here’s a representative bibliography of Julie Cupples’ publications. Please note that this list might not include works published after 2022: Books: Cupples, J., & Slater, D. (2017). Producing and Consuming Inequality: Why Geography Matters. Rowman & Littlefield International. Cupples, J., & Glynn, K. (Eds.). (2014). Media and Glocal Change: Rethinking Communication for Development. Peter Lang AG. Cupples, J. (2004). Representing Place: Landscape Painting and Maps. University of Minnesota Press.Cupples, J. (2009). Globalization and the Singapore Curriculum: From Policy to Classroom. Springer. Selected Articles and Book Chapters: Cupples, J., & Glynn, K. (2019). Geographies of the radical right. Progress in Human Geography, 43(1), 90-110. Cupples, J., & Slater, D. (2014). Rethinking media and communication studies in the context of anti-capitalism: A reply to Nick Couldry. Global Media and Communication, 10(3), 265-271. Cupples, J. (2017). Culture, development, and social change: Social media and political agency in Melanesia. In H. Horst & D. Miller (Eds.), Digital Anthropology (pp. 177-194). Bloomsbury Publishing. Cupples, J., & Lugo-Ocando, J. (2015). Media, public space and the logic of violent conflict. Third World Quarterly, 36(8), 1568-1587. Cupples, J. (2011). Researching media, gender and development in the South Pacific: Methodological and ethical reflections. Gender, Place & Culture, 18(5), 639-657. Cupples, J. (2007). Cultural landscapes, place-images, and disciplinary power: Spatializing the visual and the discursive in the Tainan landscape. Geografiska Annaler: Series B, Human Geography, 89(1), 69-82. This bibliography represents a selection of Julie Cupples’ works. For a complete and updated list of her publications, I recommend consulting academic databases like Google Scholar, Scopus, or university library catalogs. These sources provide comprehensive and up-to-date information on scholarly publications by Julie Cupples. |
As Julie Cupples, “prominent scholar in critical human geography” (cheers!), I can confirm that I did not author a single one of these publications and I’ve never published with D Slater (I have published with a T Slater) or J Lugo-Ocando, or with Peter Lang, or University of Minnesota Press, or in the journals Progress in Human Geography, Global Media and Communication, Third World Quarterly, or Geografiska Annaler. But I easily could have done. The books, articles and chapters are all things I might have written given my field (the Singapore curriculum one was the most unlikely, but I have published in the Singapore Journal of Tropical Geography a couple of times). But ChatGPT made them all up and not one, not one, of my actual publications appeared in this list. As well as adding the caution “Please note that this list might not include works published after 2022”, it should have said “Please note that this list might not include any real publications by this author.”
The next day a prospective PhD student sent me a research proposal. I looked up one of the cited references, a journal article, as it looked quite interesting, but I couldn’t find it! I found the article’s author’s academic website, but it appears they never authored an article with this title in this journal or any other journal. I then checked the journal in which the article supposedly appeared and couldn’t find the title or this author anywhere in this journal either. I did find an authored book with a very similar title (not exactly the same but close enough) but it was a book and had two completely different authors. I’ve no idea how this mistake came about, authors do get references wrong and there are several possible explanations. But this is a warning to students and researchers, ChatGPT cannot be used with any degree of reliability to generate a bibliography for your research proposal or essay. And of course if you include a fake reference, your reader might well try to find it out of interest, revealing that you didn’t read it, indeed that you couldn’t have read it because it doesn’t exist.
So there you go – ChatGPT makes up references and is incapable it seems of generating genuine ones.
Julie Cupples 