Experimental AI Meta-analysis of an Academic Journal Issue

TRANSCEND MEMBERS, 10 Jun 2024

Anthony Judge | Laetus in Praesens - TRANSCEND Media Service

Use of Facilities of ChatGPT 4 and Claude 3

Introduction

2 Jun 2024 – This exploration emerged from comparative use of the facilities of ChatGPT 4 and Claude 3 (Opus variant) in summarizing the themes evoked in a set of 35 AI-related articles presented on this site over the past year (Eliciting Experimentally an Overview by AI of AI-related Articles, 2024). This developed into a comparison of the two artificial intelligence facilities with respect to global modelling — especially from a psychosocial perspective. The extent of the responses to this distinctive focus suggested that it would be more appropriate to transfer them to a separate document (Perspectives of AI on Psychosocial Implications of Global Modelling, 2024). There the pattern of question/responses could be further developed under more appropriate section headings.

There is clearly a case for challenging contrasting artificial intelligence facilities with regard to the subtleties framed by the issues in a journal. The AI facilities previously tested both offered the possibility of uploading relatively large PDF files to enable focused analysis. This suggested a different kind of test through uploading to each the PDF version of a single issue of an academic journal (as indicated below). The interaction recorded in what follows is a report of the process of question/response relating both to the issue as a whole and to particular contributions therein. In addition to a summary of the issue, the opportunity was taken (as before) to elicit the transformation of that summary into poetic form and into images (in the case of ChatGPT’s DALL-E plugin).

Both AI facilities were challenged with the possibility of generating musical representations of the themes evoked in the issue — given the rapid development of applications with such facilities and the continuing research on sonification of data (Sonification as a mnemonic aid to global sensemaking, 2020). Whilst both engaged in an insightful speculative interaction about that possibility, and the requisite preliminary analysis, neither was able to offer directly any experimental musical rendering at this stage. The possibility elicited concluding symbolic implications of relevance to governance and interdisciplinarity.

As with the earlier exercises cited, the following exploration necessarily makes extensive use of AI. The role of such AI facilities as an “aggregator” of non-numeric information, rather than as a “computer” of numeric data was previously noted. Reservations regarding such use were also noted, both with regard to the questionable verbosity and style of responses, and what could be termed an undue degree of “algorithmic enthusiasm” for the relevance of the questions posed (Eliciting integrative insight via ChatGPT, 2024). Such enthusiasm could be readily caricatured as a form of “ingratiation” detracting from the responses. Other styles of presentation could have been requested of the AI facilities. A future technical possibility is to present the questions in a single document with links to the responses in separate documents.

Of particular interest however is any critical assessment of the extent to which the responses frame new insights rather than a preponderance of “strategic clichés” — potentially derived from the reports of that quality which feature in many authoritative references. The intent is to “bear witness” to what is increasingly possible technically with any journal, even though the responses may be questionable. This applies particularly to the use of the image generation facility whose included results are primarily indicative of future possibilities rather than useful at this time.

As previously noted, a merit of this approach is that readers can explore alternative articulations by repeating (or amending) the questions to the AI facilities to which they have access — especially as those facilities become more sophisticated and have a wider access to relevant published research. As with this experiment, this would involve uploading to other AI facility one (or more) PDF versions. As in the previous experiments, the responses of ChatGPT are distinctively presented below in grayed areas, in parallel with those of Claude 3.

As previously noted with such experiments, a particular concern is with the biases introduced in framing prompts — readily challenged to the extent that they take the form of “leading questions“. Although proposed to the editors of the journal, the opportunity of framing questions from their perspective did not evoke any response. The results presented cannot therefore be construed as reflecting their approval of the exercise in any way.

TO CONTINUE READING Go to Original – laetusinpraesens.org

Share this article: email mastodon facebook 🔗 copy link

DISCLAIMER: The statements, views and opinions expressed in pieces republished here are solely those of the authors and do not necessarily represent those of TMS. In accordance with title 17 U.S.C. section 107, this material is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. TMS has no affiliation whatsoever with the originator of this article nor is TMS endorsed or sponsored by the originator. “GO TO ORIGINAL” links are provided as a convenience to our readers and allow for verification of authenticity. However, as originating pages are often updated by their originating host sites, the versions posted may not match the versions our readers view when clicking the “GO TO ORIGINAL” links. This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of environmental, political, human rights, economic, democracy, scientific, and social justice issues, etc. We believe this constitutes a ‘fair use’ of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information go to: http://www.law.cornell.edu/uscode/17/107.shtml. If you wish to use copyrighted material from this site for purposes of your own that go beyond ‘fair use’, you must obtain permission from the copyright owner.

There are no comments so far.