[LINK] Concepts, Context, 'Truth' and Values [ WAS Re: ChatGPT's odds of getting code questions correct are worse than a coin flip

[Roger Clarke]

> On 15/08/2023 9:04 am, Kim Holburn wrote:
>> >Among other findings, the authors found ChatGPT is more likely to 
>> make conceptual errors than factual ones. "Many answers are incorrect 
>> due to ChatGPT’s incapability to understand the underlying context of 
>> the question being asked," the paper found. 

On 16/8/23 10:58 am, Bernard Robertson-Dunn wrote:
> Suppose you wanted to tell an AI system how to distinguish between right 
> and wrong (if you try it will tell you it has no opinion), correct and 
> incorrect, truth and lies? How would you do it?
> Would it work with e.g. far right extremists? MAGA supporters? Politicians?

It's interesting how many of the current issues are clustering.

We're concerned about mis- and dis-information, both uttered and 
disseminated by, and both received and misunderstood, by both humans and 
artefacts.

I have a first-cut manuscript tackling an aspect of this, and would be 
delighted to again have the benefit of the Link Institute's views on it.

The topic is 'The Authentication of Textual Assertions'.

*Textual Assertion* means a visual representation of an Assertion 
expressed in, in principle, any natural language that has a visual form

*Assertion as to Fact* means a Textual Assertion that involves at least 
one Real-World phenomenon

*Authentication* is a process that establishes a degree of confidence in 
the reliability of an Assertion   [ I avoid the assumption that 
humanly-accessible truth is a useful construct for these purposes. ]

*Evidence*, in relation to the Authentication of an Assertion, is data 
that tends to support or deny that Assertion


After recounting some bits of boring old linguistic theory, dipping into 
a bit of logic / critical thinking lit, and reviewing the scatty 
literature on mis-/disinformation and 'fake news', I propose some 
'reliability criteria' for authenticating Textual Assertions as to Fact.

The first-cut lists a baker's dozen criteria and short guidance in 
relation to each of them, grouped under the four headings of The Work, 
The Assertions, The Argument, and Evidence and Counter-Evidence:
http://rogerclarke.com/ID/ATS.html#T6

The Appendix contains some preliminary applications of the guidance to 
WMDs, the storming of the Capitol, and some more business-ish assertions.

Get stuck in and sort me out!


-- 
Roger Clarke                            mailto:Roger.Clarke at xamax.com.au
T: +61 2 6288 6916   http://www.xamax.com.au  http://www.rogerclarke.com

Xamax Consultancy Pty Ltd      78 Sidaway St, Chapman ACT 2611 AUSTRALIA 

Visiting Professor in the Faculty of Law            University of N.S.W.
Visiting Professor in Computer Science    Australian National University

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <https://mailman.anu.edu.au/pipermail/link/attachments/20230816/9aa7bb33/attachment.sig>