[LINK] SAI Generated Instructions

[Roger Clarke]

> On 5/6/25 21:03, Roger Clarke wrote:
>> ... you need to specify the 'right' answer ... *in advance of* 
>> evaluating the output. ...

On 7/5/2025 09:32, Tom Worthington wrote:
> The test was evaluation by an expert in the subject matter area, in this 
> case a tax accountant. They concluded the output was accurate and 
> suitable for presenting to clients.

Not bad, but not compliant with the principle ...


>> You've covered yourself with "[it] gave a useful answer".
> 
> Isn't useful all we need from AI?

Nope.

Useful = "able to be used for a practical purpose".

There's no element of reliability in that property.

Without data quality factors being satisfied (appropriate association, 
accuracy, timeliness, etc. - a dozen factors in all), mere usefulness 
(which I'd equate to 'apparent relevance') isn't enough.


http://www.rogerclarke.com/ID/FDIS.html#Q


-- 
Roger Clarke                            mailto:Roger.Clarke at xamax.com.au
T: +61 2 6288 6916   http://www.xamax.com.au  http://www.rogerclarke.com

Xamax Consultancy Pty Ltd      78 Sidaway St, Chapman ACT 2611 AUSTRALIA 

Visiting Professorial Fellow                          UNSW Law & Justice
Visiting Professor in Computer Science    Australian National University