How would you go about testing an AI product? Join me at QA&TEST Conferences where we will debate the topic on Oct 30.
I’ve been fortunate to work on that problem a few times in past years, in my experience this is not a straight forward answer..
Biggest reason for me – it’s not easy to establish clear oracles (I didn’t use the word requirements..) which creates a complete new dynamics to test these products.
The test objectives of these systems are therefore going to be a bit different, and wouldn’t be just about testing the actual model,
I’d stretch it from the beginning – ‘capturing of data’ and then take to the other extreme – ‘is the model predictions good enough’ – not an easy answer to give, but worth investigating.