Interesting insights<\/h2>\n

The talks and questions during the panel discussion unearthed some interesting points which I feel might be very helpful for teams exploring to test AI systems.<\/p>\n

Regulatory requirements<\/h3>\n

For safety critical devices regulatory bodies provide practices, processes and guidelines governing how safety approvals will be given. With AI products the community is debating what is more practical and pragmatic approach to certify AI systems.<\/p>\n

Due to evolving nature of AI products, it is possible the guidelines will be more process based rather than around the product\u2019s functionality itself since those are going to be a moving target. It goes without saying this is a very complicated problem to solve and the stakes are high. Take an example of self-driving cars and it\u2019s impact.<\/p>\n

Continuous learning algorithms<\/h2>\n

In certain ML models, like deep learning, are mostly ever evolving. After the product is released, it still keeps learning and changing it\u2019s algorithm. This poses a different set of challenges and the traditional test and release cycles are not enough. Observability and production tests become a must in such situations, which means testing becomes an ongoing activity happening in production.<\/p>\n

Biases in AI products<\/h3>\n

AI models build their intelligence through machine learning by consuming large amounts of training data. The kind of data we provide is going to govern the kind of assumptions the model makes. In recent years a few incidences have surfaced where the AI model was biased to a certain group of people or other variables.<\/p>\n

The challenge, many times we don\u2019t even know if a bias exists. For instance, a leading tech company had an AI program to short list resumes. Later it was discovered the program assumed highly skilled people are usually male candidates. This was perhaps due to the training data it had, since most candidates who got job offers were men, it made that assumption. Even after knowing the problems it was very hard to fix it, and eventually they had to stop using the said AI program!<\/p>\n

The most evident solution is first to figure out any biases that may exist before the training data is provided. The challenge is off course knowing about those biases. What can also help is giving a very wide range of data. Also train & test on every different data sets. This can highlight any incorrect assumptions and biases that might have been built \/ inferred from the training data set.<\/p>\n

Standardizing building and testing AI systems<\/h3>\n

While regulators and the wider community is looking for ways to have some baseline practices, however the use cases of AI are very widespread and validating the underline algorithm is challenging, it\u2019s going to be hard to reach some generic product-based validations.<\/p>\n

Having said that, there are some techniques which can help look at these solutions from a different paradigm, which can be used as good techniques to identify potential risks in such systems. One such technique is STPA [link]<\/a><\/strong> which suggests an alternative top down approach of looking at holistic systems and focusing just on safety instead of focusing on the system\u2019s functionality.<\/p>\n

Challenges ahead<\/h2>\n

The field of AI is exciting and has lots of applications. By now we are already seeing many products started to use AI in some capacity. This is going to ever increase because of AI\u2019s capability to process multi-dimension factors and process large amounts of data which can be hard for humans to do.<\/p>\n

Apart from topics discussed above, the key challenge IMHO is going to be lack of skills. Looking after the quality aspect is going to be even more challenging, these systems needs engineers who have technical insights into how the underlying technology works plus have \u2018testing Acumen\u2019 to test well. While this has always been a problem, seems with AI systems & Big data projects this will be an even bigger one.<\/p>\n<\/div>

<\/div><\/div><\/div><\/div><\/div>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":3,"featured_media":15671,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"yoast_head":"\n