Let’s take some of the engineering out of AI

AI recommendation systems are built by engineers but engineers shouldn’t be totally in charge of them. We need non-machine learning experts working on these systems too. Including regular people on AI teams will reduce bias and improve the performance of the systems.

There is no escaping recommendations. AI is used to recommend things to us in products all the time. Obvious examples are amazon recommending products to buy or Netflix recommending movies to watch. Once you know what to look for you’ll find recommendation systems everywhere you look. And despite being used everywhere, the recommendations of these systems are often poor in quality.

Computer programs using machine learning techniques are what power most recommendation systems and, at their heart, the systems are methods of transforming data about what happened in the past into predictions about what will happen in the future. Those systems are technical feats of engineering taking months or years to build. And because they require expertise in a combination of computer science and statistics, these systems are usually built by a team of engineers with a math and statistics background.

Somewhat surprisingly, the personal characters of those who build these systems have profound impacts on the recommendations provided. This is not a new insight. Much has been written on the topic of how algorithms can encode the biases of people who build them and society at large. Research papers like this one by Inioluwa Deborah Raji (https://arxiv.org/abs/2102.00813) and news articles like “Who Is Making Sure the A.I. Machines Aren’t Racist?” (https://www.nytimes.com/2021/03/15/technology/artificial-intelligence-google-bias.html) are both excellent explorations of these arguments. But there is another reason we should be concerned with who is chosen to build these systems: if non-machine learning experts are part of the teams building the systems the recommendations could be a lot better. The reason for this has to do with what machine learning practitioners refer to as “Feature Engineering”.

Feature engineering is a critical step in the development of most recommendation systems. It is a process where humans, typically data scientists and engineers, define how to process “raw data” into the “features” the system will learn from. Many people have the mistaken impression that recommendation systems consume raw data in order to learn how to make accurate recommendations. The reality is that, in order to get these systems to deliver anything better than random guesses, engineers have found that features must be defined manually. Those features are the kinds of things that we as humans understand to be important to making a prediction. So, for example, if we are building a system to recommend products we might define features related to how often users buy big-ticket items, the categories of items they have bought in the past, their hobbies, and so on. Those features are requirements for good performance because algorithms do not know on their own that hobbies tell us something about what a human is likely to buy. This process of transforming raw data into features that have information that is important is called feature engineering.

And that’s why including non-experts on the teams building these AI systems is so important. Data scientists and engineers are experts in building programs but they are not usually also experts in why people buy products, watch movies, cheat on taxes, or any other of the millions of applications for recommendation systems. I have personally seen this dynamic play out countless times. I am a sys-gender white male from an upper-middle-class background in California. To date, all of the engineering teams I have worked with that built recommendation systems were staffed with people of the same background.

Including people unfamiliar with machine learning, but with knowledge about the domain can dramatically improve the quality of the features engineered which in turn gives algorithms better data to work with and results in better recommendations overall. Including people who have experience with the problem firsthand also means considering the positionality of the team members. Because recommendation systems are frequently used by everyone, it is essential that the team that builds them be representative of a diverse set of experiences. If instead those teams continue to be composed of people from privileged backgrounds, not only will the job opportunities be unequal, but the recommendations we all receive will be worse than they need to be.