In our survey study, we paired a real page from search engine Bing and a synthesized page with more diversities in the results (i.e. less biased). Both pages show the top-10 search items given search queries. We asked participants which one they prefer and why do they prefer the selected one. Statistical analyses revealed that overall, participants prefer the original Bing pages. Additionally, the location where the diversities are introduced is significantly associated with users’ preferences.

We found out that users prefer results that are more consistent and relevant to the search queries. Introducing diversities undermines the relevance of the search results and impairs users’ satisfaction to some degree. It was interesting to see that users tend to pay more attention to the top portion of the results rather than the bottom ones, which is consistent with some previous findings.

Han, B., Shah, C., & Saelid, D. (2021). Users’ Perception of Search-Engine Biases and Satisfaction. Second International workshop on algorithmic bias in search and recommendation (Bias 2021). April 1, 2021.

Interested in learning more? Read the full paper here: https://arxiv.org/abs/2105.02898