The ACM is the world’s largest computing society. It recognizes up to 10 percent of its worldwide membership as distinguished members based on their professional experience, groundbreaking achievements and longstanding participation in computing. The ACM has three tiers of recognition: fellows, distinguished members and senior members. The Distinguished Member Recognition Program, which honors members with at least 15 years of professional experience, recognized Shah for his work at the intersection of information, access and responsible AI. Shah expressed his gratitude and appreciation for the award.
“I’m incredibly grateful for all the support I’ve received from everyone. It’s a very humbling experience,” said Shah.
Shah has contributed a great deal of research related to people-centered information access and examining how biases and issues of discrimination that are present within information systems can be counteracted. One of Shah’s significant contributions to the iSchool has been co-founding Responsibility in AI Systems and Experiences (RAISE).
“I am both astounded and unsurprised by this new effort,” says Chirag Shah at the University of Washington, who studies search technologies. “When it comes to demoing these things, they look so fantastic, magical, and intelligent. But people still don’t seem to grasp that in principle such things can’t work the way we hype them up to.”
An artificial intelligence program that has impressed the internet with its ability to generate original images from user prompts has also sparked concerns and criticism for what is now a familiar issue with AI: racial and gender bias.
And while OpenAI, the company behind the program, called DALL·E 2, has sought to address the issues, the efforts have also come under scrutiny for what some technologists have claimed is a superficial way to fix systemic underlying problems with AI systems.
“The common thread is that all of these systems are trying to learn from existing data,” Shah said. “They are superficially and, on the surface, fixing the problem without fixing the underlying issue.”
Marketing and analytics experts said marketers can choose from a number of off-the-shelf predictive analytics tools with machine learning and AI built in. However, Shah explained that the more advanced marketing operations often build their own algorithms and custom tools, seeing it as a way to differentiate their efforts and maximize the success for their own organizations. “It almost also becomes a proprietary thing. For many companies, the way they derive their insights is the ‘secret sauce,'” he said.
Dr. Yunhe Feng a fellow Infoseeking lab member and his team attended the Discovering AI@UW event co-hosted by the eScience Institute last week and presented his gender bias in image search work at the poster session.
This inspirational and educational event is a great networking opportunity for faculty, researchers, and students who are interested in using AI, finding synergies with major campus initiatives, and strategizing about the future of AI research at UW.
ExpScore: Learning Metrics for Recommendation Explanation
ABSTRACT Many information access and machine learning systems, including recommender systems, lack transparency and accountability. Highquality recommendation explanations are of great significance to enhance the transparency and interpretability of such systems. However, evaluating the quality of recommendation explanations is still challenging due to the lack of human-annotated data and benchmarks. In this paper, we present a large explanation dataset named RecoExp, which contains thousands of crowdsourced ratings of perceived quality in explaining ecommendations. To measure explainability in a comprehensive and interpretable manner, we propose ExpScore, a novel machine learning-based metric that incorporates the definition of explainability from various perspectives (e.g., relevance, readability, subjectivity, and sentiment polarity). Experiments demonstrate that ExpScore not only vastly outperforms existing metrics and but also keeps itself explainable. Both the RecoExp dataset and open-source implementation of ExpScore will be released for the whole community. These resources and our findings can serve as forces of public good for scholars as well as recommender systems users.
Bingbing Wen | University of Washington, Seattle, WA, US | email@example.com
Yunhe Feng | University of Washington, Seattle, WA, US| firstname.lastname@example.org
Yongfeng Zhang | Rutgers University, New Brunswick, NJ, US | email@example.com Chirag Shah | University of Washington, Seattle, WA, US, firstname.lastname@example.org |
Shruti Phadke is one of our lab members who is currently a Ph.D. candidate studying in Information School. Her presentation focuses on natural language processing computational methods and combining them with theories from social psychology to understand various Community practices in communities of problematic information.
In information seeking and retrieval research, researchers often collect data about users’ behaviors to predict task characteristics and personalize information for users. The reliability of user behavior may be directly influenced by data collection methods. This article reports on a mixed-methods study examining the impact of study setting (laboratory setting vs. remote setting) and task authenticity (authentic task vs. simulated task) on users’ online browsing and searching behaviors. Thirty-six undergraduate participants finished one lab session and one remote session in which they completed one authentic and one simulated task. Using log data collected from 144 task sessions, this study demonstrates that the synthetic lab study setting and simulated tasks had significant influences mostly on behaviors related to content pages (e.g., page dwell time, number of pages visited per task). Meanwhile, first-query behaviors were less affected by study settings or task authenticity than whole-session behaviors, indicating the reliability of using first-query behaviors in task prediction. Qualitative interviews reveal why users were influenced. This study addresses methodological limitations in existing research and provides new insights and implications for researchers who collect online user search behavioral data.