Sunipa Dev

Sunipa Dev

Tech Lead and Research Scientist, Google
Sunipa Dev

Building safe, beneficial, and inclusive AI for global populations.

About

I am a Senior Research Scientist at Google Research within the Future of AI and Society organization. My research is at the intersection of language, AI, and society and my core objective is to core objective is to develop safe and inclusive AI systems that are truly useful for the majority of the world's population. My current focus is on socially grounding generative models, ensuring their responses reflect and serve a diverse range of real-world user experiences and needs.

Prior to this, I was fortunate to receive the NSF Computing Innovation Fellowship to work with Dr. Kai-Wei Chang at UCLA where I also received the DAAD Ainet award for my work. Before this I completed my PhD with Dr. Jeff Phillips at the School of Computing in University of Utah, investigating various facets of high dimensional embeddings such as language models.

I organize multiple workshops at *CL conferences, particularly the Cross Cultural Considerations in NLP workshop which has helped mobilize researchers engaged with multi-cultural NLP of the world, and the HCI + NLP workshop which bridges together interdisciplinary work that is much needed to push the frontiers of language technologies. I am also an advisor and former organizer for the Widening NLP or WiNLP workshop. Reach out to know more or to join us!

News

  • I was a Keynote Panelist for the inaugural Conference on Language Modeling (COLM 2024), where I discussed the importance of multi-cultural AI. I have also discussed this at the Deployable AI Workshop at AAAI 2024, and panels at WiNLP 2024, and JMM 2025.
  • You can now read more on my work and thoughts about improving model performance on gendered translations and culturally salient user queries in this article in Nature Computational Science. A more detailed overview is also shared in The Keyword.
  • As the Ethics Chairs for EMNLP 2024, Luciana Benotti, Snigdha Chaturvedi, and I have drafted guidelines for future editions. I continue to Senior Area Chair for ACL ARR, and NAACL, EMNLP, IJCNLP-AACL, and more into 2025.
  • Along with co-organizers Luciana Benotti, Vinodkumar Prabhakaran, Guido Ivetto, I will be at Khipu AI in Santiago, Chile in March 2025 with a new tutorial demonstrating the importance of scalable, cross-cultural evaluations of generative models.
  • We will be organizing the 3rd Workshop on Cross Cultural Considerations at NAACL 2025, and the 4th Workshop on HCI + NLP at EMNLP 2025. Both workshop teams are excited to continue the fabulous conversations in the journey so far!
  • One of our papers at ACL on the robustness of bias measurements in NLP has won the Outstanding Paper Award!
  • I am pleased to be named on the list of 100 Brilliant Women in AI Ethics alongside many other amazing women! My evaluative datasets have also been contributed by Google to the ML Commons.

Publications

Download a BibTex file containing all these papers.

Publications (grouped by year): 2025, 2024, 2023, 2022, 2021, 2020, 2019

2025

Piyawat Lertvittayakumjorn, David Kinney, Vinodkumar Prabhakaran, Donald Martin Jr., Sunipa Dev

Towards Geo-Culturally Grounded LLM Generations

ACL. 2025 Conference [BibTex]

2024

A Jha, V Prabhakaran, R Denton, S Laszlo, S Dave, R Qadri, CK Reddy, S Dev

ViSAGe: A global-scale analysis of visual stereotypes in text-to-image generation

ACL. 2024 Conference [BibTex]

K Robinson, S Kudugunta, R Stella, S Dev, J Bastings

MiTTenS: A Dataset for Evaluating Gender Mistranslation

EMNLP. 2024 Conference [BibTex]

A Rathore, S Dev, JM Phillips, V Srikumar, Y Zheng, CCM Yeh, J Wang, ...

VERB: Visualizing and interpreting bias mitigation techniques geometrically for word representations

ACM Transactions on Interactive Intelligent Systems. 2024 Journal [BibTex]

2023

S Dev, J Goyal, D Tewari, S Dave, V Prabhakaran

Building socio-culturally inclusive stereotype resources with community engagement

Advances in Neural Information Processing Systems. 2023 Conference [BibTex]

A Jha, A Davani, CK Reddy, S Dave, V Prabhakaran, S Dev

SeeGULL: A stereotype benchmark with broad geo-cultural coverage leveraging generative models

ACL. 2023 Conference [BibTex]

T Hossain, S Dev*, S Singh*

MISGENDERED: Limits of large language models in understanding pronouns

ACL. 2023 Conference [BibTex]

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, ...

PaLM: Scaling Language Modeling with Pathways

JMLR. 2023 Journal [BibTex]

R Anil, AM Dai, O Firat, M Johnson, D Lepikhin, A Passos, S Shakeri, ...

Palm 2 technical report

arXiv. 2023 Preprint [BibTex]

NR Selvam, S Dev, D Khashabi, T Khot, KW Chang

The Tail Wagging the Dog: Dataset construction biases of social bias benchmarks

ACL. 2023 Conference [BibTex]

2022

D Wu, W Ahmad, S Dev, KW Chang

Representation Learning for Resource-Constrained Keyphrase Generation

EMNLP. 2022 Conference [BibTex]

S Bhatt, S Dev, P Talukdar, S Dave, V Prabhakaran

Re-contextualizing fairness in NLP: The case of India

AACL. 2022 Conference [BibTex]

S Dev, E Sheng, J Zhao, J Sun, Y Hou, M Sanseverino, J Kim, N Peng, KW Chang

On Measures of Biases and Harms in NLP

AACL. 2022 Conference [BibTex]

V Malik, S Dev, A Nishi, N Peng, KW Chang

Socially aware bias measurements for Hindi language representations

NAACL. 2022 Conference [BibTex]

2021

S Dev, M Monajatipoor, A Ovalle, A Subramonian, JM Phillips, KW Chang

Harms of gender exclusivity and challenges in non-binary representation in language technologies

EMNLP. 2021 Conference [BibTex]

S Dev, T Li, J M Phillips, V Srikumar

OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings

EMNLP. 2021 Conference [BibTex]

2020

S Dev, T Li, J Phillips, V Srikumar

On Measuring and Mitigating Biased Inferences of Word Embeddings

AAAI. 2020 Conference [BibTex]

2019

S Dev, S Hassan, JM Phillips

Closed Form Word Embedding Alignment

IEEE ICDM. 2019 Conference [BibTex]

S Dev, J Phillips

Attenuating Bias in Word Vectors

AISTATS. 2019 Conference [BibTex]

For a comprehensive list of my publications, please visit my Google Scholar page.

CV

Here's a downloadable version of my CV:

Download CV