Projects
AI for Social Science Research(Just Started)
We propose to conduct a meta-scientific exercise to determine the extent to which social scientists understand and/or endorse the enterprise of producing knowledge that can be synthesized by machines. Work under this aim will survey 150 research-active social scientists using both structured and unstructured elicitation techniques. Based on the Quantitative findings we will have a semi-structured interview to dig more details. A thematic process will be taken to analyze the qualitative data and statistical tests for the quantitative data.
Air Force Office of Scientific Research (AFOSR-Ongoing)
In this project we are enabling agents to engage in reasoning within a dynamic environment, which includes other agents and their cognitive processes. The platform is a game and agents will play with a human. I am designing a survey to understand how the human AI trust and strategy changes with different environment. The communication and trading strategies implemented by the human for competitive and cooperative scenario. In one case they know they are playing with agents and in another case, they don't know about that.
Usability Testing of AI empowered Tools for Replication Prediction(Ongoing)
We present a prototype hybrid prediction market and demonstrate the avenue it represents for meaningful human-AI collaboration. By embedding human participants in these markets alongside bot traders, we can bring together insights from both, to enhance the performance. We recruited 142 humans from 6 different research domains for this study where they have traded in the market with other agents. We highlight challenges and opportunities, share insights from semi-structured interviews with 8 hybrid market participants, and outline a vision for ongoing and future work.
Reproducibility or Replicability Crisis Across Disciplines and Impact of Geographical Diversity (Ongoing)
In pursuit of a comprehensive examination of the status of reproducibility, replicability, and open science practices across diverse academic disciplines in universities located in the United States and India, our research approach consisted of several key steps. This involved conducting interviews with faculty members and graduate students, followed by a rigorous analysis of the interview transcripts using thematic analysis techniques. Subsequently, we devised a follow-up survey, informed by the insights gleaned from the interviews, to broaden our study's reach and validate our findings among a larger and more diverse population. Currently we have 19 interviews and 350 survey responses. Furthermore, we are exploring the influence of geographical diversity on scientific research practices and outcomes.
This multifaceted research endeavors to contribute significantly to our understanding of the current state of reproducibility, replicability, and open science in academia, while also highlighting the potential impact of geographical diversity on these crucial aspects of scientific research.
​
​
​
​
​
​
​
​
​
Integrating measures of replicability into literature search (Completed)
This study delves into methods of incorporating AI-powered indicators of research confidence into literature searches. We conducted interviews with 17 PhD researchers to gain insights into their current literature search practices, and sought their feedback on a prototype replicability assessment tool. Our findings indicate that information regarding replicability can be a valuable asset in aiding researchers throughout the literature review and research design phases. Nevertheless, it is crucial to address issues of explainability and interpretability when it comes to system-generated outputs. Furthermore, potential drawbacks associated with AI-driven confidence assessments warrant further investigation before these tools can attain widespread acceptance and deployment.
Defense Advanced Research Projects Agency (DARPA-Completed)
We thoroughly experiment with an artificial prediction market model to understand the effects of market parameters on model performance for benchmark classification tasks. We then demonstrate, through simulation, the impact of exogenous agents in the market, where these exogenous agents represent primitive human behaviors. This work lays the foundation for a novel set of hybrid human-AI machine learning algorithms.
