Generative AI/Monitoring

GenAI applications in healthcare require new approaches for strong alignment, probabilistic guarantees on hallucination, and adaptive guardrails. My prior work in this area includes:

High Dimensional Data Monitoring
Detecting events from millions of data streams in real-time.

I formalized novel machine learning tasks for real-time data monitoring and developed AI methods for monitoring. These methods were deployed in an applied national public health setting where they led to a 200x increase in monitoring efficiency.

Foundations for Steering Theory
Quantify GenAI model steering capacity.

The interface between interpretability tools and successful long-term model monitoring and guardrailing is underexplored. I developed methods and metrics to quantify model steering capacity when model neurons are monosemantic (encode a single concept) that can be monitored for various adversarial attacks.

Models and Emergent Properties
Human evaluation and distance metrics

Humans are increasingly relying on Generative AI models that can be highly persuasive. These properties are rapidly changing; how do we develop evaluations that keep up?

Papers and Blogs

📄 Joshi, A. “Event Monitoring in Modern Public Health Data Streams”, Carnegie Mellon University Thesis

📄 Joshi, A.; Townes, T.; Gormley, N.; Neureiter, L.; Rosenfeld, R.; Wilder, B. “Outlier Ranking for Large-Scale Public Health Data.” Proceedings of the AAAI Conference on Artificial Intelligence, 38(20), 22176-22184, 2024.

🛠️ Outshines Package : https://github.com/Ananya-Joshi/Outshines-Documentation

📄 Joshi, A.; Mazaitis, K.; Rosenfeld, R.; Wilder, B. “Computationally Assisted Quality Control for Public Health Data Streams.” International Joint Conference on Artificial Intelligence, 32, 6004-6012, 2023.

📄 Joshi, A.; Cintas, C, Speakman, S, “Enabling Sparse Autoencoders for Topic Alignment in Large Language Models”

🛠️ LLM Steering Package: https://github.com/IBM/sae-steering

📄 Joshi, A., Wilder, B., and Rosenfeld, R. “Actionable Data Monitoring in Modern Data Streams.” Knowledge Discovery and Data Mining (SIGKDD) Doctoral Consortium, 2024.

📄 Joshi, A., “Data Monitoring for Large-scale Public Health Data.” AAAI Conference on Artificial Intelligence Doctoral Consortium 2025.

📄 Joshi, A. “Creating an Automated Ideological Transformer Using Moral Reframing”. Princeton University Thesis, A+

📄 Piangerelli, M., Prenkaj, B., Rotalinti, Y., Joshi, A., and Stilo, G. “Workshop on Discovering Drift Phenomena in Evolving Data Landscapes (DELTA).” Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 6731-6732, 2024.

✍️ Joshi, A. “Alerting Systems are Incompatible with Modern Public Health Data Monitoring.” Carnegie Mellon University.

✍️ Joshi, A; Rudow, M. “AutoPrompt: Using Synthetic Persona Generation to Improve Prompt Responses” https://github.com/Ananya-Joshi/AutoPrompt

✍️ Joshi, A; Rudow, M. “Multi-LLM Agent Meta-Benchmarking Method” https://github.com/jAMackathon/Multi-Agent-Meta-Benchmarking-Method/tree/main

✍️ Joshi, A. “Enabling New Applications with Today’s Mechanistic Interpretability Toolkit” https://www.lesswrong.com/posts/iDyf7aBdvBp9jnfuY/enabling-new-applications-with-today-s-mechanistic

📕 Time Series Data: Clarifying Practical Approaches This is an 80-minute active-learning lecture for students in Carnegie Mellon’s Machine Learning in Practice course.

đź“• MLCommons Medical Working Group Presentation: Monitoring for Health Events: Bridging Healthcare and Public Health Approaches.
Tthe lecture proposes opportunities for integrating machine learning orchestration in healthcare and public health. Details available upon request.

Presentations / Collaborators

I’ve presented at or collaborated with MLCommons, MLCommons Rising Stars Program , Computational and Data Science Rising Stars, IJCAI, AAAI, CMU’s Parallel Data Laboratory, CMU’s AI Seminar, CMU’s 3 Minute Thesis competition finalist, AAAI Doctoral Consortium, SIGKDD Doctoral Consortium, University of Ohio, University of Iowa, and the University of Utah. I’ve reviewed for AAAI, IJCAI, CHI, KDD, and VIS.