Advata Research

2021

Paper
Software as a Medical Device: Regulating AI in Healthcare via Responsible AI

Ahmad, Muhammad Aurangzeb, Overman, Steve, Allen, Christine, Kumar, Vikas, Teredesai, Ankur, and Eckert, Carly

In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining 2021

Abs Bib Video

With the increased adoption of AI in healthcare, there is a growing recognition and demand to regulate AI in healthcare to avoid potential harm and unfair bias against vulnerable populations. Around a hundred governmental bodies and commissions as well as leaders in the tech sector have proposed principles to create responsible AI systems. However, most of these proposals are short on specifics which has led to charges of ethics washing. In this tutorial we offer a guide to help navigate through complex governmental regulations and explain the various constituent practical elements of a responsible AI system in healthcare in the light of proposed regulations. Additionally, we breakdown and emphasize that the recommendations from regulatory bodies like FDA or the EU are necessary but not sufficient elements of creating a responsible AI system. We elucidate how regulations and guidelines often focus on epistemic concerns to the detriment of practical concerns e.g., requirement for fairness without explicating what fairness constitutes for a use case. FDA’s Software as a medical device document and EU’s GDPR among other AI governance documents talk about the need for implementing sufficiently good machine learning practices. In this tutorial we elucidate what that would mean from a practical perspective for real world use cases in healthcare throughout the machine learning cycle i.e., Data Management, Data Specification, Feature Engineering, Model Evaluation, Model Specification, Model Explainability, Model Fairness, Reproducibility, checks for data leakage and model leakage. We note that conceptualizing responsible AI as a process rather than an end goal accords well with how AI systems are used in practice. We also discuss how a domain centric stakeholder perspective translates into balancing requirements for multiple competing optimization criteria.
@inproceedings{ahmad2021software, title = {Software as a Medical Device: Regulating AI in Healthcare via Responsible AI}, author = {Ahmad, Muhammad Aurangzeb and Overman, Steve and Allen, Christine and Kumar, Vikas and Teredesai, Ankur and Eckert, Carly}, booktitle = {Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery \& Data Mining}, pages = {4023--4024}, year = {2021}, pptx = {SaMDKDDTutorial2021.pptx}, url = {https://dl.acm.org/doi/10.1145/3447548.3470823}, video = {https://www.youtube.com/watch?v=p-Kg27%2D%2DMII}, abbr = {Paper}, bibtex_show = {true}, responsibleai = {pubs}, healthcare = {pubs} }
Paper
Machine Learning Approaches for Type 2 Diabetes Prediction and Care Management

Lim, Aloysius, Singh, Ashish, Chiam, Jody, Eckert, Carly, Kumar, Vikas, Ahmad, Muhammad Aurangzeb, and Teredesai, Ankur

arXiv preprint arXiv:2104.07820 2021

Abs Bib PDF

Prediction of diabetes and its various complications has been studied in a number of settings, but a comprehensive overview of problem setting for diabetes prediction and care management has not been addressed in the literature. In this document we seek to remedy this omission in literature with an encompassing overview of diabetes complication prediction as well as situating this problem in the context of real world healthcare management. We illustrate various problems encountered in real world clinical scenarios via our own experience with building and deploying such models. In this manuscript we illustrate a Machine Learning (ML) framework for addressing the problem of predicting Type 2 Diabetes Mellitus (T2DM) together with a solution for risk stratification, intervention and management. These ML models align with how physicians think about disease management and mitigation, which comprises these four steps: Identify, Stratify, Engage, Measure.
@article{lim2021machine, title = {Machine Learning Approaches for Type 2 Diabetes Prediction and Care Management}, author = {Lim, Aloysius and Singh, Ashish and Chiam, Jody and Eckert, Carly and Kumar, Vikas and Ahmad, Muhammad Aurangzeb and Teredesai, Ankur}, journal = {arXiv preprint arXiv:2104.07820}, year = {2021}, pdf = {LimType2Diabetes2021.pdf}, abbr = {Paper}, bibtex_show = {true}, healthcare = {pubs}, providers = {pubs} }
Paper
Machine Learning Approaches for Pressure Injury Prediction

Ahmad, Muhammad Aurangzeb, Larson, Barrett, Overman, Steve, Kumar, Vikas, Xie, Jing, Rossington, Alan, Patel, Ankur, and Teredesai, Ankur

In 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI) 2021

Abs Bib Video PDF

Pressure Injuries are localized damages to the skin caused by sustained pressure. It is a common yet preventable disease affecting millions of patients. While there are multiple scales to determine if a patient has pressure injury, these methods suffer from high inter-rater subjectivity. To address this problem we create predictive models for pressure injury using Centers for Medicare Medicaid Services claims data. The models show relatively good predictive performance, we also explore aspects of the model where they will be deployed in a real world clinical settings.
@inproceedings{ahmad2021machine, title = {Machine Learning Approaches for Pressure Injury Prediction}, author = {Ahmad, Muhammad Aurangzeb and Larson, Barrett and Overman, Steve and Kumar, Vikas and Xie, Jing and Rossington, Alan and Patel, Ankur and Teredesai, Ankur}, booktitle = {2021 IEEE 9th International Conference on Healthcare Informatics (ICHI)}, pages = {427--431}, year = {2021}, organization = {IEEE}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {AhmadPressureInjuryICU_ICHI2021.pdf}, pptx = {AhmadPressureInjuryICU_ICHI2021.pptx}, video = {https://www.youtube.com/watch?v=1dELhvZKJ6w} }
Paper
Fairness in Healthcare AI: A practical guide

Ahmad, Muhammad Aurangzeb, Eckert, Carly, Allen, Christine, Kumar, Vikas, Hu, Juhua, and Teredesai, Ankur

In 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI) 2021

Bib
@inproceedings{ahmadfair2021ichi, title = {Fairness in Healthcare AI: A practical guide}, author = {Ahmad, Muhammad Aurangzeb and Eckert, Carly and Allen, Christine and Kumar, Vikas and Hu, Juhua and Teredesai, Ankur}, booktitle = {2021 IEEE 9th International Conference on Healthcare Informatics (ICHI)}, year = {2021}, organization = {IEEE}, abbr = {Paper}, pub_type = {Tutorial}, bibtex_show = {Tutorial} }
Paper
Machine Learning Approaches for Patient State Prediction in Pediatric ICUs

Ahmad, Muhammad Aurangzeb, Rivera, Eduardo Antonio Trujillo, Murray, Pollack MD, Carly, Eckert MD, Anita, Patel MD, and Teredesai, Ankur

In 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI) 2021

Abs Bib Video PDF

We consider the problem of characterizing and predicting the condition of pediatric patients in intensive care units (ICUs). This population is often typified by rapid changes in patient conditions which necessitate predictions that can capture transition in patient states. While the assessment of patient’s condition is currently usually done using domain based scoring systems, we employ machine learning models for predicting the state of the pediatric patient. Additionally, we explore how model explainability could affect the usage of predictive models in a real world settings.
@inproceedings{ahmad2021machinf, title = {Machine Learning Approaches for Patient State Prediction in Pediatric ICUs}, author = {Ahmad, Muhammad Aurangzeb and Rivera, Eduardo Antonio Trujillo and Murray, Pollack MD and Carly, Eckert MD and Anita, Patel MD and Teredesai, Ankur}, booktitle = {2021 IEEE 9th International Conference on Healthcare Informatics (ICHI)}, pages = {422--426}, year = {2021}, organization = {IEEE}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {AhmadRiveraPediatricICHI2021.pdf}, pptx = {AhmadPediatricICU_ICHI2021.pptx}, video = {https://www.youtube.com/watch?v=znrdVW4YOdw} }
Paper
Interpretable Phenotyping for Electronic Health Records

Allen, Christine, Hu, Juhua, Kumar, Vikas, Ahmad, Muhammad Aurangzeb, and Teredesai, Ankur

In 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI) 2021

Abs Bib Video PDF

Datasets from Electronic Health Records (EHRs) are increasingly large and complex, creating challenges in their use for predictive modeling. The two major challenges are large-scale and high-dimensionality. One of the common way to address the large-scale challenge is through use of data phenotypes: clinically relevant characteristic groupings that can be expressed as logical queries (e.g., “senior patients with diabetes”). With the increasing use of machine learning across the continuum of care, phenotypes play an important role in modeling for population management, clinical trials, observational and interventional research, and quality measures. Yet, phenotype interpretation can often be difficult and require post-hoc clarifications from experienced clinicians. For example, detailed analysis may be needed to find that all patients in a a phenotype are diabetic seniors with complications from previous surgery. Moreover, the high-dimensionality problem is often addressed either separately or simultaneously with phenotyping by dimension reduction methods that may further hinder interpretability. In this paper, we introduce the notion of interpretable data phenotypes generated by an unsupervised learning technique. Methods are designed to disambiguate relative feature memberships, thus facilitating general clinical validation, and alleviating the problem of high-dimensionality. The empirical study applies the proposed unsupervised interpretable phenotyping method to a real world healthcare dataset (MIMIC), then uses hospital length of stay as a reference prediction task. The results demonstrate that the proposed method produces phenotypes with improved interpretability and without diminishing the quality of prediction results.
@inproceedings{allen2021interpretable, title = {Interpretable Phenotyping for Electronic Health Records}, author = {Allen, Christine and Hu, Juhua and Kumar, Vikas and Ahmad, Muhammad Aurangzeb and Teredesai, Ankur}, booktitle = {2021 IEEE 9th International Conference on Healthcare Informatics (ICHI)}, pages = {161--170}, year = {2021}, organization = {IEEE}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {AllenPhenotypingICHI2021.pdf}, pptx = {}, video = {https://www.youtube.com/watch?v=mf1Z6QUsGhs} }
Tutorial
Fairness in Healthcare AI

Ahmad, Muhammad Aurangzeb, Eckert, Carly, Allen, Christine, Kumar, Vikas, Hu, Juhua, and Teredesai, Ankur

In 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI) 2021

Abs Bib Video

The issue of bias and fairness in healthcare has been around for centuries. With the integration of AI in healthcare the potential to discriminate and perpetuate unfair and biased practices in healthcare increases many folds. The tutorial focuses on the challenges, requirements and opportunities in the area of fairness in healthcare AI and the various nuances associated with it. The problem healthcare as a multi-faceted systems level problem that necessitates careful consideration of different notions of fairness in healthcare to corresponding concepts in machine learning is elucidated via different real world examples.
@inproceedings{ahmad2021fairness, title = {Fairness in Healthcare AI}, author = {Ahmad, Muhammad Aurangzeb and Eckert, Carly and Allen, Christine and Kumar, Vikas and Hu, Juhua and Teredesai, Ankur}, booktitle = {2021 IEEE 9th International Conference on Healthcare Informatics (ICHI)}, pages = {554--555}, year = {2021}, organization = {IEEE}, abbr = {Tutorial}, pub_type = {Tutorial}, bibtex_show = {true}, pptx = {FairnessInHealthcareML-ICHI2021.pptx}, video = {https://youtu.be/Ng0i0eOh02U} }
Tutorial
Fairness in Healthcare Machine Learning: A Practical Guide

Ahmad, Muhammad Aurangzeb, Eckert, Carly, Allen, Christine, Hu, Juhua, Kumar, Vikas, and Teredesai, Ankur

In The 25th Pacific-Asia Conference on Knowledge Discovery and Data Mining 2021

Abs Bib

The issue of bias and fairness in healthcare has been around for centuries. With the integration of AI in healthcare the potential to discriminate and perpetuate unfair and biased practices in healthcare increases many folds. The tutorial focuses on the challenges, requirements and opportunities in the area of fairness in healthcare AI and the various nuances associated with it. The problem healthcare as a multi-faceted systems level problem that necessitates careful consideration of different notions of fairness in healthcare to corresponding concepts in machine learning is elucidated via different real world examples.
@inproceedings{ahmad2020pakdd, title = {Fairness in Healthcare Machine Learning: A Practical Guide}, author = {Ahmad, Muhammad Aurangzeb and Eckert, Carly and Allen, Christine and Hu, Juhua and Kumar, Vikas and Teredesai, Ankur}, booktitle = {The 25th Pacific-Asia Conference on Knowledge Discovery and Data Mining}, year = {2021}, abbr = {Tutorial}, pub_type = {Tutorial}, bibtex_show = {true}, pptx = {PAKDD2020-FairnessInHealthcareML.pptx} }
Paper
Use of machine learning to assess the predictive value of 3 commonly used clinical measures to quantify outcomes after total shoulder arthroplasty

Kumar, Vikas, Roche, Christopher, Overman, Steven, Simovitch, Ryan, Flurin, Pierre-Henri, Wright, Thomas, Zuckerman, Joseph, Routman, Howard, and Teredesai, Ankur

In Seminars in Arthroplasty: JSES 2021

Abs Bib HTML PDF

An important psychometric parameter of validity that is rarely assessed is predictive value. In this study we utilize machine learning to analyze the predictive value of 3 commonly used clinical measures to assess 2-year outcomes after total shoulder arthroplasty (TSA).
@inproceedings{kumar2021use, title = {Use of machine learning to assess the predictive value of 3 commonly used clinical measures to quantify outcomes after total shoulder arthroplasty}, author = {Kumar, Vikas and Roche, Christopher and Overman, Steven and Simovitch, Ryan and Flurin, Pierre-Henri and Wright, Thomas and Zuckerman, Joseph and Routman, Howard and Teredesai, Ankur}, booktitle = {Seminars in Arthroplasty: JSES}, volume = {31}, number = {2}, pages = {263--271}, year = {2021}, organization = {Elsevier}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, html = {https://www.jsesarthroplasty.org/article/S1045-4527%2821%2900006-7/fulltext}, pdf = {KumarArthroplasty2021.pdf}, medtech = {pubs} }
Paper
Validation of a machine learning–derived clinical metric to quantify outcomes after total shoulder arthroplasty

Roche, Christopher, Kumar, Vikas, Overman, Steven, Simovitch, Ryan, Flurin, Pierre-Henri, Wright, Thomas, Routman, Howard, Teredesai, Ankur, and Zuckerman, Joseph

Journal of Shoulder and Elbow Surgery 2021

Abs Bib PDF

We propose a new clinical assessment tool constructed using machine learning, called the Shoulder Arthroplasty Smart (SAS) score to quantify outcomes following total shoulder arthroplasty (TSA).
@article{roche2021validation, title = {Validation of a machine learning--derived clinical metric to quantify outcomes after total shoulder arthroplasty}, author = {Roche, Christopher and Kumar, Vikas and Overman, Steven and Simovitch, Ryan and Flurin, Pierre-Henri and Wright, Thomas and Routman, Howard and Teredesai, Ankur and Zuckerman, Joseph}, journal = {Journal of Shoulder and Elbow Surgery}, year = {2021}, publisher = {Elsevier}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {RocheKumarArthroplasty2021} }
Paper
Shoulder Arthroplasty Smart Score

Roche, C, Kumar, V, Overman, S, Simovitch, R, Flurin, PH, Wright, T, Routman, H, Teredesai, A, and Zuckerman, J

J Shoulder Elbow Surg 2021

Abs Bib PDF

We propose and evaluate a new clinical assessment tool to quantify outcomes following anatomic total shoulder arthroplasty (aTSA) and reverse TSA (rTSA). This TSA-specific outcome measure aims to provide greater insight into both the disease-progression and treatment by utilizing both subjective and objective measures previously demonstrated to be highly predictive of post-operative TSA outcomes. Doing so, may create a novel tool that better accounts for TSA outcomes variability. The goal of this study is to quantify and compare the psychometric properties of this machine learning derived assessment tool relative to 5 commonly used clinical measures to quantify TSA outcomes.
@article{roche2021shoulder, title = {Shoulder Arthroplasty Smart Score}, author = {Roche, C and Kumar, V and Overman, S and Simovitch, R and Flurin, PH and Wright, T and Routman, H and Teredesai, A and Zuckerman, J}, journal = {J Shoulder Elbow Surg}, year = {2021}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {RocheKumarSmartScore2021.pdf} }
Paper
Using machine learning to predict clinical outcomes after shoulder arthroplasty with a minimal feature set

Kumar, Vikas, Roche, Christopher, Overman, Steven, Simovitch, Ryan, Flurin, Pierre-Henri, Wright, Thomas, Zuckerman, Joseph, Routman, Howard, and Teredesai, Ankur

Journal of Shoulder and Elbow Surgery 2021

Abs Bib HTML PDF

A machine learning analysis was conducted on 5774 shoulder arthroplasty patients to create predictive models for multiple clinical outcome measures after anatomic total shoulder arthroplasty (aTSA) and reverse total shoulder arthroplasty (rTSA). The goal of this study was to compare the accuracy associated with a full–feature set predictive model (ie, full model, comprising 291 parameters) and a minimal–feature set model (ie, abbreviated model, comprising 19 input parameters) to predict clinical outcomes to assess the efficacy of using a minimal feature set of inputs as a shoulder arthroplasty clinical decision-support tool.
@article{kumar2021using, title = {Using machine learning to predict clinical outcomes after shoulder arthroplasty with a minimal feature set}, author = {Kumar, Vikas and Roche, Christopher and Overman, Steven and Simovitch, Ryan and Flurin, Pierre-Henri and Wright, Thomas and Zuckerman, Joseph and Routman, Howard and Teredesai, Ankur}, journal = {Journal of Shoulder and Elbow Surgery}, volume = {30}, number = {5}, pages = {e225--e236}, year = {2021}, publisher = {Elsevier}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {KumarArthoMinimalFeature2021.pdf}, html = {https://www.sciencedirect.com/science/article/abs/pii/S1058274620306467} }
Chapter
Survey of explainable machine learning with visual and granular methods beyond quasi-explanations

Kovalerchuk, Boris, Ahmad, Muhammad Aurangzeb, and Teredesai, Ankur

2021

Abs Bib Video PDF

This paper surveys visual methods of explainability of Machine Learning (ML) with focus on moving from quasi-explanations that dominate in ML to domain-specific explanation supported by granular visuals. ML interpretation is fundamentally a human activity and visual methods are more readily interpretable. While efficient visual representations of high-dimensional data exist, the loss of interpretable information, occlusion, and clutter continue to be a challenge, which lead to quasi-explanations. We start with the motivation and the different definitions of explainability. The paper focuses on a clear distinction between quasi-explanations and domain specific explanations, and between explainable and an actually explained ML model that are critically important for the explainability domain. We discuss foundations of interpretability, overview visual interpretability and present several types of methods to visualize the ML models. Next, we present methods of visual discovery of ML models, with the focus on interpretable models, based on the recently introduced concept of General Line Coordinates (GLC). These methods take the critical step of creating visual explanations that are not merely quasi-explanations but are also domain specific visual explanations while these methods themselves are domain-agnostic. The paper includes results on theoretical limits to preserve n-D distances in lower dimensions, based on the Johnson-Lindenstrauss lemma, point-to-point and point-to-graph GLC approaches, and real-world case studies. The paper also covers traditional visual methods for understanding ML models, which include deep learning and time series models. We show that many of these methods are quasi-explanations and need further enhancement to become domain specific explanations. We conclude with outlining open problems and current research frontiers.
@incollection{kovalerchuk2021survey, abbr = {Chapter}, bibtex_show = {true}, title = {Survey of explainable machine learning with visual and granular methods beyond quasi-explanations}, author = {Kovalerchuk, Boris and Ahmad, Muhammad Aurangzeb and Teredesai, Ankur}, booktitle = {Interpretable Artificial Intelligence: A Perspective of Granular Computing (Eds. W. Pedrycz, SM Chen), Springer}, pages = {217--267}, year = {2021}, publisher = {Springer}, pdf = {KovalerchukAhmadBookChapter2021.pdf}, pptx = {KovalerchukAhmadTeredesaiIVIV2020.pptx}, video = {https://youtu.be/N8x2jx3zDmw}, responsibleai = {pubs}, xai = {pubs} }

2020

Software
fairMLHealth: Tools and tutorials for fairness evaluation in healthcare machine learning.

Allen, Christine, Ahmad, Muhammad Aurangzeb, Eckert, Carly, Hu, Juhua, Kumar, Vikas, and Teredesai, Ankur

2020

Bib Code
@misc{fairMLHealth, title = {fairMLHealth: Tools and tutorials for fairness evaluation in healthcare machine learning.}, author = {Allen, Christine and Ahmad, Muhammad Aurangzeb and Eckert, Carly and Hu, Juhua and Kumar, Vikas and Teredesai, Ankur}, year = {2020}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/KenSciResearch/fairMLHealth}}, abbr = {Software}, code = {https://github.com/KenSciResearch/fairMLHealth}, bibtex_show = {true}, healthcare = {pubs} }
Tutorial
Fairness in machine learning for healthcare

Ahmad, Muhammad Aurangzeb, Patel, Arpit, Eckert, Carly, Kumar, Vikas, and Teredesai, Ankur

In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 2020

Abs Bib Video PDF

The issue of bias and fairness in healthcare has been around for centuries. With the integration of AI in healthcare the potential to discriminate and perpetuate unfair and biased practices in healthcare increases many folds The tutorial focuses on the challenges, requirements and opportunities in the area of fairness in healthcare AI and the various nuances associated with it. The problem healthcare as a multi-faceted systems level problem that necessitates careful of different notions of fairness in healthcare to corresponding concepts in machine learning is elucidated via different real world examples.
@inproceedings{ahmad2020fairness, title = {Fairness in machine learning for healthcare}, author = {Ahmad, Muhammad Aurangzeb and Patel, Arpit and Eckert, Carly and Kumar, Vikas and Teredesai, Ankur}, booktitle = {Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery \& Data Mining}, pages = {3529--3530}, year = {2020}, abbr = {Tutorial}, pub_type = {Tutorial}, bibtex_show = {true}, pdf = {AhmadFairnessKDD2020.pdf}, pptx = {KDD2020-FairnessInHealthcareML.pptx}, video = {https://www.youtube.com/watch?v=MzuoWAk9_AQ} }
Abstract
What Is the Accuracy of Three Different Machine Learning Techniques to Predict Clinical Outcomes After Shoulder Arthroplasty?

Kumar, Vikas, Roche, Christopher, Overman, Steven, Simovitch, Ryan, Flurin, Pierre-Henri, Wright, Thomas, Zuckerman, Joseph, Routman, Howard, and Teredesai, Ankur

Clinical orthopaedics and related research 2020

Abs Bib HTML

Machine learning techniques can identify complex relationships in large healthcare datasets and build prediction models that better inform physicians in ways that can assist in patient treatment decision-making. In the domain of shoulder arthroplasty, machine learning appears to have the potential to anticipate patients’ results after surgery, but this has not been well explored.
@article{kumar2020accuracy, title = {What Is the Accuracy of Three Different Machine Learning Techniques to Predict Clinical Outcomes After Shoulder Arthroplasty?}, author = {Kumar, Vikas and Roche, Christopher and Overman, Steven and Simovitch, Ryan and Flurin, Pierre-Henri and Wright, Thomas and Zuckerman, Joseph and Routman, Howard and Teredesai, Ankur}, journal = {Clinical orthopaedics and related research}, volume = {478}, number = {10}, pages = {2351--2363}, year = {2020}, abbr = {Abstract}, pub_type = {Abstract}, bibtex_show = {true}, html = {https://pubmed.ncbi.nlm.nih.gov/32332242/} }
Paper
Assessing Fairness in Classification Parity of Machine Learning Models in Healthcare

Yuan, Ming, Kumar, Vikas, Ahmad, Muhammad Aurangzeb, and Teredesai, Ankur

AAAI Fall Symposium 2020 on AI for Social Good 2020

Abs Bib Video PDF

Fairness in AI and machine learning systems has become a fundamental problem in the accountability of AI systems. While the need for accountability of AI models is near ubiquitous, healthcare in particular is a challenging field where accountability of such systems takes upon additional importance, as decisions in healthcare can have life altering consequences. In this paper we present preliminary results on fairness in the context of classification parity in healthcare. We also present some exploratory methods to improve fairness and choosing appropriate classification algorithms in the context of healthcare.
@article{yuan2020fair, abbr = {Paper}, bibtex_show = {true}, title = {Assessing Fairness in Classification Parity of Machine Learning Models in Healthcare}, author = {Yuan, Ming and Kumar, Vikas and Ahmad, Muhammad Aurangzeb and Teredesai, Ankur}, journal = {AAAI Fall Symposium 2020 on AI for Social Good}, year = {2020}, month = nov, day = {13}, pdf = {https://arxiv.org/pdf/2102.03717.pdf}, pptx = {AAAI2020_MingKumar.pptx}, video = {https://youtu.be/7iYH5zOKmY4}, responsibleai = {pubs}, fairness = {pubs}, healthcare = {pubs} }
Paper
Emergency Department Optimization and Load Prediction in Hospitals

Padthe, Karthik K., Kumar, Vikas, Eckert, Carly M., Mark, Nicholas M., Zahid, Anam, Ahmad, Muhammad Aurangzeb, and Teredesai, Ankur

AAAI Fall Symposium 2020 on AI for Social Good 2020

Abs Bib Video PDF

Over the past several years, across the globe, there has been an increase in people seeking care in emergency departments (EDs). ED resources, including nurse staffing, are strained by such increases in patient volume. Accurate forecasting of incoming patient volume in emergency departments (ED) is crucial for efficient utilization and allocation of ED resources. Working with a suburban ED in the Pacific Northwest, we developed a tool powered by machine learning models, to forecast ED arrivals and ED patient volume to assist end-users, such as ED nurses, in resource allocation. In this paper, we discuss the results from our predictive models, the challenges, and the learnings from users’ experiences with the tool in active clinical deployment in a real world setting.
@article{padthe2020ed, abbr = {Paper}, bibtex_show = {true}, title = {Emergency Department Optimization and Load Prediction in Hospitals}, author = {Padthe, Karthik K. and Kumar, Vikas and Eckert, Carly M. and Mark, Nicholas M. and Zahid, Anam and Ahmad, Muhammad Aurangzeb and Teredesai, Ankur}, journal = {AAAI Fall Symposium 2020 on AI for Social Good}, year = {2020}, month = nov, day = {13}, pdf = {https://arxiv.org/pdf/2102.03672.pdf}, video = {https://youtu.be/Mn8ZptgnRv4}, providers = {pubs}, healthcare = {pubs} }
Patent

Machine learning model repository

Teredesai, Ankur, Marquardt, James Andrew, Rizzuto, Chris James, and Hughes, Tyler John

2020

PDF
Patent

Cryptographically secure machine learning

Fritchman, Kyle Josiah, Hughes, Tyler John, Teredesai, Ankur, De Cock, Martine Ivonne Leo, and Nascimento, Anderson

2020

PDF
Panel
Fairness, accountability, transparency in AI at scale: Lessons from national programs

Ahmad, Muhammad Aurangzeb, Teredesai, Ankur, and Eckert, Carly

In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency 2020

Abs Bib

The panel aims to elucidate how different national govenmental programs are implementing accountability of machine learning systems in healthcare and how accountability is operationlized in different cultural settings in legislation, policy and deployment. We have representatives from three different govenments, UAE, Singapore and Maldives who will discuss what accountability of AI and machine learning means in their contexts and use cases. We hope to have a fruitful conversation around FAT ML as it is operationalized ccross cultures, national boundries and legislative constraints.
@inproceedings{ahmad2020fairnest, title = {Fairness, accountability, transparency in AI at scale: Lessons from national programs}, author = {Ahmad, Muhammad Aurangzeb and Teredesai, Ankur and Eckert, Carly}, booktitle = {Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency}, pages = {690--690}, year = {2020}, abbr = {Panel}, pub_type = {Panel}, bibtex_show = {true} }

Talk

Foundations of Interpretable Machine Learning

Ahmad, Muhammad Aurangzeb

In Invited Talk at Chan Zuckerberg Initiative Redwood City, CA 2020

Bib

@inproceedings{cziAhmad2021,
  title = {Foundations of Interpretable Machine Learning},
  author = {Ahmad, Muhammad Aurangzeb},
  year = {2020},
  month = feb,
  booktitle = {Invited Talk at Chan Zuckerberg Initiative Redwood City, CA},
  abbr = {Talk},
  pub_type = {Talk},
  bibtex_show = {true},
  healthcare = {pubs},
  responsibleai = {pubs},
  xai = {pubs}
}

Talk

Holding Machine Learning Systems Accountable via Explanations

Ahmad, Muhammad Aurangzeb

In Invited Talk at Istanbul Technical University, Istanbul, Turkey 2020

Bib

@inproceedings{ituAhmad2020,
  title = {Holding Machine Learning Systems Accountable via Explanations},
  author = {Ahmad, Muhammad Aurangzeb},
  year = {2020},
  month = feb,
  booktitle = {Invited Talk at Istanbul Technical University, Istanbul, Turkey},
  abbr = {Talk},
  pub_type = {Talk},
  bibtex_show = {true},
  healthcare = {pubs},
  responsibleai = {pubs},
  xai = {pubs}
}

Tutorial
Deep Explanations in Machine Learning via Interpretable Visual Methods

Kovalerchuk, Boris, Aurangzeb, Muhammad, and Teredesai, Ankur

In The 24th Pacific-Asia Conference on Knowledge Discovery and Data Mining 2020

Abs Bib

Interpretability of Machine Learning (ML) models is a major area of current research, applications and debates in AI. The debates include a statement that This tutorial covers the state-of-the-art research, development, and applications in the area of Interpretable Knowledge Discovery and ML boosted by Visual Methods. The topic is interdisciplinary, bridging efforts of research and applied communities in AI, Machine Learning, Visual Analytics, Information Visualization, and HCI. This is a novel and fast-growing area with significant applications, and potential due to its importance in applications and prominence of visual ways of human cognition and perception. The recent progress in this area is very evident with a major deep learning explanation approach, based on visualization of salient areas and methods to visualize similarity of high-dimensional data in deep learning and other ML studies. Multiple techniques are emerging including lossless and reversible methods to visualize high-dimensional data, which will be presented in this tutorial to stimulate studies beyond heatmaps, t-SNE, and black-box ML models in general.
@inproceedings{kovalerchuk2020pakdd, title = {Deep Explanations in Machine Learning via Interpretable Visual Methods}, author = {Kovalerchuk, Boris and Aurangzeb, Muhammad and Teredesai, Ankur}, booktitle = {The 24th Pacific-Asia Conference on Knowledge Discovery and Data Mining}, year = {2020}, abbr = {Tutorial}, pub_type = {Tutorial}, bibtex_show = {true}, pptx = {KovalerchukAhmadTeredesaiPAKDD2020.pptx} }

2019

Paper
The Challenge of Imputation in Explainable Artificial Intelligence Models

Ahmad, Muhammad Aurangzeb, Eckert, Carly, and Teredesai, Ankur

arXiv preprint arXiv:1907.12669 2019

Abs Bib PDF

Explainable models in Artificial Intelligence are often employed to ensure transparency and accountability of AI systems. The fidelity of the explanations are dependent upon the algorithms used as well as on the fidelity of the data. Many real world datasets have missing values that can greatly influence explanation fidelity. The standard way to deal with such scenarios is imputation. This can, however, lead to situations where the imputed values may correspond to a setting which refer to counterfactuals. Acting on explanations from AI models with imputed values may lead to unsafe outcomes. In this paper, we explore different settings where AI models with imputation can be problematic and describe ways to address such scenarios.
@article{ahmad2019challenge, abbr = {Paper}, bibtex_show = {true}, title = {The Challenge of Imputation in Explainable Artificial Intelligence Models}, author = {Ahmad, Muhammad Aurangzeb and Eckert, Carly and Teredesai, Ankur}, journal = {arXiv preprint arXiv:1907.12669}, pdf = {AhmadIJCAI2019.pdf}, year = {2019}, xai = {1} }
Journal
Development and prospective validation of a machine learning-based risk of readmission model in a large military hospital

Eckert, Carly, Nieves-Robbins, Neris, Spieker, Elena, Louwers, Tom, Hazel, David, Marquardt, James, Solveson, Keith, Zahid, Anam, Ahmad, Muhammad, Barnhill, Richard, and others,

Applied clinical informatics 2019

Abs Bib HTML PDF

Thirty-day hospital readmissions are a quality metric for health care systems. Predictive models aim to identify patients likely to readmit to more effectively target preventive strategies. Many risk of readmission models have been developed on retrospective data, but prospective validation of readmission models is rare. To the best of our knowledge, none of these developed models have been evaluated or prospectively validated in a military hospital.
@article{eckert2019development, abbr = {Journal}, bibtex_show = {true}, title = {Development and prospective validation of a machine learning-based risk of readmission model in a large military hospital}, author = {Eckert, Carly and Nieves-Robbins, Neris and Spieker, Elena and Louwers, Tom and Hazel, David and Marquardt, James and Solveson, Keith and Zahid, Anam and Ahmad, Muhammad and Barnhill, Richard and others}, journal = {Applied clinical informatics}, volume = {10}, number = {2}, pages = {316}, year = {2019}, publisher = {Thieme Medical Publishers}, html = {https://www.thieme-connect.com/products/ejournals/html/10.1055/s-0039-1688553}, pdf = {EckertDevelopment2019.pdf}, providers = {pubs}, healthcare = {pubs} }

Talk

Holding Machine Learning Systems Accountable via Explanations

Ahmad, Muhammad Aurangzeb

In Invited Talk at Istanbul Technical University, Istanbul, Turkey 2019

Bib

@inproceedings{ituAhmad2021,
  title = {Holding Machine Learning Systems Accountable via Explanations},
  author = {Ahmad, Muhammad Aurangzeb},
  year = {2019},
  month = feb,
  booktitle = {Invited Talk at Istanbul Technical University, Istanbul, Turkey},
  abbr = {Talk},
  pub_type = {Talk},
  bibtex_show = {true},
  responsibleai = {pubs},
  xai = {pubs}
}

Talk

Do Algorithms Have Politics?

Ahmad, Muhammad Aurangzeb

In Invited Talk at King Abdul Aziz University Jeddah, Saudi Arabia 2019

Bib

@inproceedings{kaauAhmad2019,
  title = {Do Algorithms Have Politics?},
  author = {Ahmad, Muhammad Aurangzeb},
  year = {2019},
  month = feb,
  booktitle = {Invited Talk at King Abdul Aziz University Jeddah, Saudi Arabia},
  abbr = {Talk},
  pub_type = {Talk},
  bibtex_show = {true},
  healthcare = {pubs},
  responsibleai = {pubs},
  xai = {pubs}
}

Talk

Foundations of Machine Learning

Ahmad, Muhammad Aurangzeb

In Invited Talk at Maldives Ministry of Communication, Science and Technology Machine Learning Workshop, Male, Maldives 2019

Bib

@inproceedings{mmcstAhmad2019a,
  title = {Foundations of Machine Learning},
  author = {Ahmad, Muhammad Aurangzeb},
  year = {2019},
  month = feb,
  booktitle = {Invited Talk at Maldives Ministry of Communication, Science and Technology Machine Learning Workshop, Male, Maldives},
  abbr = {Talk},
  pub_type = {Talk},
  bibtex_show = {true},
  healthcare = {pubs},
  responsibleai = {pubs},
  xai = {pubs}
}

Talk

Accountability of Artificial Intelligence Systems

Ahmad, Muhammad Aurangzeb

In Invited Talk at Inaugural Annual Lecture of the Maldives Ministry of Communication, Science and Technology initiated Science and Technology Lecture Series, Maldives National University, Male, Maldives 2019

Bib

@inproceedings{mmcstAhmad2019b,
  title = {Accountability of Artificial Intelligence Systems},
  author = {Ahmad, Muhammad Aurangzeb},
  year = {2019},
  month = feb,
  booktitle = {Invited Talk at Inaugural Annual Lecture of the Maldives Ministry of Communication, Science and Technology initiated Science and Technology Lecture Series, Maldives National University, Male, Maldives},
  abbr = {Talk},
  pub_type = {Talk},
  bibtex_show = {true},
  healthcare = {pubs},
  responsibleai = {pubs},
  xai = {pubs}
}

Tutorial
Interpretable Machine Learning: What Clinical Informaticists Need to Know

Ahmad, Muhammad Aurangzeb, Eckert, Carly, Teredesai, Ankur, and Kumar, Vikas

In Clinical Informatics Conference, AMIA Atlanta, GA 2019

Abs Bib

Interpretable machine learning (ML) refers to machine learning models that can provide reasons or explanations for why certain patient-level predictions are made. In high-stakes domains outside of healthcare, merely providing traditional ML metrics like accuracy, precision, recall etc. are not sufficient. In healthcare, explanations for model results and patient-level risk predictions is imperative. Clinical providers and other decision makers note interpretability as a priority for implementation and use since black box machine learning models seldom engender trust. Decisions based on machine learning predictions could inform diagnoses, clinical care pathways, and patient risk stratification, among many others. It follows, that for decisions of such import, clinicians and other staff desire to know the “reasons” behind the prediction. This workshop will cover the definitions, nuances, challenges, and requirements for the design of interpretable and explainable ML models and systems in healthcare with an emphasis on the clinical application of this framework. We will discuss many uses in which interpretable machine learning models are needed in healthcare. Additionally, we explore the landscape of recent advances to address the challenges of model interpretability in healthcare and also broadly describe how to select the most appropriate interpretable ML algorithm for a given problem, given certain clinically-oriented constraints. We will engage our audience by having interactive quizzes throughout the tutorial. Based on the practice domains of our audience, we will also encourage participants to share their stories and learnings from applying ML in healthcare and how interpretability / explainability may have played a role. If the size of our audience is appropriate, we may also break the audience into groups to discuss the desiderata of explainable ML, its advantages, and potential pitfalls. We will also share additional resources and bibliography with the participants at the conclusion of the session for further learning.
@inproceedings{interpretableMLAMIA2019, title = {Interpretable Machine Learning: What Clinical Informaticists Need to Know}, author = {Ahmad, Muhammad Aurangzeb and Eckert, Carly and Teredesai, Ankur and Kumar, Vikas}, booktitle = {Clinical Informatics Conference, AMIA Atlanta, GA}, year = {2019}, month = apr, day = {30}, abbr = {Tutorial}, pub_type = {Tutorial}, bibtex_show = {true} }
Talk
Requirements and Limits of Explainable AI in Healthcare Clinical

Ahmad, Muhammad Aurangzeb

In Translational Science Institute at the George Washington University 2019

Abs Bib

As AI and machine learning are being increasingly integrated into healthcare, challenges regarding creating responsible AI systems that are interpretable, fair, transparent, unbiased, robust and reliable are coming under increasing scrutiny. In this talk I will focus on one important aspect of such systems – Explainability, explore what constitutes explainable AI in healthcare and what are the nuances, challenges, and requirements for the design of explainable AI in healthcare. Drawing on insights from the academia and experience from industry the talk will delve into why is explainability in healthcare different from other domains and how deployed systems may have counterintuitive implications e.g., explainability leading to distrust, explainable systems leading to more opaque models etc. The talk will focus on the application of explainability techniques and the practical challenges in creating effective explainable AI models in healthcare. Finally, the talk will describe open problems and research directions for the AI in healthcare community.
@inproceedings{ahmadGWU2019, title = {Requirements and Limits of Explainable AI in Healthcare Clinical}, author = {Ahmad, Muhammad Aurangzeb}, booktitle = {Translational Science Institute at the George Washington University}, year = {2019}, month = nov, abbr = {Talk}, pub_type = {Talk}, bibtex_show = {true} }

Talk

Holding Machine Learning Systems Accountable via Explanations

Ahmad, Muhammad Aurangzeb, and Eckert, Carly

In University of Washington School of Medicine’s Bioinformatics and Medical Education Seminar 2019

Bib

@inproceedings{uwTalk2019,
  title = {Holding Machine Learning Systems Accountable via Explanations},
  author = {Ahmad, Muhammad Aurangzeb and Eckert, Carly},
  booktitle = {University of Washington School of Medicine’s Bioinformatics and Medical Education Seminar},
  year = {2019},
  month = dec,
  abbr = {Talk},
  pub_type = {Talk},
  bibtex_show = {true}
}

2018

Paper
Privacy-preserving scoring of tree ensembles: A novel framework for AI in healthcare

Fritchman, Kyle, Saminathan, Keerthanaa, Dowsley, Rafael, Hughes, Tyler, De Cock, Martine, Nascimento, Anderson, and Teredesai, Ankur

In 2018 IEEE International Conference on Big Data (Big Data) 2018

Abs Bib PDF

—Machine Learning (ML) techniques now impact a wide variety of domains. Highly regulated industries such as healthcare and finance have stringent compliance and data governance policies around data sharing. Advances in secure multiparty computation (SMC) for privacy-preserving machine learning (PPML) can help transform these regulated industries by allowing ML computations over encrypted data with personally identifiable information (PII). Yet very little of SMC-based PPML has been put into practice so far. In this paper we present the very first framework for privacypreserving classification of tree ensembles with application in healthcare. We first describe the underlying cryptographic protocols that enable a healthcare organization to send encrypted data securely to a ML scoring service and obtain encrypted class labels without the scoring service actually seeing that input in the clear. We then describe the deployment challenges we solved to integrate these protocols in a cloud based scalable risk-prediction platform with multiple ML models for healthcare AI. Included are system internals, and evaluations of our deployment for supporting physicians to drive better clinical outcomes in an accurate, scalable, and provably secure manner. To the best of our knowledge, this is the first such applied framework with SMC-based privacypreserving machine learning for healthcare.
@inproceedings{fritchman2018privacy, title = {Privacy-preserving scoring of tree ensembles: A novel framework for AI in healthcare}, author = {Fritchman, Kyle and Saminathan, Keerthanaa and Dowsley, Rafael and Hughes, Tyler and De Cock, Martine and Nascimento, Anderson and Teredesai, Ankur}, booktitle = {2018 IEEE International Conference on Big Data (Big Data)}, pages = {2413--2422}, year = {2018}, organization = {IEEE}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {FritchmanPrivacyPreserving2018.pdf} }
Paper
S45 Predicting likelihood of emergency department admission prior to triage: utilising machine learning within a COPD cohort

Eckert, C, Ahmad, M, Zolfaghar, K, McKelvey, G, Carlin, C, and Lowe, D

2018

Abs Bib HTML

Acute exacerbation of COPD is one of the commonest reasons for emergency department (ED) attendance and admission. Optimising ED patient flow requires early identification of patients needing inpatient care to initiate the admissions process, improve bed management and reduce ED length of stay. Stratifying COPD patients by likelihood of admission and length of stay at triage would potentially facilitate these and other operational efficiencies, such as targeting early supported discharge team review and COPD care bundle. We propose a machine learning (ML) based approach to predicting the need for admission from ED amongst a cohort of COPD patients.
@misc{eckert2018s45, abbr = {Paper}, bibtex_show = {true}, html = {https://thorax.bmj.com/content/73/Suppl_4/A28.1}, title = {S45 Predicting likelihood of emergency department admission prior to triage: utilising machine learning within a COPD cohort}, author = {Eckert, C and Ahmad, M and Zolfaghar, K and McKelvey, G and Carlin, C and Lowe, D}, year = {2018}, publisher = {BMJ Publishing Group Ltd}, providers = {pubs}, healthcare = {pubs} }
Paper
Interpretable machine learning in healthcare

Ahmad, Muhammad Aurangzeb, Eckert, Carly, and M Teredesai, Ankur

In Proceedings of the 2018 ACM international conference on bioinformatics, computational biology, and health informatics 2018

Abs Bib PDF

The drive towards greater penetration of machine learning in healthcare is being accompanied by increased calls for machine learning and AI based systems to be regulated and held accountable in healthcare. Interpretable machine learning models can be instrumental in holding machine learning systems accountable. Healthcare offers unique challenges for machine learning where the demands for explainability, model fidelity and performance in general are much higher as compared to most other domains. In this paper we review the notion of interpretability within the context of healthcare, the various nuances associated with it, challenges related to interpretability which are unique to healthcare and the future of interpretability in healthcare.
@inproceedings{ahmad2018interpretable, abbr = {Paper}, bibtex_show = {true}, pdf = {AhmadCarlyInterpretable2018.pdf}, title = {Interpretable machine learning in healthcare}, author = {Ahmad, Muhammad Aurangzeb and Eckert, Carly and M Teredesai, Ankur}, booktitle = {Proceedings of the 2018 ACM international conference on bioinformatics, computational biology, and health informatics}, pages = {559--560}, year = {2018}, xai = {1} }
Paper
Death vs. Data Science: Predicting End of Life.

Ahmad, Muhammad A, Eckert, Carly, McKelvey, Greg, Zolfaghar, Kiyana, Zahid, Anam, and Teredesai, Ankur

In AAAI 2018

Abs Bib PDF

Death is an inevitable part of life and while it cannot be delayed indefinitely it is possible to predict with some certainty when the health of a person is going to deteriorate. In this paper, we predict risk of mortality for patients from two large hospital systems in the Pacific Northwest. Using medical claims and electronic medical records (EMR) data we greatly improve prediction for risk of mortality and explore machine learning models with explanations for end of life predictions. The insights that are derived from the predictions can then be used to improve the quality of patient care towards the end of life.
@inproceedings{ahmad2018death, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {AhmadEckertIAAI2018.pdf}, title = {Death vs. Data Science: Predicting End of Life.}, author = {Ahmad, Muhammad A and Eckert, Carly and McKelvey, Greg and Zolfaghar, Kiyana and Zahid, Anam and Teredesai, Ankur}, booktitle = {AAAI}, pages = {7719--7726}, year = {2018}, healthcare = {pubs} }
Paper
Automatic Detection of Excess Healthcare Spending and Cost Variation in ACOs

Liu, Eric, Ahmad, Muhammad Aurangzeb, Eckert, Carly, Nascimento, Anderson, De Cock, Martine, Padthe, Karthik, Teredesai, Ankur, and McKelvey, Greg

2018

Abs Bib PDF

There are more than nine hundred Accountable Care Organizations (ACOs) in the United States, both in the public and private sector, serving millions of patients across the country in a process to transition from fee-for-service to a value-based-care model for healthcare delivery in an effort to contain expenditures. Identifying fraud, waste, and abuse resulting in superfluous expenditures associated with care delivery is central to the success of ACOs and for making the cost of healthcare sustainable. In theory, such expenditures should be easily identifiable with large amounts of historical data. However, to the best of our knowledge there is no data mining framework that systematically addresses the problem of identifying unwarranted variation in expenditures on high dimensional claims data using unsupervised machine learning techniques. In this paper we propose methods to uncover unwarranted variation in healthcare spending by automatically extracting reference groups of peerproviders from the data and then detecting high cost outliers within these groups. We demonstrate the utility of our proposed framework on datasets from a large ACO in the United States to successfully identify unwarranted variation in therapeutic procedures even in low cost claims that had previously gone unnoticed.
@misc{liu2018automatic, author = {Liu, Eric and Ahmad, Muhammad Aurangzeb and Eckert, Carly and Nascimento, Anderson and De Cock, Martine and Padthe, Karthik and Teredesai, Ankur and McKelvey, Greg}, title = {Automatic Detection of Excess Healthcare Spending and Cost Variation in ACOs}, year = {2018}, publisher = {SIAM}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {LiuAhmadSIAM2018.pdf} }
Abstract
12 Predicting Patients at Risk for Leaving Without Being Seen Using Machine Learning

Casey, P, Zolfaghar, K, Eckert, C, Waters, L, Sonntag, H, McKelvey, T, and Mark, NM

Annals of Emergency Medicine 2018

Abs Bib PDF

Patients who leave without being seen (LWBS) remain a persistent challenge for emergency departments (ED). It results not only in suboptimal patient care and medicolegal risk but LWBS is also publicly reported and has revenue implications both in terms of lost encounters and as a CMS quality measure (OP-22). Patients who leave without being seen (LWBS) are a persistent problem in the emergency department. Patients who leave the emergency department without being seen by a health care provider contribute to delays and ED cost but neither benefit from health care nor provide reimbursement for the encounter. Many different strategies have been proposed to reduce LWBS including having greeters meet patient on ED arrival, physician triage/rounding in the waiting room, presenting patients with estimates of waiting time, and providing reassurances to patients as they wait. One of the greatest challenges is identifying the particular patients at highest risk for LWBS.
@article{casey201812, title = {12 Predicting Patients at Risk for Leaving Without Being Seen Using Machine Learning}, author = {Casey, P and Zolfaghar, K and Eckert, C and Waters, L and Sonntag, H and McKelvey, T and Mark, NM}, journal = {Annals of Emergency Medicine}, volume = {72}, number = {4}, pages = {S5--S6}, year = {2018}, publisher = {Elsevier}, abbr = {Abstract}, pub_type = {Abstract}, bibtex_show = {true}, pdf = {CaseyZolfagharLWBS2018.pdf}, providers = {pubs} }

Talk

Accountability in Healthcare AI

Ahmad, Muhammad Aurangzeb

In Invited Talk at Qatar Computing Research Institute Doha, Qatar 2018

Bib

@inproceedings{qcriAhmad2018,
  title = {Accountability in Healthcare AI},
  author = {Ahmad, Muhammad Aurangzeb},
  year = {2018},
  month = oct,
  booktitle = {Invited Talk at Qatar Computing Research Institute Doha, Qatar},
  abbr = {Talk},
  pub_type = {Talk},
  bibtex_show = {true},
  healthcare = {pubs},
  responsibleai = {pubs},
  xai = {pubs}
}

Tutorial
Explainable Machine Learning Models for Healthcare AI

Teredesai, Ankur, Ahmad, Muhammad Aurangzeb, Eckert, Carly, and Kumar, Vikas

In ACM Seminar 2018

Abs Bib Video

This tutorial extensively covers the definitions, nuances, challenges, and requirements for the design of interpretable and explainable machine learning models and systems in healthcare. We discuss many uses in which interpretable machine learning models are needed in healthcare and how they should be deployed. Additionally, we explore the landscape of recent advances to address the challenges model interpretability in healthcare and also describe how one would go about choosing the right interpretable machine learning algorithm for a given problem in healthcare.
@inproceedings{seminarACM2018, title = {Explainable Machine Learning Models for Healthcare AI}, author = {Teredesai, Ankur and Ahmad, Muhammad Aurangzeb and Eckert, Carly and Kumar, Vikas}, booktitle = {ACM Seminar}, year = {2018}, month = sep, day = {26}, abbr = {Tutorial}, pub_type = {Tutorial}, bibtex_show = {true}, video = {https://www.youtube.com/watch?v=4pgLsDzrlB8} }
Tutorial
Explainable Models for Healthcare AI

Ahmad, Muhammad Aurangzeb, Teredesai, Ankur, Eckert, Carly, and Kumar, Vikas

In Proceedings of the 24th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. London, United Kingdom 2018

Abs Bib

This tutorial extensively covers the definitions, nuances, challenges, and requirements for the design of interpretable and explainable machine learning models and systems in healthcare. We discuss many uses in which interpretable machine learning models are needed in healthcare and how they should be deployed. Additionally, we explore the landscape of recent advances to address the challenges model interpretability in healthcare and also describe how one would go about choosing the right interpretable machine learnig algorithm for a given problem in healthcare.
@inproceedings{xaiKDD2018, title = {Explainable Models for Healthcare AI}, author = {Ahmad, Muhammad Aurangzeb and Teredesai, Ankur and Eckert, Carly and Kumar, Vikas}, booktitle = {Proceedings of the 24th ACM SIGKDD Conference on Knowledge Discovery \& Data Mining. London, United Kingdom}, year = {2018}, month = aug, day = {19-23}, abbr = {Tutorial}, pub_type = {Tutorial}, bibtex_show = {true} }
Tutorial
Interpretable Machine Learning in Healthcare

Ahmad, Muhammad Aurangzeb, Eckert, Carly, Teredesai, Ankur, and Kumar, Vikas

In International Conference on Bioinformatics, Computational Biology, and Health Informatics Washington, DC, USA. 2018

Abs Bib

This tutorial extensively covers the definitions, nuances, challenges, and requirements for the design of interpretable and explainable machine learning models and systems in healthcare. We discuss many uses in which interpretable machine learning models are needed in healthcare and how they should be deployed. Additionally, we explore the landscape of recent advances to address the challenges model interpretability in healthcare and also describe how one would go about choosing the right interpretable machine learnig algorithm for a given problem in healthcare.
@inproceedings{interpretableMLBCB2018, title = {Interpretable Machine Learning in Healthcare}, author = {Ahmad, Muhammad Aurangzeb and Eckert, Carly and Teredesai, Ankur and Kumar, Vikas}, booktitle = { International Conference on Bioinformatics, Computational Biology, and Health Informatics Washington, DC, USA.}, year = {2018}, month = aug, day = {30}, abbr = {Tutorial}, pub_type = {Tutorial}, bibtex_show = {true} }
Tutorial
Machine Learning in Healthcare

Ahmad, Muhammad Aurangzeb, Eckert, Carly, Teredesai, Ankur, and Kumar, Vikas

In IEEE International Conference on Healthcare Informatics, New York, NY, USA 2018

Abs Bib

This tutorial extensively covers the definitions, nuances, challenges, and requirements for the design of interpretable and explainable machine learning models and systems in healthcare. We discuss many uses in which interpretable machine learning models are needed in healthcare and how they should be deployed. Additionally, we explore the landscape of recent advances to address the challenges model interpretability in healthcare and also describe how one would go about choosing the right interpretable machine learnig algorithm for a given problem in healthcare.
@inproceedings{mlInterpretableICHI2018, title = {Machine Learning in Healthcare}, author = {Ahmad, Muhammad Aurangzeb and Eckert, Carly and Teredesai, Ankur and Kumar, Vikas}, booktitle = {IEEE International Conference on Healthcare Informatics, New York, NY, USA}, year = {2018}, month = jun, day = {4-7}, abbr = {Tutorial}, pub_type = {Tutorial}, bibtex_show = {true} }

2017

Paper
Impact of a mobile health application on user engagement and pregnancy outcomes among Wyoming Medicaid members

Bush, James, Barlow, Dilek E, Echols, Jennie, Wilkerson, Jasmine, and Bellevin, Katherine

Telemedicine and e-Health 2017

Abs Bib PDF

Pregnancy and birth outcomes are a critical area of healthcare, yet negative outcomes like C-sections and preterm births remain widespread. Studies show that early and ongoing prenatal care can improve outcomes; however, in-person care is difficult to deliver in rural areas. This article examines the impact of mobile health technology on user engagement and birth outcomes in a Wyoming pilot study. The pilot did face some limitations; namely, the small app user group size and scant demographic information collected from users.
@article{bush2017impact, title = {Impact of a mobile health application on user engagement and pregnancy outcomes among Wyoming Medicaid members}, author = {Bush, James and Barlow, Dilek E and Echols, Jennie and Wilkerson, Jasmine and Bellevin, Katherine}, journal = {Telemedicine and e-Health}, volume = {23}, number = {11}, pages = {891--898}, year = {2017}, publisher = {Mary Ann Liebert, Inc. 140 Huguenot Street, 3rd Floor New Rochelle, NY 10801 USA}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, healthcare = {true}, pdf = {BushBarlowWilkersonTelemedicineeHealth2017.pdf} }

Talk

How Will Artificial Intelligence Transform Healthcare?

Ahmad, Muhammad Aurangzeb

In Invited Talk at Techstars Startup Week Seattle Seattle, WA. 2017

Bib

@inproceedings{stsAhmad2017,
  title = {How Will Artificial Intelligence Transform Healthcare?},
  author = {Ahmad, Muhammad Aurangzeb},
  year = {2017},
  month = oct,
  booktitle = {Invited Talk at Techstars Startup Week Seattle Seattle, WA.},
  abbr = {Talk},
  pub_type = {Talk},
  bibtex_show = {true},
  healthcare = {pubs},
  responsibleai = {pubs},
  xai = {pubs}
}

2016

Paper

Machine learning models for surgical site infection prediction

Mandagani, Prathyusha, Coleman, Shaun, Zahid, Anam, Ehlers, A Pugel, Roy, S Basu, and De Cock, Martine

In AMIA KDDM-WG Symposium (American medical Informatics Association Knowledge Discovery and Data Mining Working Group) 2016

Abs PDF

Surgical Site Infections (SSIs) are estimated to represent 17% of all hospital-acquired infections. Their impact on morbidity, mortality, and cost of care calls for techniques to risk stratify patients ahead of surgery. In this paper we train and evaluate machine learning models (logistic regression, decision trees, and boosted decision trees) that can flag – ahead of surgery – those patients who are likely to develop a SSI. In addition to the gender and the age of the patient at the time of surgery, our models rely only on available results from blood tests done prior to the surgery. We obtain the best results with boosted decision trees (AdaBoost), i.e. an AUC-score of 86% on a benchmark patient dataset of the University Hospital of North Norway. The outcomes of Glukose and CRP tests stand out in particular as indicative features for predicting a future SSI.
Paper
Sequence Based Prediction of Hospital Readmissions

Agrawal, Surabhi, Hon, Chun Pan, Garg, Swati, Sampath, Aadarsh, Sushmita, Shanu, and De Cock, Martine

In Proceedings of the 7th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics 2016

Abs Bib PDF

In this contribution we explore the use of an N-gram based approach to make predictions about the next event in a sequence of hospital admissions. To predict the value of a variable in the next admission (e.g. length of stay), we use only the sequence of values of the same variable for all hospital admissions of the same patient so far (e.g. the length of all previous hospital stays) and no other information from the discharge record. We validate our method on inpatient data from the California Office of Statewide Health Planning and Development (OSHPD), for the prediction of three kinds of variables: whether the patient will be readmitted within 30 days, and what the associated length of stay and cost of the next hospital admission will be. Our preliminary results show that in all cases simple 4-gram, 5-gram and 6-gram methods make more accurate predictions than the majority baseline method, by a margin that depends on the problem at hand as well as on the cohort, i.e. the patient population as a whole (“ALL-Cause”) versus a congestive heart failure (CHF) cohort
@inproceedings{agrawal2016sequence, title = {Sequence Based Prediction of Hospital Readmissions}, author = {Agrawal, Surabhi and Hon, Chun Pan and Garg, Swati and Sampath, Aadarsh and Sushmita, Shanu and De Cock, Martine}, booktitle = {Proceedings of the 7th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics}, pages = {489--490}, year = {2016}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {AgrawalShanuSequenceBased2016.pdf}, provider = {pubs} }
Paper
Predicting 30-day risk and cost of" All-Cause" hospital readmissions

Sushmita, Shanu, Khulbe, Garima, Hasan, Aftab, Newman, Stacey, Ravindra, Padmashree, Roy, Senjuti Basu, De Cock, Martine, and Teredesai, Ankur

In Workshops at the thirtieth AAAI conference on artificial intelligence 2016

Abs Bib PDF

The hospital readmission rate of patients within 30 days after discharge is broadly accepted as a healthcare quality measure and cost driver in the United States. The ability to estimate hospitalization costs alongside 30 day risk-stratification for such readmissions provides additional benefit for accountable care, now a global issue and foundation for the U.S. government mandate under the Affordable Care Act. Recent data mining efforts either predict healthcare costs or risk of hospital readmission, but not both. In this paper we present a dual predictive modeling effort that utilizes healthcare data to predict the risk and cost of any hospital readmission (“all-cause”). For this purpose, we explore machine learning algorithms to do accurate predictions of healthcare costs and risk of 30-day readmission.Results on risk prediction for “all-cause” readmission compared to the standardized readmission tool (LACE) are promising, and the proposed techniques for cost prediction consistently outperform baseline models and demonstrate substantially lower mean absolute error (MAE).
@inproceedings{sushmita2016predicting, title = {Predicting 30-day risk and cost of" All-Cause" hospital readmissions}, author = {Sushmita, Shanu and Khulbe, Garima and Hasan, Aftab and Newman, Stacey and Ravindra, Padmashree and Roy, Senjuti Basu and De Cock, Martine and Teredesai, Ankur}, booktitle = {Workshops at the thirtieth AAAI conference on artificial intelligence}, year = {2016}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {SushmitaRoRAAAI2016.pdf} }
Paper
Predicting future frequent users of emergency departments in California state

Pereira, Mayana, Singh, Vikhyati, Hon, Chun Pan, McKelvey, T Greg, Sushmita, Shanu, and De Cock, Martine

In Proceedings of the 7th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics 2016

Abs Bib PDF

A large percentage of emergency department (ED) visits originates from a small percentage of patients who keep returning to the ED. Being able to flag these frequent users in advance can help clinicians to take appropriate interventions to reduce the number of ED visits, thereby reducing cost and improving quality of care. In this paper we present machine learning models that can predict future ED utilization of individual patients, using only information from the present and the past. We train decision trees (DT), boosted decision trees (AdaBoost) and logistic regression (LR) models on discharge records from California-licensed hospitals from the years 2009 and 2010, and evaluate their predictive accuracy for the years 2011-2013. We also study the impact of including di↵erent groups of demographic, frequency of ED visits, distance to emergency department, and clinical features on the accuracy of our predictive models. Overall there are three key findings of this study. First, all three techniques (LR, DT and AdaBoost) have a strong predictive ability to discriminate frequent ED users (number of visits 5). Second, our models show consistent outcomes across all three test years in our dataset, which is a desired property when a predictive tool that is stable and consistent year over year is required. Third, least and most frequent ED users are comparatively easier to predict when compared to moderate ED users (with higher sensitivity and AUC scores).
@inproceedings{pereira2016predicting, title = {Predicting future frequent users of emergency departments in California state}, author = {Pereira, Mayana and Singh, Vikhyati and Hon, Chun Pan and McKelvey, T Greg and Sushmita, Shanu and De Cock, Martine}, booktitle = {Proceedings of the 7th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics}, pages = {603--610}, year = {2016}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {PereiraEDFrequentUse2016.pdf}, providers = {pubs} }
Paper
Risk stratification for hospital readmission of heart failure patients: A machine learning approach

Hon, Chun Pan, Pereira, Mayana, Sushmita, Shanu, Teredesai, Ankur, and De Cock, Martine

In Proceedings of the 7th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics 2016

Abs Bib PDF

Being able to stratify patients according to 30-day hospital readmission risk, anticipated length and cost of stay can guide clinicians in discharge planning and intervention recommendation, leading to an increase of quality of care, and a decrease of healthcare cost. We present a comparative performance analysis of decision trees, boosted decision trees and logistic regression models that can flag, at the time of discharge, patients with an anticipated early, lengthy and expensive readmission. We validate our models using discharge records of 500K congestive heart failure patients from California-licensed hospitals.
@inproceedings{hon2016risk, title = {Risk stratification for hospital readmission of heart failure patients: A machine learning approach}, author = {Hon, Chun Pan and Pereira, Mayana and Sushmita, Shanu and Teredesai, Ankur and De Cock, Martine}, booktitle = {Proceedings of the 7th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics}, pages = {491--492}, year = {2016}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, providers = {pubs}, pdf = {HonRoRHFBCB2016.pdf} }

2014

Paper

Prediction and Management of Readmission Risk for Congestive Heart Failure.

Roy, Senjuti Basu, and Chin, Si-Chi

In HEALTHINF 2014

Abs Bib PDF

@inproceedings{roy2014prediction,
  title = {Prediction and Management of Readmission Risk for Congestive Heart Failure.},
  author = {Roy, Senjuti Basu and Chin, Si-Chi},
  booktitle = {HEALTHINF},
  pages = {523--528},
  year = {2014},
  abbr = {Paper},
  pub_type = {Paper},
  bibtex_show = {true},
  pdf = {RoyChinCHFRoR2014.pdf}
}

Paper
A framework to recommend interventions for 30-day heart failure readmission risk

Liu, Rui, Zolfaghar, Kiyana, Chin, Si-chi, Roy, Senjuti Basu, and Teredesai, Ankur

In 2014 IEEE International Conference on Data Mining 2014

Abs Bib PDF

In this paper, we describe a novel framework to recommend personalized intervention strategies to minimize 30- day readmission risk for heart failure (HF) patients, as they move through the provider’s cardiac care protocol. We design principled solutions by learning the structure and parameters of a multi-layer hierarchical Bayesian network from underlying high-dimensional patient data. Next, we generate and summarize the rules leading to personalized interventions which can be applied to individual patients as they progress from admit to discharge. We present comprehensive experimental results as well as interesting case studies to demonstrate the effectiveness of our proposed framework using large real-world patient datasets on Microsoft Azure for Research platform.
@inproceedings{liu2014framework, title = {A framework to recommend interventions for 30-day heart failure readmission risk}, author = {Liu, Rui and Zolfaghar, Kiyana and Chin, Si-chi and Roy, Senjuti Basu and Teredesai, Ankur}, booktitle = {2014 IEEE International Conference on Data Mining}, pages = {911--916}, year = {2014}, organization = {IEEE}, pdf = {LiuKiyanaAnkurICDM2014.pdf}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, payer = {pubs} }
Paper
Divide-n-Discover: Discretization based data exploration framework for healthcare analytics

Chin, Si Chi, Zolfaghar, Kiyana, Roy, Senjuti Basu, Teredesai, Ankur, and Amoroso, Paul

In 7th International Conference on Health Informatics, HEALTHINF 2014-Part of 7th International Joint Conference on Biomedical Engineering Systems and Technologies, BIOSTEC 2014 2014

Abs Bib PDF

Insightful and principled visualization techniques may successfully help complex clinical data exploration tasks and aid in the process of knowledge discovery. In this paper, we propose a framework Divide-n-Discover to visualize and explore clinical data effectively, and demonstrate its effectiveness in predicting readmission risk for Congestive Heart Failure patients. Our proposed method provides clinicians a mechanism to dynamically explore the data and to understand how a single factor may influence the risk of readmission for a given patient. For example, our study indicates that patients between age 47 and 48 have 2.63 time higher chance of getting readmitted to the hospital within 30 days, compared to other patients; likewise, patients with length of stay above 1 ays are 2.27 times more likely to be readmitted within 30 days. The finding suggests that hospitals might be under pressure to discharge patients within two week while some patients may benefit from a longer stay. These observations may become valid hypotheses leading to further clinical investigation or discoveries. To the best of our knowledge, this is the first ever work that proposes principled discretization and visualization techniques in the hospital readmission risk prediction problem.
@inproceedings{chin2014divide, title = {Divide-n-Discover: Discretization based data exploration framework for healthcare analytics}, author = {Chin, Si Chi and Zolfaghar, Kiyana and Roy, Senjuti Basu and Teredesai, Ankur and Amoroso, Paul}, booktitle = {7th International Conference on Health Informatics, HEALTHINF 2014-Part of 7th International Joint Conference on Biomedical Engineering Systems and Technologies, BIOSTEC 2014}, pages = {329--333}, year = {2014}, organization = {SciTePress}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, providers = {pubs}, pdf = {ChinZolfagharHealthInf2014.pdf} }
Paper
Work in progress-in-memory analysis for healthcare big data

Mian, Muaz, Teredesai, Ankur, Hazel, David, Pokuri, Sreenivasulu, and Uppala, Krishna

In 2014 IEEE International Congress on Big Data 2014

Abs Bib PDF

Advances in healthcare data management and analytics have opened new horizons for healthcare providers such as cost effective treatments, ability to detect medical fraud, and diagnose diseases at an early stage. Central to these abilities is the need for fast ad-hoc query processing of large volumes of complex healthcare datasets. End users who work with healthcare databases spend enormous effort in data exploration since exploration is the first step to any subsequent predictive modeling to generate actionable insights for patients, providers and physicians. Unfortunately, unlike other domains the complexity and volumes of claims (ICD9 or 10) as well as clinical (HL7) healthcare datasets results in data exploration solutions being extremely slow and cumbersome when attempted using traditional disk resident data warehousing approaches. In this paper we describe the first ever attempt of real-time data exploration for healthcare datasets using in-memory databases. We benchmark and compare two such in-memory database systems to study responsiveness and ability to handle complexity of typical health data exploration tasks. We share our work in progress results and outline key issues that need to be addressed for forthcoming advances in this very important big data vertical.
@inproceedings{mian2014work, title = {Work in progress-in-memory analysis for healthcare big data}, author = {Mian, Muaz and Teredesai, Ankur and Hazel, David and Pokuri, Sreenivasulu and Uppala, Krishna}, booktitle = {2014 IEEE International Congress on Big Data}, pages = {778--779}, year = {2014}, organization = {IEEE}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, providers = {pubs}, pdf = {MianTeredesaiHazelICBD2014.pdf} }
Paper
Pathway-finder: An interactive recommender system for supporting personalized care pathways

Liu, Rui, Srinivasan, Raj Velamur, Zolfaghar, Kiyana, Chin, Si-Chi, Roy, Senjuti Basu, Hasan, Aftab, and Hazel, David

In 2014 IEEE International Conference on Data Mining Workshop 2014

Abs Bib PDF

Clinical pathways define the essential component of the complex care process, with the objective to optimize patient outcomes and resource allocation. Clinical pathway analysis has gained increased attention in order to augment the patient treatment process. In this demonstration paper, we propose Pathway-Finder, an interactive recommender system to visually explore and discover clinical pathways. The interactive web service efficiently collects and displays patient information in a meaningful way to support an effective personalized treatment plan. Pathway-Finder implements a Bayesian Network to discover causal relationships among different factors. To support real-time recommendation and visualization, a key-value structure has been implemented to traverse the Bayesian Network faster. Additionally, Pathway-Finder is a cloud based web service hosted on Microsoft Azure which enables the health providers to access the system without the need to deploy analytics infrastructure. We demonstrate Pathway-Finder to interactively recommend personalized interventions to minimize 30-day readmission risk for Heart Failure (HF).
@inproceedings{liu2014pathway, title = {Pathway-finder: An interactive recommender system for supporting personalized care pathways}, author = {Liu, Rui and Srinivasan, Raj Velamur and Zolfaghar, Kiyana and Chin, Si-Chi and Roy, Senjuti Basu and Hasan, Aftab and Hazel, David}, booktitle = {2014 IEEE International Conference on Data Mining Workshop}, pages = {1219--1222}, year = {2014}, organization = {IEEE}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, providers = {true}, pdf = {LiuSrinivasanAnkurICDM2014.pdf} }
Paper
Healthscope: An interactive distributed data mining framework for scalable prediction of healthcare costs

Marquardt, Ames, Newman, Stacey, Hattarki, Deepa, Srinivasan, Rajagopalan, Sushmita, Shanu, Ram, Prabhu, Prasad, Viren, Hazel, David, Ramesh, Archana, De Cock, Martine, and others,

In 2014 IEEE International Conference on Data Mining Workshop 2014

Abs Bib PDF

In this demonstration proposal we describe HealthSCOPE (Healthcare Scalable COst Prediction Engine), a framework for exploring historical and present day healthcare costs as well as for predicting future costs. HealthSCOPE can be used by individuals to estimate their healthcare costs in the coming year. In addition, HealthSCOPE supports a population based view for actuaries and insurers who want to estimate the future costs of a population based on historical claims data, a typical scenario for accountable care organizations (ACOs). Using our interactive data mining framework, users can view claims (sample files will be provided), use HealthSCOPE to predict costs for the upcoming year, interactively select from a set of possible medical conditions, understand the factors that contribute to the cost, and compare costs against historical averages. The back-end system contains cloud based prediction services hosted on the Microsoft Azure infrastructure that allow the easy deployment of models encoded in Predictive Model Markup Language (PMML) and trained using either Spark MLLib or various non-distributed environments.
@inproceedings{marquardt2014healthscope, title = {Healthscope: An interactive distributed data mining framework for scalable prediction of healthcare costs}, author = {Marquardt, Ames and Newman, Stacey and Hattarki, Deepa and Srinivasan, Rajagopalan and Sushmita, Shanu and Ram, Prabhu and Prasad, Viren and Hazel, David and Ramesh, Archana and De Cock, Martine and others}, booktitle = {2014 IEEE International Conference on Data Mining Workshop}, pages = {1227--1230}, year = {2014}, organization = {IEEE}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, payers = {pubs}, pdf = {MarquardtHealthscopeICDM2014.pdf} }

2013

Paper
Risk-o-meter: an intelligent clinical risk calculator

Zolfaghar, Kiyana, Agarwal, Jayshree, Sistla, Deepthi, Chin, Si-Chi, Basu Roy, Senjuti, and Verbiest, Nele

In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining 2013

Abs Bib PDF

We present a system called Risk-O-Meter to predict and analyze clinical risk via data imputation, visualization, predictive modeling, and association rule exploration. Clinical risk calculators provide information about a person’s chance of having a disease or encountering a clinical event. Such tools could be highly useful to educate patients to understand and monitor their health conditions. Unlike existing risk calculators that are primarily designed for domain experts, RiskO-Meter is useful to patients who are unfamiliar with medical terminologies, or providers who have limited information about a patient. Risk-O-Meter is designed in a way such that it is flexible enough to accept limited or incomplete data inputs, and still manages to predict the clinical risk efficiently and effectively. Current version of Risk-O-Meter evaluates 30-day risk of hospital readmission. However, the proposed system framework is applicable to general clinical risk predictions. In this demonstration paper, we describe different components of Risk-O-Meter and the intelligent algorithms associated with each of these components to evaluate risk of readmission using incomplete patient data inputs.
@inproceedings{zolfaghar2013risk, title = {Risk-o-meter: an intelligent clinical risk calculator}, author = {Zolfaghar, Kiyana and Agarwal, Jayshree and Sistla, Deepthi and Chin, Si-Chi and Basu Roy, Senjuti and Verbiest, Nele}, booktitle = {Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining}, pages = {1518--1521}, year = {2013}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {ZolfagharRiskometerKDD2013.pdf} }
Paper
Predicting risk-of-readmission for congestive heart failure patients: A multi-layer approach

Zolfaghar, Kiyana, Verbiest, Nele, Agarwal, Jayshree, Meadem, Naren, Chin, Si-Chi, Roy, Senjuti Basu, Teredesai, Ankur, Hazel, David, Amoroso, Paul, and Reed, Lester

In CoRR 2013

Abs Bib PDF

Mitigating risk-of-readmission of Congestive Heart Failure (CHF) patients within 30 days of discharge is important because such readmissions are not only expensive but also critical indicator of provider care and quality of treatment. Accurately predicting the risk-of-readmission may allow hospitals to identify high-risk patients and eventually improve quality of care by identifying factors that contribute to such readmissions in many scenarios. In this paper, we investigate the problem of predicting risk-of-readmission as a supervised learning problem, using a multi-layer classification approach. Earlier contributions inadequately attempted to assess a risk value for 30 day readmission by building a direct predictive model as opposed to our approach. We first split the problem into various stages, (a) at risk in general (b) risk within 60 days (c) risk within 30 days, and then build suitable classifiers for each stage, thereby increasing the ability to accurately predict the risk using multiple layers of decision. The advantage of our approach is that we can use different classification models for the subtasks that are more suited for the respective problems. Moreover, each of the subtasks can be solved using different features and training data leading to a highly confident diagnosis or risk compared to a one-shot single layer approach. An experimental evaluation on actual hospital patient record data from Multicare Health Systems shows that our model is significantly better at predicting risk-of-readmission of CHF patients within 30 days after discharge compared to prior attempts.
@inproceedings{zolfaghar2013predicting, title = {Predicting risk-of-readmission for congestive heart failure patients: A multi-layer approach}, author = {Zolfaghar, Kiyana and Verbiest, Nele and Agarwal, Jayshree and Meadem, Naren and Chin, Si-Chi and Roy, Senjuti Basu and Teredesai, Ankur and Hazel, David and Amoroso, Paul and Reed, Lester}, year = {2013}, booktitle = {CoRR}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {ZolfagharRoRCHFCorr2013.pdf}, providers = {pubs} }
Paper
Big data solutions for predicting risk-of-readmission for congestive heart failure patients

Zolfaghar, Kiyana, Meadem, Naren, Teredesai, Ankur, Roy, Senjuti Basu, Chin, Si-Chi, and Muckian, Brian

In 2013 IEEE International Conference on Big Data 2013

Abs Bib PDF

Developing holistic predictive modeling solutions for risk prediction is extremely challenging in healthcare informatics. Risk prediction involves integration of clinical factors with socio-demographic factors, health conditions, disease parameters, hospital care quality parameters, and a variety of variables specific to each health care provider making the task increasingly complex. Unsurprisingly, many of such factors need to be extracted independently from different sources, and integrated back to improve the quality of predictive modeling. Such sources are typically voluminous, diverse, and vary significantly over the time. Therefore, distributed and parallel computing tools collectively termed big data have to be developed. In this work, we study big data driven solutions to predict the 30-day risk of readmission for congestive heart failure (CHF) incidents. First, we extract useful factors from National Inpatient Dataset (NIS) and augment it with our patient dataset from Multicare Health System (MHS). Then, we develop scalable data mining models to predict risk of readmission using the integrated dataset. We demonstrate the effectiveness and efficiency of the open-source predictive modeling framework we used, describe the results from various modeling algorithms we tested, and compare the performance against baseline non-distributed, non-parallel, non-integrated small data results previously published to demonstrate comparable accuracy over millions of records.
@inproceedings{zolfaghar2013big, title = {Big data solutions for predicting risk-of-readmission for congestive heart failure patients}, author = {Zolfaghar, Kiyana and Meadem, Naren and Teredesai, Ankur and Roy, Senjuti Basu and Chin, Si-Chi and Muckian, Brian}, booktitle = {2013 IEEE International Conference on Big Data}, pages = {64--71}, year = {2013}, organization = {IEEE}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, providers = {pubs}, pdf = {ZolfagharRoRCHFICBD2013.pdf} }
Paper
Exploring preprocessing techniques for prediction of risk of readmission for congestive heart failure patients

Meadem, Naren, Verbiest, Nele, Zolfaghar, Kiyana, Agarwal, Jayshree, Chin, Si-Chi, and Roy, Senjuti Basu

In 2013

Abs Bib PDF

Congestive Heart Failure (CHF) is one of the leading causes of hospitalization, and studies show that many of these admissions are readmissions within a short window of time. Identifying CHF patients who are at a greater risk of hospitalization can guide the implementation of appropriate plans to prevent these readmissions. Developing predictive modeling solutions for such disease related risk of readmissions is extremely challenging in healthcare informatics. It involves integration of socio-demographic factors, health conditions, disease parameters, hospital care quality parameters, and a variety of variables specific to health care providers making the task immensely complex. This work, in collaboration with experts from Multicare Health Systems (MHS), describes a soon to be deployed prototype to predict risk of readmission within 30 days of discharge for CHF patients at MHS. We focus on data extraction and data preprocessing steps to improve prediction outcomes, including feature selection, missing value imputation and data balancing. We perform comprehensive empirical evaluations using the realworld health care data set provided by MHS. Our empirical evaluation demonstrates that we outperform one of the nearest competing previous results
@inproceedings{meadem2013exploring, title = {Exploring preprocessing techniques for prediction of risk of readmission for congestive heart failure patients}, author = {Meadem, Naren and Verbiest, Nele and Zolfaghar, Kiyana and Agarwal, Jayshree and Chin, Si-Chi and Roy, Senjuti Basu}, year = {2013}, organization = {Citeseer}, abbr = {Paper}, pub_type = {Paper}, bibtex_show = {true}, pdf = {MeademZolfaghar2013.pdf} }