Glossary of Key Terms

*Artificial Intelligence (AI)*	A computer system that can simulate or perform human tasks.
*Machine Learning (ML)*	A discipline within artificial intelligence (AI) dedicated to the development of algorithms that improve their performance on a human task without requiring explicit instructions.
*ML Algorithm/Model*	A learned representation of the patterns inherent with the input data that can be used to generate predictions.
*Pipeline*	A full sequence of steps to convert input data into output predictions. Typically involving loading, reformatting, and transforming data and predictions so that they can be integrating into real-time workflows.
*Development Operations (DevOps)*	A philosophical framework combining best practices in information technology operations and software engineering to rapidly and robustly build and implement high-quality informatics solutions.
*Machine Learning Operations (MLOps)*	A framework of technical best practices for deploying and maintaining machine learning applications efficiently and effectively.
*Target/Ground-Truth*	The “gold-standard” definition to which ML pipeline predictions will be compared.
*Discriminative Performance*	The degree to which predictions match the ground-truth labels, often measured by sensitivity, specificity, positive predictive value, and negative predictive value.
*Implementation Efficacy*	The degree to which the final implementation of the ML pipeline satisfies the original need for which it was built.
*Receiver Operating Characteristic (ROC Curve)*	A visualization of the trade-off between sensitivity and specificity across the full breadth of possible decision thresholds for a continuous output.
*Class Imbalance*	The degree to which the proportion of class labels are skewed towards one label or the other.
*Precision and Recall Curve*	A visualization of the trade-off between precision (positive predictive value) and recall (sensitivity) across the full breadth of possible decision thresholds for a continuous output.
*F₁ Score*	The harmonic mean of precision and recall. See more here.
*Matthews Correlation Coefficient*	The Pearson correlation coefficient for two binary variables. See more here.
*Cost-Sensitive Learning*	An learning approach where each classification error is assigned its own weight to fine-tune the model’s predilection towards certain error type¹ ².
*Equivocal Zone*	An interval of continuous output in which no binary class label is assigned.
*Applicability Assessment*	The determination of whether a new input is similar enough to a model’s training data for a reliable prediction.
*Demographic Parity*	A fairness criterion used to assess whether the outputs of a predictive model is independent of demographic groups (e.g. race, gender, or age).
*Predictive Parity*	A fairness criterion used to assess whether the outputs of a model have equal positive and negative predictive values across demographic groups (e.g. race, gender, or age).
*Equalized Odds*	A fairness criterion used to assess whether the outputs of a model have equal true positive rates (sensitivity) and false positive rates (specificity) across demographic groups (e.g. race, gender, or age).
*Global Explainability*	The ability to estimate each feature’s impact on model outputs aggregated across an entire data set or feature space.
*Local Explainability*	The ability to estimate each feature’s impact on model outputs for any given individual prediction.
*Governance*	The processes by which organizational responsibilities and decisions are divided, evaluated, and executed.
*Deployment*	Making a model or application accessible to other computers within a network.
*Production Environment*	The software systems and infrastructure in which live applications are hosted and run for day-to-day operations.
*Development Environment*	An isolated copy of the production environment where software changes can be tested without risk of impacting live operations.
*Application-Programming Interface (API)*	A protocol or framework by which various software applications can communicate with each other and exchange data or predictions.
*Human-in-the-Loop*	An implementation paradigm where model outputs are directed towards an expert user for incorporating into their decision-making before an action is taken.
*Data Drift*	Divergence in input data away from the initial model training data set.
*Concept Drift*	Divergence away from the training data in the target labels or context in which predictions are to be made.
*Continuous Integration (CI)*	A DevOps principle in which changes to software are incorporate in small, manageable chunks continuously rather than large overhauls.
*Continuous Deployment (CD)*	A DevOps principle in which updates to software a pushed to live environments without large periods of maintenance or down-time.

References

Ling CX, Sheng VS. Cost-Sensitive Learning [Internet]. In: Sammut C, Webb GI, editors. Boston, MA: Springer US; 2011. p. 231–5.Available from: https://link.springer.com/10.1007/978-0-387-30164-8_181

Mienye ID, Sun Y. Performance analysis of cost-sensitive learning methods with application to imbalanced medical data. Informatics in Medicine Unlocked [Internet] 2021;25:100690. Available from: https://linkinghub.elsevier.com/retrieve/pii/S235291482100174X

Glossary of Key Terms

Artificial Intelligence (AI)

Machine Learning (ML)

ML Algorithm/Model

Pipeline

Development Operations (DevOps)

Machine Learning Operations (MLOps)

Target/Ground-Truth

Discriminative Performance

Implementation Efficacy

Receiver Operating Characteristic (ROC Curve)

Class Imbalance

Precision and Recall Curve

F1 Score

Matthews Correlation Coefficient

Cost-Sensitive Learning

Equivocal Zone

Applicability Assessment

Demographic Parity

Predictive Parity

Equalized Odds

Global Explainability

Local Explainability

Governance

Deployment

Production Environment

Development Environment

Application-Programming Interface (API)

Human-in-the-Loop

Data Drift

Concept Drift

Continuous Integration (CI)

Continuous Deployment (CD)

References

F₁ Score