Takefuji Yoshiyasu

This paper outlines key machine learning principles, focusing on the use of XGBoost and SHAP values to assist researchers in avoiding analytical pitfalls. XGBoost builds models by incrementally adding decision trees, each addressing the errors of the previous one, which can result in inflated feature importance scores due to the method's emphasis on misclassified examples. While SHAP values provide a theoretically robust way to interpret predictions, their dependence on model structure and feature interactions can introduce biases. The lack of ground truth values complicates model evaluation, as biased feature importance can obscure real relationships with target variables. Ground truth values, representing the actual labels used in model training and validation, are crucial for improving predictive accuracy, serving as benchmarks for comparing model outcomes to true results. However, they do not ensure real associations between features and targets. Instead, they help gauge the model's effectiveness in achieving high accuracy. This paper underscores the necessity for researchers to recognize biases in feature importance and model evaluation, advocating for the use of rigorous statistical methods to enhance the reliability of analyses in machine learning research.

Gomez-Flores et al. proposed a Long Short-Term Memory Neural Network (LSTM-NN) for predicting the flotation behavior of battery active materials using various physicochemical and hydrodynamic variables. While they achieved high prediction accuracy, validated through Mean Relative Error (MRE) and Mean Squared Error (MSE) metrics, concerns arise regarding the integrity of feature importance assessments derived from SAGE and SHAP methodologies. Specifically, the reliance on these model-specific techniques can introduce biases, obscuring the true relationships between features. Additionally, while Spearman's correlation elucidated significant relationships among variables, the absence of discussion on p-values left gaps in interpretation. This study emphasizes the need for cautious interpretation of feature importance metrics and the elimination of less significant variables, aiming to enhance model robustness and improve actionable insights in machine learning contexts.