ScaFE: Data-Efficient Scar Classification with LLM-Generated Clinical Feature Programs

Imagine you're a healthcare provider trying to classify different types of scars from clinical photographs. The challenge lies in accurately distinguishing between keloids and hypertrophic scars, especially when you have limited expert-labeled data and varying image quality from different hospitals. Current methods often rely on complex end-to-end image models that require large amounts of data, which can be a problem when data governance restricts sharing sensitive patient information. This is what's called a data dependency issue, and it can lead to inconsistent and hard-to-audit decisions when using hosted vision-language models (VLMs).

To address these challenges, the authors propose a method called ScaFE, which stands for Scar Feature Engineering. Instead of directly asking a model to diagnose images, ScaFE uses a large language model (LLM) to extract clinical knowledge and generate deterministic feature programs that measure visible scar attributes. These programs run in a secure local environment, ensuring that patient data remains private while still allowing for the extraction of useful insights. The results are promising: ScaFE achieved an 81.0% accuracy on a diverse dataset, outperforming existing methods by a significant margin. This approach not only enhances classification accuracy with less data but also provides a more transparent and auditable process for medical image analysis, which is crucial in healthcare settings.

The approach of using LLMs to create executable feature programs for medical image classification is a meaningful extension of existing methods.

The evaluation on diverse hospital data with clear metrics supports the reliability of the results.