A Common Measure of Communication for Speech Brain-Computer Interfaces

Imagine you're working on a device that helps people with paralysis communicate by translating their brain activity into speech. The challenge is that different systems use various datasets and methods, making it hard to compare their effectiveness. For instance, one system might report high accuracy, but that could be misleading if it only recognizes a limited vocabulary. This situation is what's called a measurement problem, where the reported scores don't truly reflect a user's ability to communicate their intended speech. To address this, the authors propose a new metric called open-vocabulary mutual information (OVMI). This metric measures how much information a speech BCI can convey based on a reference distribution of words that a user might want to communicate. By using OVMI, systems can be evaluated on a common scale, allowing for fair comparisons even when they operate under different conditions. The authors found that optimizing vocabulary based on OVMI can lead to significant improvements in accuracy, showing that this approach not only clarifies how systems perform but also guides better vocabulary design for future developments. Overall, this work provides a structured way to assess and compare speech BCIs, which could lead to more effective communication aids for those in need.

The introduction of open-vocabulary mutual information provides a new framework for evaluating speech BCIs.

The method is backed by comparisons across existing systems, though it may still rely on specific datasets.