High-Performance Computing Framework Based on Distributed Systems for Large-Scale Neurophysiological Data
حوزههای تخصصی:
Recent advancements in neurophysiological recording technologies have led to significant complexities in managing large-scale neural data, creating potential bottlenecks in the storage, sharing, and processing within the neuroscience community. To address these challenges, we developed the Big Neuronal Data Framework (BNDF), a distributed high-performance computing (HPC) solution. BNDF leverages open-source big data frameworks, Hadoop and Spark, to offer a flexible and scalable architecture. We tested BNDF on three large-scale electrophysiological datasets from nonhuman primate brains, demonstrating improved runtimes and scalability due to its distributed design. In comparative analyses against MATLAB, a widely used platform, BNDF showcased over five times faster performance in spike sorting, a common task in neuroscience. This significant speed advantage highlights BNDF’s potential to enhance the efficiency of neural data processing and analysis, making it a valuable tool for researchers navigating the complexities of modern neural datasets. Overall, BNDF represents a promising approach to streamline the handling of extensive neural data in the field of neuroscience.