Abstract

Image processing tasks in Computer vision such as segmentation, object detection, and classification are foundational in geosciences, driving advancements by enabling precise analysis and interpretation of Earth’s dynamic systems and landscapes. The processing of high-resolution aerial images for object detection presents significant challenges, including the need for high detection accuracy and the ability to handle vast datasets effectively. Traditional methods often struggle with the scale and complexity of such tasks, necessitating innovations that can leverage distributed computing to meet these demands. This study introduces a groundbreaking framework that integrates Vision Transformers, a cutting-edge architecture for object detection, with PySpark’s distributed computing capabilities. This inference model significantly enhances batch inference processing efficiency of voluminous datasets, enabling the analysis of high-resolution aerial imagery with notable accuracy. By utilizing Resilient Distributed Datasets (RDDs), the research offers a detailed algorithmic analysis that reveals the computational advantages of this PySpark-based approach. The proposed Vision Transformer-PySpark framework is evaluated on the DOTA benchmark dataset for aerial images, demonstrating its scalability and superior performance as the amount of computing nodes rise, achieving improved scalability. The comparison of this framework against cutting edge object detection models underscores it’s effectiveness and scalability, setting a new standard for efficient, large-scale aerial image analysis in distributed computing environments.