OpenEarthAgent links AI reasoning with executable geospatial tools

A team affiliated with Mohamed bin Zayed University of Artificial Intelligence, IBM Research, Linköping University, Technical University Munich and Australian National University has developed OpenEarthAgent. The framework breaks complex geospatial questions into steps, calls satellite-imagery and GIS tools, and uses each result to decide what to do next.

Rather than offering a one-shot description of an image, the system connects natural-language planning with executable analysis. MBZUAI says OpenEarthAgent uses Qwen3-4B-Instruct-2507 as its base model and can return text, maps or other images. The paper appears in the ECCV 2026 proceedings.

OpenEarthAgent brings perceptual, GIS, spectral and GeoTIFF operations into one structured tool registry. A central orchestrator checks a tool’s arguments, runs the operation, caches intermediate outputs and adds the result to working memory. The model reasons from that observation before choosing another tool or returning an answer. The loop supports inputs including RGB imagery, synthetic-aperture radar, GIS layers and multispectral data.

Available operations include object detection and segmentation, distance and area calculations, zonal statistics, and indices such as NDVI for vegetation, NBR for burn severity and NDBI for built-up areas. A question can therefore become a sequence of steps: locate objects in imagery, measure their geometry and compare the result with another spatial layer, with each executable output feeding the next step.

The researchers used supervised fine-tuning on structured, multi-step trajectories. According to MBZUAI, another language model generated prompts and reasoning traces that were checked before inclusion. The paper record and repository describe deterministic replay checks for whether trajectories execute, produce valid geometry and use tools correctly.

Project records list 14,538 training instances and 1,169 evaluation instances. The paper’s current version reports 100,656 reasoning steps in the training split and 7,064 in the evaluation split. The corpus covers urban analysis, environmental monitoring, disaster assessment or response, and infrastructure analysis.

The evaluation used two modes. A step-by-step test measured procedural reasoning without running tools. An end-to-end test required models to issue live tool calls, provide arguments and reason from the returned outputs.

The benchmark comparisons are the researchers’ results and were not independently replicated in the reviewed evidence. In the step-by-step evaluation, they report that OpenEarthAgent beat GPT-5, GPT-4o and o4-mini on four of five measurements. The exception was summary accuracy: o4-mini scored 89.48, while OpenEarthAgent scored 83.64.

For the end-to-end evaluation, the researchers report 45% answer accuracy for OpenEarthAgent and 44% for GPT-5. They also report 77% image-generation accuracy for GPT-4o and 76% for OpenEarthAgent. MBZUAI notes that OpenEarthAgent’s answer accuracy remained below 50%, leaving substantial room for improvement.

The team reports an average of about 22 seconds per query in its experiment, compared with about 42 seconds for the next-fastest tested model, Qwen3-4B. The reviewed evidence does not independently validate the timing setup or result.

The public project repository includes code, training and evaluation instructions, tool-server setup guidance, and links to the model and technical report. The research team says it plans to use reinforcement learning to reduce tool-selection errors and improve reasoning trajectories.