South Korean researchers have developed robotic artificial intelligence (AI) technology that remembers information gathered from its surroundings and independently retrieves it when needed. Rather than guessing when information is uncertain, the system checks stored memories to plan its actions, with potential applications in household and service robots.
The Electronics and Telecommunications Research Institute (ETRI) said on the 26th that it had jointly developed an AI agent called SceneQueryPlanner with the Gwangju Institute of Science and Technology (GIST). The technology combines direct observation of the environment with active retrieval of stored memories.
Recent advances in large language models (LLMs) have enabled robots to understand human instructions and perform multistep tasks. However, carrying out instructions in real-world environments requires robots to understand not only object names but also materials, states and spatial relationships. They must also remember information gathered earlier when moving between different spaces.
Existing AI agents typically receive observed environmental information all at once in lengthy text descriptions, making it harder to identify relevant details as information accumulates. When necessary information is missing, language models may also make incorrect assumptions based on their pretrained knowledge.
To address these limitations, the researchers developed a system that stores observations of rooms, furniture, object locations and other attributes in the form of a “3D scene graph.” The approach allows robots to remember their surroundings like a three-dimensional map. This structured memory organizes information about object materials and states, such as whether something is open or closed or switched on or off, along with relationships between objects. The memory is updated whenever new information is observed.
The key innovation lies in how the stored memories are used. SceneQueryPlanner combines “environment grounding,” in which a robot moves around and directly inspects its surroundings, with “memory grounding,” in which it retrieves only the information needed for the current task from stored memories. Instead of reading every stored detail at once, the system selectively queries object attributes and spatial relationships relevant to the task. This allows it to plan not only what to do but also what additional information it needs to verify.
For example, when instructed to “put the glass cups on the table on the shelf and the plastic cups in the cabinet,” the robot first identifies the cups on the table and then retrieves information about the material of each cup from its memory. It then distinguishes the glass cups from the plastic ones and moves them to their designated locations. Rather than guessing the materials, the robot acts on verified information.
To evaluate the technology, the researchers also developed AtPlan-Bench, a benchmark for assessing AI task performance in complex household environments. Experiments showed that SceneQueryPlanner achieved higher task success rates than the existing ReAct approach with both open-source and commercial models. For Meta’s Llama and Alibaba’s Qwen open models, success rates increased from 46.8% to 69.6% and from 44.0% to 68.0%, respectively. The system also outperformed the existing approach across all evaluation environments when tested with OpenAI’s commercial models. Its success rate on complex tasks involving object attributes improved by 17.2 percentage points.
The technology is expected to contribute to advances in physical AI, enabling robots to make decisions and act autonomously in real-world settings, including household, caregiving and service applications.
The research team plans to further develop the technology so it can build and update scene-graph memories in real time from incomplete environmental information and enable physical robots to carry out extended tasks in complex everyday environments.
“The key to this technology is enabling robots to systematically remember information about their surroundings and independently retrieve and use it when needed,” said Choi Jae-woo, a senior researcher at ETRI’s Social Robotics Research Laboratory.
“This research is significant because it advances memory beyond simple storage, enabling agents to actively query and retrieve the information they need,” said Kim Eui-hwan, a professor at GIST.
The researchers have released the code for SceneQueryPlanner and the AtPlan-Bench benchmark for use in follow-up research. The findings were presented on the 25th at Findings of EMNLP 2026, a leading international conference in natural language processing.