IWAI 2026KEYNOTE EXPLORERA COMPANION TO THE KEYNOTE
The theoretical ideas behind the talk, with their mathematical formulations and connections to the slides.
Notation: i ∈ A indexes agents; oᵢ denotes observations, zᵢ internal representations, aᵢ actions, and w a shared external symbol. Bold symbols collect the variables of all agents. The formulations below retain their stated conditioning and approximation assumptions.
Symbols, minds and society
A symbol emergence system links individual learning from embodied experience with the social creation and use of symbols. Shared language influences how individuals perceive and categorize the world, while their interactions continually reshape the shared symbol system. This reciprocal process connects cognition, language and society.
Learning meanings through experience
Symbol emergence in robotics investigates how robots can acquire, invent and share symbols through physical and social experience. A robot integrates vision, sound, touch and action to learn internal categories, then connects them with words through interaction. The goal is to construct and study the mechanisms linking embodied cognition with language.
Building systems that learn and develop
Cognitive and developmental robotics studies how cognitive abilities arise through a robot’s interaction with its body, environment and other agents. Development includes perception, action, representation learning and social communication. Robots provide a constructive way to investigate how these abilities develop together, rather than treating language learning as an isolated task.
Understanding through building
A constructive approach investigates a phenomenon by building a system that can reproduce relevant aspects of it. In this keynote, robots and multi-agent models make proposed mechanisms of symbol emergence concrete and testable. Their behavior can reveal what an assumed learning or interaction mechanism explains, and where it falls short.
Communication protocols arise through interaction
Emergent communication studies how interacting agents develop a communication protocol instead of receiving a fully specified language. Agents may learn signals that help solve a task, or co-construct symbols through language games. The keynote asks how such communication can be grounded in each agent’s experience and connected to shared world modeling.
A sender, a message and a target
In a typical referential game, a sender observes a target and sends a message. A receiver uses the message to select the target among candidates. Reward-based implementations train agents using feedback on successful identification. The keynote contrasts this task objective with learning a shared language through joint attention and generative inference.
Sharing attention to a referent
Joint attention involves attending to an object or event together, with awareness of the other person’s attention. In human development, it provides a foundation for early word learning: words can be related to a jointly attended referent. The naming game assumes this shared referent; learning joint attention itself remains a developmental challenge for robots.
Language learning as generative inference
Generative emergent communication approaches the emergence of symbols through probabilistic generative models and inference. Agents learn from their own observations while communication couples their representations. The Metropolis–Hastings naming game supplies a concrete mechanism: local proposals and acceptance decisions implement inference over shared symbols, linking communication with representation learning.
Predicting the consequences of action
A world model represents how observations and latent states change, including the effects of actions. It can support learning, prediction and planning. In the keynote, individual world modeling learns from embodied experience, while CPC connects these private models through shared symbols. The expression below is a schematic sequential factorization.
World structure carried by language
The collective world model hypothesis proposes that language carries structure learned collectively through people’s embodied experiences and symbolic interactions. An LLM trained on language may therefore learn aspects of world structure indirectly. This is a hypothesis about the collective origins of linguistic information; grounding and predictive capability still require empirical evaluation.
A collective that models the world
CPC proposes that the co-construction of shared symbols can be understood as decentralized Bayesian inference. Individual agents learn from their own embodied observations; language games couple these internal models through a shared external variable. A collection of predictive-coding agents becomes a collective predictive coder when its inference includes this coupling.
This is the generative-model and variational-inference factorization used in the keynote. The variational distribution approximates the collective posterior; its factorization is an assumption, not an identity for the true posterior.
An objective for the coupled system
Collective free energy (CFE) evaluates the joint generative and inference models, including both private internal representations and shared symbols. Minimizing it makes approximate inference approach the collective posterior while trading off predictive fit and model complexity.
Here observations and actions are fixed. This is variational free energy; −F is the ELBO. The CPC-MS paper also gives a formulation over anticipated observations and preferences. Those two conditioning conventions should be kept distinct.
What couples the agents
Under the factorization above, the free-energy objective separates into a collective regularization term, individual predictive likelihoods, and terms relating each inferred internal representation to the symbol-conditioned model. The collective regularization term depends on the internal representations of the group; it cannot generally be replaced by independent per-agent costs.
Here, the collective regularization term retains the prior p(w), including terms that may be constant under a particular prior. It is considered as part of the complete coupled free-energy objective.
Exploration with a collective dimension
The CPC-MS formulation extends the collective regularization term into prospective action evaluation. Exploration can reshape the shared representations of the community, not only increase what one individual knows. Scientific experiments and dialogue can therefore have a collective epistemic role.
This follows the signed, cost-minimization convention of CPC-MS Eq. 2.10. Its first term is labelled collective epistemic value in that paper. It should not be silently identified with a standard positive mutual-information quantity. Tildes denote future variables; C denotes preferences/reward-related conditioning in this formulation.
Local exchanges, collective inference
A speaker proposes a symbol from its own distribution. A listener decides whether to accept it using its own internal representation. Under the specified model and proposal rule, these local exchanges implement MCMC inference over a shared symbol without directly reading another agent’s latent state.
The cancellation in this acceptance ratio depends on using the speaker’s distribution as the proposal. This simplified two-agent product target represents the model-specific naming game, rather than a rule for every possible language game.
Science as a collective world-modeling process
Collective Predictive Coding as Model of Science maps shared scientific knowledge to external representations w, a scientist’s internal model to zᵢ, and experimental observations to oᵢ. Publication, criticism, interpretation and experimentation couple these levels. Science is viewed as a community improving predictive models through distributed interaction.
From shared representations to action
CPC explains how agents can co-construct representations. Collective active inference additionally asks how prospective actions are selected within the coupled system. MARL-CPC and Dreamer-CPC provide concrete bridges: CPC-based message learning supports decentralized reinforcement learning, while the keynote distinguishes these systems from a complete realization of collective active inference.
This is a schematic action-selection statement. The precise feasible actions, planning horizon, preferences and decentralized optimization procedure must be specified for an implementation.
Cognition across bodies, minds and collectives
System 0 describes precognitive embodied processes; System 1, fast intuitive processes; System 2, slower deliberative processes; and System 3, collective intelligence and symbol emergence. The framework links these levels through multiple timescales and develops a connection with Bergson’s philosophy, in collaboration with Yasushi Hirai.
The four systems are levels and timescales of an interacting cognitive system, rather than four independent modules with a single fixed update rate.
Human–AI co-creation with plurality
Symbiotic alignment proposes mutual adaptation in a human–AI ecosystem. Agents participate in the co-construction of shared symbols while retaining distinct internal models. The aim is coherence that can support cooperation across differences, rather than requiring every agent to hold the same worldview.
A multimodal collective belief distribution can represent distinct perspectives. Preserving plurality and ensuring healthy interactions are design and governance questions; they are not automatic consequences of a free-energy objective.