PMID- 35462780
OWN - NLM
STAT- PubMed-not-MEDLINE
LR  - 20220716
IS  - 2296-9144 (Electronic)
IS  - 2296-9144 (Linking)
VI  - 9
DP  - 2022
TI  - Active Inference and Epistemic Value in Graphical Models.
PG  - 794464
LID - 10.3389/frobt.2022.794464 [doi]
LID - 794464
AB  - The Free Energy Principle (FEP) postulates that biological agents perceive and 
      interact with their environment in order to minimize a Variational Free Energy 
      (VFE) with respect to a generative model of their environment. The inference of a 
      policy (future control sequence) according to the FEP is known as Active 
      Inference (AIF). The AIF literature describes multiple VFE objectives for policy 
      planning that lead to epistemic (information-seeking) behavior. However, most 
      objectives have limited modeling flexibility. This paper approaches epistemic 
      behavior from a constrained Bethe Free Energy (CBFE) perspective. Crucially, 
      variational optimization of the CBFE can be expressed in terms of message passing 
      on free-form generative models. The key intuition behind the CBFE is that we 
      impose a point-mass constraint on predicted outcomes, which explicitly encodes 
      the assumption that the agent will make observations in the future. We interpret 
      the CBFE objective in terms of its constituent behavioral drives. We then 
      illustrate resulting behavior of the CBFE by planning and interacting with a 
      simulated T-maze environment. Simulations for the T-maze task illustrate how the 
      CBFE agent exhibits an epistemic drive, and actively plans ahead to account for 
      the impact of predicted outcomes. Compared to an EFE agent, the CBFE agent incurs 
      expected reward in significantly more environmental scenarios. We conclude that 
      CBFE optimization by message passing suggests a general mechanism for 
      epistemic-aware AIF in free-form generative models.
CI  - Copyright (c) 2022 van de Laar, Koudahl, van Erp and de Vries.
FAU - van de Laar, Thijs
AU  - van de Laar T
AD  - Department of Electrical Engineering, Eindhoven University of Technology, 
      Eindhoven, Netherlands.
FAU - Koudahl, Magnus
AU  - Koudahl M
AD  - Department of Electrical Engineering, Eindhoven University of Technology, 
      Eindhoven, Netherlands.
AD  - Nested Minds Network Ltd., Liverpool, United Kingdom.
FAU - van Erp, Bart
AU  - van Erp B
AD  - Department of Electrical Engineering, Eindhoven University of Technology, 
      Eindhoven, Netherlands.
FAU - de Vries, Bert
AU  - de Vries B
AD  - Department of Electrical Engineering, Eindhoven University of Technology, 
      Eindhoven, Netherlands.
AD  - GN Hearing Benelux BV, Eindhoven, Netherlands.
LA  - eng
PT  - Journal Article
DEP - 20220406
PL  - Switzerland
TA  - Front Robot AI
JT  - Frontiers in robotics and AI
JID - 101749350
PMC - PMC9019474
OTO - NOTNLM
OT  - active inference
OT  - constrained bethe free energy
OT  - free energy principle
OT  - message passing
OT  - variational optimization
COIS- BdV was employed by the GN Hearing Benelux BV. MK was employed by Nested Minds 
      Network Ltd. The remaining authors declare that the research was conducted in the 
      absence of any commercial or financial relationships that could be construed as a 
      potential conflict of interest.
EDAT- 2022/04/26 06:00
MHDA- 2022/04/26 06:01
PMCR- 2022/04/06
CRDT- 2022/04/25 05:11
PHST- 2021/10/13 00:00 [received]
PHST- 2022/01/27 00:00 [accepted]
PHST- 2022/04/25 05:11 [entrez]
PHST- 2022/04/26 06:00 [pubmed]
PHST- 2022/04/26 06:01 [medline]
PHST- 2022/04/06 00:00 [pmc-release]
AID - 794464 [pii]
AID - 10.3389/frobt.2022.794464 [doi]
PST - epublish
SO  - Front Robot AI. 2022 Apr 6;9:794464. doi: 10.3389/frobt.2022.794464. eCollection 
      2022.