Detection of distant affordances


Previous experiments have shown that only two type of information are required to localize an affordance in space: the interaction allowing to move closer the most to this affordance, and an estimation of its distance. I thus developed a structure that can detect distant affordances, and store and track them while the agent is moving, under the form of couples (interaction, distance).

The first step thus consists to detect distant affordances using signatures of interactions learned by the agent. We use here a property of signatures: indeed, a signature designates contexts of interactions that characterize an affordance, and these interactions can have their own signatures. Thus, by replacing the content of the signature S of an interaction i by the sum of signatures of interactions designated by S and associated to tha same primary interaction j, we can obtain a signature that characterizes a context that, after enacting j, allows to enact i. As this principle can recursively be applied, it is possible to "backmove" the interactional context affording an interaction with a sequence of interactions. We can then consider that the position of an object instance is characterized by the sequence of interactions allowing to reach it. Note that when more than one sequences allow to reach the object instance, we only consider the shortest. Similarly, when an object can be interacted from more than one position, we only consider the closest position (interactionaly). These sequences carry the two required types of information: the interaction allowing to move closer is given by the first interaction of the sequence (as it is the first "step" of the shortest path), and the distance, characterized by the length of the sequence, in number of interactions. This principle thus define a strong correlation between the notion of distance and possibilities of interactions of the agent.

signatures
Figure 1 : Signatures obtained after 40 000 steps. Each column represents the signature of an interaction. As we observed previously, "move forward" is related to the absence of green and blue object in front of the agent (dark red blob). "Bump" is related to a green object and "eat" to a blue object. On signatures of visual interactions, we can observe that movements produced by their associated primary interactions (in these examples, move forward and turn right 90°) are encoded in signatures. We use these properties to "backmove" signatures.

rétropropagation
Figure 2 (click for full size) : backmovement of signature of interaction "bump" with an increasingly long sequence (bottom). We observe that the green blog characterizing the affordance of "bump" is moved on visual space. These backmovements of the signature allow to detect distant affordances in the environment

I developed and tested mechanisms implementing this principle n our simulated agents. We first let the agent learns signatures of interactions, then, when the majority of signatures are stabilized (nearly 40 000 steps), we test the object instance detection mechanism. At each detection cycle, the agent "scans" its environment by backmoving signatures. After eliminating longest and redundant sequences, we observe that the agent can characterize its environment in terms of sequences of interactions: in example given in Figure 3, the agent has detected walls, preys and empty spaces in its environment.

detection1
Figure 3 : the agent detects a set of sequences of interactions characterizing the presence of object instances. Each sequence is here represented with a circle indicating the real position and orientation characterized by the sequence. The agent detects object affording "bump" (in front of green circles), "eat" (blue circles) and "move forward" (red circles). Couples (interactions;distance) of detected instances are displayed.

By comparing similar environments, we observe that the precision of object detection is directly correlated to interactional possibilities of the agent. Thus, close objects affording the same interaction will be considered as the same object, as they are characterized by the same couple (interaction;distance). However, if an object affording an other interaction is set between these two objects, the agent will consider two separated objects: this new object will separates sequences characterizing the two other objects.

detection2

detection3
Figure 4 : the object detection depends of the interactional possibilities of the agent. Top: the two preys are considered as the same instance localized at ("move forward";4), as they can be reached by similar sequences. Bottom: the presence of a wall block between the preys allows a clear separation. Preys are localized at ("move forward";5) and ("move forward";6)



This mechanism thus allows to characterize the environment of the agent under the form of interactions. However, it is limited to the observable space. An additional structure must be added to keep and update the positions of these objects, even when they escape from sensory system of the agent.


Previous    Back    Next