Agent-mediated application emergence through reinforcement learning from user feedback

Cyber-physical and ambient systems surround the human user with applications that should be tailored as possible to her/his preferences and the current situation. We propose to build them automatically and on the fly by composition of software components present at the time in the environment, but without prior expression of the user's needs or process specification or composition model. In order to produce knowledge useful for automatic composition in the absence of an initial guideline, we have developed a generic solution based on lifelong online reinforcement learning. It is decentralized within a multi-agent system where agents learn incrementally from user feedback to satisfy her/him. Different use cases have been experimented in which applications, adapted to the user and the situation, are composed and emerge automatically and continuously.