This paper deals with the design of a multi-modal system for pervasive context-aware service provision and human-environment interaction in augmented environments by the use of Personal Digital Assistants (PDA) or SmartPhones. The system enables mobile devices and remote displays to perform as interaction devices with pervasive applications which run on a dynamically composed server network. Visual interaction for service setup and provision are driven by appropriate graphical interfaces and XML-based protocols, which are dynamically composed according to the type of service and to the user current position by means of a mobile agent-based framework. The paper discusses both protocols, hardware and software system components. The first part of the document gives a general description of the system, which is managed by an entity-driven organization in augmented reality.The mobile and reference devices of the system framework are then discussed, along with the mobile agent software which is used to manage connections among them and with system entities. The paper also gives some details about the ad-hoc protocols for entity interaction. Next, a case study is discussed dealing with service provision in a campus augmented environment which has been arranged according to service requirements. Finally the paper discusses some user experiences while using trial services.
This article is authored also by Synbrain data scientists and collaborators. READ THE FULL ARTICLE