Decentralized Architecture and Organizing Mechanisms for Distributed Terminal System WANG Xu, JI Yang, TANG Xiaosheng, LI Yinong and ZHANG Ping Wireless Technology Innovation (WTI) Institute School of Telecommunications Engineering Beijing University of Posts and Telecommunications (BUPT) 10 Xitucheng Road, Beijing, P.R.China 100876 Email: [email protected], {jiyang,txs,hoplee,pzhang}@bupt.edu.cn

Abstract— Our objective is to build a distributed terminal system to provide smart, context-aware, rich-experienced applications upon personal environment networking technologies, such as WLAN, IEEE 802.15.3 series, ZigBee, Bluetooth, etc. For the proliferation of smart devices with autonomous applications, the users can get much more service experiences than before. While smart devices facilitate human operation, the coordination of devices through networks may provide applications proactively by gathering much more service context, which indicates the emergence of the pervasive computing age. Hence, we proposed the distributed terminal system for the cooperating of smart devices. In this paper, we analyzed the organization architecture of universal service terminal (UST), the distributed terminal system proposed by us before. In UST project, we have abstracted and encapsulated the capabilities of devices as servers for remote invocation by applications, moreover, the framework functionalities have been introduced for the organization of the distributed system. Though the architecture of UST has been validated feasible in a demonstration, the centralized control mechanisms in the heterogeneous environment are inefficient and unreliable. Thus, we propose an evolved scheme by introducing decentralized mechanisms in this paper. The devices around the user are organized in an overlay peer-to-peer network, and some powerful nodes of them provide the decentralized mechanisms for resource management, service discovery, etc.

I. I NTRODUCTION In recent years, information technologies around human beings, e.g. Personal Area Network (PAN), wearable network, Intelligent Apparatus (IA), sensor technology, were developed rapidly, and couples of applications built on them were deployed. These applications can get much more service context information from the environments or interactions with the user and bring him or her much more service experiences than before. Some of the applications can even communicate with servers via Internet for certain supports. While these applications were almost working on single or few devices, to implement the vision of pervasive computing [1] or ubiquitous computing [2] and bring smart, context-aware [3], richexperienced and customized services to users, the devices in personal environment should work cooperatively and proactively by learning the behavior or getting the information from other devices or the network services. Inspired by the extensibility of the open API [4] in the developing Next Generation Network (NGN) architecture, the authors introduced an extensible framework to personal area

to aggregate smart devices around the user as a virtual terminal and provide the capabilities of them to the applications effectively in the project UST [5], which was sponsored by National 863 High-Tech R&D program of China in 2004. In the UST project, a C/S based distributed architecture was designed, and a set of mechanisms were developed to validate the feasibility of the architecture and show the potential in improving service experiences. A UST is a distributed system which is the aggregation of devices around the user, and it provides an adaptive executing environment to applications according to the dynamically changing service context. A prototype system based on WLAN and a video conversation application were implemented to evaluate the feasibility of the concept and basic mechanisms. Though the demonstration showed that a distributed terminal could provide more service experiences to the user by exploiting the resources around him or her effectively, several problems restrict the utilization of the potential capacity of cooperative working of devices. Firstly, UST could employ several diverse access networks for more bandwidth and reliability, while the personal area is highly heterogeneous and the UST did not have an effective approach to working on heterogeneous networks. Moreover, though several nodes in UST might have the capability to become a master node in UST, only one of them would be elected as the master, which might be in much heavier load than others and lead to “single-point failure”. As the second phase of the research of UST, which is named as UST+[6], one of the most important working item is to improve the communication mechanism in personal environment. In the past five years, the peer-to-peer (P2P) technologies have been increasing incredibly over Internet. As each node in P2P overlay networks is both client and server and does not suffered from “single-point failure”, P2P systems are very reliable, and in some of the P2P systems, e.g. Skype [7], certain nodes are able to relay traffic from one network domain to another, which can cope with interconnected heterogeneous networks. Thus, we introduce the decentralized methods into UST+ for more effective and efficient cooperation among devices. The rest of this article was organized as following. Firstly, the basic mechanisms of UST were introduced in section II;

then the decentralized architecture and improved mechanisms were presented in the successive section, and the developing status and open issues were figured out also; at last, the authors drawn a conclusion about their work. II. UST: T HE D ISTRIBUTED T ERMINAL S YSTEM In the UST project, we proposed an extensible architecture (Fig. 1) for the distributed terminal system, in which service capabilities of devices were abstract and encapsulated as capability servers [5]. A device in UST system could provide several capability servers, which could be invoked remotely by the applications. According to the dynamic nature of the personal area networks, the framework functionalities including resource management, AAA, service discovery, etc., was introduced for the organization of UST. At least one device in UST system had the capability to provide them, and one of the devices would be elected as the master to provide the functionalities. It is obvious that the master device plays a significant role for extensibility of the architecture.

•

•

functionalities in the service environment, if it is found that one of the devices is more suitable to be the master than the current one according to the capabilities of devices and the user preference, its framework functionalities will be activated and it will become the new master. The challenge in the shift of framework functionalities is the migration of the context information of executing environment, i.e. how to move a running application, and we had never implemented this procedure in our demonstration yet. The attaching, detaching, or capability changing of the devices procedure will occur when the user’s service context (e.g. location, status, or environment) has changed. Some devices will attach to or detach from the UST system and others may modify their capabilities, which may influence the status of the framework and running applications. The Attach/Detach/Change procedure includes two sub-functions, one is update the device capability database maintained by framework, and the other is updating the running applications. Both subfunctions are charged by the master device. The invocation of capabilities includes the discovery, acquisition and invocation of the capabilities. The application discoveries and acquires capabilities from the framework based on the description of the capabilities, and then it can access them via remote method invocation.

B. Analysis of the Organizing Mechanisms in UST

Fig. 1.

The Architecture of UST

A. Organizing Mechanisms in UST The basic procedures of the UST include the aggregation of the terminal, the migration of the framework capability, the attaching/detaching or capability changing of the devices, and the invocation of terminal capabilities. By all of the procedures we mentioned, the UST could provide best capabilities to support applications according to the users preference under different service contexts. • The terminal aggregating procedure is the construction procedure of the application execution environment of a UST. The primary device, which contains the identity information of the user and has the framework functionalities, plays the key role in this procedure, which registers the capabilities of devices and add them into the system and builds a distribution system. The first step of aggregating is to build a minimum UST containing the only primary component. It provides some capability servers, runs the execution environment, loads User information to framework, and activates it. Then the other components can join into the system and provide their capabilities. • The framework migration procedure is defined as when there are several devices that can provide framework

In the UST system, the master device, which provides the framework functionalities, maintains the global information of the system, and takes part in all system-wise operations. Since the overall informations are stored in the dynamic database in the master device, the system could be extended by update the database, which is extensible. However, since all the interactions are related with the master, its capability and connectivity with other devices may become the bottleneck of the system. It is assumed that every device in UST system could setup a connection with the master device, even if they cannot communicate with each other directly. Hence, the logical topology of UST is in star shape, however, in personal environment, the networks are highly heterogeneous because sorts of networking technologies e.g. IEEE 802.15 series, IEEE 802.11 series, Ethernet, power line, etc. may be exploited. Therefore, a UST operation message may lead to many hops inefficiently. On the other hand, the master device undertakes heavy traffic load and storage or control job, which affect its communication and energy efficiency. Moreover, as a centralized distributed system, the risk of “single point failure” should be taken into consideration, i.e. the failure of the master device may lead to crash of UST system.Unfortunately, for personal and portable devices, the failure is occurred much more frequently than those in telecommunication devices. Actually, there often exist more than one devices which can provide framework functionalities, and employing of those devices cooperatively may be able to enhance the reliability

and efficiency of UST system. Thus we proposed a decentralized extensible personal service environment in UST+ as an enhanced version of the UST project. III. D ECENTRALIZED O RGANIZING S CHEME OF D ISTRIBUTED T ERMINAL S YSTEM IN UST+ Considering the diversity of devices, we classify the devices as senior nodes (SN) and basic nodes (BN) according to whether they have the capability to provide framework functionalities. While one of the senior nodes provides framework functionalities in UST system, all of them provide framework functionalities jointly via decentralized mechanisms, which will be described in subsection III-B and III-C.

with heterogeneous wired or wireless networks. Therefore, the underlying networks of UST+ is assumed as a heterogeneous ad hoc network, we also assumed that all the SNs and their network connections could be described as a connected graph (shown in Fig. 3) for the simplification of design, i.e. there must exists at least a path, though which one SN could find another, and all the nodes on the path are SNs. As most of the devices have multi-mode connections are sophisticated enough to provide framework functionalities, the assumption is acceptable.

A. Decentralized Architecture of Distributed Terminal System In the organization architecture of UST+, every SN works as both client and server, while a BN works as client only, i.e. both the capability servers of devices and applications running on nodes need interact with the decentralized framework functionalities provided by SNs for system organizing, e.g. service advertising, service discovery, resource allocation or release, etc. To support the requirements, a middleware architecture of the device was designed shown in Fig. 2 around with dashed line. Every node has an component named as “aggregation control” (AC), which is responsible for interacting with framework functionalities as client on behalf of the device, e.g. registering the capability servers (CS), which provide the remote invocation interfaces to applications; an SN also has another component named as framework, which is responsible for organizing the SNs as a peer-to-peer overlay network and providing the functionalities to the clients. Moreover, applications running on the distributed terminal systems and serving users also get the knowledge of terminal capabilities and the handle of them from the decentralized framework functionalities provided by SNs.

Applications Framework Function AC

Provided by SN

CS

CS

Provided by Middleware

Provided by each node Physical Resources

AC − Aggregation Control Function CS − Capability Server SN − Senior Node

Fig. 2.

Architecture of the Device in UST+

In personal area, e.g. resident environment, office or vehicle, etc., there are diverse smart devices or sensors connected

Basic Node (BN)

Connection between BN and SN

Senior Node (SN)

Connection between SNs

Fig. 3.

The Topology of UST+

Hence, the system could be described in a layered model shown in Fig. 4. In the top layer, applications and ACs on behalf of the capabilities of devices run as the clients of framework functionalities. Although a BN does not have the framework component, the AC and/or applications on it still need access framework functionalities provided by the peer-to-peer network made up by SNs for register its capabilities or acquiring service capabilities etc. The framework functionalities were separated into two sub-layers, the upper one provides application programming interfaces (API) to support the semantics of the primitive operations of distributed terminal system, which is independent from the peer-to-peer networking mechanisms; on the other hand, the lower sublayer provides common services of decentralized lookup and storage services and deal with the topology changing, which is based on the peer-to-peer lookup service [8], [9]. The separation of the two sub-layer of framework functionalities facilitates the improvement of the basic algorithm, for the improvement of common services can impact the upper layers little vice versa. B. Common Service Sub-layer in the Decentralized Organizing Architecture The framework mechanisms in UST+ are built upon a peer-to-peer overlay network, which provides several common services (shown as Fig. 4) as the fundamental of the framework functionalities. The objective of the common service sub-layer

Client

Client

TABLE I

Application or AC of Device

I NFORMATION ABOUT A C APABILITY S ERVER

Framework Mechanisms Framework Functionalities Common Services for framework

Physical Connection BN

SN

Fig. 4.

Data Item ServiceTypeID ServerID DeviceID NormalizedCapa UsedBy TTL Capability

Definition Type of the service Identifier of the server The device providing this server Normalized Capabilities of server What applications are using it Time to live of this description Capability description of Server

Data Type string string string integer string integer XML

SN

Layered model for UST+

is to provide a common infrastructure for the development of the framework functionalities independently from the peerto-peer lookup or routing mechanisms, by which they can be developed and improved simultaneously. All the services are decentralized and with the characteristic of load balance, failure tolerance, and scalability. A chord [8], [9] based protocol was selected as chord was efficient and scalable in peer-to-peer lookup, and similar protocol such as Pastry [10] in PAST [11] was considered also. While all the nodes in chord are equalized, only senior nodes in UST+ can provide services. The services provided by this sub-layer include search, retrieval and modification of value by a key in certain database, which are all based on the lookup functions. While the IPv4 address was used in chord as the identifier of node, IPv6 address was selected in the UST+ for the large address space and the convenience of stateless address autoconfiguration [12] mechanism. And the SHA-1 [13] message digest function was selected to guarantee the randomization of the keys. The following results are proven in the paper that introduced chord [9] Theorem 1: In an N -node network based on chord protocol, 1) each node maintains information about O(log N ) other nodes; 2) lookup by a given key need about O(log N ) messages; and 3) a join or leave requires O(log2 N ) messages. As only the SNs provide the framework functionalities, the N stands for the number of SNs in the distributed terminal system. For the devices or sensors around user are much less reliable than those in the machine room of operators, we must save replicas of data to prevent system from crash when nodes failure, and chord also guarantee [9] that the organization information can be recovered with high probability even with a 1/2 probability of SN failure. C. Basic Mechanisms in the Decentralized Architecture Based on the common services, a suit of decentralized mechanisms was developed to cover the essential framework functionalities of UST. Since the framework functionalities was provided decentralized, the migration mechanism of frameworks was obsoleted and the leaving or joining of a nodes was a little more sophisticated than before.

TABLE II I NFORMATION ABOUT AN A PPLICATION Data Item AppID Location Required Recommended Using

Definition Identifier of the application Where the application was initialized Services required by the application Services recommended by it What server was used by it

Data Type string string string string string

All the framework mechanisms for organizing the UST+ system deal with the data about capability servers and/or applications, which are described in Table I and II. For the capability server, which provides certain type capability that could be exploited by applications, not only has an ID stand for itself (ServerID), but also has an ID for the type of service it providing. And the database of the capability servers is indexed by the service type, by which facilitates the acquisition of capability servers. Moreover, a normalized capability value was presented here, by which simplified the task of finding a capability server with additional capability requirements. The TTL item is a watchdog timer for monitoring the activity of devices. For the application, not only the application ID and capability servers in use are recorded, but also the required and recommended services and normalized capability value are recorded. Thus, the application will be interrupted if no server can provide required service with acceptable capability; or may be notified if new servers can provide some recommended services. And the location is used for the call back. //finding a SN as proxy sn=this.find sn(); //register CS to framework m=cs.length(); //the number of CSs for i=1 to m; //register the ith capability server sn.register cs(cs[i]); Fig. 5.

Pseudocode of a BN Joining UST+ system

If a SN joins the UST+ system, the framework component will be active and it will join the peer-to-peer network first and some data will be written to it, which will be done in the common service sub-layer. Then its own AC will register its capabilities to the framework functionalities, the framework mechanisms method will write the data into capability server database, and then they can be used by any authorized

applications in UST+ system. For BNs, it is similar to SNs in joining the UST+ system. While an SN can interact with framework locally, a BN must find an SN firstly since it does not have. After having found an SN, the BN will interact with framework provided by the SN to register its capability servers. Although a BN need the proxy function provided by an SN to interact with the functionalities, the connect between application and the capability server are not essentially through the SN, i.e. they may interact each other directly for efficiency. The pseudo code of a BN joins the system was shown in Fig. 5. //notify the the connecting BNs m=bn.length(); for i= 1 to m; bn[i].message("SN REMOVAL"); //terminate all the applications m=app.length(); for i= 1 to m; app[i].signal(TERMINATE); //deregister CS to framework m=cs.length(); for i=1 to m; deregister cs(cs[i]); //detach itself from the network node detach(this); Fig. 6.

Pseudocode of an SN leaving UST+ system

On the other hand, if a BN detaches from the UST+ system, it should stop the applications running on it if any, and remove the entries of its capability servers from the database, which may lead to the application employing its capability servers changing to other servers or hanging up. Then the data about the node in the SN working as its proxy will be removed, and the node will be free. For a SN, before the detachment, it should ask the BNs connecting with UST+ system through it for finding a new SN for that, if they cannot find one, they will remove themselves at first. Then the SN will terminate the applications running on it and removal the data entry of capability servers in the UST+ database just like the BN does. And then the decentralized system will shift the data stored in the SN originally to other nodes and adjust the topology information consequently. The pseudocode of an SN leaves the system is shown in Fig 6. The implicit detachment mechanisms are also designed for the lack of reliability and dynamic nature of terminal environment. If a node was prevented from refreshing its data entry stored in the decentralized database, it would be considered as implicit detached when the time-to-live (TTL) timer has overflowed. Then the detach procedure will be performed in the peer-to-peer network. When an application for UST+ are initiating on some nodes, it should query the framework for capability servers and indicate the essential and recommended capability server

//get the handle of framework fw=get fw(); //query for the lists of CSs cs list=fw.query cs(required, recommended); if cs list is nil return f alse; //negotiate with framework cs list’=select cs(cs list); cs list"=fw.negotiate(cs list’); if cs list" is nil return f alse; //get the handles of CSs m=cs list".length(); for i= 1 to m cs[i]=fw.get cs(cs list"[i]); if cs[i] is nil return f alse; Fig. 7.

Pseudocode of an application initialization

it need, and then it will retrieve a list of severs and their capability description. After having negotiated with framework and capability servers, the application gets the handles of the selected capability server. The pseudocode of an application initialization is shown in Fig. 7. And the framework also maintains lists for each type of capability about the the application interesting in it, thus, the application will be able to get notifications when a related capability servers are being changed. IV. S TATUS AND F UTURE W ORK The system described is under development. A SIP [14] based protocol are being developed, and it will be implemented and evaluated in a prototype system as the enhancement of the the demonstration of the UST project in heterogeneous environment. The SIP was selected as its simple, extensible and widely adopted in maintenance of multi-party session. While the evaluation platform and prototype of the system is being developed, we are working for the improvement of the mechanisms simultaneously. In the system described, a normalized value was used to denote the capability of a device in certain aspect, which is a benchmark for it, i.e. the normalized value is a value depending on the device itself. However, the capability of a device was restricted by the environment significant. For example, a video output device with a MPEG-2 decoder, which has only a connection restricted to below 500 kbps with the media source, may be trivial, while it can bring perfect experiences when it is connected with the media source through Fast Ethernet. Therefore, we are going to develop a more reasonable evaluation and denotation scheme for UST+ in the next step.

As the distributed terminal in UST+ is a peer-to-peer system in nature, it could be deployed across WAN potentially and constructed based on the service context for the user. Hence, it may enable the smart, context-aware, rich experienced, proactive application for the user, which is just our objective and will be researched in the future. V. C ONCLUSION We presented the design of the architecture of the distributed terminal system in UST+ , which aimed to facilitate the cooperatively working of smart devices and sensors. As in the heterogeneous and dynamic personal environment, the sophisticated decentralized mechanisms are more scalable, efficient and failure tolerant against the centralized. We presented the architecture of system and the layered decentralized model, moreover, a suite of chord based decentralized mechanisms to maintain the organization, e.g. node joining and detaching, service discovery, etc., and related data structure of system was designed and discussed in detail. ACKNOWLEDGMENT This work is partially supported by National 863 High Tech R&D Program of China, No. 2004AA119030 and National Natural Science Foundation of China (NSFC), No. 60432010. R EFERENCES [1] M. Satyanarayanan, “Pervasive computing: Vision and challenges,” IEEE Personal Commun. Mag., vol. 8, pp. 10–17, Aug. 2001.

[2] M. Weiser, “The computers for the twenty-first century,” Scentific American, vol. 265, Sept. 1991. [3] B. Schilit, N. Adams, and R. Want, “Context-aware computing applications,” in Proc. Workshop on Mobile Computing Systems and Applications, Dec. 1994, pp. 85–90. [4] A.-J. Moerdijk and L. Klostermann, “Opening the networks with parlay/osa: standards and aspects behind the apis,” IEEE Network, vol. 17, pp. 58–64, June 2003. [5] X. Wang, Y. Ji, and L. Liu, “Universal service terminal: evolving towards the TSE in MUSE,” World Wireless Research Forum (WWRF 11), Oslo, Norway, June 2004. [6] X. Wang, Z. Hu, X. Tang, and Y. Li, “UST+: Enabling the service capabilities of pervasive computing in personal environment for the NGN service architecture,” unpublished. [7] S. Baset and H. Schulzrinne. (2004) An analysis of the skype peer-to-peer internet telephony protocol. Columbia University. [Online]. Available: http://www1.cs.columbia.edu/ salman/publications/cucs-03904.pdf [8] R. Cox et al. (2005) The chord project. [Online]. Available: http://pdos.csail.mit.edu/chord/ [9] I. Stoica, R. Morris, M. Kaashoek, and H. Balakrishnan, “Chord: A scalable peer-to-peer lookup service for internet applications,” in Proc. ACM SIGCOMM’01, San Diego, CA, Aug. 2001. [10] A. Rowstron and P. Druschel, “Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems,” in Proc. IFIP/ACM Middleware 2001, Heidelberg, Germany, Nov. 2001. [11] P. Druschel and A. Rowstron, “Past: A persistent and anonymous store,” in Proc. HotOS VIII, Schloss Elmau, Germany, May 2001. [12] S. Thomson and T. Marten, “IPv6 stateless address autoconfiguration,” RFC 2462, Dec. 1998. [13] Secure Hash Standard, U.S. Department of Commerce/NIST, National Technical Information Service Std. FIPS 180-1, Apr. 1995. [14] J. Rosenberg et al., “SIP: Session initiation protocol,” RFC 3261, June 2002.