XOE Package Management

Erik Wright, Peter Mehlitz
Transvirtual Technologies Inc.

Abstract:

XOE [1] is a software architecture for Internet appliances. These devices are usually resource constraint, operated by users without specific computer skills, and run applications showing various different, device-independent content types. As a consequence, XOE includes a mechanism to automatically query, install and update software components on a per-demand basis. This paper describes how the XOE package management differs from other systems, what the underlying package model is, and how it can be used inside of a Java-based application framework.

1 Introduction

The post PC area comes with new challenges for a successful software architecture. So far, these architectures were mainly characterized by

Figure 1: PC versus Post-PC Internet topology

With the advent of devices like PDAs, web-pads, high end cell-phones, set-top boxes, telematics systems etc. this is rapidly changing. A generic architecture for these target systems has to support the following network structure.


The XOE [1] architecture meets these requirements by means of featuring

  1. a portable middleware / programming environment (PersonalJava compliant KAFFE [#KAFFE##1###] system, adaptable to various operating systems and hardware architectures)
  2. a web-like application model, with

    1. binary compatible application logic (device independent Java services)
    2. device independent data (XML) and view specification (HTML)
  3. an automated on-demand detection and installation of software components via a dedicated, configured server component
On demand software installation plays a pivotal role in adapting the device to new content, and to reduce its cost in terms of storage and bug fixes.

Even though measures (1.) and (2.) help to minimize the overall system complexity by reducing the number of involved platform dependent components, there still remains a significant challenge in terms of software configuration management, which has to cope with several dependency relevant dimensions like architecture, localization etc.

Figure 2: Software Configuration Dimensions

This problem is solved by the XOE [1] package management subsystem.

2 Package Manager Goals

Package managers are already available for a variety of operating systems and platforms, but so far have not been integrated into application frameworks. These legacy systems usually are

Figure 3: Conventional Package Management Systems

As a consequence, such systems require a system administrator to identify and install packages prior to processing the corresponding content, and to guarantee that install pre-conditions are fulfilled (disk space, architecture etc.). General file-based package management systems also do not provide the required support to install specific Java components, which have to be initialized from inside a Java runtime environment.

In contrast, the XOE [1] package management system has been designed to meet the following goals:

Figure 4: XOE Package Management System

What really sets the XOE [1] package manager apart from legacy systems is that it is extensible in itself, uses a remotely running query system, and features much more powerful install actions than simple file copies. Even though it is a integral part of the XOE [1] architecture, it can be used in any Java based system. This explicitly includes platforms with minimal file systems, or no filesystem at all. Since components are installed when they are actually used, these components can be directly instantiated (e.g. native libraries or Java classes), and don't need to be stored as files. This does not only save memory, but also enables more sophisticated, application framework specific install procedures (e.g. registering services implemented by Java objects, instantiated from package classes).

3 Package Model

XOE [1] packages are stored as standard zip (or jar) archives with a *.xoe extension. There is no restriction about what can go into such an archive, except of that it has to include a XML [3] document package.xml, which contains the meta data of the package. It is perfectly fine to have packages which just contain a package.xml (e.g. to describe platform configuration). The package.xml document itself is extensible, but contains four major sections describing purpose, features and constraints of the package.

Figure 5: XOE Package Structure

A listing of a example package to provide support for the https protocol could look like this:

  Archive: libhttps_0.1.xoe

  Length   ..  Name
  --------     ----
  1269     ..  package.xml
     0     ..  com/
     0     ..  com/transvirtual/
     0     ..  com/transvirtual/protocol/
     0     ..  com/transvirtual/protocol/https/
   426     ..  com/transvirtual/protocol/https/Handler.class
   715     ..  com/transvirtual/protocol/https/HttpsURLConnection.class
  1080     ..  com/transvirtual/protocol/https/HttpsProtocolHandlerFactory.class

This example archive contains only two component types: package meta data (package.xml) and Java class files (*.class). Additional lower level components (e.g. architecture dependent native libraries) could be distributed as a separate package. The corresponding package.xml document for our https package therefor might have the following structure:

  <?xml version="1.0"?>

  <package name="libhttps" version="0.1">

    <info>
      <description>
        java library for secure web transactions
      </description>
      <author>Erik Wright</author>
      <copyright>..</copyright>
      <license>..</license>
    </info>

    <provides>
      <proviso name="protocolhandler"
                  ns="http://www.xoe.org/installer/service">
        <functionality>protocolhandler</functionality>

        <class>
          com.transvirtual.protocol.https.HttpsProtocolHandlerFactory
        </class>
        ..
        <feature name="protocol" value="https" />
      </proviso>

      <proviso name="libhttps"
                  ns="http://www.xoe.org/installer/base/package">
        <version>0.1</version>
        <name>libhttps</name>
        ..
      </proviso>
    </provides>

    <requires>
      <dep ns="http://www.xoe.org/installer/base/package"
              name="libhttps-native" predepends="true" />
    </requires>

    <conflicts />
  </package>

The top node of the package.xml document is the <package> tag itself, which has to have a name and a version attribute. There are 4 subsections: <info>, <provides>, <requires> and <conflicts>.

The <info> section is used to set a informal package description, and additional information about author, copyright and license.

The <provides> section is usually the biggest one, and consists of <proviso> specifications. Provisos are the constructs describing the logical entities which are provided by the package. They can refer to physical entities like single Java classes, or logical ones like packages themselves. Provisos are the key constructs to lookup packages providing requested features, which is done via XPath [5] queries into the package.xml document. Provisos must have a name and a ns (namespace) attribute, the latter one being used to determine a suitable installer for the corresponding entity.

The <requires> section specifies directly or indirectly packages this one depends on. These dependency packages are identified via namespace/name attribute pairs checked against their proviso sets. Dependencies can be further qualified by a predepends attribute, which is used to build a install order of packages. If this attribute is set, the dependency package is completely installed before the current one. If it is not set, the order is unspecified, but it is still guaranteed that all required packages install or get reverted.

The <conflicts> section finally is the place to list namespace/name attribute pairs which collide with the current package.

4 Package Query and Install

XOE [1] package management starts with a request from arbitrary applications. This request is defined in terms of a Java object which can be turned into a XPath [5] query, which is a symbolic expression used to locate nodes inside of XML [3] documents. The object used to query a package has to have a type which implements a simple interface to obtain this XPath expression.

  public interface IPackageQuery { public String toXPath (); }

For instance, a class to query a specific protocol support (e.g. https) could be implemented like:

  class ProtocolRequest implements IPackageQuery {
    String protoName;
    Protocol (String proto) { this.protoName = protoName; }
    ..

    public String toXPath () {
      return ( "package[provides[proviso"
                   + ( "[feature"
                         + ("[@name='protocol']")
                         + ("[@value='" + protoName + "']]]]")));
    }
  }

This could be used in an application (e.g. a web browser) to detect and install a set of required packages in the following way:

  URL url =..;
  IPackageQuery query = new ProtocolRequest(url.getProtocol());

  SearchResults res = PackageManager.search(pq);
  InstallOption[] options = res.getInstallablePackages();

  if ( options.length > 0 ) {
    try {
      PackageManager.commit(options[0]);
      // go on processing content
    }

    catch ( PackageException x ){
      // report error, abort
    }
  }
  else {
    // report missing support, abort
  }

A real world application would probably check for alternatives, and let the user confirm before it starts to install, but the whole process can be automated to the extend which makes sense for the application.

Even though the XOE [1] package management provides a rich API, e.g. to

the main application interface consists of the two static methods search() and commit() in class PackageManager.

The first step of a install process is to create a appropriate IPackageQuery object, which might involve application specific classes. Once this object is initialized, it can be used to perform the package lookup by calling the PackageManager method

  SearchResults search (IPackageQuery in_q)

The lookup process is performed by means of running the XPath query (defined by the IPackageQuery object) on data which is created from package.xml documents. The query server mechanism is accessed via a interface

  public interface IPackageQueryHandler {
    public void performQuery (IPackageQuery in_q,
                              Vector out_results);
  }

This abstraction enables a variety of different repository implementations. In particular, it is possible to use different servers for the package query and the package repository itself.

The local package database that comes with the XOE [1] client is implemented as a set of PackageDescription objects, containing DOM [4] tree objects built from package.xml data. It caches query results to increase efficiency, and uses a XPath [5] search engine to lookup package information.

The remote package database, which is part of the XOE server, is a LDAP [6] repository. It's query system is implemented as a Java-servlet based SOAP service, i.e. it can be easily integrated into existing server architectures (like Tomcat). The XOE client includes a IPackageQueryHandler implementor which transparently turns package queries into SOAP [7] messages, waits for the remote response, and extracts the query result objects from the response message.

The query response itself consists of a SearchResult object, which contains sets of

Installable packages can be obtained by calling the SearchResult method

  InstallOption[] getInstallablePackages()

They are stored as InstallOption objects, which contain not only the relevant package descriptions, but also the actions which are required to install the package. Such actions can include installing, upgrading, or even un-installing other packages.

To finally install packages, the application calls the static PackageManager method

  void commit ( InstallOption opt ) throws PackageException

This eventually retrieves the package data, which is identified by a unique URI containing of package name and version, and installs it by processing its provisos. Each proviso namespace is checked against the configured install service objects, which are used to perform the appropriate install actions (load classes, copy files, load native libraries etc.).

If the install process fails, a PackageException is thrown. All actions related to a package install are part of a single transaction, i.e. are guaranteed to completely succeed, or to be completely reverted in case of a failure.

5 Size and Dependencies

The client side XOE [1] package management consists of about 50 Java classes and interfaces, with about 170 kB of class files. A sample SOAP/LDAP [7,6] based package repository implementation (without user account support) includes about 20 additional classes.

The client components can be used in a PersonalJava 1.2 environment, i.e. they are suitable for embedded devices.

The package management is a integral part of the XOE [1] client architecture, but can be extracted and incorporated into other Java environments. External dependencies consist mainly of a XPath[5] search engine and a DOM [4]implementation, which are both provided by XOE.

6 Conclusions

The XOE [1] package management system successfully demonstrates that a automated, robust, Java-based software configuration system can be used to provide rich functionality on resource constrained devices.

The system heavily utilizes XML [3], DOM [4], XPath [5]and SOAP [7] to achieve a high degree of flexibility and network transparent operation. The use of a dedicated server component makes it possible to implement on-demand package installation without requiring a system administrator. The strict separation of package data and meta data reduces bandwidth requirements.

By using XPath based queries to lookup relevant packages, the system avoids the fragile package format problem, and enables powerful, extensible package descriptions.

Last not least, the XOE package management system features a high degree of modularity, which makes it possible to customize the install process, and to interface different types of package repositories.

By means of these virtues, the XOE package management plays a major role in reducing device manufacturing and maintenance costs, and in adapting devices to a increasing variety of content types.

Bibliography

1
XOE - Extensible Operating Environment
http://www.transvirtual.com

2
Kaffe - the Open Source Java Virtual Machine
http://www.transvirtual.com/kaffe.htm

3
XML - Extensible Markup Language
http://www.w3.org/XML

4
DOM - Document Object Model
http://www.w3.org/DOM

5
DOM - Document Object Model
http://www.w3.org/DOM

6
LDAP - Lightweight Directory Access Protocol
http://www.umich.edu/ dirsvcs/ldap/doc

7
SOAP - Simple Object Access Protocol
http://www.w3.org/TR/SOAP

About this document ...

XOE Package Management

This document was generated using the LaTeX2HTML translator Version 99.2beta8 (1.43)

Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney.

The command line arguments were:
latex2html -no_subdir -split 0 -show_section_numbers /tmp/lyx_tmpdir2273UggAE6/lyx_tmpbuf2273WGHycc/pkgmgnt-whitepaper.tex