This section describes and illustrates how the various components of the Cloud4all/GPII system participate in the auto-personalization process. This is based on the scenarios of use described at user scenario examples.
Jim: Trusted Device, Local Flow Manager
Jim uses the system on a trusted device with most of the architectural components running locally. He uses a platform-specific RFID User Listener to log in to the system.
Overview of the Application Flow (In Brief)
When Jim scans his RFID tag, which containing his user token, an HTTP request is sent from the User Listener to the local Flow manager. The request payload contains his user token. This request triggers two asynchronous, simultaneous actions on the system:
- The device reporter is invoked, which returns the current operating system and the list of available applications and built-in features on the system
- A request is made to the preferences server with his user token, which returns his needs and preferences (NP) set.
Once the device information is retrieved, an event is triggered and an additional call is made by the Flow manager to the Solution Registry, retrieving information about the solutions installed on his device.
When all of these three pieces of information have been retrieved (the preferences, solutions information and device information), and their event callbacks have been fired, the Flow manager calls the Matchmaker Framework, which in turn invokes a Matchmaker. Based on the device, solutions, and preferences information, the Matchmaker subsystem will return a set of system configuration instructions.
The Flow manager then sends the resulting configuration payload to the Context Evaluator, which will filter out any setup instructions that don’t apply to the current context.
Subsequently, the system configuration instructions are converted to platform- and application-specific instructions by the Transformer. This payload is then forwarded to the Lifecycle Manager, which is responsible for configuring the system.
The process of retrieving the preferences is illustrated in the sequence diagrams below. This process consists of several steps:
- Verifying and authorizing access to the application that request access to an NP set
- Retrieving the preferences and filtering them based on the user privacy policies, and
- Converting the preference into the ontology required by the requesting application (if necessary)
Firstly, any consumer of the preferences server needs to be verified before being given access to a preference set. The application (in this case, the local Flow manager) requests authorization from the Authorization Server (labelled in the diagram as the “OAuth Server”).
The Authorization Server initiates a separate connection to the requesting application with an authorization grant; in order to verify that the requesting party is actually the service they say they are. The requesting application then makes a new request to the Authorization Server for a time-limited access token to fetch a preference set for this particular user.
The preferences framework can now be contacted with the access token and, optionally, a desired ontology. The preferences framework retrieves the full preferences set associated with the access token from the “raw” Preferences Server, which uses a CouchDB database as its persistent backend.
The final step in the process is the Ontology Handler, which converts the filtered NP set into the requested ontology using the Transformer and related conversion rules. Finally the filtered, ontologized NP set is passed back to the requesting application.
The Matchmaking Process
When the user first keys in to the system, the primary task of the Matchmaker Framework is to produce a set of context-indexed system configuration instructions inferred from the user’s NP set.
The first step in this task is to gather any additional information that might be useful to the Matchmaker. When the Matchmaker Framework is initially called, it receives all the information gathered by the Flow manager (i.e. the solution information, device reporter data, the user’s NP set). This is then supplemented by information about the current context, as well as inferred user preferences. The inferred user preferences are general (common term) preferences calculated from any application specific preferences that may be stored in the user’s NP set.
All this information is then passed to the Matchmaker Dispatcher, which is responsible for analysing the available data and delegating to the most suitable Matchmaker instance — either the Rule-Based or Statistical Matchmaker, which are both available in the cloud. The Dispatcher makes an HTTP request containing all the available information to the selected Matchmaker, which then calculates the most suitable system configurations for the user’s machine within each of the predicted contexts that the user may encounter. The system configuration is passed back, to be processed in the second phase of the Matchmaking Framework, namely the Context Evaluator (previously called the Mini Matchmaker).
Evaluating the Contexts and Generating Configuration Instructions
The second task of the Matchmaker Framework deals with determining the current context, deciding which of the system configuration instructions should be applied to the system, and converting this information to platform and application settings.
The first step of this phase is to decide which of the context-indexed system configuration instructions should be applied right now. This is done by the Context Evaluator, which looks at the current context and priorities in the NP set. Based on this, a single contextual system configuration is selected. Once a configuration is selected, the format of this data depends on whether a local, hybrid or cloud-based Flow Manager configuration is being used. In the case of both the hybrid and local modes, the required output is a format that can be interpreted by the Lifecycle Manager, whereas in the cloud-based Flow Manager mode, the result should be a collection of settings that is directly interpretable by the requesting application.
In the case of the Jim scenario, the local Flow Manager is used, so the contextual system configurations instructions are converted by the Lifecycle Manager format via the Transformer component.
The final step in the process is to actually configure the system. When the Lifecycle Manager receives the system configuration instructions produced by the Matchmaking process, it iterates through each of the listed solutions and sets them up. This generally involves two steps:
- Configuring the solution
- Triggering any lifecycle actions, such as launching the application
For each method of configuration that a solution requires, a corresponding Settings Handler is called with the desired settings as calculated by the Matchmaker. Each type of configuration method (such as Windows Registry values, INI settings files, etc.) requires a specialized Settings Handler, but once one has been written, it can be used by any application that shares this method of storing it settings.
Before a setting is written, its current value is stored in a cache by the Lifecycle Manager. This ensures that the system can be restored to its previous state when the user keys out.
Lifecycle Actions work in a similar manner to Settings Handlers. They are responsible for any actions that need to be taken when setting up a solution, such as launching it, writing additional files or making network requests, system calls, etc. If required, the system state related to each Lifecycle Action is recorded in order to be able to restore the system later. Once all the Settings Handlers and Lifecycle Actions for all the desired solutions have been run, the system is configured.
The Five-Layer Configuration
In Jim’s scenario, in addition to the basic use case of configuring native applications and built-in access features, the Cloud4all/GPII system also needs to adapt web- and browser-related settings.
Configuring a browser and its content differs from the standard use cases in several ways:
- a browser shouldn’t necessarily be launched on login, and adaptation should be deferred until the user opens their browser and goes to a particular page
- web content needs a means to query the current settings so that it can adapt itself (such as swapping out images for high-contrast equivalents, etc.)
- in cases where a web application can’t adapt itself, there needs to be means by which the service synthesis tool (SST) can modify it on the fly
The following figure illustrates the process of configuring and adapting browser-based content.
Two browser extensions have been developed in Cloud4all/GPII, which serve to:
- Configure the browser’s own built-in accessibility features
- Provide a controlled means to share user preferences with scripts running inside the web browser’s context “sandbox.”
If the Cloud4all/GPII browser extension has been installed on the device, the Device Reporter will report it as an available solution for the Matchmakers to create configuration instructions for.
When the browser is started, the extension will make a Web Socket request to the local Flow Manager in order to request its settings. This call is passed on to the Web Socket Settings Handler, which in turn returns the settings retrieved by the Lifecycle Manager. This will take care of browser adaptations like enabling a high-contrast stylesheet, etc.
The SST works by being available as a solution for the Matchmakers to configure as soon as a Cloud4all/GPII browser extension is available. If the user wants to take advantage of the SST’s capabilities, they can specify a preference for it, which will then be passed as a setting to the browser extension. When a user browses to any web page, the browser extension will rewrite the page’s URL in order to redirect it to the SST, with an ID denoting the SST service to use and the page that is being visited. The SST will then run the page through its service pipe and return a URL to the newly generated page to the browser extension. The end result is that the user will see the page that has been adapted by the SST service as opposed to the original page.
Web Content Adaptation
Some web applications provide self-adaptation features, such as those provided by Fluid Infusion’s UI Options component. These, too, can be configured by Cloud4All/GPII. For this purpose, the browser extension exposes an API to web content, which can be used to retrieve settings suitable for the application. When this happens, the extension calls the Flow Manager via a Web Socket to request the desired configuration of the website from the Matchmakers. Since it is impossible to calculate all the potential settings for every web page at login, a separate matchmaking process will be performed for the solution. This is the standard “pull” model described above. This is then passed back by the Flow manager to the browser extension, which then forwards the information across the browser sandbox to the web content.
The Cloud-based Flow manager diagram can be seen below. This mode of the Flow manager is used for pull-based settings requests, such as:
- Adaptable web content (as described in the previous section)
- Standalone applications that can’t support a locally installed personalization workflow. For example this includes low-powered embedded devices such as the Java Simple Phone, standalone web applications like EASIT, and self-adaptive applications, such as Mobile Accessibility.
The Cloud-based Flow manager allows an application to request settings that should enact itself. An application ID and user token must be sent to the Flow manager, as described earlier in the Authorization Server workflow.
The first phase of the Matchmaking is identical to the local Flow manager mode, though it only includes one solution (the requesting application). The second phase, context evaluation, is somewhat different from the local Flow manager scenario, in that the result is not a set of instructions to the Lifecycle Manager, but rather a list of settings in a format that is directly operable by the requesting application (i.e. a raw set of settings). The application is itself responsible for writing the settings returned by the Cloud-based Flow Manager.