Documentation

Platform Content Module

The Platform Content Module (Bundle-ID: org.apache.clerezza.platform.content) provides tools to upload, view and edit structured and binary content. It provides basic templates for displaying content as well as an editor (the Discobits Editor) to edit content. It also allows putting binary content using tools that support HTTP PUT (such as curl).

Uploading Binary Content

The attribute "binary" may sound a bit confusing when referring to content as all content stored on a computer is binary. We use the term here for arbitrary content which can be stored on the platform and retrieved as binary identical copy. For example if an RDF/XML file is uploaded as binary content one will be able to retrieve a copy that is identical to the original file, i.e. including comments and possible syntax error, would the same file be uploaded as graph (rather than binary content) the file would be parsed on upload (producing an exception in the case of syntax error) and one will be able to access a serialization of that graph in any supported RDF format, the downloaded documents will describe the same graph but not (necessarily) be binary identical to the uploaded document.

The Discobits Editor

The discobits editor allow editing stuctured content. The overall structure is stored in RDF the actual bits of rich text are stored in the XHTML format. The editor refects this by providing widgets for editing the structure and an XHTML editor (mozile) for editing bits of XHTML.

SCB Web

The bundle SCB Web allows access to SCB graphs over HTTP with JAX-RS.

Upload Triples with a POST Request

To upload triples with a POST request, a client can use the URI path "/graph" and place the triples and other required information into the body as multipart/form-data which consists of
  • a file labeled "graph" containing the triples and specifying the mime-type.
  • a text field labeled "name" specifying the name of the MGraph. If an MGraph with this name does not already exist, a new one will be created.
  • an optional text field labeled "mode" specifying the mode. If the mode is "replace", existing triples of the MGraph will be deleted before new triples are added. If the mode is not specified or is "append", posted triples will be added to the MGraph.
  • an optional text field labeled "redirection" specifying an URI which the client should be redirected to in case of success.
A response with the status code BAD REQUEST (400) is returned if the required data are missing. If the request can be satisfied, one of the following responses is returned:
  • SEE OTHER (303), if redirection is specified.
  • CREATED (201), if redirection is not specified and a new MGraph is created.
  • NO CONTENT (204), if redirection is not specified and no new MGraph is created.

For your convenience you may access a web-form at the Uri-Path /graph/upload-form.

Backup of Triple Collections

The platform allows the current user to download all triple collections that he has access to through the URI path "/admin/backup/download". The resulted file is a compressed archive in zip format. All triple collections in this file are serialized in N-Triples format. Furthermore, a file called "triplecollections.nt" is contained in backup.zip, which describes the mapping of file names to triple collection names.

Platform Documentation Aggregator

Any bundle may provide some user documentation. This documentation is provided by a file named "documentation.nt" in the META-INF directory of the bundle-jar. This file can provide multiple documentation resources described using the discobits ontology. The resources have UriRefs starting with "bundle:///", this will be replaced with http://<hostname>/bundle-doc/<Bundle-SymbolicName> by the documentation aggregator which provides the graph urn:x-localinstance:/documentation.graph containing the documentation of all installed bundles. To reference a documentation entry from another bundle the bundle-uri can contain a Borundle-SymbolicName after the second slash, e.g. bundle://org.apache.clerezza.platform.content/intro.
By convention any bundle documentation provides a resource named bundle:///intro introducing the functionality of the bundle.

Writing Documentation

The recommended way to create and edit the documentation of a bundle is as follows:
  • Launch clerezza (in the following we assume its running on localhost:8080)
  • Install and start the stable n-triples serialization provider (to make the stored files more suitable for version control) by entering on the commmand line: start("mvn:org.apache.clerezza/rdf.stable.serializer")
  • Install and start the file storage provider(to access local rdf files as named graphs) by entering on the commmand line: start("mvn:org.apache.clerezza/rdf.file.storage")
  • If it doesn't exist yet create an empty file src/main/resources/META-INF/documentation.nt in the project to be documented and
  • Open http://clerezza.apache.org/tools/editor/, enter the file-uri of the previously created file as Graph and a bundle-uri (such as bundle:///intro) as resource
  • edit, save, ship your bundle

Documentation Viewer

The documentation viewer provides access to the documentation (provided by the different modules) at /documentation.
The documentation is ordered so that the after properties http://clerezza.org/2009/08/documentation#after are satisfied. When the documentation is written with the discobits editor such a property is typically set by adding something like the folleowing in the RDF editing mode to the xml element representing the resource: <after xmlns="http://clerezza.org/2009/08/documentation#" rdf:resource="bundle://org.apache.clerezza.platform.content/discobits-editor"/ />

overview

Introduction

Smart Content Binding (SCB) is an open source framework developed by clerezza.org aiming primarily at providing a java implementation of the graph data model specified by W3C RDF and functionalities to operate on that data model. SCB offers a service interface to access multiple named graphs and it can use various providers to manage RDF graphs in a technology specific manner, e.g., using Jena (TBD) or Sesame . It also provides façades that allow an application to use Jena or Sesame (TBD) APIs to process RDF graphs (note that the choice of a façade is independent of the chosen backend; you can for example use the Jena façade to write your code against the Jena API while using the Sesame provider to store your data in a Sesame store). Furthermore, SCB offers a serialization and a parsing service to convert a ImmutableGraph into a certain representation (format) and vice versa. In order to support ontologies usage, SCB provides a tool to convert ontologies defined in various formats including RDF/XML into Java classes.

Default 404 Response Provider

The default 404 Response Provider budle with id org.apache.clerezza.platform.content.default404 provides an implementation of the PageNotFoundService (package/bundle: org.apache.clerezza.platform.content) that the delivers the response retrieved from the uri with the same protocol and host and the /page-not-found. For example if 404 is to be generated for http://example.org/foo/bar the entity obtained by dereferencing http://example.org/page-not-found will be returned.

Uploading 404 response page with curl

example: curl -u admin:admin -H "Content-type: text/html" -T tmp.html http://127.0.0.1:8080/page-not-found

ScalaServerPages

ScalaServerPages allow using scala to render a response resource to a particular output format.

Example

The following shows a simple ScalaServerPage:
//a ScalaServePage to render a http://clerezza.org/2009/05/usermanager#UserPermissionPage def um(s: Any) = new UriRef("http://clerezza.org/2009/05/usermanager#"+s) def perm(s: Any) = new UriRef("http://clerezza.org/2008/10/permission#"+s) <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>Permissions for user {res/um("user")/FOAF.name}</title> </head> <body> {for (permission <- res/um("permission")) yield <div id="permission"> {permission/perm("javaPermissionEntry")*} </div> } </body> </html>
The following iterates over an rdf:List represented by the root-resource
//a ScalaServePage to render a http://clerezza.org/2009/05/renderletmanager#RenderletManagerPage def typerendering(s: Any) = new UriRef("http://clerezza.org/2009/04/typerendering#"+s) <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>Listing renderlets</title> </head> <body> {for (renderlet <- res!!) yield <div id="renderlet"> type: {renderlet*} </div> } </body> </html>
You can sort rdf:List using the sort-method:
//sorting the URIs of a http://clerezza.org/2009/05/renderletmanager#RenderletManagerPage <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>Listing renderlets</title> </head> <body> {for (renderlet <- (res!!).sort((a,b) => ((a*) < (b*)))) yield <div id="renderlet"> type: {renderlet*} </div> } </body> </html>
same for properties:
//a ScalaServePage to render a http://clerezza.org/2009/05/usermanager#UserPermissionPage //sorting by java-permission-entry def um(s: Any) = new UriRef("http://clerezza.org/2009/05/usermanager#"+s) def perm(s: Any) = new UriRef("http://clerezza.org/2008/10/permission#"+s) <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>Permissions for user {res/um("user")/FOAF.name}</title> </head> <body> {for (permission <- (res/um("permission")) .sort((a,b) => (a/perm("javaPermissionEntry")*) < (b/perm("javaPermissionEntry")*))) yield <div id="permission"> {permission/perm("javaPermissionEntry")*} </div> } </body> </html>

How does it work

A ScalaServerPages is transformed into a Scala Source file and compiled. The content of the ScalaServerPage becomes the content of a method returning AnyRef, the returned Object will be transformed to a String and the to a byte-array to be written to the response stream

Avialable values

  • renderer: CallbackRenderer, used to delegate to another Renderlet usally not used directly but wia the render method
  • res: GraphNode, the main response resource, as a GraphNode it is dynamically converted to a RichGraphNode allowing the functions provided by org.apache.clerezza.utils.scala
  • val context:GraphNode, a GraphNode with contextual information not specifically related to the current request, such as description on the current user
  • val mode: String, the rendering mode
  • val uriInfo: UriInfo, the UriRinf of the request, allows access to the request URI and query parameters
  • val sharedRenderingValues: java.util.Map[String, Object], a map used to share values across the different renderlets and ScalaServerPages involved in the creation of a representation, typically used to prevent repeated computation of the same values. Typically this map is not accessed directly, instead values are retrived with $("key") and set with $("key") = newvalue

Accessing OSGi services

ScalaServerPages are used for rendering information, therefore services are typically accessed when producing the RDF and not from the ScalaServePage. Still you can access designated services from the ScalaServerPages, you get an instance with $[serviceInterface], eg.:
$[AdvertisingService].getBanner
You can only access services that are annotated with @org.apache.clerezza.platform.typerendering.WebRenderingService

Clerezza - Platform Type Rendering Utilities for Scala

The Type-Rendering Utilities for Scala allow to write Renderlets in Scala more easily. Apart from a having to add a bit of code to produce complete Scala classes they provide the same functionality as available in ScalaServerPages, additionally the Renderlet can register itself for a specified RDF type and rendering mode.
See the ScalaDoc for PageRenderlet and its superclass AbstractRenderlet for usage instrcutions. Note that the resulting rendelerlets need to be registered as service, in serviceComponents.xml the respective section would look like this:
<scr:component enabled="true" name="org.example.html.BookFormRenderlet">
	<implementation class="org.example.html.BookFormRenderlet"/>
	<service servicefactory="false">
		<provide interface="org.apache.clerezza.platform.typerendering.Renderlet"/>
	</service>
	<property name="service.pid" value="org.example.html.BookFormRenderlet"/>
	<reference name="renderletManager"
	interface="org.apache.clerezza.platform.typerendering.RenderletManager"
	cardinality="1..1" policy="static" bind="bindRenderletManager" unbind="unbindRenderletManager"/>
</scr:component>

Smart Content Binding

Smart Content Binding is just the name for the RDFsupport in clerezza.
The RDF support in Apache Clerezza

Overview

Introduction

Smart Content Binding (SCB) is an open source framework developed by clerezza.org aiming primarily at providing a java implementation of the graph data model specified by W3C RDF and functionalities to operate on that data model. SCB offers a service interface to access multiple named graphs and it can use various providers to manage RDF graphs in a technology specific manner, e.g., using Jena (TBD) or Sesame . It also provides façades that allow an application to use Jena or Sesame (TBD) APIs to process RDF graphs (note that the choice of a façade is independent of the chosen backend; you can for example use the Jena façade to write your code against the Jena API while using the Sesame provider to store your data in a Sesame store). Furthermore, SCB offers a serialization and a parsing service to convert a ImmutableGraph into a certain representation (format) and vice versa. In order to support ontologies usage, SCB provides a tool to convert ontologies defined in various formats including RDF/XML into Java classes.

Architecture

SCB comprises the following architectural components as depicted in Fig. 1:

  • Core
  • Facades
  • Utilities
  • Storage Providers
  • Parsing Providers
  • Serializing Providers
  • Ontologies Tool

SCB Architecture
Figure 1: SCB Architecture

The Core contains interface definitions of the RDF ImmutableGraph data model and its implementation. The three main classes are ImmutableGraph, Graph, and Graph. The class ImmutableGraph represents an immutable RDF ImmutableGraph, as such its identity criterion is defined in terms of graph-isomorphism. The class Graph represents a mutable RDF ImmutableGraph, which enables triples to be added to or removed from a ImmutableGraph. The class Graph is the super class of both the class ImmutableGraph and MImmutableGraph. SCB Core provides three services: TcManager [4] allows access to the various Graphs, Parser [5] and Serializer [6] to allow reading and writing graphs from and to various formats. In an OSGi environment these services are accessed using the service registry or injected using OSGi Declarative Services. In a non OSGi environment static factory methods are used to return an instance. The TcManager delegates actual processing tasks to a specific Storage Provider chosen from a set of Storage Providers based on their priority number (weight). Storage Providers can be dynamically bound to or unbound from the Core. The functionality required by the Parser and Serializer is delegated to registered Parsing and Serializing Providers respectively, according to their capability (supported formats). Later registered providers shadow previous ones for the same format.

The current implementation of SCB includes a Jena Façade. The Jena Façade allows an application to use Jena API to manipulate a TC.

In order to ease operations on a resource in a TC, the Utilities component provides a class with a set of useful methods, e.g., to delete all triples (statements) with the resource as subject and a specified predicate.

Finally, the Ontologies Tool contains a standalone application called SchemaGen to generate the Java source code with constants from an ontology description.

Artifacts

Each architectural component described above comprises one or more artifacts as listed in Table 1.

Table 1: Artifacts of each architectural components

Architectural Component Artifact Artifact Type
Core org.apache.clerezza.rdf.core Jar and OSGi bundle
org.apache.clerezza.rdf.core.test Jar and OSGi bundle
Façades org.apache.clerezza.rdf.jena.facade Jar and OSGi bundle
Utilities org.apache.clerezza.rdf.utils Jar and OSGi bundle
Storage Providers org.apache.clerezza.rdf.sesame.storage Jar and OSGi bundle
org.apache.clerezza.rdf.jena.storage Jar and OSGi bundle
Parsing Providers org.apache.clerezza.rdf.jena.parser Jar and OSGi bundle
Serializing Providers org.apache.clerezza.rdf.jena.serializer Jar and OSGi bundle
Ontologies Tool org.apache.clerezza.rdf.tool.schemagen Executable jar

Building SCB

If you want to build SCB from its sources, you need JDK version 1.6 to compile. Maven version 2.0.9 has been used to build SCB.

Developing Applications using SCB

To develop an application on top of SCB, you need to have the Core and optionally Utilities and Façades.

Deploying an Application Requiring SCB

To deploy an application which uses SCB, Java(TM) SE Runtime Environment 6 or higher is needed. If the application does not use OSGi, then the SCB Core, the jars required for compilation, as well as the needed implementations (typically one provider for storage, serialization, and parsing) must be in the classpath. If the application uses OSGi, these jars are loaded as bundles, and to run in OSGi, Declarative Services must be enabled (see the documentation of your OSGi container).

The use of SCB in an OSGi environment has been tested with Apache Felix [7] and Eclipse Equinox [8].

Extending and Customizing SCB

Since SCB applies a Service Oriented Architecture approach, following components can be easily extended: Storage Providers, Parsing Providers, and Serializing Providers. Using OSGi, new bundles just need to implement the required service interfaces. To be located outside an OSGi environment, they should also provide files in "META-INF/services" for the services they expose. Consult the SCB JavaDoc for more information [9].

Developing with SCB

Once you have an overview over SCB the best resource is the javadoc API documentation. The following gives hints and describes best prectices on particular toopics.

When to use locks?

The MGraphs returned by the TcManager are lockable. A LockableMGraph has a getLock()-Method returning a ReadWriteLock. Write-Locks can be used if the applications logic requires it, i.e. when it wants to ensure that the Graph isn't modified by another thread while some triples are being added or removed bading on the presence or absence of other triples (note that its a violation of the open world assumption if the absence of some triples causes some actions to be done).
Readlocks must be used when multiple threads access an Graph and a method returning an Iterator is used. In this case a read-lock must be obtained before calling the method on Graph and released only after the last usage of the returned iterator. Failing to do so may result in ConcurrentModification when another threads modifies the Graph while we are iterating over it.
Currently (this might change infuture versions), it is also necessary to lock on the MGraphs against which a sparql select queries is directed. The lock on the MGraphs should be kept till iteration over the resultset is compleeted.
How do I create deadlock-safe locks?
There is no such thing, if you try to get a write-lock withing a read-locked section you'll have a deadlock.
What is the best way to Lock a ImmutableGraph in a service? (the developers, wich use the service know nothing about the lock and set own locks - so the service needs to check something)
The service documentation should be clear about which MGarphs the service might be writing to, so the caller can make sure it holds no read-lock on one of these Graph.
Where do I find additional documenation and source-code examples?
See the documentation and tutorials for the java.util.concurrent.locks package.

Offline-Site generator

The offline-site generator allows to generate representations of the resources described in the content graph for offline usage of for deployment on a traditional file-based webserver.
An offline site can be retrieved at /admin/offline/download with the following query parameters:
  • 1 baseUri: for all resources with a URI starting with this Uri creation of the specified target fomats is attempted
  • 1 targetUri: where base-uri appears in the representations it is replaces with target-uri
  • 0 or 1 rootLinkPrefix: prefix to be prepended to links with Uri reference starting with a slash
  • 1 - n formatExtension: the file- extensions represing the different formats to be produced, the extensions are added to the generated files where the file would not otherwise end with the extension.
Example: http://clerezza.apache.org/admin/offline/download?baseUri=http://clerezza.apache.org/&targetUri=http://incubator.apache.org/clerezza/test&formatExtension=xhtml&formatExtension=rdf&formatExtension=png&formatExtension=html&formatExtension=js&rootLinkPrefix=/clerezza/test