Redland librdf RDF API Library - Building and Installing from Source

1. Getting the sources

There are several ways to get the sources. The most stable and tested versions are the sources shipped with each release and these are recommended as the first place to start. For the latest developent sources, anonymous GIT access is available but this may require some configuring of developer tools that are not needed for the releases.

The source bundle and package files contain all the HTML files and documentation provided on the web site.

Redland (librdf) requires the Raptor and Rasqal to be already built and installed (libraries, headers and pkg-config .pc files). See the Raptor install document and Rasqal install document for the details of installing them.

1.1. Getting released sources

Every release comes with full sources and these are available from http://download.librdf.org/source/.

1.2. Getting the sources from GIT

  git clone git://github.com/dajobe/librdf.git
  cd librdf

At this stage, or after a git pull you will need to create the automake and autoconf derived files, as described below in Create the configure program by using the autogen.sh script.

Building Redland in this way requires some particular development tools not needed when building from snapshot releases - automake, autoconf, libtool and dependencies. The autogen.sh script looks for the newest versions of the auto* tools and checks that they meet the minimum versions.

2. Configuring and building

Redland uses the GNU automake and autoconf to handle system dependency checking. It is developed and built on x86 Linux and x86 OSX but is also tested on other systems occasionally.

configure tries very hard to find several programs and libraries that Redland might need. These include the storage modules (Berkeley/Sleepycat DB, MySQL, PostgreSQL 3store) and various others. A summary of the modular parts found is given at the end of the configure run. Several options to configure given below can be used to point to locations or names of dependencies that cannot be automatically determined.

2.1. Create configure program

If there is no configure program, you can create it by running the autogen.sh script, as long as you have the automake and autoconf tools. This is done by:

  ./autogen.sh

and you can also pass along arguments intended for configure (see below for what these are):

  ./autogen.sh --prefix=/usr/local/somewhere

On OSX you may have to explicitly set the LIBTOOLIZE variable for the libtoolize utility since on OSX libtoolize is a different program. The full path to the utility should be given:

  LIBTOOLIZE=/opt/local/bin/glibtoolize ./autogen.sh

Alternatively you can run the automake and autoconf programs by hand with:

  aclocal; autoheader; automake --add-missing; autoconf

The automake and autoconf tools have many different versions and autogen.sh enforces the minimums. At present development is being done with automake 1.10.2 (minimum version 1.7), autoconf 2.63 (minimum version 2.54) and libtool 2.2.6 (minimum version 2.2.0). These are only needed when compiling from GIT sources. autogen.sh enforces the requirements.

2.2. Options for configure

See also the generic GNU installation instructions in INSTALL for information about general options such as --prefix etc.

--disable-assert

Disable compiling run-time assertions. In maintainer mode, assertion failures are fatal.

--disable-assert-messages

Disable compiling run-time assertion failure messages. In maintainer mode, assertion failures are fatal after the assertion failure is reported.

--enable-debug

Enable debug messages (default not enabled). Maintainer mode automatically enables this.

--enable-digests=LIST

Does nothing - only builtin content digests are available now: MD5 and SHA1.

--enable-parsers=LIST

Select the list of RDF parsers to be included if the are availble. The valid list of RDF parsers is currently only raptor (the default) since the older repat parser has been removed. Raptor uses either of libxml2 (prefered) or expat. Redland requires the Raptor parser for other functionality, so it cannot be disabled.

--with-bdb= ROOT or no

Enable use of the Berkeley DB library installed at ROOT. That means ROOT/include must contain the BDB header db.h and ROOT/lib must contain the library libdb.a (or whatever shared library version/name your system uses).

If the value is no, the BDB backend store is disabled.

Berkeley DB (also known as Sleepycat DB) is distributed and supported by Oracle. Versions between 2.4 to 5.3 should work. Version 6 onwards will not be supported due to the change to the AGPL license, which would probably force Redland librdf to be AGPL licensed. Some systems do not come installed with a working Berkeley DB so on those systems, Redland will have no persistent storage unless BDB is built separately and enabled via this option.

Note: If you change installed versions of BDB then you will need to re-configure Redland carefully to let it discover the features of the newer BDB as follows:

  rm -f config.cache
  make clean
  ./configure ... # any configure arguments here

(plus you might need to use the dbX_upgrade utility to update the BDB database files to the formats supported by the newer version X - see the BDB documentation to find out if this is required.)

If the BerkeleyDB is installed in different places from ROOT/lib (library) and ROOT/include (header) or the library name is something that can't be worked out automatically, then you can use the next set of options to specify them.

If all of the BDB options are omitted, Redland will do a best efforts guess to find the newest BDB installation but this may not work for all configurations.

--with-bdb-lib=LIBDIR

--with-bdb-include=INCDIR

--with-bdb-name=NAME

Use Berkeley DB with the installed library in LIBDIR and the db.h header in INCDIR and the installed library called NAME like -lNAME. This is relative to LIBDIR. All of these options can be omitted and configure will try to find or guess the values from the system.

For example, to compile redland on OSX with fink might require a configure line something like this:

  ./configure --with-bdb-lib=/sw/lib \
              --with-bdb-include=/sw/include/db3

The name of the BDB library was correctly discovered for this configuration, as db-3.3.

If all of the BDB options are omitted, Redland will do a best efforts guess to find the newest BDB installation but this may not work for all configurations.

(At present, Redland knows of the default /sw Fink installation directory and will look there for BDB installs)

--with-mysql(=CONFIG|yes|no)

Enable use of the Redland MySQL 3.x, 4.x triple store backend using CONFIG for the mysql_config program. The default when either no argument is given, or --with-mysql alone, is to search for mysql_config on the search PATH. With --with-mysql=no, this store is disabled.

Versions 3.23.58 and 4.0.4 have been tested and work.

--with-openssl-digests

Enable the content digests provided by the OpenSSL libcrypto library (MD5, SHA1 and RIPEMD160) if the library is available. configure will automatically enable this unless disabled by setting this option to no.

--with-postgresql(=CONFIG|yes|no)

Enable use of the Redland PostgreSQL triple store backend using CONFIG for the pg_config program. The default when either no argument is given, or --with-postgresql alone, is to search for pg_config on the search PATH. With --with-postgresql=no, this store is disabled.

--with-sqlite=(yes|no|2|3)

Enable use of SQLite triple store backend with a particular version V2 or V3, an automatically chosen one with yes or disable it (with no).

--with-threestore(=CONFIG|yes|no)

Enable use of the AKT project 3store triple store backend using CONFIG for the 3store-config program. The default when either no argument is given, or --with-threestore alone, is to search for the 3store-config on the search PATH. With --with-threestore=no, this store is disabled.

--with-xml-parser=NAME

Pick an XML parser to use for Raptor - either libxml (default) or expat. If this option is not given, either will be used, with libxml preferred if both are present. One of these much be available for Raptor to parse XML syntaxes.

Raptor has been tested with various combinations of these libraries that are described further in the Raptor install documentation.

WARNING If the Sleepycat/Berkeley DB library is installed in a non-default directory, when the final linking occurs, the libraries may not be found at run time. To fix this you will need to use a system-specific method of passing this information to the run-time loader. On most systems you can set the LD_LIBRARY_PATH environment variable to include the directory where the librdf shared library is found. (On OSX this is DYLD_LIBRARY_PATH). You can also configure it via a system wide file - see the ld, ld.so, orld.so.1 or dyld manual pages for details.

2.3 Configuring

The default configuration will install into /usr/local:

   ./configure

To install into the standard Unix / Linux (and also Cygwin) system directory, use:

   ./configure --prefix=/usr

Append to the line any additional options you need like this:

   ./configure --prefix=/usr --with-bdb=/usr/local/berkeleydb

If you are having problems with configuring several times when adding or removing options, you may have to tidy up first with either of these:

   make clean
   rm -f config.cache

2.4 Compiling

Compile the library and the rdfproc utility with:

   make

Note: GNU make is probably required which may be called gmake or gnumake if your system has a different make available too.

2.5. Testing

You can build and run the built-in tests for Redland with:

    make check

which should emit lots of exciting test messages to the screen but conclude with something like:
All n tests passed
if everything works correctly.

(If you have got all the required subsidiary development tools, you can also do make distcheck which does a longer check that the distribution installation, configuring and building works. This does not perform any additional core testing).

2.6 Installing the library

To install the C library (static and shared typically) plus the interface header (.h) files do:

   make install

3. Using the library

Once the library has been configured and built, there are several C example programs that can be used. They are in the examples sub-directory and can be built with:

   cd examples

   make EXAMPLE
   # or on cygwin
   make EXAMPLE.exe

or to build all of them

   make examples

If no Berkeley DB was found by configure, some of the examples will fail since there is no on-disk storage system available. To change them to use the in-memory hashes, edit the lines reading something like

  storage=librdf_new_storage("hashes",
                             "test", 
                             "hash_type='bdb',dir='.'");

to read

  storage=librdf_new_storage("hashes", 
                             "test",
                             "hash_type='memory',dir='.'");

3.1 rdfproc

The rdfproc utility in the utils directory exercises the majority of the useful parts of the Redland API and can demonstrate many ways to store, search and manipulate the graph from C.

3.2. example1.c

example1 uses a RDF parser, if you have one available, to parse a URI of RDF/XML content, store it in multple Berkeley DB hashes on the disk and run queries against them. It takes two arguments, the first the URI of the RDF/XML content (or file:filename) and the second, optional one, is the name of the RDF parser to use.

3.3. example2.c

example2 does not use a RDF parser, but reads from a simple triple dump format and again stores the data on disk in multiple Berkeley DB hashes.

3.4. example3.c

example3 contains a 10 line main program that creates an RDF model, a statement, adds it to the model and stores it on disk.

3.5. example4.c

example4 contains an example of how to serialize an RDF model to a syntax.