Redland

Dave Beckett

 
 
Hosted by
Dreamhost

since 2005.

Data

DOAP
(See DOAP Project)

Raptor RDF Syntax Library - Release Notes

Raptor2 2.0.16 changes

Release because ... it's been a while

CVE-2017-18926 fixed in commit 590681e546cd9aa18d57dc2ea1858cb734a3863f

CVE-2020-25713 fixed in commit 4f5dbbffcc1c6cf0398bd03450453289a0979dea

Issues Fixed:

  • 0000600: Can't compile on Cygwin: sort_r.h "Cannot detect operating system"
  • 0000606: "JSON" format outputs invalid Unicode escapes
  • 0000617: heap buffer overflow in raptor_qname_format_as_xml
  • 0000618: heap buffer overflow in raptor_xml_writer_start_element_common
  • 0000650: Out of bounds read leads to segfault in raptor_xml_writer_start_element_common
  • 0000652: Manpage for the rapper utility describes the wrong Atom version

Parser changes

The Turtle parser was improved to process input in chunks, rather than all in one step (with a large memory buffer). This allows Turtle to read and parse files much larger than the system memory; it splits the inputs after the end of a turtle "sentence" that ends in a .. Includes memory usage minimizing with AVLTree pruning during serializing and addition of turtle serializer flush implementation. This improvment across multiple patches was contributed by Sebastian Freundt. Thanks Pull Request #37 Pull Request #38

Turtle parser improvements for Bison 3+ and removing deprecated or older style directives. Thanks to fix from Akim Demaille Pull Request #43

Turtle parser fixes for cleaning up va_arg on failure paths. [coverity CID 343351]

grddl parser: fix leak on failure paths of WWW model methods [coverity CID 343353]

Serializer changes

Added a serializer for the mKR (my Knowledge Representation) language with the name mkr Patch developed by Richard H. McCullough based on the Turtle serializer. Thanks. Pull Request #25

XML writer calculate max nspace declarations correctly. Fixes Issues #0000617 and #0000618

JSON writer changed to write UTF-8 for JSON literals Add RAPTOR_ESCAPED_WRITE_BITFLAG_UTF8 to RAPTOR_ESCAPED_WRITE_JSON_LITERAL Fixes Issue #0000606

rdfxmla serializer was fixed to handle leak of iter object on some failure pathes. [coverity CID 343354] [coverity CID 343358]

Fix for RSS 1.0 serializing default namespaces.

Writer changes

XML Writer: compare namespace declarations correctly. Applied a patch from 0001-CVE-2020-25713-raptor2-malformed-input-file-can-lead.patch.1 that fixes Issue #0000650 which overwrote heap during XML writing in parse type literal content. This was detected with clang asan. Thanks to fix from Michael Stahl / mst2.

Build and portability fixes

CMake fixes

  • Fix cmake builds for time.h check change, generated pc file. Thanks to fix from Artturin. Pull Request #52
  • Multiple cmake build fixes. Thanks to multiple changes from mathvich. Pull Request #50
  • Multiple fixes for Appveyor's Windows build using CMake Thanks to fix from 0u812.
  • Multiple fixes for Cmake build system to allow building from source including running flex, bison and running the tests. Thanks to fix from 0u812.
  • Made Raptor build again with CMake under Windows via CI as a service Appveyor. The resulting binaries and libraries were not tested.

Remove and fix obsolete autoconf: AC_PROG_LEX AC_HEADER_TIME AC_HEADER_STDC.

Update bison check to find first new enough GNU bison

Fix configure.ac -Wimplicit-function-declaration in HAVE___FUNCTION__ test to provide Clang 16 support. Fix from Sam James, thanks.

Support newer ICU UC version (>=56) for newer NFC check interface unorm2_quickCheck()..

Several minor fixes and workarounds for clang ASAN warnings or false warnings - not able to understand raptor's reference model.

Updated fix-flex.pl script for newer flex naming conventions.

Portability fixes found with GCC 10 and newer compilers for wrong error message callbacks in turtle and grddl parsers

Later, updated to make bison 3.4 the minimum.

Update turtle parser and support scripts to work with Bison 3.0.5 Patch contributed by Adam Novak. Thanks. Pull Request #39

Fix GNUInstallDirs for install paths to be able to install libraptor2.a and raptor2.pc to a specific directory when cross-compiling. Thanks to fix from Eric Le Lay. Pull Request #36

Made some code changes to quiet GCC extra warnings.

Portability fixes for OpenBSD

Some portability for building under cygwin by Richard H. McCullough while developing the mkr serializer.

Portability fixes in the raptor2.spec file. Thanks to fix by philjohn

Enable pkg-config to work when cross-building such as with yocto. Thanks to fix by Andreas Müller.

Use only pkgconfig for libxslt and ICU UC. Patches from Hugh McMaster. Thanks.

Updated Travis configuration to try builds across a wider range of compilers and with clang ASAN enabled.

Other changes

Handle deleting empty avltree. Fixes invalid pointer reference reported by Li Qiang via email. Thanks.

Change rapper(1) utility to count in longs instead of ints. Because: billions and billions of triples. Thanks to patch from Jerven Bolleman.

Accept cygwin as a Linux-like. Fixes Issue #0000600.

Mention Atom 0.3 in docs and manual page Fixes Issue #0000652.

Check curl_easy_setopt() return values for errors In several places check for curl_easy_setopt() erorr returns and fail. [coverity CID 343360]

Fixed internal ICU string NFC check to convert to UTF-16 first before trying to do a NFC normalization check. This allows returning an error response.

raptor_uri_counted_filename_to_uri_string() fix to copy NUL when copying the file://code prefix. Coverity false positive since the following lines of code always add more characters and a NUL. "Fixes" [Coverity CID 353858]

Raptor2 2.0.15 changes

This release mainly made general bug fixes as well as several fixes to the Turtle / N-Triples family of parsers and serializers. It added utility functions for re-entrant sorting of objects and sequences and a few other useful methods.

Issues Fixed:

  • 0000574: Language tags with underscore
  • 0000575: Wrong API or wrong API documentation
  • 0000576: Wrong assert for a counted string being nul terminated
  • 0000577: iri parsing does not conform to REC-n-quads-20140225
  • 0000579: raptor_world_generate_bnodeid accepts world not parser
  • 0000581: My patch for parser and serializers count
  • 0000584: raptor fails to parse trig files with dos line endings

Parser changes

Fixed raptor_grddl_filter_triples() check for three URIs which has been broken since 2007 but just did more work. Found via gcc5 warning.

The TRiG parser now accepts \r newlines between the graph name and the following '{'. Fixes Issue #0000584

Turtle, TRiG, N-Triples and N-Quads parsers now check escapes correctly for URI strings. \t \b \n \r \f are forbidden. Check that raw ' ' or \u0020, \u003C or \u003E are also not accepted. Fixes Issue #0000577

N-Triples / N-Quads parsers nwo accept '_' as an alias for '-' in lang strings. '_' is not legal. Fixes Issue #0000574

Updated RDFa parser (librdfa) to support full URLs for typeof from upstream. Re-checked several tests which have been passing for some time but the expected output was not a correct conversion from the testsuite sparql.

Serializer changes

Turtle serializer now writes () instead of rdf:nil via Pull Request #16 from Richard H. McCullough. Thanks!

I/O-stream class changes

raptor_iostream_decimal_write() and raptor_iostream_hexadecimal_write now return non-0 if they do not succeed in writing successfully. Fixes Issue #0000575

Term class changes

raptor_new_term_from_blank() now also accepts "" as the same as a NULL blank node ID argument.

URI class changes

Fixed raptor_uri_counted_filename_to_uri_string() to use the passed in len, not look for end NUL char and count lengths. Fixes Issue #0000576.

raptor_new_uri_from_counted_string() no longer assumes a NUL terminated string. Remove a debug assert and use fwrite() to emit the counted URI string to the debug file handler.

Fixed raptor_uri_normalize_path() to check the size of output buffer is big enough.

Writer class changes

Fixed raptor_turtle_writer_quoted_counted_string() to use the passed in length, drop the strlen() call and use the length internally. Fixes Issue #0000576.

World class changes

Added raptor_world_get_parsers_count() and raptor_world_get_serializers_count() with patch from Victor Porton. Thanks! Fixes Issue #0000581.

Other Changes

Made raptor work with Travis CI again; have to download and install Bison 3.

Added raptor_sort_r() and raptor_sequence_sort_r() re-entrant sort utility methods based on the public domain sort_r code by Isaac Turner.

Ensure raptor_locator_format() always adds a NUL to terminate the output string.

Fix a -99 <= var <= 99 timezone interval check. Thanks to Richard Trieu for the report.

Improve configure warning check using AC_LANG_WERROR

Use C99 __FUNCTION__ replacing __func__ from C90.

Improve build-time flex and bison output to be more silent.

Added Intel C compiler (icc) support from Sebastian Freundt (hroptatyr on GitHub) - Thanks.

Raptor2 2.0.14 changes

Many code quality fixes from clang and coverity primarily in error path cleanups (out of memory cases) and dead code.

Parser changes

Turtle 1.1 parser: fixes about triple quoting alignment with SPARQL from the www-rdf-comments list post REC report of problem and expected answer.

Other changes

Fixed RDFa 1.0 and RDFa 1.1 test suites to properly report failures.

Updated the fix-flex and fix-bison scripts to remove dead code that clang and coverity complain about.

Raptor2 2.0.13 changes

Issues Fixed:

  • 0000562: N-Quads parsing of URIs with ' fails

Parser changes

The N-Triples / N-Quads parser was updated to allow ' in URIs. Fixes Issue #0000562.

Configuration changes

Use the discovery of the xml2-config(1) / curl-config(1) programs as test for presence of the libraries. This fixes the issue where that after configuring, curl was present but was not used for WWW fetching. This is seen when the output of curl-config --cflags is empty / whitespace.

Updated to use the AC_CHECK_PROGS macro to find jing since AC_CHECK_PROG doesn't default to setting it as found when present!

Internal changes

The Turtle and parsedate parsers and the turtle lexer were updated to use and require Bison 3.0.0.

The JSON and turtle writers were updated to save several expensive strlen() calls.

Raptor2 2.0.12 changes

Parser changes

The N-Triples / N-Quads parser was updated to pass all the two test suites for the updated specifications:

The main fixes were:

  • allow \b and \f escapes (from Turtle, SPARQL)
  • fix the allowed blank node identifier characters
  • require all URIs to be absolute
  • handle a single short document with no end of line
  • support NULs inside literals
  • handle unescaped newlines inside literals

Term class changes

Added raptor_new_term_from_counted_string() to create a term from a Turtle or N-Triples string syntax. This is the opposite of raptor_term_to_turtle_counted_string() and raptor_term_to_turtle_string() and also reads the strings created by raptor_term_to_counted_string() and raptor_term_to_string().

Raptor2 2.0.11 changes

Issues Fixed:

  • 0000556: Problem with relative URI in rdf:about, rdf:resource etc.
  • 0000554: RDF/XML serializer does not canonicalize attribute order in XMLLiterals

Parser changes

N-Triples parser was updated to support all of RDF 1.1 N-Triples - A line-based syntax for an RDF graph (aka N-Triples 2013) It now supports # comments after the end of lines, checks for illegal unicode escapes, checks for bad URIs and checks language tags. Also removed support for legacy N-Triples "foo"-LANGTOKEN. Many internal changes were also made.

RDF/XML parser fixed the XML Canonicalization so that it writes the XML attributes in sorted order. Fixes Issue #0000554.

URI class changes

Added raptor_uri_uri_string_is_absolute() to check if a URI is absolute.

QName class changes

Added raptor_qname_format_as_xml() to turn a qname into an XML declaration.

Configuration and build changes

Restructured code to build an internal librdfa convenience library (never installed) so that newer automake will stop moaning about subdir-objects.

configure now checks for clang to correctly find supported warnings.

Various portability and build fixes (Daniel Richard G)

Build fixes: make check will now work if jing is not installed (for feed XML generation validation).

Other changes

Fixed a few issues found with LLVM 3.4 (SVN) run with address sanitizing.

Raptor2 2.0.10 changes

Issues Fixed:

  • 0000532: configure.ac: required file `src/raptor_config.h.in' not found
  • 0000535: configure accepts bogus values for --with-xml2-config
  • 0000545: Reopen issue 503
  • 0000548: xml:lang inside rdf:parseType="Literal" are removed in parsed output

Parser changes

Turtle parser was updated to handle Turtle Terse RDF Triple Language W3C Candidate Recommendation 19 February 2013 (Turtle 2013) The main changes were to align with SPARQL-style blank node names, prefixes and local names.

N-Triples parser was updated to handle N-Triples - A line-based syntax for an RDF graph W3C Working Group Note 09 April 2013 (N-Triples 2013) The main changes were to allow \b \f, UTF-8 in strings and URIs and the SPARQL-style blank node names.

RDF/XML parser updated to understand the new RDF 1.1 datatypes: rdf:HTML and rdf:langString

Serializer changes

Updated N-Triples and Turtle serializers (via writers) for 2013 versions based on SPARQL 1.1 definitions. These new formats have incompatible changes.

Added functions for writing escaped Turtle / N-Triples terms:

int raptor_string_escaped_write(const unsigned char *string,
  size_t len, const char delim, unsigned int flags, raptor_iostream *iostr);

int raptor_term_escaped_write(const raptor_term *term, unsigned int flags,
  raptor_iostream* iostr);

int raptor_uri_escaped_write(raptor_uri* uri, raptor_uri* base_uri,
  unsigned int flags, raptor_iostream *iostr);

These uses the new new raptor_escaped_write_bitflags enum bitflag values for the flags argument.

Deprecated raptor_string_python_write() for raptor_string_escaped_write() with flags.

Configuration and build changes

configure now recommends flex 2.5.36

configure was updated to switch configuring libxml and libcurl to use the PKG_PROG_PKG_CONFIG and PKG_CHECK_MODULES macros. The preference remains to prefer looking for the xml2-config and curl-config programs before trying pkg-config unless --with-curl-config=no --with-xml2-config=no are used.

configure now dies if xml2-config or xslt-config point at non executable files. Patch from Michael Stahl - thanks. Fixes Issue #0000534

configure now adds curl or libxml pkg-config requires only if they were found via pkg-config; this prevents unnecessary dependency on pkg-config files if they are not needed.

Updated autogen.sh and code to handle variations of header macro. Fixes Issue #0000532

Raptor2 2.0.9 changes

Issues Fixed:

  • 0000499: Turtle parser fails to correctly parse valid syntax
  • 0000508: Raptor objects to possibly valid Turtle syntax
  • 0000520: compilation failure
  • 0000521: Python RDF 1.0.14.1 segfault when one tries to parse xhtml+rdf using the RDFa parser
  • 0000526: Document how to find gtkdocize in INSTALL.html

Parser Changes

RDFa parser: Handle non-namespaced elements without crash in RDFa. Fixes Issue #0000521

Turtle Parser: Added full support for Turtle Terse RDF Triple Language W3C Candidate Recommendation 19 February 2013

This includes accepting new keywords from SPARQL (BASE and PREFIX), triples forms and details of the characters allowed in URIs, Literals, Prefix Names, Local Names and Blank Node identifiers. Some existing documents that used \-escapes in URIs to encode characters not allowed in URIs, are now forbidden. For example ASCII 32 (0x20) - space, which should be written as %20.

Serializer changes

Turtle Serializer: Added full support for Turtle Terse RDF Triple Language W3C Candidate Recommendation 19 February 2013 which particular effects the encoding of URIs in the serializer.

Configuration and build changes

Added CMake build framework for building Raptor on Microsoft Windows. Removed the old win32/ area and win32_raptor_config.h references. Contributed by Daniel Richard G. for Teragram Inc.

Ensure that a small system BUFSIZ does not affect parser guessing Patch from Daniel Richard G for Teragram Inc.

Added a compile-time check for raptor_world_guess_parser_name() to ensure that RAPTOR_READ_BUFFER_SIZE is at least as large as FIRSTN, because otherwise the guesser sees fewer than FIRSTN bytes from the document. Patch from Daniel Richard G for Teragram Inc.

Update configure to allow xml2-config and curl-config to work for libxml and libcurl. It now report on the source of xml, curl libraries in the configuration status.

autogen.sh updates to abort the run if a configuring program fails, Generate NEWS with old timestamp if missing so automake can run.

Handle the libxml2 ret->checked field not being present in older libxml2 versions. The entities checked field was added 2006-10-10 in libxml2 GIT commit a37a6ad91a61d168ecc4b29263def3363fff4da6and released in libxml2 2.6.27 on 2006-10-25. Fixes Issue #0000520

Other changes

raptor_www_set_ssl_verify_options() now supports the Curl 7.28.1 removal of CURLOPT_SSL_VERIFYHOST with a value of 2. Now verify_host non-0 means to verify CN, 0 means to not verify.

Remove -m MODE from the rapper(1) help message which was removed in commit f94fa561db05b21132b14a2b72f05b77e666c252 on Wed Apr 28 21:31:54 2010 -0700 as part of the Raptor V2 work.

Raptor2 2.0.8 changes

Issues Fixed:

  • 0000381: Raptor incorrectly serializes turtle lists when list nodes are URIs.
  • 0000487: Does raptor_world_guess_parser_name [librdf_parser_guess_name2] ever return NULL?
  • 0000505: Parsing certain escaped unicode strings in Turtle cases an error
  • 0000507: Turtle parse error causes fatal error

Parser class changes

Guessing a parser with raptor_world_guess_parser_name() now returns NULL failure when the guess is very poor, rather just return the first bad result. Fixes Issue #0000487 which was reported in Redland librdf but is implemented here.

Guess parser: now returns error file and line location information from the guessed parser.

RDFa parser: Updated librdfa to the latest GIT supporting RDFa 1.1 with 30 tests still not passing - mostly issues in the core librdfa. Made several resilience and crash fixes. Updated the RDFa 1.0 test suite to latest tests and made fixes. The RDFa parser now accepts aliases 'rdfa10', 'rdfa11' and the default 'rdfa' is RDFa 1.1

RSS tag soup parser: Use time() when gettimeofday() is not available. Fix several reference leaks.

RDF/XML parser: The range of RDF/XML entity recognizing heuristic was broadened to allow recognition of documents produced by Stanford's Protege software (and possibly others). Fixed some parser memory leaks / double frees.

Turtle parser: Do not report multiple errors for Turtle string decoding problems. When a Turtle qname cannot be found, a fatal error is no longer generated but a regular error message giving information on the qname that failed (usually due to unknown prefix). Fixes Issue #0000507

SAX2 API changes

raptor_sax2_inscope_xml_language() will now return "" for explicit no language (xml:lang="") as well as NULL for undefined language (no xml:lang present).

Serializer class changes

Turtle: Do not emit a Turtle (...) collection if the list item is a URI. This fixes Issue #0000381.

Term class changes

Added raptor_term_to_turtle_counted_string() counted string form of raptor_term_to_turtle_string.

Unicode class changes

raptor_unicode_utf8_string_put_char() and raptor_unicode_utf8_string_get_char() now allow reading / writing U+D800 to U+DFFF (UTF-16 surrogates). rather than returning failure. BOMs remain forbidden - definitely not UTF-8. Fixes Issue #0000505

URI class changes

Fixed URI resolving with reference (relative) URIs that are bare queries like '?y' with no path. Now matches the specification RFC3986 section 5.4.1 Normal Examples.

Added raptor_new_uri_relative_to_base_counted() to construct URI relative to current base.

Added raptor_uri_to_turtle_counted_string() counted string form of raptor_uri_to_turtle_string.

WWW class changes

The Curl WWW implementation now interprets Content-Locationcode> header as absolute or relative URI.

Configuration and build changes

All configure --with-foo options now handle --without-foo (and --with-foo=no) to disable attempting to find the value in the PATH. This can be used with --with-xml2-config, --with-xslt-config, --with-curl-config, --with-icu-config and --with-libwww-config where the value 'no' or --without-foo can be used to disable it and prevent automatic searches for the config script in the PATH.

The -DRAPTOR_DEBUG and -DMAINTAINER_MODE flags are now written to the config header instead of added to CPPFLAGS.

The -g flag is no longer added to CFLAGS or CPPFLAGS with --enable-debug.

The existing --disable-release (default) now correctly removes -O options in flag variables for the maintainer.

autogen.sh now looks for the environment variable NOCONFIGURE to prevent it running configure at the end of the auto generation run.

The configure vsnprintf() check was made more comprehensive.

Other changes

Multiple portability improvements to vsnprintf code and macros. Also fixed uninitialized variable problems in non-c99 variant of raptor_vsnprintf2(). Thanks to John Emmas for reporting.

Multiple portability fixes for building out of the source tree. Out of source tree 'make check' and 'make dist' should both work. Thanks to Daniel Richard G. for the patches.

Multiple portability fixes for building on old Solaris versions. Thanks to Daniel Richard G. for the patches.

Multiple portability fixes for building on Windows including strcasecmp(), windows headers, configuration fixes, parsedate code, URI test builds, vsnprintf building. Thanks to Daniel Richard G. for the patches.

Updated the example rdfguess to accept a file called or no arguments to read from stdin.

Raptor2 2.0.7 changes

CVE-2012-0037 fixed

Issues Fixed:

  • 0000448: Turtle parser does not return error status from turtle_parse_chunk()
  • 0000469: Allow rapper to bypass server SSL certs checks in libcurl

Removed Expat support since expat has not had a release in years and libxml2 works well. This allows some code simplification. Updated configure so that if Raptor is configured with no parser that requires an XML parser, libxml2 will not be required.

Removed internal Unicode NFC checking code used for checking RDF/XML literals conformance which was expensive to check and a large of compiled-in static dataset that was rather out of date. Replaced with optional compiled use of ICU. If ICU is not explicitly configured, no literal checking is done.

Options changes

Added new options:

RAPTOR_OPTION_NO_FILE
Deny file requests during parsing. Enabled by default.
RAPTOR_OPTION_LOAD_EXTERNAL_ENTITIES
Allow loading of XML external entity loading. Disabled by default.
RAPTOR_OPTION_WWW_SSL_VERIFY_PEER
Controls verifying an SSL peer during parsing / WWW. Takes an integer value: non-0 to verify peer SSL certificate (default 1).
RAPTOR_OPTION_WWW_SSL_VERIFY_HOST
Controls verifying an SSL host during parsing / WWW. Takes an integer value: 0 none, 1 CN match, 2 host match (default). Other values are ignored.

Parser class changes

The RDF/XML, RSS Tag Soup and RDFa parsers now pass on network, file and entity loading parser options to the internal SAX2 to enable enforcing of network, file and entity loading policy.

RDF/JSON parser handles an API change between YAJL V1 and V2.

Turtle parser now returns parser errors to raptor_parse_chunk(). Fixes Issue #0000488

SAX2 class changes

Added raptor_sax2_set_uri_filter() to set a URI filter for any SAX2 calls that do internal lookups of URIs.

Control file and network loading inside SAX2. Option RAPTOR_OPTION_LOAD_EXTERNAL_ENTITIES now enforces loading external XML entities and is by default enabled. If enabled, RAPTOR_OPTION_NO_FILE and RAPTOR_OPTION_NO_NET are also checked. All URIs loaded are also passed through any URI filter, if set by raptor_sax2_set_uri_filter().

URI class changes

Added new URI constructor raptor_new_uri_from_uri_or_file_string() to build a URI from a URI string or a filename string, normalizing the result to a file: URI.

Added raptor_uri_uri_string_to_counted_filename_fragment() to turn a URI string to a filename and URI fragment along with string output string counts.

Added utility methods for checking if a file: URI or filename is a file that exists: raptor_uri_file_exists() and raptor_uri_filename_exists()

WWW class changes

Added raptor_www_set_ssl_verify_options() to set SSL verify options. Fixes Issue# 0000469.

The raptor_www_fetch() call now returns the status from any URI filter that returns a non-0 response.

Build changes

Added --with-icu-config=PATH option to enable use of ICU for NFC checking.

Requires automake 1.11.2+ for -Wextra-portability

configure uses AM_PROG_AR to make automake -Wextra-portability happy.

Raptor2 2.0.6 changes

Fixed expat support which was broken in 2.0.5

Handle libCurl SSL options before 7.16.4 (that's 2007) since old libcurl is still around on Enterprise (that means old) linux systems.

Sequence Changes

Added utility functions to swap elements, reverse the sequence and generate permuations:

    int raptor_sequence_swap(raptor_sequence* seq,
      int i, int j);

    int raptor_sequence_reverse(raptor_sequence* seq,
      int start_index, int length);

    int raptor_sequence_next_permutation(raptor_sequence *seq,
      raptor_data_compare_handler compare);

Raptor2 2.0.5 changes

Issues Fixed:

  • 0000465: c99 snprintf usage
  • 0000476: Add Format URIs to raptor_syntax_descriptions
  • 0000479: raptor 2.0.4 : -i nquads fails to treat the context term as optional bugs.librdf.org
  • 0000481: Invalid unicode characters cause raptor to emit infinite output while converting n-quads to n-triples

Parser class changes

Updated all parser URIs to use the W3C Format URIs as the primary URIs. Existing URIs become aliases. (Nicholas J Humfrey)

GRDDL parser: Correctly set the base_uri when resolving the sheet URI.

N-Quads parser: Make context optional. (Lauri Aalto) Fixes Issue #0000479.
When guessing formats, make N-Quads always beat N-Triples since since now all ntriples parse fine with the nquads parser. (Lauri Aalto)

RDFA parser: fix when building with 64-bit systems to prevent value truncation.

Turtle parser: now uses the official text/turtle mime type in the syntax recognizing code.

Serializer class changes

Updated all serializer URIs to use the W3C Format URIs as the primary URIs. Existing URIs become aliases. (Nicholas J Humfrey)

Turtle serializer now uses the official text/turtle mime type in the syntax recognizing code. Do not generate infinite output when the input Unicode UTF-8 encoding is bad. Fixes Issue #000481.

Unicode class changes

Added new utility functions for calculating number of Unicode characters in a UTF-8 string raptor_unicode_utf8_strlen() and getting a subset of a UTF-8 string raptor_unicode_utf8_substr():

    int raptor_unicode_utf8_strlen(const unsigned char *string,
        size_t length);

    size_t raptor_unicode_utf8_substr(
        unsigned char* dest, size_t* dest_length_p,
        const unsigned char* src, size_t src_length,
        int startingLoc, int length);

URI class changes

raptor_uri_string_to_relative_uri_string() now compares URI paths not files. (Joe Presbrey) Fixes Issue #0000472

Build changes

The library no longer needs trunc(), lround(), round() or anything else from libm. Removed the checks for these functions from configure.

configure no longer enables debug messages by default for --enable-maintainer-mode but now requires the --enable-debug option to be given.

Other changes

Records GIT version in the version string when building from GIT sources with --enable-maintainer-mode. This makes it clearer when an non-released version is being used.

Added internal raptor_format_integer() which enabled the removal of all internal use of snprintf().

Added raptor_vsnprintf2() deprecating raptor_vsnprintf() which does not actually have the vsnprintf() calling contention. Added raptor_snprintf() with snprintf() calling convention. Added raptor_vasprintf() with vasprintf() (GNU) calling convention.

    int raptor_vsnprintf2(char *buffer, size_t size, 
			  const char *format, va_list arguments);
    int raptor_snprintf(char *buffer, size_t size, const char *format, ...);

    int raptor_vasprintf(char **ret, const char *format, va_list arguments);

raptor_locator_format() now picks a large enough buffer size if snprintf() is likely not portable, when HAVE_C99_VSNPRINTF is not defined. Fixes Issue #0000465

Internal code style changes for how allocation is done and casting with new macros.

RAPTOR_GOOD_CAST: code checks or logic ensures cast will not truncate

RAPTOR_BAD_CAST: value may be truncated; may require API change/break. Might be unrealistic e.g. a >4G error message, qname prefix.

Some good uses:

  • narrowing a known, checked unicode char to a U16

Some 'bad' uses:

  • only handing error messages, literal language, qname prefixes of a max len constrained by int
  • passing in data to libxml constrained by int max len
  • locator column field constrained to int size

Some bad uses:

  • iostream read_bytes and write-bytes methods return int but could easily return a lot more in the size_t range (compare to fread). API change needed.
  • locator byte field constrained to int size. should be size_t
  • raptor_nfc_check returns int offset into a buffer that could be larger
  • raptor_ntriples_parser_context changed line_length and offset to size_t
  • raptor_turtle_parser changed buffer_length to size_t

Updated code to use size_t for sizes such as those from strlen() and ptrdiff_t for pointer differences so that on 64-bit architectures, values are not potentially truncated to size of int.

Raptor2 2.0.4 changes

Issues Fixed:

  • 0000308: undefined reference to round and trunc while cross compiling for mipsel
  • 0000449: ntriples serializer and parser inconsistent w.r.t. _ in name tokens
  • 0000451: Incorrect qname definition in TRiG parser
  • 0000455: Incorrect AVL tree operations. [ with fix ]
  • 0000456: raptor-2.0.3 fails to build against yajl-2.0.2 API
  • 0000457: raptor-2.0.3 fails to build with curl-7.21.7

Parser class changes

The raptor_option enumeration gains values for setting SSL client side certificates: RAPTOR_OPTION_WWW_CERT_FILENAME for the certificate filename, RAPTOR_OPTION_WWW_CERT_TYPE for the certificate type and RAPTOR_OPTION_WWW_CERT_PASSPHRASE for the certificate passphrase.

raptor_parser_parse_uri_with_connection() (which is called by raptor_parse_uri()) now uses raptor_www_set_ssl_cert_options() to turn the parser options above into settings on the WWW object.

RDF/JSON parser: Gains support for building with YAJL V2. Note that YAJL V1 and V2 both install the same library name 'libyajl' even though they have different ABI and APIs.

TRiG parser: Fixed to support the legal uri : { ... } syntax naming a graph where the ':' is optional.

Turtle parser: Modified to not include the internal input() function in the lexer which is never needed.

Serializer class changes

N-Triples and Turtle serializers: Now use raptor_bnodeid_ntriples_write() to always write legal blank node IDs.

WWW class changes

Added support for raptor_www to handle setting SSL client certificate options during WWW retrieval.

Added raptor_www_set_ssl_cert_options() method to set the SSL client certificates on a WWW object.

Other changes

Make Raptor build against Curl 7.21.7 which removed a header file that was made an empty file in the libcurl source code on April 26 2004 around version 7.12.0 which is now the minimum version Raptor supports..

Fixed an AVL Tree issue during deletion that messes up some pointers. Patch from 'v-for-vandal'. Thanks.

Added a utility function raptor_bnodeid_ntriples_write() to write a N-Triples blank node ID in legal form, replacing any letters not in the allowed set.

Expanded GCC warnings and corrected a few internal uses of int when size_t was meant. Others remain.

Reduced stack use of raptor_www_file_handle_fetch() and raptor_parser_parse_iostream() by moving I/O buffer to the www or parser objects respectively.

Raptor2 2.0.3 changes

The main change is to add the new main header file raptor2.h. The new header has been added to allow applications to be sure they have got the raptor V2 header (with #include <raptor2.h> and not the raptor V1 header file (#include <raptor.h>). The raptor.h header will NOT be removed until the next major raptor release (V3).

Turtle / TRiG parser changes

Alter the parser to not use a large token stack when parsing TRiG graphs, enabling the parser to handle much larger files. The parser still gathers all input into a single memory segment so the maximum amount of input is probably memory size/3.

TRiG parser now allows a QName for the graph name.

Attempted to make the parser handle streaming lexing and parsing but only partially successful; bison could be made to stream parse but flex could not be made to stream lex. This meant it could not be convinced to return a "need more input" response at the end of a chunk of data and continue lexing later.

Fix value ownership fix graph name. Fixes Issue #0000437

Other Changes

Raptor was ran through the LLVM clang static code analyzer and several issues fixed, mostly in unused variables and in unlikely error recovery conditions. Some actual potential crash bugs were found:

  1. The workaround for ancient libxml2 error handling in raptor_libxml_xmlStructuredError_handler_parsing() was broken for some time, so that probably means nobody uses old libxml2.
  2. librdfa URI resolving in rdfa_resolve_uri() used unallocated memory in some relative URI cases.

Bison 2.4 is required to build Raptor from GIT source with no pregenerated files.

Raptor2 2.0.2 changes

Fixed a too strict version checking bug in raptor_new_world()

Raptor 2.0.1 Changes

DO NOT USE. Use 2.0.2 which fixes a too strict version checking bug in raptor_new_world()

Parser class changes

The internal librdfa parser that uses Raptor internals now exports symbols prefixed with raptor_librdfa so that Raptor can be linked with the standard librdfa in applications without symbol clashes. Fixes Issue #0000416

Serializer class changes

The N-Quads serializer was fixed to output the graph name in the fourth column, instead of repeating the object.

Internal changes

Replaced all internal fatal errors that went to an abort() with regular logged error.

Raptor 2.0.0 Changes

Raptor V2 final release. See the release notes for 1.9.0 and 1.9.1 for the major changes in Raptor V2.

API changes since 1.9.1 beta

Added raptor_syntax_description_validate() to public API to validate and compute counts for syntax descriptions. Primarily this is for use by Rasqal.

The raptor_syntax_description now has an array of URI strings instead of a single one, to allow URI aliasing. This is primarily for use by Rasqal.

Fixed guessing by URI in raptor_world_guess_parser_name() which was not working.

Fixed Issues:

  • 0000405: RDFa parser does not handle base href with single quoted value

Raptor 1.9.1 Changes

1.9.1 is the second beta release of Raptor 2 and intended to be the last release before 2.0.0.

Raptor V1 to V2 upgrading changes

Improved the upgrade-script.pl raptor V1 to V2 upgrade script to handle changes to enums and macros as well as handle statement field renaming. Updated the generated documentation to also include enum and macro changes.

Added defensive code in each constructor to check the passed-in raptor world object is a valid Raptor V2 world rather than V1. It generates a warning once per run of the program.

The raptor_new_world() function is now a macro that calls an internal function raptor_check_world_internal() with a version macro argument to detect when Raptor V2 is invoked with an inconsistent header and library.

AVLTree class changes

Free the iterator at the end of debug method raptor_avltree_print().

I/O Stream class changes

Fix major copying too much error in the internal raptor_read_string_iostream_read_bytes() memcpy that could cause crashes - the function always copied the maximum amount the user requested in the read method even if the string was smaller.

Parser class changes

JSON parser: Made relative URIs option work.

N-Quads parser: Added. The serializer already existed.

librdfa parser: Updated to latest librdfa GIT 1a1a08c790b7649a7f6c12fb9e40c0d3dbe70481

Serializer class changes

Fix raptor_serializer_start_to_iostream() to prevent crash when this method is used more than once on the same serializer.

RSS/Atom serializer: Fix crashes with feeds containing entries with blank node items.

Stringbuffer class changes

Added utility methods raptor_stringbuffer_append_uri_escaped_counted_string() and raptor_stringbuffer_append_hexadecimal().

Term class changes

Deleted raptor_term_as_counted_string() and raptor_term_as_string(). Since 1.9.0 was an unstable API, this is allowed.

Added raptor_term_to_counted_string() replacing deleted raptor_term_as_counted_string().

Added raptor_term_to_string() replacing deleted raptor_term_as_string().

URI class changes

Added convenience methods for constructing URI terms from strings: raptor_new_term_from_counted_uri_string() and raptor_new_term_from_uri_string().

Use string lengths in equals and comparisons for faster checking. Use memcmp() instead of strncmp() when lengths are known and the same.

WWW class changes

The CURL WWW module now looks for and uses the HTTP response Content-Location: header to get the base URI for content retrievals. This allows these base URIs to be used in parsing content retrived from URIs. Fixes Issue #0000402

Raptor 1.9.0 Changes

1.9.0 is the first beta release of Raptor 2. There may be changes and additional releases before version 2.0.0 with the final stable API.

This is a major update and cleanup to the Raptor API and ABI with additions, removals and changes. There are also major restructurings of the internal code and cleanups. There are also some new features in the form of additional APIs, new serializers and parsers.

The details of the additions, removals and changes of functions, structures, typedefs and enums are listed in the Raptor 2 API reference manual section on Changes between raptor 1.4.21 and 1.9.0. The upgrading document explains how to upgrade existing code that uses the V1 APIs to use the new APIs.

The major changes in this release are:

  • Removed all deprecated functions and typedefs.
  • Renamed all functions to the standard raptor_class_method form.
  • All constructors take a raptor_world argument.
  • URIs are interned and there is no longer a swappable implementation.
  • Statement is now an array of 3-4 RDF Terms to support triples and quads.
  • World object owns logging, blank node ID generation and describing syntaxes.
  • Features are now called options and have typed values.
  • GRDDL parser now saves and restores shared libxslt state.
  • Added serializers for HTML 'html' and N-Quads 'nquads'.
  • Added parser 'json' for JSON-Resource centric and JSON-Triples.
  • Switched to GIT version control hosted by GitHub.
  • Added memory-based AVL-Tree to the public API.

Fixed Issues:

  • 0000357: raptor_term_as_string does not return N-Triples escaped string, although API says so
  • 0000361: HTML Table serializer
  • 0000369: Unescaped quotes in long literals (Turtle/N3) cause parser failure if at the end of the literal
  • 0000370: Odd behaviour regarding the parsing of Trig files
  • 0000373: Remove deprecated GTK symbols
  • 0000379: raptor libxslt security policy conflicts with apps that also use libxslt and set security policy

General changes

Renamed the library name and package tarball name to a new separate names (libraptor2, raptor2-X.tar.gz) so they can be distinguished and installed into the same system as raptor 1 without file clashes. Note: This does not apply to the rapper(1) utility and it's manual page which are in both packages. Packagers should stop packaging the rapper(1) binary and the rapper manual page from raptor 1, in preference to the raptor 2 version.

Nicholas J Humfrey is a new committer.

A utility script docs/upgrade-script.pl is automatically generated from the changes data to aid migrating from Raptor 1 to Raptor 2 APIs. If it is not a simple rename, the script annotates the change as a comment near the code.

Raptor's Statement, Term and URI classes can fully replace librdf's librdf_statement, librdf_node and librdf_uri classes. This has been tested with GIT versions when both Rasqal and librdf are configured with --with-raptor2. When raptor 2 is stable, Rasqal and librdf will default to use these..

Documentation changes

The API reference manual now includes an automatically generated reference section on the functions and types that were added, removed and changed between raptor 1 and raptor 2 APIs. It also includes an automatically generated section on the parser and serializers with their names, mime types and Q values.

The tutorial examples were updated to the Raptor 2 APIs.

NOTE: At this time, the descriptive parts of the tutorial have NOT been updated to use the Raptor 2 API.

World class changes

A log handler function for receiving warnings and errors (fatal or other) are now all sent to a single log handler of type raptor_log_handler registered to the world class with raptor_world_set_log_handler(), rather than handlers set for each parser, sax2, serializer or www instance. Several constructors lose error handler and error data parameters consequently.

The enumerating approach for listing the known parsers and serializers and writing into variables passed in, was replaced with methods that return pointers to static raptor_syntax_description objects to allow more parameters to be returned. This allowed the removal of multiple older calls that kept having to gain new return arguments. The description also returns a list of (mime type, qname) pairs using a new typedef raptor_type_q. It includes a set of bitflags for some simple information such as RAPTOR_SYNTAX_NEED_BASE_URI to describe when a syntax requires a base URI passed in, otherwise is not needed or optional.

raptor_world_set_libxml_flags() is replaced with raptor_world_set_flag() which are simple flags that require no initialisation of the world since they are used before it is opened. This means that it cannot use the Options module for world flags. New flag RAPTOR_WORLD_FLAGS_URI_INTERNING allows disabling of the URI interning using an AVL-Tree which can use up memory.

raptor_world_set_libxslt_security_preferences() and raptor_world_set_log_handler() gains an int return value to report failure if world is already opened.

Moved the generation of blank node IDs from the parsers (many objects) to the world class (1 object). Cannot think of good reasons why the blank node ID generation policy needs to be different per-parser. Enumeration raptor_genid_type was deleted since only the blank node type was ever used.

AVL Tree changes

The internal memory-based AVL Tree implementation is exposed as a public API.

Features changes

Replaced with raptor_options

Identifier class changes

Replaced with Term class.

raptor_identifier_type enum was removed and replaced with enum raptor_term_type and a smaller set of 3 possible term types. The enum values match the int values used in librdf_node_type.

I/O Stream class changes

Constructors all gain a world argument.

Some methods now return an int error status such as raptor_iostream_write_end()

Many classes iostream writing methods and methods of the iostream class itself were changed to follow the standard raptor_CLASS_write naming convention and to put the iostream argument at the end.

Deprecated typedef raptor_iostream_handler was removed

Deprecated functions were removed: raptor_new_iostream_from_handler(), raptor_iostream_get_bytes_written_count() and raptor_iostream_write_string_turtle()

Locator changes

Renamed some methods to follow the raptor_locator_method standard.

Namespaces and namespace stack classes changes

Constructors all gain a world argument and lose error handler, error data arguments.

Options

A new name for what were called features in Raptor 1 and this was expanded to allow use of values types string, integer and boolean. Options can be set on multiple classes such as parser, serializer but are centrally described using the new raptor_option_description structure.

The option get / set operations were reduced from multiple methods per class for getting / setting an int, int / string to just one get and one set, with optional parameters. It can also handle adding new types later, which will be passed in as strings and converted internally.

The options code introduces a new enum raptor_domain which essentially is for describing a class such as parser. For example options listed for domain RAPTOR_DOMAIN_PARSER will be usable with the parser option get and set methods raptor_parser_get_option(parser, ...) and raptor_parser_set_option(parser, ...).

Parser class changes

raptor_parser_set_strict() was removed and replaced with a new parser option RAPTOR_OPTION_STRICT.

Parser object methods to return information about the parser class were removed since they can be done instead via fields of the static object returned from raptor_world_get_parser_description(). Deleted functions include raptor_parser_get_label(), raptor_parser_get_mime_type(), and raptor_parser_get_need_base_uri().

Parsing may generate start graph and end graph events for syntaxes that return graph names such as TRiG. These are returned by the handler of typedef raptor_graph_mark_handler set by new function raptor_parser_set_graph_mark_handler() which returns a start or end event of type enum raptor_graph_mark_flags.

raptor_parser_get_accept_header() fixed length of static string to ensure NUL is copied.

Many functions were renamed to match the raptor_parser_method naming standard. Some of these get a long longer, but are at least consistent. For example raptor_parse_uri() becomes raptor_parser_parse_uri().

Deprecated typedef raptor_ntriples_parser was removed.

Deprecated functions were removed: raptor_ntriples_term_as_string(), and raptor_ntriples_string_as_utf8_string()

The librdfa parser was updated with the latest updates to the RDFa syntax (first, rel/rev, whitespace, xml:lang) and fixes issues with non-terminated strings.

The GRDDL parser now saves and restores the libxslt global state (handlers, security preferences) so that it does not affect other applications in the same memory space. Part of this can be overridden with raptor_world_set_libxslt_security_preferences() which allows user code to disable raptor touching the security preferences entirely. Fixes Issue #0000379

The RDF/XML parser never generated special names for bag IDs (bagidNNN), so all support for that has been removed.

The RDF/XML parser uses the new reference-counted URIs and raptor terms to prevent a lot of malloc/free for temporary uses.

Removed RDF/XML parser feature RAPTOR_FEATURE_ASSUME_IS_RDF since it did nothing since at least 2003, maybe earlier.

Added a JSON parser (name 'json') that can read both the resource-centric and triples JSON RDF formats. Requires the YAJL JSON parser library. Contributed by Nicholas J Humfrey.

Moved the generation of blank node IDs from the parsers (many objects) to the world class (1 object). Cannot think of good reasons why the blank node ID generation policy needs to be different per-parser.

QName class changes

Constructors all gain a world argument and lose error handler, error data arguments.

Sequence changes

Use new generic raptor_data_free_handler, raptor_data_print_handler, raptor_data_context_free_handler, raptor_data_context_print_handler data-focused handlers which are shared with the AVL Tree.

Serializer class changes

Added raptor_serializer_flush() for user code to indicate to a serializer to flush state since the application knows a block has been ended, or it needs to save memory. No serializer currently implements this, but it is expected the abbreviated serializers that use a lot of state - Turtle, RDF/XML-Abbrev - would benefit from this.

Added an HTML Table serializer with name 'table' contributed by Nicholas J Humfrey.

Removed the obsolete 'simple' serializer from the source code. N-Triples is an appropriate simple (and well-defined) replacement syntax.

Statement class changes

This was made a more regular class that can be created, copied and freed using reference counting. The structure changed to be an array of 4 raptor terms rather than a complex union focused on the allowed RDF Statements. This change allows representing non-RDF triples as well as triples with named graphs (also known as quads).

raptor_statement_part_as_counted_string() turns into a method of the new Term class, as a "statement part" is now represented as a Term.

Term class changes

This new class with an open structure replaces raptor_identifier and was created to represent an RDF Term: literal, URI or blank node and is reference counted and dynamic. It is a union of fields for three types.

This class can fully replace librdf's librdf_node when both Rasqal and librdf are built with --with-raptor2. The semantics of comparison (ordering) and equality are the same.

Unicode changes

Several utility functions were renamed to match their functionality more accurately and be associated with the Unicode module set of functions. For example raptor_utf8_check() was renamed to raptor_unicode_check_utf8_string()

URI class changes

The whole concept of URI handlers was removed, there is no longer a way to replace the implementation of the URI class. The implementation built in now interns URIs string so that a single URIs for a string is only ever in memory once, but reference counted. This is what Redland librdf has done for many years, and librdf can fully use this.

Deprecated functions were removed: raptor_uri_is_file_uri().

The raptor_uri_resolve_uri_reference() now returns the length of the URI written into the buffer.

WWW class changes

Rename several of the functions to the standard constructor, destructor and method naming format.

XML Writer class changes

Constructors all gain a world argument and lose error handler, error data arguments.

Configuration changes

Added configure argument --with-yajl=DIR (or 'no') to set the prefix where YAJL libraries and headers are installed.

Raptor headers are now installed in a sub-directory of PREFIX/include.

raptor-config was removed. Building against raptor 2 should be done only using pkg-config with either --cflags or --libs arguments like this:

  cc -o prog prog.c `pkg-config raptor2 --cflags`  `pkg-config raptor2 --libs`

Internal changes

Code style: lots of changes mostly adding lots more whitespace such as to show control change (if, while) or early function exit (return).

All strcpy() with known lengths and strncpy() were replaced with memcpy() which is quicker and compilers optimize better.

Code style: use TYPE* var_name = RAPTOR_CALLOC(TYPE, 1, sizeof(*var_name)) rather than repeat the type name which can be error-prone.

Code style: put whitespace round assignment and operators i.e. use c = a + b rather than c=a+b.

Code style: removed some strdup()s.

Code style: try to remove un-necessary strlen() by tending to pass around a string with it's length, a counted string. This meant adding a few new constructors such as for the URI and Term classes.

Code style: methods called with a NULL object return failure and destructors called with a NULL object return silently rather than the former mix of crashing or abort()ing.

Code style: copy constructor is now called raptor_class_copy() rather than raptor_new_class_from_class() for brevity.

Code style: print methods return an int return status indicating success.

rapper Utility changes

Print out the option types next to the description in the -f help usage message.

Removed unused -a flag from code.

Did NOT rename -f option to match the renaming of features to options in the API. Not needed.

Removed long deprecated -n flag which was shortcut for selecting the N-Triples parser and can be done with -i ntriples

Removed -s/--scan flag since it is an option of a single parser (RDF/XML) which can be done with -f/--feature scanForRDF.

Other changes

Removed support for building raptor with dmalloc debugging (removing --with-dmalloc configure argument). Valgrind wins.

The example 'grapper' GNOME GUI utility for raptor was updated to the latest GTK APIs (2.5.0+ but still quite old) so that it builds with all the GTK 'disable deprecated' flags set. The UI is not quite correct though - the preferences menu is broken.

Switched to GIT version control, hosted by GitHub. The latest raptor 1 code is available on branch 'raptor1' as well as from the release tags for specific versions.

Raptor 1.4.21 Changes

This is a bug fix only release with no new features. New development has moved to raptor 2 where a planned ABI and API break is underway.

Fixed Issues:

  • 0000318: Cannot end a Turtle literal with \\ inside triple-quotes
  • 0000319: ntriples parser does not register that it accepts ntriples mime type - text/plain
  • 0000326: Turtle parser allows '.' in qnames which is not-to-spec
  • 0000331: Turtle long literals with raw newlines do not count line numbers correctly
  • 0000332: RDFXML parser finds duplicates and misbehaves (when it shouldn't)
  • 0000337: raptor/turtle outputs invalid qnames which cause syntax errors on parsing

Parser changes

N-Triples parser: Declare acceptance of text/plain mime type with q=0.1

RDFA parser (via librdfa): Updated to fix some buffer management problems when it was passed large blocks (4096 bytes or more), a few memory leaks and some other minor bugs.

RDF/XML parser: Properly reset the ID-checking set at the start of each parse.

Turtle parser: Allow \\ at the end of triple-quoted literals. Forbid '.'s in prefixes and qnames (follow specification). Properly count newlines inside the literals for error reporting.

Serializer changes

Turtle serializer: Forbid '.'s in prefixes and qnames (follow specification).

Other Changes

Updated configure and the build system to use silent rules for the maintainer (by default), or when --enable-silent-rules is passed to configure. This feature requires building with automake 1.11 which requires autoconf 2.62 or newer when building from GIT.

autogen.sh script was updated to enforce the autotools versions above.

Raptor 1.4.20 Changes

This is a bug fix only release with no new features. New development has moved to raptor 2 where a planned ABI and API break will happen. There may be preview releases of raptor 2 with 1.9.x numbering.

Fixed Issues:

  • 0000306: rapper doesn't handle datatype=""and xml:lang="" properly with RDFa
  • 0000307: configure fails at vnsprintf test when cross compiling
  • 0000310: Raptor does not like single character namespaces with RDFa
  • 0000312: Ununitialized pointer in example rdfserialize.c causes crash

Parser Changes

GRDDL parser: Fix XML parser context resource leak if raptor_grddl_fetch_uri() fails. Save and restore error handlers properly to prevent crashes when an error is reported during parsing.

RDFA parser (via librdfa): Update to latest librdfa GIT sources with head a438ce68a40e04b399ec2b2c613d0c867d9315c7
now moved to https://github.com/msporny/librdfa to fix handling single character namespaces (Issue #0000310), empty datatype attribute and empty xml:lang attributes (Issue #0000306)

Added three unapproved RDFa tests 0172, 0173 and 0174 to cover the fixes above.

Serializer Changes

Turtle serializer: Applied scalability patch from Chris Cannam. This switches the serializer to use a raptor_avltree instead of a raptor_sequence for the subject and blanks used with raptor_abbrev_node_lookup(). This fixes a performance problem in the serializing and moves lookups from O(N) to O(log N) - from list to balanced tree.

Other Changes

If cross compiling, check for vsnprintf() C99 compatible at runtime by setting define CHECK_VSNPRINTF_RUNTIME during configuration. Fixes Issue #0000307

Use calloc() for allocating a raptor_statement in rdfserializer.c example code to properly initialise state. Fixes Issue #0000312

Use AC_SYS_LARGEFILE to get large file IO checks which allows 32-bit systems to read multi-gigabyte files.

autogen.sh script fix for if test when uname is not in standard OSX dir.

Raptor 1.4.19 Changes

WARNING: FUTURE ABI and API CHANGES. The next release of raptor 1.4.x will include bug fixes only and no new features. New development will move to raptor 2 where a planned ABI and API break will happen. There may be preview releases of raptor 2 with 1.9.x numbering.

Fixed Issues:

  • 0000259: Fix NFC check for legal combiner sequence
  • 0000262: Error when raptor_new_uri() fails in Turtle parser
  • 0000263: Invalid turtle output syntax on empty integer/double/decimal literals
  • 0000266: Default/atom namespace in atom serializer output
  • 0000269: strstr is called in raptor_parse_chunk() on a buffer string, where it should be called on a null-terminating string.
  • 0000270: RSS serializer fixes for g++
  • 0000276: Fix raptor_sequence_set_at() when setting beyond end
  • 0000277: broken collection abbreviation in turtle serialization
  • 0000287: Fix raptor_sax2_parse_chunk() calling raptor_log_error_to_handlers() with expat
  • 0000288: raptor_get_feature function does not return feature value
  • 0000289: Fix RDFa parser problem when there is a subject and predicate specified on an element, but no child nodes for the object literal
  • 0000290: Fix performance problems when turtle parsing with lots of namespaces
  • 0000293: Fix RDF/XML Parser problem with legacy ordinal predicates
  • 0000296: Avoid calling xsltSetDefaultSecurityPrefs()
  • 0000299: Avoid calling xmlSetStructuredErrorFunc() and xmlSetGenericErrorFunc()
  • 0000303: rdfa parser does not parse content as RDFa which librdfa+expat alone handles

Parser Changes

raptor_get_feature() now returns the integer value rather than just 1 or 0.
Issue #0000288

Guess parser: return name of guessed parser not 'guess'.

N-Triples parser: Produce error messages when raptor_new_uri() fails.
Issue #0000262

RDFa parser: Fix problem when there is a subject and predicate specified on an element, but no child nodes for the object literal using latest librdfa GIT source with head 2ddcb3f9e010d0b3d9ee546e807539be5da1b14a
Issue #0000289

RSS tag soup parser: Huge internal changes:
Recording more atom core structures in triples (such as author, contributor - person) rather than only channels and items
Introduced a new 'blocks' concept to record single element structured items such as atom category, link and rss enclosure
Added itunes namespace and container.

RDF/XML Parser: Adjust predicate_type when removing ordinal identifier type from predicate.
Issue #0000293

Serializer Changes

Atom 1.0 serializer: Now tested and takes more care to try to generate valid Atom 1.0

Turtle serializer: Validate XSD integer, decimal and double literal output. Emit special short forms only if the whole literal value is consumed by strtol() (for integers) or strtod() (for decimals and doubles). Otherwise produce a warning and emit literal in the normal "value"^^<datatype_uri> format.
Issue #0000263
Fix broken collection abbreviation
Issue #0000277

RSS serializer: Fixes for g++
Issue #0000270
Added a new serializer feature RAPTOR_FEATURE_PREFIX_ELEMENTS (short name prefixElements) for atom and rss 1.0 serializers to decide whether core elements in the default namespace are declared with the prefix or without a prefix.
Removed generation of deprecated predicate ordinals of type RAPTOR_IDENTIFIER_TYPE_ORDINAL and replace with resource type URIs

XML Support Changes

Removed generic calls to xmlSetStructuredErrorFunc() and xmlSetGenericErrorFunc() which can be a problem when libxml is shared with other code in memory. They may be called optionally but will do a save/restore of the existing functions. This protection is enabled by the new API call raptor_set_libxml_flags() to set the flags from values in enum raptor_libxml_flags.
Issue #0000299

Use context-specific libslt security configuration to avoid calling generic call xsltSetDefaultSecurityPrefs() which can be a problem when libxslt is shared with other code in memory. Allow the user to set the policy for raptor globally with new API function raptor_set_libxslt_security_preferences().
Issue #0000296

Make libxml SAX2 structured errors register parser-specific handler function raptor_libxml_xmlStructuredErrorFunc() instead of libxml global structured error handler. The libxml flag method above can still enable registerding the global error handlers.

In raptor_sax2_parse_chunk() fixed calls to raptor_log_error_to_handlers() when built with expat.
Issue #0000287

Other Changes

Win32 portability fixes from Lou Sakey:

  • Handle absence of gettimeofday()
  • Call xmlCleanupParser() libxml call last to avoid an access violation on windows.
  • Windows vsnprintf() portability patch
  • raptor_sequence_set_at() fixed to maintain the design contract: provide "size" consecutive items in "sequence" starting from "start" even when setting items more than +1 offset beyond the end of sequence.
    Issue #0000276

rapper(1) utility changes: if counting, do not use a serializer at all.

Internal Changes

More internal changes to be more resiliant after allocation failure (Lauri Aalto)

Reorganised tests in source tree to pull out specific directories for RDF/XML, Turtle, etc.

Use a DJ Bernstein hash to replace a linked list for storing a stack of namespaces. This makes turtle parsing with lots of namespaces (100s) much faster. Based on the initial patch in the bug.
Issue #0000290

Use new internal raptor_memstr() function to compare a string against a buffer that may not be NUL terminated.
Issue #0000269

raptor_error_handlers: API structure gains world field. BINARY COMPATIBILITY BREAK: sizeof(raptor_error_handlers) changed. Source compatibility not broken.

raptor_identifier: API structure gains world field. BINARY COMPATIBILITY BREAK: sizeof(raptor_identifier) changed. Source compatibility not broken.

More fixes for compiling with C++

Move some more static data as constant to enable more efficient compilation - moves to data segment of object binary.

Use memmove for overlapping copy, not memcpy when doing relative URI resolving.

Raptor V2 Preparation Changes

Lots of internal changes were made by Lauri Aalto preparing for Raptor V2 to fully attach all static data and config to a new raptor_world object. A new static instance of this class is now used internally behind the existing V1 API and will be required to be constructed by the library user for the V2 API with a new constructor/destructor.

NOTE: The method names here are illustrative of the final V2 names but are not confirmed - These functions are not supported in the 1.4.x series. Some methods will still be altered for fields and (raptor_world) parameters. All constructors should have it as a parameter. All methods will not have them (so for example, all the URI methods named _v2 will lose the world parameter and just have the URI parameter - but that is also because the URI handler/context part will go into raptor_world in V2)

To use the unsupported and experimental V2 functions, define -DRAPTOR_V2_EXPERIMENTAL=1 when building with raptor.

Added raptor_world typedef.

Added raptor world class constructor raptor_world* raptor_new_world(void) and initializer: int raptor_world_open(raptor_world* world)

Added world class destructor: void raptor_free_world(raptor_world* world)

Added new V2 methods:

  void raptor_world_set_libxslt_security_preferences(raptor_world *world,
    void *security_preferences)

  void raptor_world_set_libxml_flags(raptor_world *world,  int flags)

  void raptor_error_handlers_init_v2(raptor_world* world,
    raptor_error_handlers* error_handlers);

Added V2 methods that are versions of existing methods, named with _v2 suffix:

  int raptor_parsers_enumerate_v2(raptor_world* world,
    const unsigned int counter, const char **name, const char **label)

  int raptor_syntax_name_check_v2(raptor_world* world, const char *name);

  void raptor_print_locator_v2(raptor_world* world, FILE *stream,
    raptor_locator* locator);

  const char *raptor_locator_uri_v2(raptor_world* world,
    raptor_locator *locator);

  int raptor_features_enumerate_v2(raptor_world* world,
    const raptor_feature feature, const char **name,
    raptor_uri **uri, const char **label);

  int raptor_serializers_enumerate_v2(raptor_world* world,
    const unsigned int counter, const char **name, const char **label,
    const char **mime_type, const unsigned char **uri_string);

  int raptor_serializer_syntax_name_check_v2(raptor_world* world,
    const char *name);

  int raptor_serializer_features_enumerate_v2(raptor_world* world,
    const raptor_feature feature, const char **name,  raptor_uri **uri,
    const char **label);

Added world pointer to raptor_identifier object

Added V2 identifier class constructor:

  raptor_identifier* raptor_new_identifier_v2(raptor_world* world,
    raptor_identifier_type type, raptor_uri *uri,
    raptor_uri_source uri_source, const unsigned char *id,
    const unsigned char *literal, raptor_uri *literal_datatype,
    const unsigned char *literal_language);

Added V2 parser class method:

  raptor_parser* raptor_new_parser_for_content_v2(raptor_world* world,
    raptor_uri *uri, const char *mime_type, const unsigned char *buffer,
    size_t len, const unsigned char *identifier)
  raptor_world* raptor_parser_get_world(raptor_parser* rdf_parser);

Added V2 serializer class constructor and method:

  raptor_serializer* raptor_new_serializer_v2(raptor_world* world,
    const char *name)
  raptor_world* raptor_serializer_get_world(raptor_serializer* rdf_serializer)

Added V2 statement class raptor_statement_v2 typdef for future replacing of raptor_statement

Added V2 statement class methods:

  void raptor_print_statement_v2(const raptor_statement_v2 * statement,
    FILE *stream);

  unsigned char* raptor_statement_part_as_counted_string_v2(raptor_world* world,
    const void *term, raptor_identifier_type type,
    raptor_uri* literal_datatype, const unsigned char *literal_language,
    size_t* len_p);

  unsigned char* raptor_statement_part_as_string_v2(raptor_world* world,
    const void *term, raptor_identifier_type type,
    raptor_uri* literal_datatype, const unsigned char *literal_language);  

  int raptor_statement_compare_v2(const raptor_statement_v2 *s1,
    const raptor_statement_v2 *s2);

Added V2 uri class methods:

  unsigned char* raptor_uri_as_counted_string(raptor_uri *uri, size_t* len_p);

  raptor_uri* raptor_new_uri_v2(raptor_world* world,
    const unsigned char *uri_string);

  raptor_uri* raptor_new_uri_from_uri_local_name_v2(raptor_world* world,
    raptor_uri *uri, const unsigned char *local_name);

  raptor_uri* raptor_new_uri_relative_to_base_v2(raptor_world* world,
    raptor_uri *base_uri, const unsigned char *uri_string);

  raptor_uri* raptor_new_uri_from_id_v2(raptor_world* world,
    raptor_uri *base_uri, const unsigned char *id);

  raptor_uri* raptor_new_uri_for_rdf_concept_v2(raptor_world* world,
    const char *name);

  void raptor_free_uri_v2(raptor_world* world, raptor_uri *uri);

  int raptor_uri_equals_v2(raptor_world* world, raptor_uri* uri1,
    raptor_uri* uri2);

  int raptor_uri_compare_v2(raptor_world* world, raptor_uri* uri1,
    raptor_uri* uri2);

  raptor_uri* raptor_uri_copy_v2(raptor_world* world, raptor_uri *uri);

  unsigned char* raptor_uri_as_string_v2(raptor_world* world, raptor_uri *uri);

  unsigned char* raptor_uri_as_counted_string_v2(raptor_world* world,
    raptor_uri *uri, size_t* len_p);

  raptor_uri* raptor_new_uri_for_xmlbase_v2(raptor_world* world,
    raptor_uri* old_uri);

  raptor_uri* raptor_new_uri_for_retrieval(raptor_uri* old_uri);

  raptor_uri* raptor_new_uri_for_retrieval_v2(raptor_world* world,
    raptor_uri* old_uri);

  raptor_uri* raptor_new_uri_for_xmlbase_v2(raptor_world* world,
    raptor_uri* old_uri);

  raptor_uri* raptor_new_uri_for_retrieval_v2(raptor_world* world,
    raptor_uri* old_uri);

  unsigned char* raptor_uri_to_relative_counted_uri_string_v2(raptor_world* world,
    raptor_uri *base_uri, raptor_uri *reference_uri, size_t *length_p);

  unsigned char* raptor_uri_to_relative_uri_string_v2(raptor_world* world,
    raptor_uri *base_uri,  raptor_uri *reference_uri);

  void raptor_uri_print_v2(raptor_world* world,
    const raptor_uri* uri, FILE *stream);

  unsigned char* raptor_uri_to_counted_string_v2(raptor_world* world,
    raptor_uri *uri, size_t *len_p);

  void raptor_uri_set_handler_v2(raptor_world* world,
    const raptor_uri_handler *handler, void *context);

  void raptor_uri_get_handler_v2(raptor_world* world,
    const raptor_uri_handler **handler, void **context);

Added V2 www class methods:

  raptor_www *raptor_www_new_with_connection_v2(raptor_world* world,
    void* connection);

Added V2 qname class methods:

  raptor_qname* raptor_new_qname_from_namespace_local_name_v2(raptor_world* world,
    raptor_namespace *ns, const unsigned char *local_name,
    const unsigned char *value);

Added V2 namespace class methods:

  raptor_namespace_stack* raptor_new_namespaces_v2(raptor_world* world,
    raptor_simple_message_handler error_handler, void *error_data, int defaults);

  int raptor_namespaces_init_v2(raptor_world* world,
    raptor_namespace_stack *nstack,
    raptor_simple_message_handler error_handler, void *error_data,
    int defaults);

Added V2 sequence class typedefs and methods:

  typedef void (raptor_sequence_free_handler_v2(void* context, void* object));

  typedef void (raptor_sequence_print_handler_v2(void *context, void *object,
    FILE *fh));

  raptor_sequence* raptor_new_sequence_v2(raptor_sequence_free_handler_v2* free_handler,
    raptor_sequence_print_handler_v2* print_handler, void* handler_context);

  void raptor_sequence_set_print_handler_v2(raptor_sequence *seq,
    raptor_sequence_print_handler_v2 *print_handler);

Added V2 iostream class methods:

  int raptor_iostream_write_uri_v2(raptor_world* world,
    raptor_iostream *iostr,  raptor_uri *uri);

  void raptor_iostream_write_statement_ntriples_v2(raptor_world* world,
    raptor_iostream* iostr, const raptor_statement *statement);

Added V2 xml writer class methods:

  raptor_xml_writer* raptor_new_xml_writer_v2(raptor_world* world,
    raptor_namespace_stack *nstack, raptor_iostream* iostr,
    raptor_simple_message_handler error_handler, void *error_data,
    int canonicalize);

  int raptor_xml_writer_features_enumerate_v2(raptor_world* world,
    const raptor_feature feature, const char **name, raptor_uri **uri,
    const char **label);

Raptor 1.4.18 Changes

Fixed Issues:

  • 0000186: Add RDFa support to Raptor
  • 0000255: rss-tag-soup serializer does not generate namespaces so re-serializing in rdf/xml looks wierd for atom

Parser Changes

A new RDFa parser was added (name rdfa) using librdfa to implement it. librdfa is linked as part of Raptor and written by Manu Sporny of Digital Bazaar, licensed with the same license as Raptor.

The RDFa test suite was added to the test and (via librdfa) Raptor passes all but 4 tests which fail due to different output xmlns attribute ordering (which does not matter to XML parsers).

Serializer Changes

Added new function raptor_serialize_start_to_iostream() to have the new semantics of not owning and destroying the passed-in iostream. This allows the caller to serialize to an existing iostream and then to continue to write to it. raptor_serialize_start() owns and then closes the iostream that is passed in.

A new Atom Syndication Format 1.0 (RFC 4287) serializer was added (name atom) using the RSS 1.0 RDF triple model with mapping to atom terms and consideration of atom output format conditions.

RSS 1.0 serializer

  • Allow setting output namespaces for the serializer
  • Allow writing extra RDF triples as RDF/XML attached to RSS items.
  • Recognize predicates with XML Literal and emit as parseType="Literal" or when content:encoded, as a CDATA block, by RSS 1.0 convention.
  • Removed code assumptions about triples appearing in a certain order.
  • Free namespaces and stack in correct order

Turtle serializer now respects the writeBaseURI feature to control generating the @base directive.

Abbreviated serializers (RDF/XML Abbrev and Turtle) now remove duplicate triples.

Added feature RAPTOR_FEATURE_RSS_TRIPLES to add RDF triples to RSS 1.0 or Atom serializer output with values 'rdfxml' or 'atom-triples'. Atom triples writes at:map sections for the atom:entry elements and at:feedmap and at:entrymap sections to the atom:feed elements. at:contentType is used to provide a type attribute value for an atom:content that has a URI value.

Added RAPTOR_FEATURE_ATOM_ENTRY_URI for the Atom serializer to set the URI of an atom entry. If the URI matches the URI of an entry item in the RDF mode of the channel, then an Atom Entry document is generated rather than an Atom Feed document.

QName Class Changes

Added raptor_qname_to_counted_name() to get a formatted qname for a name.

raptor_new_qname_from_namespace_local_name() will accept a NULL namespace to construct a namespace-less qname.

Sequence Class Changes

raptor_sequence_set_at() now handles setting an item at an index in the sequence beyond capacity+1 to automatically extend.

Added raptor_sequence_delete_at() to delete an item at a position in a sequence and return it.

URI Class Changes

raptor_uri_to_relative_counted_uri_string() now has support for a base URI with scheme and authority but no path, so the result can be a relative URI starting with '/'.

XML Writer Class Changes

XML Writer allows adding newlines via raptor_xml_writer_newline() which requires use of raptor_xml_writer_flush() to indicate when XML writer output is finished.

Added raptor_xml_writer_get_depth() to get the current XML writer element stack depth.

Other Changes

Many more resiliance checks were added.

Removed all calls to abort() in code on fatal errors. This requires using setjmp and longjmp inside parsers built with flex and bison.

The Turtle writer may optionally generate @base depending on flags. (This is used by Turtle serializer to handle the writeBaseURI feature)

Tidied error messages for rapper(1) when parsing stdin.

raptor_init() and raptor_finish() use a reference count to ensure initialising and terminating happen at most once each.

Raptor 1.4.17 Changes

The main changes to this release are:

Added two new JSON serializers: resource-centric 'json' (Talis RDF/JSON) and triple-centric 'json-triples'.

Made I/O Stream class raptor_iostream support reading as well as writing with new constructors and new methods.

Added a new public SAX2 API class raptor_sax2 exposing the existing internal API which has been around since the first release of Raptor 8 years ago and runs on top of either expat or libxml2.

Added new public error handlers structure raptor_error_handlers containing a set of (function, data pointers) pairs called raptor_handler_closure for each error log level. Added raptor_log_level enum for the error log level. Added an initialization function for the structure, raptor_error_handlers_init().

Several other API changes, fixes and improvements were made.

Fixed Issues:

  • 0000252: Allow controlling of cache headers in Raptor
  • 0000245: Extra classes added to an OWL object

I/O Stream class changes

Made I/O Stream class raptor_iostream support reading data in addition to writing. (Dave B):

  • Deprecated raptor_iostream_handler structure for new raptor_iostream_handler2 structure which contains the new factory functions for reading.
  • Added new read I/O stream factory handler typedefs raptor_iostream_read_bytes_func and raptor_iostream_read_eof_func.
  • Added new raptor_new_iostream_from_handler2() I/O stream constructor to allow building of read and write iostreams deprecating raptor_new_iostream_from_handler().
  • Added new raptor_iostream_tell() deprecating raptor_iostream_get_bytes_written_count.
  • Added new read I/O Stream constructors: raptor_new_iostream_from_sink(), raptor_new_iostream_from_filename() raptor_new_iostream_from_file_handle() and raptor_new_iostream_from_string()
  • Added new read I/O Stream methods raptor_iostream_read_bytes() and and raptor_iostream_read_eof().

Added new write I/O Stream method raptor_iostream_write_string_python() to write an encoded string to an I/O stream using python / JSON / Turtle / N-Triples / SPARQL escaping rules. (Dave B)

Serializer Class Changes

Added two new JSON serializers (Dave B):

  1. Resource-centric serializer named json based on Talis RDF/JSON Specification
  2. Triple-centric serializer named json-triples based on the SPARQL results in JSON format.

Added new serializer features for the JSON serializers (DaveB):

  • RAPTOR_FEATURE_JSON_CALLBACK (name 'jsonCallback') to set the top-level callback function name wrapper above the outer object.
  • RAPTOR_FEATURE_JSON_EXTRA_DATA (name 'jsonExtraData') to add extra top-level JSON object data.

Example of using the resource-centric serializer while defining a callback:

$ rapper -q -o json -f jsonCallback=foo https://librdf.org/raptor/raptor.rdf
foo(
{
  "https://librdf.org/raptor/#raptor" : {
    "http://usefulinc.com/ns/doap#description" : [ {
...

Statement Class Changes

Added raptor_statement_compare() to provide an ordering between raptor_statement objects. (Dave B)

Parser Class Changes

Added new parser features to control HTTP headers in web requests (Dave B, based on a patch in the bug):
Also never Pragma: header with libcurl ever.
Fixes Issue #0000252

  • RAPTOR_FEATURE_WWW_HTTP_CACHE_CONTROL to control sending Cache-Control (default: none)
  • RAPTOR_FEATURE_WWW_HTTP_USER_AGENT to control sending User-Agent (default: none)

Turtle parser:

  • Added tests to forbid ' and '''-quoted strings and to forbid () in triple predicate position following the updated Turtle spec. (Dave B)
  • Write ';' statement terminators with a leading space for consistency with '.' terminator. (Dave R)
  • Remove canonicalisation of integer and double to match Turtle latest spec. (Dave B)

QName Class Changes

Added new methods raptor_qname_get_local_name(), raptor_qname_get_value() and raptor_qname_get_counted_value(). (Dave B)

SAX2 Class Changes

Added new public SAX2 API class raptor_sax2 exposind th existing internal one which has been around since the first release of Raptor 8 years ago and runs on top of either expat or libxml2. (Dave B)

  • Constructor: raptor_new_sax2()
  • Destructor: raptor_free_sax2()
  • XML handler methods: raptor_sax2_set_start_element_handler(), raptor_sax2_set_end_element_handler(), raptor_sax2_set_characters_handler(), raptor_sax2_set_cdata_handler(), raptor_sax2_set_comment_handler(), raptor_sax2_set_unparsed_entity_decl_handler() and raptor_sax2_set_external_entity_ref_handler().
  • XML handler factory typedefs: raptor_sax2_start_element_handler, raptor_sax2_end_element_handler, raptor_sax2_characters_handler, raptor_sax2_cdata_handler, raptor_sax2_comment_handler, raptor_sax2_unparsed_entity_decl_handler and raptor_sax2_external_entity_ref_handler.
  • Set XML Namespace handler method: raptor_sax2_set_namespace_handler()
  • Parsing methods: raptor_sax2_parse_start() and raptor_sax2_parse_chunk()
  • Other methods: raptor_sax2_inscope_xml_language() and raptor_sax2_inscope_base_uri()

Serializer Class Changes

Abbreviated serializers (RDF/XML-Abbrev and Turtle):

  • Switched from using a sequence to using an AVL Tree with a cursor to more efficiently (faster) group/sort triples by subject. This changes the previous syntax output order but has no semantic difference. (Dave B)
  • Use the AVL Tree to remove duplicate triples. (Dave B)
    Fixes Issue #0000245

Turtle serializer:

  • Feature RAPTOR_FEATURE_WRITE_BASE_URI added to control writing @base directive to Turtle. (Dave R)
  • Remove canonicalisation of integer and double to match Turtle latest spec. (Dave B)

URI Class Changes

Update URI resolving for RFC3986 changes (Dave B)

WWW Class Changes

Added new method raptor_www_set_http_cache_control() to set the HTTP Cache-Control: header in requests. (Dave B, based on a patch in the bug)
Fixes Issue #0000252

XML Class Changes

Added new method raptor_xml_element_get_language() to get the language associated with an element. (Dave B)

Portability and Resilience Changes

Pass on error failures in parser and serializer factory construction. (Lauri)

Abbreviated serializers (RDF/XML-abbrev and Turtle): low memory and allocation failure fixes. (Lauri)

Altered API function signatures of raptor_uri_set_handler(), raptor_uri_get_handler(), raptor_new_namespaces(), raptor_namespaces_init() and raptor_new_xml_writer() to add appropriate consts. (Lauri)

Portability fixes for RAPTOR_API and other macros. (Lauri)

Removal of many sets of writable static data in N-Triples parser, URI class, Unicode NFC code, libxml support, Turtle writer and XML writer. (Lauri)

Portability fixes for round() and trunc() that are not always available in libc but might be in libm. (Dave B)

Turtle/N3 parsers and serializers, RDF/XML_Abbrev serializer: many low memory fixes and better out of memory errors. (Lauri)

Other Changes

Rewrote internal error log functions to use new error handlers structures and simplify the calls. (Dave B)

Expanded internal raptor_avltree datatype support to add a cursor, allowing it to be used for creating large ordered sequences that need to be walked. (Dave B)

Updated rdfdiff utility to handle duplicate triples in inputs. (Dave B)

raptor_sequence_shift() and raptor_sequence_unshift() are now as efficient as the sequence push and pop operations: O(1). (Lauri)

autogen.sh was updated.

rapper utility can now accept multiple -f / --feature options; previously only one parser and one serializer feature was possible.

Raptor 1.4.16 Changes

The main changes to this release are:

Provide 100% support for the GRDDL W3C Recommendation of 2007-09-11.

The Turtle parser and serializer were updated to support @base for specifying a base URI, following Turtle of 2007-09-11.

The Turtle and RDF/XML serializers had performance improvements for large graphs.

Added a TRiG Parser based on Turtle with named graph support.

Several other API changes, fixes and improvements were made.

Fixed Issues:

  • 0000188: Wrong RAPTOR_API definition for mingw
  • 0000192: raptor_uri_filename_to_uri_string() - getcwd() loop error
  • 0000194: parser and serializer don't recognize the same mime types
  • 0000195: Compile error in raptor_serialize.c debug code
  • 0000207: RDF file can be parsed, but not then serialised.
  • 0000210: RAPTOR_FEATURE_WRITER_XML_DECLARATION broken in Ruby
  • 0000214: Empty rdf:about, plus base-uri, produces incorrect turtle output
  • 0000216: flickrdf segfaults at raptor_serialize_end!
  • 0000217: flickrdf segfaults at raptor_serialize_end!
  • 0000232: libraptor does not correctly free up libxml error handler, causing crashes in subsequent calls to libxml error handlers
  • 0000237: raptor_sequence robustness: item ownership on insert error
  • 0000238: GRDDL parser in SVN overwrites blank nodes when merging graphs
  • 0000239: GRDDL parser in SVN returns 60 less triples with https://www.w3.org/

Namespaces Class Changes

raptor_namespaces_init() now returns an integer status.

Parser Class Changes

Added raptor_graph_handler typedef and raptor_set_graph_handler() to return named graph identifiers during parsing, initially for the TRiG parser.

These were added the GRDDL parser:

  • RAPTOR_FEATURE_MICROFORMATS (microformats) to enable hCard and hCal microformats
  • RAPTOR_FEATURE_HTML_TAG_SOUP (htmlTagSoup) to use the HTML tag soup parser if the XML parsing fails
  • RAPTOR_FEATURE_HTML_LINK (htmlLink) to enable html <link>
  • RAPTOR_FEATURE_WWW_TIMEOUT (wwwTimeout) for setting URI retrieval timeouts during processing

XML Element Class Changes

Added raptor_new_xml_element_from_namespace_local_name() constructor to make an XML element from a local name relative to a raptor_namespace.

Unicode Class Changes

Defined a new raptor_unichar typedef for a Unicode codepoint defined as unsigned long which was the previous type used. Altered the Unicode function to take it as a parameter. raptor_unicode_char_to_utf8(), raptor_utf8_to_unicode_char(), raptor_unicode_is_xml11_namestartchar(), raptor_unicode_is_xml10_namestartchar(), raptor_unicode_is_xml11_namechar() and raptor_unicode_is_xml10_namechar().

URI Class Changes

Added raptor_uri_compare() and raptor_uri_compare_func function pointer for implementing it in the raptor_uri_handler. The handler now has a version field initialised to trigger the new factory method for uri compare when the version is 2 or more.

WWW Class Changes

Added raptor_www_set_connection_timeout() to set the WWW retrieval connection timeout in seconds.

Added raptor_www_final_uri_handler typedef and raptor_www_set_final_uri_handler() to return the final URI seen during WWW retrieval such as after redirects.

Added raptor_www_get_final_uri() to return the final URI after a WWW retrieval which might include redirects.

Parser Changes

The GRDDL parser/processor was substantially updated and now supports 100% of the Gleaning Resource Descriptions from Dialects of Languages (GRDDL) syntax, W3C Recommendation of 2007-09-11:

  • Transforming XML with XSLT 1.0
  • Processing XML namespaces.
  • Transforming XHTML with XSLT 1.0
  • Processing HTML profiles.
  • Handling of base URIs and URI redirects.
  • XInclude processing.
  • Parsing as RDF/XML when it is recognised after a transform.

it also:

  • Handles hCard and hCal microformats when feature RAPTOR_FEATURE_MICROFORMATS is enabled (default enabled).
  • Handles <link type="application/rdf+xml" href="URI" /> to RDF/XML content when feature RAPTOR_FEATURE_HTML_LINK is enabled (default enabled).
  • Attempts parsing with libxml's HTML parser if XML parsing fails, when feature RAPTOR_FEATURE_HTML_TAG_SOUP is enabled (default enabled).
  • Discards errors during recursive processing such as 404s, failure to parse, failure of XSLT processing.
  • Uses XSLT security - denies reading, writing to files, directories or writing to network.
  • Accepts the RAPTOR_FEATURE_NO_NET feature to prevent all networking.
  • Allows fine-grained URI filtering with raptor_parser_set_uri_filter().

RDF/XML parser recognising was updated to just the start of the document for guessing if it should handle content and to try to avoid html URLs.

RSS Tag soup parser recognising was updated to accept with the string 'feed' in the identifier.

TRiG Parser was added based on the Turtle parser, adding named graphs. It returns name graph URis via a callback set with new API call raptor_set_graph_handler()

Turtle parser added @base support, fixed turtle escapes to URIs. Recognising was updated to look for @prefix early in the document.

Serializer Changes

Turtle serialiser changes:

  • Generate @base when an output base URI is given.
  • Properly format Turtle XSD doubles using new snprintf code.
  • Fix unwanted blank line at end of Turtle list abbreviation.
  • Use AVL Tree rather than sequence for significant performance improvement for large serialisations.

RDF/XML serialiser was changed to emit a legal empty RDF/XML document when no triples are serialised and to skip emitting statements with bad predicate uris rather than returning an error.

RDF/XML Abbrev serialiser was changed to use an AVL tree rather than sequence for significant performance improvement for large serialisations.

rapper Utility Changes

Added an --show-graphs option to print named graph URIs as seen (such as with TRiG).

Added -I / --input-uri and -O / --output-uri options to set the input / parsing and output / serializing base URIs separately. Defaults remain the same - the serializer base URI defaults to the input base URI, however it was set.

Portability Changes

Fixes for when building from Subversion on cygwin (EOL issues, Makefiles).

Remove unused semicolons for prevention of compiler warnings.

Fix some uninitialized variables that some compilers complain about.

Allow RAPTOR_ASSERT_DIE to be externally defined.

Allow RAPTOR_WWW_BUFFER_SIZE to be externally defined.

Other Changes

autogen.sh was updated to handle program versions better using an inline perl helper.

Start to add resiliance to memory allocation failures and errors inside the library.

Added AVL Tree code to make much faster key:value lookups. This is used for RDF/XML parser XML ID checks and in the 'abbrev' serializers - Turtle and RDF/XML-Abbrev for looking up nodes.

Better libxml error messages are now returned, mentioning some of the names and values that caused the error.

Raptor 1.4.15 Changes

General Changes

GRDDL parser now passes the (unapproved) test suite for the GRDDL W3C Working Draft 2 March 2007 except for two tests that have been reported as having errors.

When using libcurl as the WWW retrieval library, errors in resolving a URI such as not found (404) are now reported as proper errors and cause parsing to fail rather than just return no triples.

Some improvments where made to guessing for a parser to match some content. Firstly, any mime type with Q <10 is added to the score, don't lose the influence of the mime type entirely. The consequence of this is that Turtle can pretend to be a partial N3 parser. Secondly, the XHTML mime type is now correctly recognised by the GRDDL parser rather than the RSS Tag Soup parser.

Fixed Issues:

  • #0000174: Serializing to rdfxml* with a base_uri doesn't set the xml:base attribute, but does truncate rdf:about and rdf:resource values
  • #0000177: Some URI references mis-resolved
  • #0000178: No errors from accessing 404 URIs
  • #0000180: messages garble output to stdout

Parser and Serializer Changes

Added better error reporting for XML errors using the libxml structured error reporing api. From

$ rapper -i grddl https://librdf.org/LICENSE.txt
rapper: Parsing URI https://librdf.org/LICENSE.txt
rapper: Error - URI https://librdf.org/LICENSE.txt - XML error - https://librdf.org/LICENSE.txt:2: 
rapper: Error - URI https://librdf.org/LICENSE.txt - XML error - parser 
rapper: Error - URI https://librdf.org/LICENSE.txt - XML error - error : 
rapper: Error - URI https://librdf.org/LICENSE.txt - XML error - Document is empty
rapper: Error - URI https://librdf.org/LICENSE.txt - XML error -                   Redland librdf RDF API Library - License
rapper: Error - URI https://librdf.org/LICENSE.txt - XML error -                   ^
rapper: Failed to parse URI https://librdf.org/LICENSE.txt grddl content
rapper: Parsing returned 0 triples

To this:

$ rapper -i grddl https://librdf.org/LICENSE.txt
rapper: Parsing URI https://librdf.org/LICENSE.txt
rapper: Error - URI https://librdf.org/LICENSE.txt:1 - XML parser error: Document is empty
rapper: Error - URI https://librdf.org/LICENSE.txt:1 - XInclude processing failed for GRDDL document
rapper: Failed to parse URI https://librdf.org/LICENSE.txt grddl content
rapper: Parsing returned 0 triples

GRDDL parser updated to support the GRDDL W3C Working Draft 2 March 2007:

  • Namespace and profile URI handling now works.
  • Run XML Include processing
  • Throw away XML validation errors
  • When a namespace URI is seen that was RDF/XML Mime type, run the RDF/XML parser on the content.
  • Look for substrings inside rel attributes when looking for profiles.
  • Return a warning and do not fail if XSLT sheet is not found
  • Use libxml structured errors for better reporting
  • Removed old hard-coded xslt scripts

Turtle parser was changed to accept the N3 mime type text/rdf+n3 at low Q(quality) so it might work for N3 that is the RDF subset - quite common.

Changed the RSS Tag Soup parser and RSS 1.0 serializer to stop sharing use of the declared namespaces so that when using both at the same time, there is no double-free of the same objects.

Correct the content: namespace URI in the RSS parser and serializers.

Other Changes

rapper gains a -t/--trace option to show URIs traversed. Handy for GRDDL.

raptor_uri_resolve_uri_reference() no longer goes past the end of buffer when the relative URI is ,/

Added an internal API for capturing parsed data as it is seen. Use by GRDDL parser but with no public API.

Added an internal API for structured error reporting. Updates made throughout the library but with no public API.

Internal API raptor_new_sax2() signature changed to just have an error_handlers pointer argument rather than multiple function / user_data pairs.

Raptor 1.4.14 Changes

General Changes

Added a Turtle Terse RDF Triple Language serialiser by Dave Robillard based on the existing RDF/XML-Abbrev serialiser.

Added a GraphViz DOT format serialiser by Evan Nemerson.

The GRDDL parser now does namespace and profile URI recursion and has other improvements and fixes.

Fixed Issues:

  • #0000032: GRDDL indirection feature request
  • #0000141: Crash when GRDDL parser is used with a used-generated blank node ID prefix.
  • #0000143: Crash when GRDDL parser fails to retrieve URI.
  • #0000148: A public function to generate a blank ID would be nice though.
  • #0000155: entity processing in literal property elements (with libxml)
  • #0000157: Crash when RDF/XML Abbrev serializer sees a rdf:type predicate with a literal object.

Configuration Changes

raptor-config gains a --private-libs for the internal libraries used in building raptor, with the public ones only emitted with --libs.

raptor.pc now uses Libs.private for internal dynamically linked libraries.

The libxml minimum version is now 2.6.8 since 2.6.7 crashes on PPC64 Linux. 2.6.8 was released March 2004 so this should be no burden.

Do not use PATH_MAX so raptor can build on Hurd.

Parser Changes

RDF/XML parser now looks for the RDF/XML root element and namespace declaration in the initial bytes of content when guessing. This allows content that is in other mime types such as application/xml to be more likely guessed as RDF/XML.

When guessing a parser to use, if an an exact match is found for the mime type (q=10), then that parser is used.

The GRDDL parser has several changes:

  • Recurses through the root element's namespace URI and the profile URIs. It excludes several common namespace URIs from processing (XHTML, RDF/XML, XML Schema) and does not traverse the GRDDL profile URI itself.
  • Tries to guess which of the RDF/XML or Turtle parser is wanted from an XSLT result. Guessing is performed because not all the XSLT sheets used in the demonstrations set the mime type to match Turtle's unregistered type, or because the return no mime type, or return an XML one, when it was expected RDF/XML would be received.
  • Watches the processed URIs and never visits the same URI more than once in a session.
  • Passes on general XSLT errors to raptor rather than letting the default (printing to stderr) work.
  • Declares XSLT 'base' and 'Base' parameters to allow some XSLT sheets to work - pragmatism.

Serializer Changes

Added a new Turtle Terse RDF Triple Language serializer and two new internal APIs based on the existing RDF/XML-Abbrev serialiser, written by Dave Robillard:

  • turtle_writer for serializing triples to Turtle This may be moved to the public API in a future release.
  • raptor_abbrev for the common 'abbreviated serializer' core that is shared between the RDF/XML-Abbrev and Turtle serializer.

Added a new GraphViz DOT format serialiser writen by Evan Nemerson.

Note that testing the turtle serializing (make test) requires the rdfdiff -u and a few of the tests take some time to run.

Other Changes

Added raptor_home_url_string and raptor_license_string exported strings.

Added raptor_parser_generate_id() as a public function to generate an identifier for a parser.

rdfdiff gains the -u/--base-uri option to specify the from file base URI so that if the from file is a local file or relative URI, it can be given an absolute base.

Failures to retrieve content from a URI using the raptor_www class implementations now return a failure as well as setting the HTTP status code to 403 or 404 as appropriate. Previously success may have been returned with no bytes.

Raptor 1.4.13 Changes

General Changes

Prevent losing memory for a raptor_xml_writer when a serializer is reused several times.

Fixed issues reported on the Redland Issue Tracker:

  • Issue #0000134: Check for equal scheme and authority during construction of relative URIs from two absolute URIs.

Configuration Changes

In maintainer mode, add all the supported compiler -W warning flags to the CFLAGS.

Allow LEX to be set to things that aren't exactly 'flex'.

Documentation Changes

Added single triple serializing example to the tutorial to demonstrate serializing without parsing and building a raptor_statement.

Other Changes

Declare several raptor functions with GCC printf-formatting attributes when using a new enough GCC.

RDF/XML parser now creates literals with raptor_stringbuffer so that it does a lot less copying when constructing longer literals.

Added single raptor_statement serializing example to demonstrate serializing alone without parsing.

Raptor 1.4.12 Changes

Restored the order of serialized syntaxes back to the same as in Raptor 1.4.10 which Redland was relying on - asking to serialize to mime type 'application/rdf+xml' without specifying a parser name in Redland with Raptor 1.4.11 wrote it in XMP instead of RDF/XML as it used to. This happened more often with language bindings. That problem will be fixed in a future release of Redland but for now, this stops wierd things like that happening.

Raptor 1.4.11 Changes

General Changes

Added raptor_get_feature_count() to return the count of features, in preference to using the macro value RAPTOR_FEATURE_LAST.

Added raptor_www_set_uri_filter() method of the WWW class (raptor_www) objects to have an optional URI filter function that checks if the URL given is allowed to be retrieved, or denied entirely.

Fixed issues reported on the Redland Issue Tracker:

  • #0000112: raptor_namespaces_qname_from_uri not public API?
  • #0000110: strcasecmp problem under windows (raptor_rss.c does not compile)
  • #0000091: guess parser should guess the syntax each time it is run, not be fixed
  • #0000089: Add a NONET feature to prevent network fetches
  • #0000041: Allow multiple transformation URLs in data-view:transformation property
  • #0000014: bNode content written twice in rdfxml-abbrev output mode

Documentation Changes

The Raptor Reference Manual now includes descriptions of all the parsers and serializers and the tutorial has a new section describing how to filter URIs and deny network requests.

Parser Changes

Added functionality to prevent network requests either via setting a new feature RAPTOR_FEATURE_NO_NET that denies network requests during a parser operation or with a URI filter function raptor_parser_set_uri_filter(). This function uses raptor_www_set_uri_filter() internally.

Added raptor_get_need_base_uri() to tell if a parser requires a base URI argument. Presently the N-Triples parser is the only parser that does not require a base URI. raptor_start_parse() will now throw an error if no base URI is given and it is needed.

The GRDDL parser was changed to handle a list of URIs in the profile so it now can support dataview:transformation in XML taking a list of transformations as defined in The GRDDL profile for XHTML part of the GRDDL specification. It now also recognises Embedded RDF and HCalendar using well known XPaths and transforms them to RDF triples using well known XSLT sheet URIs.

The Guess parser now resets after each parse and does a fresh guess on the syntax based on the incoming information. Fixes Issue #0000091

The Turtle parser (and experimental N3 parser) were changed to now require base URIs as they always should have. The error messages when reporting problems with grammar tokens now return better responses. Added better memory cleanup during parser error recovery.

Serializer Changes

The RSS 1.0 Serializer now works again.

Updated the RDF/XML Abbreviated serializer to do proper reference counting on the blank/resource nodes used as subjects and objects to prevent dual-triple generation. Fixes the reported Issue #0000014

Other Changes

The internal SAX2 API can also prevent network fetches with the feature RAPTOR_FEATURE_NO_NET.

Fixed a SAX2 problem that caused parsers that use it to leak memory for 1 URI, affected RDF/XML and RSS Tag Soup.

rapper help and verbose message formats were tidied.

Raptor 1.4.10 Changes

General Changes

No parser will now generate a triple with an identifier type RAPTOR_IDENTIFIER_TYPE_ORDINAL. Only identifier type resource, anonymous (blank node) and literal will be generated. All serializers will convert any RAPTOR_IDENTIFIER_TYPE_ORDINAL type on input to type resource.

Configuration Changes

No longer adds LDFLAGS to pkgconfig file raptor.pc and raptor-config fixing Issue #0000097.

Parser Changes

All parsers no longer generate a triple with an identifier type RAPTOR_IDENTIFIER_TYPE_ORDINAL, as deprecated in 1.4.8. The replacement type generated is RAPTOR_IDENTIFIER_TYPE_RESOURCE.

The RSS Tag Soup (rss-tag-soup) parser now makes the triples appear before parser destruction. This caused odd symptoms like parsing in python returning no triples and the parser then crashing during object destruction.

The RDF/XML (rdfxml) parser no longer crashes if a comment is seen outside an element, such as before or after the root element.

Serializer Changes

The RDF/XML (rdfxml) serializer no longer crashes if the serializer is used more than once.

Raptor 1.4.9 Changes

Configuration and Build Changes

Now using Subversion for version control and the Raptor installation instructions explain how to get Raptor from Subversion.

configure now allows --enable-parsers=node and --enable-serializers=none. Using both is possible!

No longer require libxml2 for the RSS Tag Soup parser

Various Win32 fixes and VC build files updates from John Barstow.

Documentation Changes

A new Raptor Tutorial was written covering using all parsing and serializing functions along with example code.

The Raptor Reference Manual now covers 100% of all functions, structs and defines with gtkdoc generated documentation.

rapper utility Changes

rapper now uses namespaces found in parsing to give hints to the serializer as to how to format the output. The result of this is that rapper can be used as an RDF pretty-printer and is especially good at such things as turning flat N-Triples to RDF/XML or RDF/XML-Abbrev. such as:

rapper -q -i ntriples -o rdfxml-abbrev example.nt

Parser Changes

All parsers no longer generate RAPTOR_IDENTIFIER_TYPE_PREDICATE as the statement predicate type, as deprecated in 1.4.8. The replacement type generated is RAPTOR_IDENTIFIER_TYPE_RESOURCE.

The Turtle parser now has true and false boolean literals, which were accidently omiited from the parser in the 1.4.8 update.

Parsers can register capabilities for handling multiple mime types with Q values. These are then used in WWW requests for content in the Accept: header for HTTP. Added raptor_parser_factory_add_mime_type for registering, raptor_parser_get_accept_header to get the accept header values for the types supported by one parser.

From the previous change, the RSS parser now accepts several unregistered RSS mime types as well as the registered Atom one; the RDF/XML parser accepts unregistered mime type text/rdf seen occasionally; the Turtle parser accepts several experimental mime types. All unregistered or experimental types are accepted with lower Q than any registered type.

The RSS Tag Soup parser for RSS* and Atom no longer requires libxml2 (for it's XML Reader API). Internal changes mean that it will fully work on top of expat.

Serializer Changes

The RSS/Atom serializer now uses input namespace declarations to choose namespaces on output.

Added raptor_serialize_set_namespace_from_namespace to set a namespace for serializing from an existing raptor_namespace.

Serializing to RDF/XML (or RDF/XML Abbrev) now does not double-free URI strings. Fixes Issue #0000065

RSS serializer no longer writes the XML header twice.

IOStream Class Changes

Added raptor_iostream_write_uri to directly write a URI to an iostream without the need to go via a string.

Fixed bug in raptor_iostream_write_xml_any_escaped_string failing to write ';' after escaping U+0009 and U+000A

Namespaces Class Changes

Added raptor_namespaces_qname_from_uri to do URI splitting into qname prefering to use the current in-scope namespaces before having to search.

raptor_namespaces_format now NULL-terminates the namespace string. Fixes Issue #0000062

Added raptor_namespace_get_counted_prefix to return a namespace prefix and it's length.

QName Class Changes

Added raptor_qname_get_namespace to get the namespace associated with a QName.

StringBuffer Class Changes

raptor_stringbuffer_append_counted_string and raptor_stringbuffer_append_string now Do nothing on appending a NULL string or a string of length 0. Fixes Issue #0000073

Unicode Class Changes

raptor_utf8_to_unicode_char now also checks for overlong UTF-8 sequences, illegal code positions or out of range codes.

URI Class Changes

Deprecated raptor_uri_is_file_uri which takes a URI string argument for new function raptor_uri_string_is_file_uri which more clearly says that.

Changed all URI string calloc/mallocs to add enough room for a full pointer at the string end to stop valgrind moaning on 64bit systems when looking for the end of string NUL.

raptor_uri_set_handler and raptor_new_iostream_from_handler now take const handler arguments.

WWW Class Changes

Get the curl success status into a long, not an int which causes failure on 64 bit. Fixes Issue #0000075

WWW requests for content to parse now always send an appropriate Accept: header with Q values for the parser, or for the guess parser, all supported mime types.

Internal Changes

Added XML element methods raptor_xml_element_get_attributes and raptor_xml_element_get_attributes_count, raptor_xml_element_is_empty to the SAX2 API.

Many internal changes were made to the SAX2 API to finally separate XML and RDF/XML parts. The SAX2 API is now fully usable on either libxml2 or expat. That last sentence implies a lot of work, by the way.

Raptor 1.4.8 Changes

General Changes

A large source re-arrangement was performed. All C sources and headers that build the library are now in the src dir, general documentation in the doc dir and utilities in the utils dir. This both tidied up the mixture of files at the top level and also enabled better use with gtk-doc.

Future API change: From the next release of Raptor, raptor_statement predicates will return identifiers of type RAPTOR_IDENTIFIER_TYPE_RESOURCE instead of RAPTOR_IDENTIFIER_TYPE_PREDICATE. Identifiers of type RAPTOR_IDENTIFIER_TYPE_ORDINAL may no longer be returned in any statement position (to be confirmed).

Version Control change: Raptor will be switching to use Subversion for version control after the 1.4.8 release. Please check the Redland Subversion site for the latest status or the online Raptor installation notes for the raptor specific subversion installation information.

Configuration Changes

The autogen.sh script for building from CVS was revamped to be more modular.

configure now takes an --enable-gtk-doc option to enable building of the documentation using the gtk-doc utility. It is by default enabled only if the utility is available.

Added a new configure option --enable-serializers (in 1.4.7) to allow the selection of the required RDF serializers from any of those supported.

raptor-config now has a --options argument to list the configured or discovered options of the library such as parsers, serializers and other choices.

Documentation Changes

The GNOME gtk-doc program is now used to automatically extract documentation from source comments into reference documentation. This is then merged with templates and additional documentation to provide a reference manual for raptor as XML document which is turned into HTML along with GNOME devhelp support.

This new documentation intended to replace the libraptor manual page/web page as easier to read document with scope for better expanding with more detail of raptor including examples and tutorial information. The manual page will continue to contain the summary information for the present.

Portability Changes

Fixed a long-standing URI resolution bug on win32 - only remove leading / if there is one present (patch from John C. Barstow)

rapper utility Changes

Altered the -g argument to invoke the guessing parser rather than guess on file/URI name alone. This is now equivalent to choosing an input syntax of guess with -i guess.

Added a --show-namespaces long option (no short version) to show namespaces that are declared in the parsed content.

Parser Changes

A new guessing parser was added, picking the actual parser to use at run-time based on protocol or other information.

Allow a content type returned by a protocol (such as HTTP) to enable choosing of parser at run-time. Added a new optional parser factory method content_type_handler to return this.

Allow parsers to handle several syntaxes rather than only 1 or 2.

Parsers can now return namespace prefix/URI declarations as they are given in the content by the means of a new handler type raptor_namespace_handler and parser method raptor_set_namespace_handler. Duplicate namespace prefix/URIs can be returned.

GRDDL Parser Changes

Bug fix when the entire content is in one chunk (René Puls).

Guessing Parser Changes

A new parser that guesses the actual parser to use at run-time based on a combination of MIME Content-Type, file or URI name and in future, iniital bytes of the content. If the Content-Type is an exact match to a known parser, it is always chosen before trying heuristics.

RDF/XML Parser Changes

When emitting literals, handle a datatyped empty literal. This is a post-REC errata for the revised RDF/XML recommendation. See archived example for further information.

RSS Tag Soup Parser Changes

Added atom 1.0 support including use of the new namespace. Atom 0.3 namespace terms are turned into new properties. Replace atom copies of Dublin Core or RSS properties with the original terms:

Atom 1.0 term Original term
atom:content rss:description
atom:id rss:link
atom:published dc:date
atom:rights dc:rights
atom:title rss:title

Apply the in-scope base URI (such as from xml:base) to atom 1.0 fields that take URI values: atom:id, atom:icon and atom:logo.

Added optional date parsing code to turn XML RSS date fields into ISO format ones, suitable for Atom and XML schema datatypes format. Will use library parsedate code from curl or INN if available.

XML RSS field pubDate is now turned into Dublin Core dc:date field in the ISO format.

XML RSS field content is turned into content:encoded in RDF triples on output with escaping.

Turtle Parser Changes

Updated to support Turtle version 2006-01-02 (announcement).

Switch qname, blank node and prefix definitions to SPARQL ones.

Check for illegal not-hexadecimal \u and \U escape values.

Fix greedy matching of long literals ("""....""") that ended on the last """ found rather than the first.

Added double and decimal constants.

Added optional +- sign to all numeric constants.

Allow \" escape inside long strings.

Take care to reset the generated raptor_statement language and datatype fields when not used.

Serializer Changes

Added a new Atom 1.0 serializer (name atom) by parameterising the RSS 1.0 serializer.

Added a new Adobe XMP compatible serializer (name rdfxml-xmp) by parameterising the RDF/XML Abbreviated serializer. Patch provided by Sid Steward.

All serializers can be chosen at configure time from those available using configure option --enable-serializers.

The RSS parser and serializer can now be independently enabled or disabled. The RSS serializer no longer requires an XML parser.

RDF/XML Serializer / XML Writer Changes

A new XML Writer feature RAPTOR_FEATURE_WRITER_XML_VERSION was added to allow chosing XML 1.0 (value 10) or XML 1.1 output (value 11). This feature is also accepted by serializers as an option and used by the RDF/XML and RDF/XML-Abbrev serializers.

A new XML Writer feature RAPTOR_FEATURE_WRITER_XML_DECLARATION was added to allow omitting the XML declaration (default true).

Added functions raptor_xml_any_escape_string() and raptor_iostream_write_xml_any_escaped_string() which take an XML version. The XML 1.0 functions give errors when attempting to write #x1-#x1f (excluding #x9, #xA, #xD) or #x7F.

Atom 1.0 Serializer Changes

Added a new serializer using the Atom 1.0 format and namespace. This reads RDF triples in the RSS 1.0 model, along with any additional atom 1.0 properties and serializes an Atom 1.0 feed file.

Adobe XMP Serializer Changes

Added a new serializer writing RDF/XML in the profile used by Adobe XMP. Note that this does require RDF triples to be used in a certain style; for example all triple subjects are the "current documment" giving rdf:about="".

URI Class Changes

Fix a bug when adding a default path of / to a URI in functions raptor_new_uri_for_xmlbase() and raptor_new_uri_for_retrieval(). (Bug #0000045)

raptor_uri_equals was altered to accept NULL pointers, which do not compare equal to a non-NULL URI. NULL does equal NULL.

Internal Changes

The internal SAX2 class was extensively changed so that remaining interdependencies with the RDF/XML parser were removed and it can now be re-used for other syntaxes cleanly. Several functions were modified or added.

Removed old and hardly-tested internal support for XML entity resolution (libxml only).

Various fixes for GCC 4 warnings.

Raptor 1.4.7 Changes

Fix a couple of crashes in the RSS tag soup parser / serializer (Dave Beckett, Suzan Foster).

configure now looks for the libxslt/xslt.h header as well as the libxslt library and disables XSLT and GRDDL support it if is missing. This catches systems with the libraries without headers as has happened on some OSX versions.

In serializers rdfxml and rdfxml-abbrev, report failure to serialize to RDF/XML if the predicate URI is not absolute.

Raptor 1.4.6 Changes

Added --with-xslt-config configure option

Added a new parser for Gleaning Resource Descriptions from Dialects of Languages (GRDDL) which allows reading XHTML and XML as RDF triples by using profiles in the document that declare XSLT transforms from the XHTML/XML content into RDF/XML which is the RDF content. It does not support all the GRDDL styles, for example dataview:namespaceTransformation, or perform recursive transformations.

The turtle parser now accepts """long literals"""

XML writer feature support were added in 1.4.5 and not documented. The new functions are: raptor_xml_writer_features_enumerate, raptor_xml_writer_set_feature, raptor_xml_writer_set_feature_string, raptor_xml_writer_get_feature and raptor_xml_writer_get_feature_string. The three XML writer features added are \fBRAPTOR_FEATURE_WRITER_AUTO_INDENT\fR with boolean value (default true) to auto-indent the XML, \fBRAPTOR_FEATURE_WRITER_AUTO_EMPTY\fR with boolean value (default true) to automatically generate empty elements if a start/end element sequence has no content and \fBRAPTOR_FEATURE_WRITER_INDENT_WIDTH\fR with an integer value (default 2) to set the indenting level for the XML.

New build configuration and portability fixes for win32 (John Barstow)

Portability fixes for win32 - added SIZEOF_UNSIGNED_SHORT (Dave Viner, others)

Added a signing memory debugging system to aid checking when raptor-allocated memory is freed in another library or vice-versa enabled by --with-memory-signing configure option (defaults to on in maintainer mode).

Fixed a few internal malloc/frees to use RAPTOR_MALLOC / RAPTOR_FREE so that the above signed memory system worked.

RDF/XML serializer: Use the maximal name when splitting a predicate.
Turn datatyped literals that are rdf:XMLLiteral into inline XML with rdf:parseType="Literal" rather than XML-escaped.

RDF/XML abbreviated serializer: Fix a crash when there is a NULL base URI. Use the maximal name when splitting a predicate.
Turn datatyped literals that are rdf:XMLLiteral into inline XML with rdf:parseType="Literal" rather than XML-escaped.

RSS tag soup parser: Fix crash with unexpected use of alternate attribute.
Update from Suzan Foster to reflect the latest status of the enclosure vocabulary and allow multiple common items and fields.

RSS 1.0 serializer: Added RSS enclosures serializing.

grapper example GTK program now stores the window width and height using gconf2.

Raptor 1.4.5 Changes

Added a new RDF/XML with abbreviations serializer rdfxml-abbrev written by Steve Shepard which handles several of the abbreviations specified by the RDF/XML Syntax Specification (Revised) W3C Recommendation. It is suitable for writing small documents as there are known scaling issues.

The RSS tag soup parser was updated to work better when there is no base URI given. It also now supports reading the RSS 1.1 format and turning it into RSS 1.0 model triples.

Deprecated raptor_ntriples_string_as_utf8_string as rather too internal to be useful, since it only works with a parser.

More fixes to work around the broken libxml2 on Apple OSX 10.3.x with inconsistent shared libraries / headers.

Experimental and incomplete Notation 3 parser - updated to match changes to Turtle. CVS changes only, not enabled in standard builds.

Raptor 1.4.4 Changes

Make the RSS tag soup parser handle RSS 0.9 namespace elements by turning them into RSS 1.0.

Fix a couple of crashes in the RSS 1.0 serialiser when no base URI is used.

Make raptor_uri_to_relative_counted_uri_string work when the base or reference URI have no paths such as like http://example.org

Added portability fixes for Win32 to get Raptor 1.4.3 building with MS Visual Studio using expat and libcurl. The RAPTOR_INTERNAL define was moved to the build configuration and defines added for integral type sizes. Patch from Dave Viner (dviner at apache dot org).

Raptor 1.4.3 Changes

A release with the major new feature of an XML writer API. This is now used along with a new supporting XML element class to improve the existing RDF/XML serializer and to provide a new RSS 1.0 serializer.

This API it is also used by the next release of Rasqal to provide serializing of query results to XML.

The new raptor_xml_writer class functions added are: raptor_new_xml_writer (constructor), raptor_free_xml_writer (destructor), raptor_xml_writer_empty_element, raptor_xml_writer_start_element, raptor_xml_writer_end_element, raptor_xml_writer_cdata, raptor_xml_writer_cdata_counted, raptor_xml_writer_raw, raptor_xml_writer_raw_counted, raptor_xml_writer_comment and raptor_xml_writer_comment_counted.

The new raptor_xml_element class functions added are: raptor_new_xml_element (constructor), raptor_free_xml_element (destructor), raptor_xml_element_get_name, raptor_xml_element_set_attributes, raptor_xml_element_declare_namespace and raptor_iostream_write_xml_element.

Parser Changes

RSS tag soup parser now works with older libxml2s (2.5.10+), including the one shipped with some Apple OSX versions that has an inconsistent header file and library.

RSS tag soup parser recognises/scores more common XML RSS file names.

RSS tag soup parser turns XML RSS <guid isPermaLink="true">val</guid> into RDF/XML form <guid rdf:resource="val"/>, leaving the non isPermaLink form to be a literal value.

A bug was found in libxml2 that causes double expanding of XML entities in RDF/XML. This has been reported but cannot be worked around from raptor. The expat XML parser can be used as an alternative, as it does not have this problem. A test was added for this bug but it will not cause the test suite ('make check') to fail.

Added additional Turtle parser tests that cover Notation 3 syntax that is not part of the Turtle language.

Added raptor_parser_set_feature_string and raptor_parser_get_feature_string methods to set/get string feature values.

Serializer Changes

Added feature relative_uris for serializers. This is used by the RDF/XML serializer and enabled by default.

Added feature start_uri for serializers with a string value to set the start URI for serializing. Not used at present.

Added new methods raptor_serializer_features_enumerate to list serializer features and functions to set/get serializer feature integer or strings values: raptor_serializer_set_feature, raptor_serializer_get_feature, raptor_serializer_set_feature_string and raptor_serializer_get_feature_string.

Added raptor_serialize_set_namespace to allow user declaration of prefix/URI namespaces pairs as serializing hints.

the RDF/XML serializer was improved using the new XML Writer class so it now uses any user-declared namespace hints in it's output and emits relative URIs whenever possible. The latter was provided by a patch from René Puls.

A new RSS 1.0 serializer was added, using the new XML Writer class and using the same structures, classes and properties as the RSS tag soup parser.

URI class changes

Added relative URI generating code from a patch written by René Puls and provide this with two new methods raptor_uri_to_relative_uri_string and raptor_uri_to_relative_counted_uri_string.

Added raptor_uri_print to print a URI to a file handle.

Added methods raptor_uri_to_string and raptor_uri_to_counted_string to return a URI as newly allocated strings.

I/O Stream Changes

Many classes gained methods to write to iostreams, supporting the new XML Writer class functionality. The added methods are: raptor_iostream_write_namespace, raptor_iostream_write_ntriples_string, raptor_iostream_write_qname, raptor_iostream_write_statement_ntriples, raptor_iostream_write_stringbuffer, raptor_iostream_write_xml_element and raptor_iostream_write_xml_escaped_string.

Namespace Class Changes

Added raptor_namespace_copy copy constructor and raptor_new_namespace_from_uri constructor to build a namespace from a raptor_uri object.

Added utility function raptor_new_namespace_parts_from_string to decode syntax of the form xmlns:prefix="uri" into prefix and uri string pairs.

Added raptor_namespaces_find_namespace_by_uri method for namespace stack to find a declared namespace by URI. This complements raptor_namespaces_find_namespace which already provides searching by prefix.

Unicode and UTF-8 Changes

Added several methods for checking characters forming parts of XML 1.0 or XML 1.1 names: raptor_unicode_is_xml10_namestartchar, raptor_unicode_is_xml11_namestartchar, raptor_unicode_is_xml10_namechar and raptor_unicode_is_xml11_namechar.

Added a function raptor_utf8_check to check that a string is legal UTF-8 and all the encoded Unicode characters are in the range U+0 <= character <= U+10FFFF

Added a function raptor_xml_name_check to check that a string is a legal XML name (1.0 or 1.1) as well as legal UTF-8.

Other Changes

Feature support: Added raptor_feature_value_type to determine value of a feature - either integer (most) or string.

XML QName class: Added raptor_qname_copy copy constructor.

Sequence class: Added raptor_sequence_join to join two sequences of items, leaving one empty.

Statement class: Added raptor_statement_copy copy constructor and raptor_free_statement destructor. Previously these were internal to raptor.

The rapper utility was modified to add a feature form: -f xmlns:PREFIX="URI" allowing the setting of output serializer namespaces.

The namespace URI string constants exported by raptor are now of type unsigned char*.

Raptor 1.4.2 Changes

Make raptor_xml_escape_string fail correctly when given bad UTF-8 to escape.

Raptor 1.4.1 Changes

Fixed a buffer overrun in decoding a URI scheme in raptor_uri constructors such as raptor_new_uri.

Fixed a crash in RSS enclosures crash when the url attribute seen on a non-<enclosure> element

raptor_xml_escape_string return value has changed to be an int, returning <0 on failure. This allows the empty string encoding an empty string case to work and be distinguished from an error.

Raptor 1.4.0 Changes

A release with the major new feature of providing serializing of RDF triples to syntaxes. It also added a new support class for I/O streams and had other minor fixes.

Added a Raptor Serializer class (raptor_serializer) with similar style to Parser (raptor_parser). Two serializers are provided, for RDF/XML and N-Triples. The serializing can be done to files, C FILE* or to strings. The raptor_iostream class that provides this also allows writing to any other form by creating a custom iostream.

The new raptor_serializer class functions added are: raptor_serializers_enumerate, raptor_serializer_syntax_name_check, raptor_new_serializer, raptor_free_serializer, raptor_serialize_start, raptor_serialize_start_to_filename, raptor_serialize_start_to_string, raptor_serialize_start_to_file_handle, raptor_serialize_statement, raptor_serialize_end, raptor_serializer_get_iostream, raptor_serializer_set_error_handler, raptor_serializer_set_warning_handler and raptor_serializer_get_locator

Added a Raptor I/O stream abstraction in raptor_iostream class to support serializing of RDF to multiple output streams such as to filenames, to C standard I/O FILE* handles and to strings especially for cross-language use. A raptor_iostream_handler can be used to construct a user-defined iostream.

The new raptor_iostream class functions added are: raptor_new_iostream_from_handler, raptor_new_iostream_to_sink, raptor_new_iostream_to_filename, raptor_new_iostream_to_file_handle, raptor_new_iostream_to_string, raptor_free_iostream, raptor_iostream_write_bytes, raptor_iostream_write_byte, raptor_iostream_write_end, raptor_iostream_write_string, raptor_iostream_write_counted_string, raptor_iostream_get_bytes_written_count, raptor_iostream_write_decimal and raptor_iostream_format_hexadecimal.

The rapper utility was modified to use serializer class so that the output formats supported are now N-Triples (-o ntriples) - the default, and RDF/XML (-o rdfxml).

Raptor now exports more static namespace URI strings for general application use: raptor_xml_namespace_uri, raptor_rdf_namespace_uri, raptor_rdf_schema_namespace_uri, raptor_xmlschema_datatypes_namespace_uri, raptor_owl_namespace_uri, and the length raptor_rdf_namespace_uri_len.

The raptor_stringbuffer class gained a new method raptor_stringbuffer_copy_to_string which allows efficient copy-out of a constructed string.

The raptor_www class gained a new method raptor_www_fetch_to_string to allow retrieving of web content as a single string.

RSS tag soup parser gained support for generating triples for enclosures, after a patch from Suzan Foster. Changes made include correcting the enclosures namespace and tidying some memory leaks.

Raptor 1.3.3 Changes

A release with major improvements along with several minor fixes.

Raptor's License was changed from LGPL 2.1/MPL 1.1 to LGPL 2.1/Apache 2

Thanks to Chris Pointon for several patches to make Raptor easier to build under Win32 which were applied, with some slight modifications.

Increased WWW content retrieval buffer size from 256 bytes to 4K since this was causing problems for even moderate size documents.

After testing raptor on a very large RDF/XML file with many rdf:ID values, the check for duplicate values was found to be inefficient in memory and slow. The implementation was improved to be more memory efficient and a new parser feature check_rdf_id was added to disable checking (default is enabled).

Added a new Unicode NFC checker to replace the functionality formally available by calling the GNOME glib function g_utf8_normalize. This new checker is done via several tables and adds approximately 50K to the object size of the library when compiled on x86. This code and tables can be disabled with configure option --disable-nfc-check causing all checks to succeed.

Fix the exporting of raptor_xml_literal_datatype_uri_string and raptor_xml_literal_datatype_uri_string_len as constants for use by applications. Previously raptor.h wasn't doing this correctly.

Added raptor_calloc_memory for allocating zeroed memory inside raptor, for use by applications passing memory in/out of raptor.

Added a new configure option --enable-parsers to allow the selection of the required RDF parsers from any of those supported (RDF/XML, Turtle, N-Triples, RSS tag soup).

Reorganised the sources to split parsing support from RDF/XML to support compiling without this parser.

Updated the RSS Tag Soup parser to start to handle the Atom 0.3 currently being standardised by the IETF Atom Publishing Format and Protocol working group.

Altered the Turtle parser to work with large source documents that exceeded bison limits. Thanks to Geoff Chappell for providing a fix for this.

Rewrote the URI parsing to create an internal structure and improved the relative URI resolving in preparation for future work such as potentially supporting URI canonicalisation such as proposed to be used by Atom.

Raptor 1.3.2 Changes

A release with some minor fixes.

Added a new configure option --with-expat-source=DIR to allow the use of external expat source trees in either the old or newer directory structure style. (Patch from Mark Smith).

Added raptor_alloc_memory for handlers that need to allocate memory in the same heap as raptor uses for raptor_free_memory. This is mostly useful for allocating memory that is freed by raptor in error, ID and statement handlers on win32 which has separate heaps for different DLLs.

A bug was fixed where errors which happened when fetching WWW content were always printed to stderr. They are now passed to the main error routines which allows applications to retrieve them.

Accessor functions were added for parts of the public raptor_locator structure which makes it possible to get structured error information from language bindings via Redland (Patch from Edd Dumbill). The new functions are:

  • int raptor_locator_line(raptor_locator *locator);
  • int raptor_locator_column(raptor_locator *locator);
  • int raptor_locator_byte(raptor_locator *locator);
  • const char * raptor_locator_file(raptor_locator *locator);
  • const char * raptor_locator_uri(raptor_locator *locator);

The Unicode Normal Form C (NFC) checking via the GNOME glib library function g_utf8_normalize is broken, comparing the data it says is failed against other NFC checkers. It is also slower than need be since it is doing full normalizing rather than just checking for NFC, and adds a rather large dependency for just one function. A new portable checker will be added in a later release.

Raptor 1.3.1 Changes

A release primarily to fix some win32 and portability issues.

raptor.h now includes stdarg.h

Corrected the raptor_print_statement declaration in raptor.h for the argument statement to have one less 'const' which matches the actual code.

Made several portability fixes for compiling natively on win32 which doesn't quite do POSIX or C99.

Changed the support for file: URIs and converting to and from filenames. It now %-escapes spaces and % characters on conversion to and from filenames with raptor_uri_uri_string_to_filename, raptor_uri_uri_string_to_filename_fragment and raptor_uri_filename_to_uri_string. For Win32, more tests were added and the format of URIs supported corrected to use the file:///c: form rather than file://c|/

URIs that resolve to directories now return an error when lstat is available to check.

Parser Changes

The Turtle parser was updated to only allow language with non-datatyped literals, allow a '_' immediately after a ':' in qnames and to make a bare ':' qname work correctly.

The Turtle parser was fixed to re-initialise correctly when performing multiple parsings. The other parsers already did this correctly.

Added a warning to the RDF/XML parser for unknown rdf:parseType values, when parsing in lax mode - which is the default. It now tells the user when the parsing is working as 'Literal' mode by finding an unknown value. This is controlled by a new parser feature warn_other_parsetypes which is default set true in lax mode. Parser modes are controlled by the raptor_set_parser_strict method.

Raptor 1.3.0 Changes

A release primarily to provide support for the new Rasqal RDF query library but with some new features and fixes.

Parser Changes

Added a new constructor raptor_new_parser_for_content to guess the parser to use from hints in URIs or content, using a new utility function raptor_guess_parser_name.

Additional checks were added to the RDF/XML parser for RDF-namespaced names in element and attributes and if they are forbidden giving an error otherwise if unknown, giving a warning.

The Turtle parser was updated to correct the collections syntax, allow '-' in names and QNames and to add integer literals. This parser now correctly uses raptor_generate_id when a blank identifier name is needed.

Completed parser feature support by adding raptor_get_feature, raptor_feature_from_uri, and raptor_features_enumerate to get values and enable discovery of supported features at run time. raptor_set_feature was changed to give return a success value

Added a new method raptor_get_mime_type to get the MIME type of the syntax for a parser

raptor_parse_uri_with_connection (which is called by raptor_parse_uri) now sets the HTTP Accept: header to the MIME type of the parser in WWW requests using the new raptor_www_set_http_accept().

rapper changes

Added options -f/--feature for setting features and -g/--guess for guessing syntax from some content or identifiers. See rapper(1) for all rapper options.

Utility function changes

Added raptor_syntax_name_check to check for valid syntax language names.

Added raptor_free_memory to free memory returned by raptor functions.

Added Unicode utility functions raptor_unicode_char_to_utf8 and raptor_utf8_to_unicode_char.

Exported URI string raptor_xml_literal_datatype_uri_string.

Deprecated raptor_print_statement_detailed always intended to be internal.

WWW Class changes

Added support to set the HTTP Accept: header for curl and libxml2 when retrieving HTTP content by the new raptor_www_set_http_accept method.

New classes - Sequence and Stringbuffer

Added a utility class raptor_sequence providing simple sequences that can handle stacks and queues

Added a utility class raptor_stringbuffer for constructing strings from substrings appended or prepended.

Raptor 0.9.0 - Raptor 1.2.0 Changes

Release notes for 1.2.0 and earlier are in the NEWS page or ChangeLog