6. Retrieval Facility

YAZ version 2.1.20 or later includes a Retrieval facility tool which allows a SRU/Z39.50 to describe itself and perform record conversions. The idea is the following:

The Retrieval facility is driven by an XML configuration. The configuration is neither Z39.50 ZeeRex or SRU ZeeRex. But it should be easy to generate both of them from the XML configuration. (unfortunately the two versions of ZeeRex differ substantially in this regard).

6.1. Retrieval XML format

All elements should be covered by namespace http://indexdata.com/yaz . The root element node must be retrievalinfo.

The retrievalinfo must include one or more retrieval elements. Each retrieval defines specific combination of syntax, name and identifier supported by this retrieval service.

The retrieval element may include any of the following attributes:

syntax (REQUIRED)

Defines the record syntax. Possible values is any of the names defined in YAZ' OID database or a raw OID in (n.n ... n).

name (OPTIONAL)

Defines the name of the retrieval format. This can be any string. For SRU, the value, is equivalent to schema (short-hand); for Z39.50 it's equivalent to simple element set name. For YAZ 3.0.24 and later this name may be specified as a glob expression with operators * and ?.

identifier (OPTIONAL)

Defines the URI schema name of the retrieval format. This can be any string. For SRU, the value, is equivalent to URI schema. For Z39.50, there is no equivalent.

The retrieval may include one backend element. If a backend element is given, it specifies how the records are retrieved by some backend and how the records are converted from the backend to the "frontend".

The attributes, name and syntax may be specified for the backend element. These semantics of these attributes is equivalent to those for the retrieval. However, these values are passed to the "backend".

The backend element may includes one or more conversion instructions (as children elements). The supported conversions are:

marc

The marc element specifies a conversion to - and from ISO2709 encoded MARC and MARCXML/MarcXchange. The following attributes may be specified:

inputformat (REQUIRED)

Format of input. Supported values are marc (for ISO2709), xml (MARCXML/MarcXchange) and json (MARC-in_JSON).

outputformat (REQUIRED)

Format of output. Supported values are line (MARC line format); marcxml (for MARCXML), marc (ISO2709), marcxhcange (for MarcXchange), or json (MARC-in_JSON ).

inputcharset (OPTIONAL)

Encoding of input. For XML input formats, this need not be given, but for ISO2709 based inputformats, this should be set to the encoding used. For MARC21 records, a common inputcharset value would be marc-8.

outputcharset (OPTIONAL)

Encoding of output. If outputformat is XML based, it is strongly recommened to use utf-8.

select

The select selects one or more text nodes and decodes them as XML. The following attributes may be specified:

path (REQUIRED)

X-Path expression for selecting text nodes.

This conversion is available in YAZ 5.8.0 and later.

solrmarc

The solrmarc decodes solrmarc records. It assumes that the input is pure solrmarc text (no escaping) and will convert all sequences of the form #XX; to a single character of the hexadecimal value as given by XX. The output, presumably, is a valid ISO2709 buffer.

This conversion is available in YAZ 5.0.21 and later.

xslt

The xslt element specifies a conversion via XSLT. The following attributes may be specified:

stylesheet (REQUIRED)

Stylesheet file.

6.2. Retrieval Facility Examples

Example 7.19. MARC21 backend

A typical way to use the retrieval facility is to enable XML for servers that only supports ISO2709 encoded MARC21 records.

     <retrievalinfo>
       <retrieval syntax="usmarc" name="F"/>
       <retrieval syntax="usmarc" name="B"/>
       <retrieval syntax="xml" name="marcxml"
		  identifier="info:srw/schema/1/marcxml-v1.1">
         <backend syntax="usmarc" name="F">
	   <marc inputformat="marc" outputformat="marcxml"
		 inputcharset="marc-8"/>
	 </backend>
       </retrieval>
       <retrieval syntax="xml" name="dc">
         <backend syntax="usmarc" name="F">
	   <marc inputformat="marc" outputformat="marcxml"
		 inputcharset="marc-8"/>
           <xslt stylesheet="MARC21slim2DC.xsl"/>
	 </backend>
       </retrieval>
     </retrievalinfo>

     

This means that our frontend supports:

  • MARC21 F(ull) records.

  • MARC21 B(rief) records.

  • MARCXML records.

  • Dublin core records.


Example 7.20. MARCXML backend

SRW/SRU and Solr backends returns records in XML. If they return MARCXML or MarcXchange, the retrieval module can convert those into ISO2709 formats, most commonly USMARC (AKA MARC21). In this example, the backend returns MARCXML for schema="marcxml".

     <retrievalinfo>
       <retrieval syntax="usmarc">
         <backend syntax="xml" name="marcxml">
	   <marc inputformat="xml" outputformat="marc"
		 outputcharset="marc-8"/>
	 </backend>
       </retrieval>
       <retrieval syntax="xml" name="marcxml"
		  identifier="info:srw/schema/1/marcxml-v1.1"/>
       <retrieval syntax="xml" name="dc">
         <backend syntax="xml" name="marcxml">
           <xslt stylesheet="MARC21slim2DC.xsl"/>
	 </backend>
       </retrieval>
     </retrievalinfo>

     

This means that our frontend supports:

  • MARC21 records (any element set name) in MARC-8 encoding.

  • MARCXML records for element-set=marcxml

  • Dublin core records for element-set=dc.


6.3. API

It should be easy to use the retrieval systems from applications. Refer to the headers yaz/retrieval.h and yaz/record_conv.h.