omnipotentdatatranslator

Translate API About

Translate Schema.org data

API

The omnidator provides a set of simple CORS-enabled APIs to translate from various data formats that use Schema.org terms to various data formats:

• Main API
• Low-level API
• Microdata API

Main API

The {from} and {to} specifies the respective input and output format and can take the following values:

INPUT FORMATS:
 microdata
 csv

OUTPUT FORMATS:
 json (served as: application/json) 
 rdf-xml (served as: application/rdf+xml) 
 rdf-turtle (served as: text/turtle)

The {URL} is the HTTP URI (address) of the data source, such as a HTML+microdata document or a CSV file.

For example, invoking the omnidator API as follows (for translating a CSV file that uses Schema.org terms in the header):

yields the following JSON data:

{
  "@": [
    {
      "a": "<http://schema.org/URL>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:3", 
      "rdf:value": "<http://dbpedia.org/resource/Earth>"
    }, 
    {
      "a": "<http://schema.org/Mass>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:2", 
      "rdf:value": "1.988435x10^30 kg"
    }, 
    {
      "http://purl.org/dc/terms/title": "Thing", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:1"
    }, 
    {
      "http://purl.org/dc/terms/title": "row 2", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2", 
      "http://purl.org/NET/schema-org-csv#cell": [
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:1>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:2>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:3>"
      ], 
      "a": "<http://purl.org/NET/schema-org-csv#Row>"
    }, 
    {
      "a": "<http://schema.org/Thing>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:1", 
      "rdf:value": "Mars"
    }, 
    {
      "http://purl.org/dc/terms/title": "row 3", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3", 
      "http://purl.org/NET/schema-org-csv#cell": [
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:2>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:3>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:1>"
      ], 
      "a": "<http://purl.org/NET/schema-org-csv#Row>"
    }, 
    {
      "a": "<http://schema.org/URL>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:3", 
      "rdf:value": "<http://dbpedia.org/resource/Sun>"
    }, 
    {
      "a": "<http://schema.org/URL>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:3", 
      "rdf:value": "<http://dbpedia.org/resource/Mars>"
    }, 
    {
      "http://purl.org/dc/terms/title": "row 4", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4", 
      "http://purl.org/NET/schema-org-csv#cell": [
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:3>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:2>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:1>"
      ], 
      "a": "<http://purl.org/NET/schema-org-csv#Row>"
    }, 
    {
      "a": "<http://schema.org/Mass>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:2", 
      "rdf:value": "6.4191x10^23 kg"
    }, 
    {
      "a": "<http://schema.org/Mass>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:2", 
      "rdf:value": "5.9721986x10^24 kg"
    }, 
    {
      "a": "<http://schema.org/Thing>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:1", 
      "rdf:value": "Sun"
    }, 
    {
      "http://purl.org/dc/terms/title": "header", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1", 
      "http://purl.org/NET/schema-org-csv#cell": [
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:2>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:1>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:3>"
      ], 
      "a": "<http://purl.org/NET/schema-org-csv#HeaderRow>"
    }, 
    {
      "a": "<http://schema.org/Thing>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:1", 
      "rdf:value": "Earth"
    }, 
    {
      "http://purl.org/dc/terms/title": "solar-system.csv", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#table", 
      "http://purl.org/NET/schema-org-csv#row": [
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4>"
      ], 
      "http://purl.org/dc/terms/source": "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv>", 
      "a": "<http://purl.org/NET/schema-org-csv#Table>"
    }, 
    {
      "http://purl.org/dc/terms/title": "Mass", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:2"
    }, 
    {
      "http://purl.org/dc/terms/title": "URL", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:3"
    }
  ], 
  "#": {
    "xml": "http://www.w3.org/XML/1998/namespace", 
    "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#", 
    "rdfs": "http://www.w3.org/2000/01/rdf-schema#"
  }
}

Low-Level API

The omnidator also has a low-level API that takes data formats (the same as the above INPUT FORMATS) as input. We 'guess' the type of input format via sniffing, the output is fixed to RDF/Turtle - as you may guess, the low-level API is more for experimenting with omnidator's internal workings.

For example, invoking the omnidator API as follows (for a CSV file that uses Schema.org terms in the header):

yields the following RDF/Turtle data:

@prefix dcterms: <http://purl.org/dc/terms/> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix scsv: <http://purl.org/NET/schema-org-csv#> .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#table> a <http://purl.org/NET/schema-org-csv#Table>;
    scsv:row <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4>;
    dcterms:source <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv>;
    dcterms:title "solar-system.csv" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1> a <http://purl.org/NET/schema-org-csv#HeaderRow>;
    scsv:cell <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:1>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:2>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:3>;
    dcterms:title "header" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:1> dcterms:title "Thing" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:2> dcterms:title "Mass" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:3> dcterms:title "URL" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2> a <http://purl.org/NET/schema-org-csv#Row>;
    scsv:cell <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:1>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:2>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:3>;
    dcterms:title "row 2" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:1> a <http://schema.org/Thing>;
    rdf:value "Earth" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:2> a <http://schema.org/Mass>;
    rdf:value "5.9721986x10^24 kg" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:3> a <http://schema.org/URL>;
    rdf:value <http://dbpedia.org/resource/Earth> .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3> a <http://purl.org/NET/schema-org-csv#Row>;
    scsv:cell <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:1>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:2>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:3>;
    dcterms:title "row 3" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:1> a <http://schema.org/Thing>;
    rdf:value "Mars" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:2> a <http://schema.org/Mass>;
    rdf:value "6.4191x10^23 kg" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:3> a <http://schema.org/URL>;
    rdf:value <http://dbpedia.org/resource/Mars> .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4> a <http://purl.org/NET/schema-org-csv#Row>;
    scsv:cell <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:1>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:2>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:3>;
    dcterms:title "row 4" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:1> a <http://schema.org/Thing>;
    rdf:value "Sun" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:2> a <http://schema.org/Mass>;
    rdf:value "1.988435x10^30 kg" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:3> a <http://schema.org/URL>;
    rdf:value <http://dbpedia.org/resource/Sun> .

Another example, a HTML page with Schema.org terms encoded in microdata:

yields the following RDF/Turtle data:

@prefix ns1: <http://schema.org/Event#> .
@prefix ns2: <http://schema.org/Place#> .
@prefix ns3: <http://schema.org/AggregateOffer#> .
@prefix ns4: <http://schema.org/PostalAddress#> .

<http://example.org/event123> a <http://schema.org/Event>;
    ns1:location [ a <http://schema.org/Place>;
            ns2:address [ a <http://schema.org/PostalAddress>;
                    ns4:addressLocality "Philadelphia";
                    ns4:addressRegion "PA" ];
            ns2:url <wells-fargo-center.html> ];
    ns1:offers [ a <http://schema.org/AggregateOffer>;
            ns3:lowPrice "$35";
            ns3:offerCount "1,938" ];
    ns1:startDate "2011-04-21T20:00";
    ns1:url <nba-miami-philidelphia-game3.html> .

[] a <http://schema.org/Event> .

Microdata API

You can use omnidator also to inspect microdata, like so:

For example, invoking the omnidator API as follows:

yields the following (text/plain) rendering of the contained items:

********************************************************************************
7 data items found in total:
--------------------------------------------------------------------------------
ITEM (http://example.org/event123) OF TYPE (http://schema.org/Event) {
  url = nba-miami-philidelphia-game3.html
  startDate = 2011-04-21T20:00
  offers ->
  ITEM (anonymous::cb1f06fa-94f2-11e0-a963-0d6c9d121c6d) OF TYPE (http://schema.org/AggregateOffer) {
    lowPrice = $35
    offerCount = 1,938
  }
  location ->
  ITEM (anonymous::cb1f0a87-94f2-11e0-be0d-0d6c9d121c6d) OF TYPE (http://schema.org/Place) {
    url = wells-fargo-center.html
    address ->
    ITEM (anonymous::cb1f0d11-94f2-11e0-8a1d-0d6c9d121c6d) OF TYPE (http://schema.org/PostalAddress) {
      addressLocality = Philadelphia
      addressRegion = PA
    }
  }
}
--------------------------------------------------------------------------------
ITEM (http://example.org/event456) {
}
--------------------------------------------------------------------------------
ITEM (anonymous::cb1f123a-94f2-11e0-a890-0d6c9d121c6d) OF TYPE (http://schema.org/Event) {
}
--------------------------------------------------------------------------------
ITEM (anonymous::cb1f1423-94f2-11e0-916f-0d6c9d121c6d) {
}

What is omnidator about?

Conceptually, omnidator takes whatever kind of data format that uses Schema.org terms and turns it into any other kind of data format. The following overview diagram renders omnidator's functionality on an abstract level:

The omnidator eats any kind of data formats - well, actually at the time only microdata and CSV - and translates it to the data lingua franca RDF. It then translates the data to any other target data formats. Currently JSON and RDF serialisations (RDF/XML and RDF/Turtle) are supported for the output, but we're working on the others, seriously ;).

Components

Technically, omnidator is a web application written in Python, deployed on Google App Engine, using the following components:

• Schema.org gateway, which, in turn, uses:
  • Ed Summers' rdflib-microdata rdflib-plugin for processing microdata.
  • A bespoke rdflib-plugin for processing CSV data (part of the Schema.org gateway) along with a vocabulary to express the structure of the table.
• The RDFLib library for processing RDF, the internal data representation.
• The Schema.org terms, to be more precise, the Schema.RDFS.org reformulations in RDFS.