Fork me on GitHub

omnipotentdatatranslator

Translate Schema.org data

Examples: CSV to JSON ยท Microdata to RDF/Turtle
from: to:

API

The omnidator provides a set of simple CORS-enabled APIs to translate from various data formats that use Schema.org terms to various data formats:

Main API

http://omnidator.appspot.com/{from}/{to}/?url={URL}

The {from} and {to} specifies the respective input and output format and can take the following values:

INPUT FORMATS:
 microdata
 csv

OUTPUT FORMATS:
 json (served as: application/json) 
 rdf-xml (served as: application/rdf+xml) 
 rdf-turtle (served as: text/turtle)

The {URL} is the HTTP URI (address) of the data source, such as a HTML+microdata document or a CSV file.

For example, invoking the omnidator API as follows (for translating a CSV file that uses Schema.org terms in the header):

http://omnidator.appspot.com/csv/json/?url=https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv

yields the following JSON data:

{
  "@": [
    {
      "a": "<http://schema.org/URL>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:3", 
      "rdf:value": "<http://dbpedia.org/resource/Earth>"
    }, 
    {
      "a": "<http://schema.org/Mass>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:2", 
      "rdf:value": "1.988435x10^30 kg"
    }, 
    {
      "http://purl.org/dc/terms/title": "Thing", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:1"
    }, 
    {
      "http://purl.org/dc/terms/title": "row 2", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2", 
      "http://purl.org/NET/schema-org-csv#cell": [
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:1>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:2>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:3>"
      ], 
      "a": "<http://purl.org/NET/schema-org-csv#Row>"
    }, 
    {
      "a": "<http://schema.org/Thing>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:1", 
      "rdf:value": "Mars"
    }, 
    {
      "http://purl.org/dc/terms/title": "row 3", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3", 
      "http://purl.org/NET/schema-org-csv#cell": [
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:2>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:3>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:1>"
      ], 
      "a": "<http://purl.org/NET/schema-org-csv#Row>"
    }, 
    {
      "a": "<http://schema.org/URL>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:3", 
      "rdf:value": "<http://dbpedia.org/resource/Sun>"
    }, 
    {
      "a": "<http://schema.org/URL>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:3", 
      "rdf:value": "<http://dbpedia.org/resource/Mars>"
    }, 
    {
      "http://purl.org/dc/terms/title": "row 4", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4", 
      "http://purl.org/NET/schema-org-csv#cell": [
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:3>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:2>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:1>"
      ], 
      "a": "<http://purl.org/NET/schema-org-csv#Row>"
    }, 
    {
      "a": "<http://schema.org/Mass>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:2", 
      "rdf:value": "6.4191x10^23 kg"
    }, 
    {
      "a": "<http://schema.org/Mass>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:2", 
      "rdf:value": "5.9721986x10^24 kg"
    }, 
    {
      "a": "<http://schema.org/Thing>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:1", 
      "rdf:value": "Sun"
    }, 
    {
      "http://purl.org/dc/terms/title": "header", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1", 
      "http://purl.org/NET/schema-org-csv#cell": [
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:2>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:1>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:3>"
      ], 
      "a": "<http://purl.org/NET/schema-org-csv#HeaderRow>"
    }, 
    {
      "a": "<http://schema.org/Thing>", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:1", 
      "rdf:value": "Earth"
    }, 
    {
      "http://purl.org/dc/terms/title": "solar-system.csv", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#table", 
      "http://purl.org/NET/schema-org-csv#row": [
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1>", 
        "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4>"
      ], 
      "http://purl.org/dc/terms/source": "<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv>", 
      "a": "<http://purl.org/NET/schema-org-csv#Table>"
    }, 
    {
      "http://purl.org/dc/terms/title": "Mass", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:2"
    }, 
    {
      "http://purl.org/dc/terms/title": "URL", 
      "@": "https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:3"
    }
  ], 
  "#": {
    "xml": "http://www.w3.org/XML/1998/namespace", 
    "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#", 
    "rdfs": "http://www.w3.org/2000/01/rdf-schema#"
  }
}

Low-Level API

The omnidator also has a low-level API that takes data formats (the same as the above INPUT FORMATS) as input. We 'guess' the type of input format via sniffing, the output is fixed to RDF/Turtle - as you may guess, the low-level API is more for experimenting with omnidator's internal workings.

http://omnidator.appspot.com/translate?url={URL}

For example, invoking the omnidator API as follows (for a CSV file that uses Schema.org terms in the header):

http://omnidator.appspot.com/translate?url=https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv

yields the following RDF/Turtle data:

@prefix dcterms: <http://purl.org/dc/terms/> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix scsv: <http://purl.org/NET/schema-org-csv#> .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#table> a <http://purl.org/NET/schema-org-csv#Table>;
    scsv:row <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4>;
    dcterms:source <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv>;
    dcterms:title "solar-system.csv" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1> a <http://purl.org/NET/schema-org-csv#HeaderRow>;
    scsv:cell <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:1>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:2>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:3>;
    dcterms:title "header" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:1> dcterms:title "Thing" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:2> dcterms:title "Mass" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:1,col:3> dcterms:title "URL" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2> a <http://purl.org/NET/schema-org-csv#Row>;
    scsv:cell <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:1>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:2>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:3>;
    dcterms:title "row 2" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:1> a <http://schema.org/Thing>;
    rdf:value "Earth" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:2> a <http://schema.org/Mass>;
    rdf:value "5.9721986x10^24 kg" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:2,col:3> a <http://schema.org/URL>;
    rdf:value <http://dbpedia.org/resource/Earth> .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3> a <http://purl.org/NET/schema-org-csv#Row>;
    scsv:cell <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:1>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:2>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:3>;
    dcterms:title "row 3" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:1> a <http://schema.org/Thing>;
    rdf:value "Mars" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:2> a <http://schema.org/Mass>;
    rdf:value "6.4191x10^23 kg" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:3,col:3> a <http://schema.org/URL>;
    rdf:value <http://dbpedia.org/resource/Mars> .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4> a <http://purl.org/NET/schema-org-csv#Row>;
    scsv:cell <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:1>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:2>,
        <https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:3>;
    dcterms:title "row 4" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:1> a <http://schema.org/Thing>;
    rdf:value "Sun" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:2> a <http://schema.org/Mass>;
    rdf:value "1.988435x10^30 kg" .

<https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/solar-system.csv#row:4,col:3> a <http://schema.org/URL>;
    rdf:value <http://dbpedia.org/resource/Sun> .

Another example, a HTML page with Schema.org terms encoded in microdata:

http://omnidator.appspot.com/translate?url=https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/md-test-1.html

yields the following RDF/Turtle data:

@prefix ns1: <http://schema.org/Event#> .
@prefix ns2: <http://schema.org/Place#> .
@prefix ns3: <http://schema.org/AggregateOffer#> .
@prefix ns4: <http://schema.org/PostalAddress#> .

<http://example.org/event123> a <http://schema.org/Event>;
    ns1:location [ a <http://schema.org/Place>;
            ns2:address [ a <http://schema.org/PostalAddress>;
                    ns4:addressLocality "Philadelphia";
                    ns4:addressRegion "PA" ];
            ns2:url <wells-fargo-center.html> ];
    ns1:offers [ a <http://schema.org/AggregateOffer>;
            ns3:lowPrice "$35";
            ns3:offerCount "1,938" ];
    ns1:startDate "2011-04-21T20:00";
    ns1:url <nba-miami-philidelphia-game3.html> .

[] a <http://schema.org/Event> .

Microdata API

You can use omnidator also to inspect microdata, like so:

http://omnidator.appspot.com/mdpretty?url={URL}

For example, invoking the omnidator API as follows:

http://omnidator.appspot.com/mdpretty?url=https://raw.github.com/mhausenblas/schema-org-rdf/master/tools/schema-gateway/test/md-test-1.html

yields the following (text/plain) rendering of the contained items:

********************************************************************************
7 data items found in total:
--------------------------------------------------------------------------------
ITEM (http://example.org/event123) OF TYPE (http://schema.org/Event) {
  url = nba-miami-philidelphia-game3.html
  startDate = 2011-04-21T20:00
  offers ->
  ITEM (anonymous::cb1f06fa-94f2-11e0-a963-0d6c9d121c6d) OF TYPE (http://schema.org/AggregateOffer) {
    lowPrice = $35
    offerCount = 1,938
  }
  location ->
  ITEM (anonymous::cb1f0a87-94f2-11e0-be0d-0d6c9d121c6d) OF TYPE (http://schema.org/Place) {
    url = wells-fargo-center.html
    address ->
    ITEM (anonymous::cb1f0d11-94f2-11e0-8a1d-0d6c9d121c6d) OF TYPE (http://schema.org/PostalAddress) {
      addressLocality = Philadelphia
      addressRegion = PA
    }
  }
}
--------------------------------------------------------------------------------
ITEM (http://example.org/event456) {
}
--------------------------------------------------------------------------------
ITEM (anonymous::cb1f123a-94f2-11e0-a890-0d6c9d121c6d) OF TYPE (http://schema.org/Event) {
}
--------------------------------------------------------------------------------
ITEM (anonymous::cb1f1423-94f2-11e0-916f-0d6c9d121c6d) {
}

What is omnidator about?

Conceptually, omnidator takes whatever kind of data format that uses Schema.org terms and turns it into any other kind of data format. The following overview diagram renders omnidator's functionality on an abstract level:

omnidator concept

The omnidator eats any kind of data formats - well, actually at the time only microdata and CSV - and translates it to the data lingua franca RDF. It then translates the data to any other target data formats. Currently JSON and RDF serialisations (RDF/XML and RDF/Turtle) are supported for the output, but we're working on the others, seriously ;).

Components

Technically, omnidator is a web application written in Python, deployed on Google App Engine, using the following components: