Enterprise-class repositories

Introduction

The platform’s repository is the database that stores all of its data.

Out of the box the platform comes with a file-based repository that is suitable for development and low-demand usage. This repository doesn’t offer high availability (HA) nor disaster recovery (DR) procedures. That’s why we don’t recommend it for production environments (although in theory some basic DR procedures could be configured).

For production environments (especially with high-loads) we recommend configuring an external repository for the platform.

There are several options to choose from, in this document we’ll cover them along with their configuration.

Stardog

Stardog is a leading repository used by multiple enterprise organizations. You can read more about it on their website: stardog.com.

To configure a Stardog server as the platform’s repository you need to follow these steps:

  1. Create a database in the Stardog server for the platform to use
  2. Set query.all.graphs to true for the database you created. This can be achieved running the following command in the Stardog’s bin directory:

    ./stardog-admin metadata set --option "query.all.graphs=true" name-of-the-database-you-created

    Note that you need to replace name-of-the-database-you-created with the name of the database you created on the previous step.

  3. Configure the following platform’s configuration parameters:

    • carbonldp.repository.type => stardog
    • carbonldp.repository.url => Stardog’s server url (by default it listens on http://localhost:5820/
    • carbonldp.repository.username => Stardog’s user username (if you are only testing you can use Stardog’s default username admin
    • carbonldp.repository.password => Stardog’s user password (if you are only testing you can use Stardog’s default password admin
    • carbonldp.repository.id => The id of the database you created on the previous step

    Here’s an example of a command to run a platform instance configured to use a Stardog server:

    docker run -d --name carbonldp -p 8083:8083 carbonldp/carbonldp-platform:5.0.0 \
        --carbonldp.repository.type="stardog" \
        --carbonldp.repository.url="http://stardog-instance:5820/" \
        --carbonldp.repository.username="admin" \
        --carbonldp.repository.password="admin" \
        --carbonldp.repository.id="carbonldp"
    

    Note that this can also be achieved by changing the config.properties file. See: Platform’s configuration.

Common pitfalls

Unable to obtain a connection to the Stardog database

If you are running the platform as a Docker container and the Stardog server as a service of the Docker host (accessing it as http://localhost:5820/ or something similar), the platform will not be able to access the server.

Docker containers are “somewhat” independent systems from the host. They are their own localhost. So by telling the platform to connect to http://localhost:5820/ the platform will actually look for that service inside of the container.

We won’t offer a solution here as it greatly depends on your Docker installation and most of the times the steps required are os-specific.

To get more information you can try to search for: docker container accessing host service

Although we won’t detail a solution, there are a couple of workarounds you can take to avoid the problem all together:

  • Option 1:

    Configure the Stardog server so it is accessible through a resolvable hostname.

    E.g. http://my-stardog-instance:5820/
  • Option 2:

    Run the Stardog server as a container and link it to the platform’s container

    docker run -d --name carbonldp -p 8083:8083 \
        --link stardog:stardog \                                 # Stardog's container name
        carbonldp/carbonldp-platform:5.0.0 \
            --carbonldp.repository.type="stardog" \
            --carbonldp.repository.url="http://stardog:7200/" \
            --carbonldp.repository.username="admin" \
            --carbonldp.repository.password="admin" \
            --carbonldp.repository.id="carbonldp"
    

GraphDB

GraphDB is an enterprise ready repository developed by Ontotext. You can read more about it on their website: ontotext.com.

To configure a GraphDB server as the platform’s repository you need to follow these steps:

  1. Create a “repository” in the GraphDB server for the platform to use
  2. Configure the following platform’s configuration parameters:

    • carbonldp.repository.type => remote
    • carbonldp.repository.url => GraphDB’s server url (by default it listens on http://localhost:7200/
    • carbonldp.repository.username => GraphDB’s user username (if you are only testing you can use GraphDB’s default username admin
    • carbonldp.repository.password => GraphDB’s user password (if you are only testing you can use GraphDB’s default password admin
    • carbonldp.repository.id => The id of the database you created on the previous step

    Here’s an example of a command to run a platform instance configured to use a GraphDB server:

    docker run -d --name carbonldp -p 8083:8083
        carbonldp/carbonldp-platform:5.0.0 \
            --carbonldp.repository.type="remote" \
            --carbonldp.repository.url="http://graphdb-instance:7200/" \
            --carbonldp.repository.username="admin" \
            --carbonldp.repository.password="admin" \
            --carbonldp.repository.id="carbonldp"

    Note that this can also be achieved by changing the config.properties file. See: Platform’s configuration.

Common pitfalls

Unable to obtain a connection to the remote database

If you are running the platform as a Docker container and the GraphDB server as a service of the Docker host (accessing it as http://localhost:7200/ or something similar, the platform will not be able to access the server.

Docker containers are “somewhat” independent systems from the host. They are their own localhost. So by telling the platform to connect to http://localhost:7200/ the platform will actually look for that service inside of the container.

We won’t offer a solution here as it greatly depends on your Docker installation and most of the times the steps required are os-specific.

To get more information you can try to search for: docker container accessing host service

Although we won’t detail a solution, there are a couple workarounds you can take to avoid the problem all together:

  • Option 1:

    Configure the GraphDB server so it is accessible through a resolvable hostname.

    E.g. http://my-graphdb-instance:7200/
  • Option 2:

    Run the GraphDB server as a container and link it to the platform’s container

    docker run -d --name carbonldp -p 8083:8083 \
        --link graphdb:graphdb \                                 # GraphDB's container name
        carbonldp/carbonldp-platform:5.0.0 \
            --carbonldp.repository.type="remote" \
            --carbonldp.repository.url="http://graphdb:7200/" \
            --carbonldp.repository.username="admin" \
            --carbonldp.repository.password="admin" \
            --carbonldp.repository.id="carbonldp"

Note that in the example above, the name defined by the link parameter establishes the URL to be used by the carbonldp.repository.url parameter; so they must match.