Reading documents

So far we have covered how to retrieve single documents using the $get() method, and how to retrieve (or list) multiple documents taking advantage either of the parent-child relationship or the membership relationship (with $listChildren(), $getMembers(), etc). Still, applications normally need more than just retrieving a single document or a list of documents.

In this document we’ll cover common use cases and how to accomplish them with the SDK.

The query function

We’ve been using the methods $get(), $getChildren() and $getMembers() with minimal configuration. But these methods also accept an additional parameter that allows us to specify constraints for the retrieval: the query parameter.

The query parameter must be a function that returns a very particular object. To help with the creation of said object, the function will receive as its only parameter a Builder object. It has the methods required to specify the desired constraints and return the object the function is expected to return.

We’ll call this function the query function.

To make things easier, here’s the general pattern to follow when using this function:

// ...
carbonldp.documents.$get( "parent/", _ => _
    .someConstraint()
    .someOtherConstraint()
);
// ...
// ...
carbonldp.documents.$get<Something>( "parent/", _ => _
    .someConstraint()
    .someOtherConstraint()
);
// ...
// ...
carbonldp.documents.$get( "parent/", function( _ ) {
    return _
        .someConstraint()
        .someOtherConstraint()
    ;
} );
// ...

Retrieving only specific properties

It is fairly common for applications to require only certain properties of a document. This can happen, for example, when rendering a list of objects and just displaying their name.

To retrieve only specific properties of a document, the Builder object provides the method properties().

Example

Let’s imagine that an application needs to render a list of employees using only their name and their profilePicture. This can be achieved doing the following:

// ...
carbonldp.documents.$getMembers( "employees/", _ => _
    .properties( {
        "name": {
            "@type": "string"
        },
        "profilePicture": {
            "@type": "string"
        }
    } )
);
// ...
// ...
carbonldp.documents.$getMembers<Employee>( "employees/", _ => _
    .properties( {
        "name": {
            "@type": "string"
        },
        "profilePicture": {
            "@type": "string"
        }
    } )
);
// ...
// ...
carbonldp.documents.$getMembers( "employees/", function( _ ) {
    return _
        .properties( {
            "name": {
                "@type": "string"
            },
            "profilePicture": {
                "@type": "string"
            }
        } )
    ;
} );
// ...

As you can see, the properties() method receives an object that looks very similar to the one used by the Object schema. The properties of this object define the property names to retrieve. Each property can specify the same things as in the Object schema, @id, @type, etc..

Now you may be wondering:

“Well, I already specified how I expect certain properties to be interpreted in my Object schema. Do I really need to declare them again?”

Of course not. If the properties were specified in the general Object schema, then you can point them to _.inherit:

.properties( {
    "name": _.inherit,
    "profilePicture": _.inherit
} )

_.inherit specifies that the property’s details should be pulled from the Object schema.

“But what if I don’t want to declare my properties on the general object schema?”

Like we said in the Object schema document, instead of declaring them in the general object schema it is a good practice to declare properties in the object schema of a specific type. But if you do that, the Builder object won’t have information to use for the properties specified.

We will address this problem in the next section.

Retrieving only documents of a specific type

If you have not worked with object schemas before or have no idea of what they are, please refer to the What is an object schema? section, particularly the Data Types (@type) section, to better understand the use of types while querying.

 

A document can store any type of documents as its children/members, but most of the time, applications only care about documents that follow a particular shape. Or said in other words, have a specific @type.

The Builder object provides the method withType() to specify only documents that have a desired @type that should be retrieved.

// ...
carbonldp.documents.$getMembers( "employees/", _ => _
    .withType( "Employee" )
);
// ...
// ...
carbonldp.documents.$getMembers<Employee>( "employees/", _ => _
    .withType( "Employee" )
);
// ...
// ...
carbonldp.documents.$getMembers( "employees/", function( _ ) {
    return _
        .withType( "Employee" )
    ;
} );
// ...

Filtering documents based on their type has an additional advantage, it provides additional information about the desired documents to the Builder object and because of this, the _.inherit object can now be used with properties that are located in that @type‘s schema:

// ...
carbonldp.extendObjectSchema( "Employee", {
    "name": {
        "@type": "string"
    },
    "profilePicture": {
        "@type": "string"
    }
} );

carbonldp.documents.$getMembers( "employees/", _ => _
    .withType( "Employee" )
    .properties( {
        "name": _.inherit,
        "profilePicture": _.inherit
    } )
).then( ( employees ) => {
    console.log( employees );
} );
// ...
// ...
carbonldp.extendObjectSchema( "Employee", {
    "name": {
        "@type": "string"
    },
    "profilePicture": {
        "@type": "string"
    }
} );

carbonldp.documents.$getMembers<Employee>( "employees/", _ => _
    .withType( "Employee" )
    .properties( {
        "name": _.inherit,
        "profilePicture": _.inherit
    } )
).then( ( employees:(Employee & Document)[] ) => {
    console.log( employees );
} );
// ...
// ...
carbonldp.extendObjectSchema( "Employee", {
    "name": {
        "@type": "string"
    },
    "profilePicture": {
        "@type": "string"
    }
} );

carbonldp.documents.$getMembers( "employees/", function( _ ) {
    return _
        .withType( "Employee" )
        .properties( {
            "name": _.inherit,
            "profilePicture": _.inherit
        } )
    ;
} ).then( function( employees ) {
    console.log( employees );
} );
// ...

Filtering documents

The method withType() can filter documents based on their types, but it is fairly common for applications to need to filter them based on their properties instead (or more complex conditions).

The Builder object provides the method filter() which can be used to filter the documents to be retrieved based on a custom condition.

This method accepts a SPARQL FILTER expression (you can find more about them here). If you don’t
know SPARQL, don’t worry. The Builder object also provides methods to create those expressions without SPARQL knowledge.

Comparison operators

Filter expressions accept the following comparison operators:

= Equal to
!= Not equal to
> More than
< Less than
>= More than or equal to
<= Less than or equal to

Using JavaScript values

Values in filter expressions need to be wrapped using the _.value() method. This makes the Builder object convert the native JavaScript value to a SPARQL value. Example:

_.value( "Hello!" )
_.value( 1289.9 )
_.value( someVariable )
_.value( new Date() )
// etc

Referring to properties

To reference a property of the document in the filter expression, the Builder object provides the method: _.property(). It accepts the name of the property to reference:

_.property( "name" )

Comparing values

Using these three tools comparison expressions can be used in the filter expression. For example, if we wanted to get only blog posts that had a publishedOn value previous to the current date (meaning they are already published), we could accomplish it using the following code:

// ...
carbonldp.documents.$getMembers( "posts/", _ => _
    .properties( {
        "title": {
            "@type": "string"
        },
        "content": {
            "@type": "string"
        },
        "publishedOn": {
            "@type": "dateTime"
        }
    } )
    .filter( `${_.property( "publishedOn" )} < ${_.value( new Date() )}` )
);
// ...
// ...
carbonldp.documents.$getMembers<BlogPost>( "posts/", _ => _
    .properties( {
        "title": {
            "@type": "string"
        },
        "content": {
            "@type": "string"
        },
        "publishedOn": {
            "@type": "dateTime"
        }
    } )
    .filter( `${_.property( "publishedOn" )} < ${_.value( new Date() )}` )
);
// ...
// ...
carbonldp.documents.$getMembers( "posts/", function( _ ) {
    return _
        .properties( {
            "title": {
                "@type": "string"
            },
            "content": {
                "@type": "string"
            },
            "publishedOn": {
                "@type": "dateTime"
            }
        } )
        .filter( _.property( "publishedOn" ) + " < " + _.value( new Date() ) )
    ;
} );
// ...

Parenthesis and logical operators

Filter expressions accept parenthesis to wrap comparisons or arithmetic expressions. They can be used together with the following supported logical operators:

&& And
|| Or

For example, to retrieve published blog posts that don’t have the property deleted set to true, the following code could be used:

// ...
carbonldp.documents.$getMembers( "posts/", _ => _
    .properties( {
        "title": {
            "@type": "string"
        },
        "content": {
            "@type": "string"
        },
        "publishedOn": {
            "@type": "dateTime"
        },
        "deleted": {
            "@type": "boolean"
        }
    } )
    .filter( `( ${_.property( "publishedOn" )} < ${_.value( new Date() )} ) && ( ${_.property( "deleted" )} != ${_.value( true )} )` )
);
// ...
// ...
carbonldp.documents.$getMembers<BlogPost>( "posts/", _ => _
    .properties( {
        "title": {
            "@type": "string"
        },
        "content": {
            "@type": "string"
        },
        "publishedOn": {
            "@type": "dateTime"
        },
        "deleted": {
            "@type": "boolean"
        }
    } )
    .filter( `( ${_.property( "publishedOn" )} < ${_.value( new Date() )} ) && ( ${_.property( "deleted" )} != ${_.value( true )} )` )
);
// ...
// ...
carbonldp.documents.$getMembers( "posts/", function( _ ) {
    return _
        .properties( {
            "title": {
                "@type": "string"
            },
            "content": {
                "@type": "string"
            },
            "publishedOn": {
                "@type": "dateTime"
            },
            "deleted": {
                "@type": "boolean"
            }
        } )
        .filter( "( " + _.property( "publishedOn" ) + " < " + _.value( new Date() ) + " ) && ( " + _.property( "deleted" ) + " != " + _.value( true ) + " )"  )
    ;
} );
// ...

Paging through ordered documents

Sometimes collections can get very big and retrieving them all together on a single request can take a long time (or even crash). To avoid this, the Builder object provides three methods that let you page through the documents of a collection:

orderBy()

Orders the documents to be retrieved based on one of their properties. This method accepts two parameters:

  • The property name
  • Either "ascending"/"ASC" or "descending"/"DESC" to specify the order
limit() Number of documents to retrieve
offset() Indicates the number of documents to ignore when retrieving results. An offset of 0 has no effect

Note: Using limit() and offset() without orderBy() can have unpredictable results as the order will be decided by the platform (or there may be no order at all).

Example

Continuing with the Blog example, let’s imagine that we needed to retrieve the 10 most recent posts for the post feed. This could be achieved with the following code:

// ...
carbonldp.documents.$getMembers( "posts/", _ => _
    .properties( {
        "title": {
            "@type": "string"
        },
        "content": {
            "@type": "string"
        },
        "publishedOn": {
            "@type": "dateTime"
        },
        "deleted": {
            "@type": "boolean"
        }
    } )
    .filter( `( ${_.property( "publishedOn" )} < ${_.value( new Date() )} ) && ( ${_.property( "deleted" )} != ${_.value( true )} )` )
    .orderBy( "publishedOn", "descending" )
    .limit( 10 )
);
// ...
// ...
carbonldp.documents.$getMembers<BlogPost>( "posts/", _ => _
    .properties( {
        "title": {
            "@type": "string"
        },
        "content": {
            "@type": "string"
        },
        "publishedOn": {
            "@type": "dateTime"
        },
        "deleted": {
            "@type": "boolean"
        }
    } )
    .filter( `( ${_.property( "publishedOn" )} < ${_.value( new Date() )} ) && ( ${_.property( "deleted" )} != ${_.value( true )} )` )
    .orderBy( "publishedOn", "descending" )
    .limit( 10 )
);
// ...
// ...
carbonldp.documents.$getMembers( "posts/", function( _ ) {
    return _
        .properties( {
            "title": {
                "@type": "string"
            },
            "content": {
                "@type": "string"
            },
            "publishedOn": {
                "@type": "dateTime"
            },
            "deleted": {
                "@type": "boolean"
            }
        } )
        .filter( "( " + _.property( "publishedOn" ) + " < " + _.value( new Date() ) + " ) && ( " + _.property( "deleted" ) + " != " + _.value( true ) + " )"  )
        .orderBy( "publishedOn", "descending" )
        .limit( 10 )
    ;
} );
// ...
After that, if the applicat

After that, if the application needed to retrieve the next 10 posts the only change needed would be to add the offset() method:

// ...
.orderBy( "publishedOn", "descending" )
.limit( 10 )
.offset( 10 )
// ...

Retrieving nested objects

So far we have covered only simple object structures. But sometimes an application needs to retrieve a more complex one, like for example the following objects tree:

{
    "$id": "employees/john-snow/",
    "types": [ "Employee" ],
    "name": "John Snow",
    "profilePicture": "http://.../profile.png",
    "projects": [
        {
            "$id": "projects/website-redesign/",
            "types": [ "Project" ],
            "title": "Website redesign"
        },
        {
            "$id": "projects/acme-intranet/",
            "types": [ "Project" ],
            "title": "ACME's Intranet"
        }
    ]
}

Retrieving connected objects can be accomplished by specifying that the value of a property (of @id type) should be included in the response. To do it, a query property needs to be declared in a property description object.

This query property accepts another query function that will be used to apply constraints to the values of that
property.

In this query function the desired properties of the nested object can be specified too (along with everything we have included covered so far).

To retrieve the previous objects tree the following code could be used:

// ...
carbonldp.extendObjectSchema( "Employee", {
    "name": {
        "@type": "string"
    },
    "profilePicture": {
        "@type": "string"
    },
    "projects": {
        "@type": "@id"
    }
} );
carbonldp.extendObjectSchema( "Project", {
    "title": {
        "@type": "string"
    }
} );

carbonldp.documents.$get( "employees/john-snow/", _ => _
    .withType( "Employee" )
    .properties( {
        "name": _.inherit,
        "profilePicture": _.inherit,
        "projects": {
            "query": _ => _
                .withType( "Project" )
                .properties( {
                    "title": _.inherit
                } )
        }
    } )
).then( ( employee ) => {
    console.log( employee.projects.length ); // 2
    console.log( employee.projects[ 0 ].title ); // Website redesign
} );
// ...
// ...
carbonldp.extendObjectSchema( "Employee", {
    "name": {
        "@type": "string"
    },
    "profilePicture": {
        "@type": "string"
    },
    "projects": {
        "@type": "@id"
    }
} );
carbonldp.extendObjectSchema( "Project", {
    "title": {
        "@type": "string"
    }
} );

carbonldp.documents.$get<Employee>( "employees/john-snow/", _ => _
    .withType( "Employee" )
    .properties( {
        "name": _.inherit,
        "profilePicture": _.inherit,
        "projects": {
            "query": _ => _
                .withType( "Project" )
                .properties( {
                    "title": _.inherit
                } )
        }
    } )
).then( ( employee:Employee & Document ) => {
    console.log( employee.projects.length ); // 2
    console.log( employee.projects[ 0 ].title ); // Website redesign
} );
// ...
// ...
carbonldp.extendObjectSchema( "Employee", {
    "name": {
        "@type": "string"
    },
    "profilePicture": {
        "@type": "string"
    },
    "projects": {
        "@type": "@id"
    }
} );

carbonldp.documents.$get( "employees/john-snow/", function( _ ) {
    return _
        .withType( "Employee" )
        .properties( {
            "name": _.inherit,
            "profilePicture": _.inherit,
            "projects": {
                "query": function( _ ) {
                    return _
                        .withType( "Project" )
                        .properties( {
                            "title": _.inherit
                        } )
                    ;
                }
            }
        } )
    ;
} ).then( function( employee ) {
    console.log( employee.projects.length ); // 2
    console.log( employee.projects[ 0 ].title ); // Website redesign
} );
// ...

Even though the projects property isn’t pointing to _.inherit, it will still inherit whatever wasn’t specified on the description object from the object schema. In this case, the @type gets inherited since it wasn’t explicitly defined.

Note that nested queries can be as deep as you need them to be but the requests may become slow.

Conclusion

By specifying a query function, the application can control the data it is going to retrieve and reduce the number of requests it executes.

But sometimes these capabilities aren’t enough. After all, the query function isn’t as flexible as a fully-fledged query language.

In the next document we’ll cover how to use SPARQL, a query language that can cover the use cases the query function can’t.