Olingo provides an Java implementation of OData for both client and server sides. Regarding the server side, it provides a frame to handle OData requests, specially the queries described with the OData within the query parameter $filter.
We dont provide here a start guide to implement an OData service with Olingo (it will be the subjet of another post) but focus on the way to handle queries. We first deal with the basic frame in Olingo to implement queries and then how to translate them to ElasticSearch ones. To finish, we also tackle other query parameters to control the entity fields returned ($select) and the data set returned and pagination ($top and $skip).
Handling OData queries in Olingo
Olingo is based on the concept of processor to handle OData requests. The library allows to register a processor class that implements a set of interfaces describing what it can handle. In the following snippet, we create a processor that can handle entity collection, entity collection count and entity requests.
public class ODataProviderEntityProcessor
implements EntityCollectionProcessor,
CountEntityCollectionProcessor,
EntityProcessor {
@Override
public void readEntityCollection(final ODataRequest request,
ODataResponse response, final UriInfo uriInfo,
final ContentType requestedContentType)
throws ODataApplicationException, SerializerException {
(...)
}
@Override
public void countEntityCollection(ODataRequest request,
ODataResponse response, UriInfo uriInfo)
throws ODataApplicationException, SerializerException {
(...)
}
@Override
public void readEntity(final ODataRequest request, ODataResponse response,
final UriInfo uriInfo, final ContentType requestedContentType)
throws ODataApplicationException, SerializerException {
(...)
}
}
Imagine that we have a entity set called products of type Product. When we access the OData service with the URL http://myservice.org/odata.svc/products, Olingo will route the request to the method readEntityCollection of our processor. The objects provided as parameters will contain of the hints regarding the request and allow to set elements to return within the response.
If we want to use queries, we simply need to leverage the query parameter $filter. So if we want to get all the products with name MyProductName, we can simply use this URL: http://myservice.org/odata.svc/products?$filter=name eq 'MyProductName'. Within the processor the query expression can be reached using the parameter uriInfo, as described below:
@Override
public void readEntityCollection(final ODataRequest request,
ODataResponse response, final UriInfo uriInfo,
final ContentType requestedContentType)
throws ODataApplicationException, SerializerException {
FilterOption filterOption = uriInfo.getFilterOption();
(...)
}
The query support of Olingo doesnt stop here, since it parses the query string for us and allows to based on the classical pattern Visitor. To implement such processing, we simply need to create a class that implements the interface ExpressionVisitor and uses it on the parsed expression, as described below:
Expression expression = filterOption.getExpression();
QueryBuilder queryBuilder = expression
.accept(new ElasticSearchExpressionVisitor());
The visitor class contains the methods that can will be called when an element of the parsed expression is encountered. A sample empty implementation is described below with the main methods:
public class ElasticSearchExpressionVisitor implements ExpressionVisitor {
@Override
public Object visitBinaryOperator(BinaryOperatorKind operator,
Object left, Object right)
throws ExpressionVisitException,
ODataApplicationException {
(...)
}
@Override
public Object visitUnaryOperator(UnaryOperatorKind operator, Object operand)
throws ExpressionVisitException, ODataApplicationException {
(...)
}
@Override
public Object visitMethodCall(MethodKind methodCall, List parameters)
throws ExpressionVisitException, ODataApplicationException {
(...)
}
@Override
public Object visitLiteral(String literal)
throws ExpressionVisitException, ODataApplicationException {
(...)
}
@Override
public Object visitMember(UriInfoResource member)
throws ExpressionVisitException, ODataApplicationException {
(...)
}
}
This approach allows to handle several levels within queries. The returned elements of methods corresponds to the elements that will be passed as parameters to other method calls. Lets take a simple example based on the expression name eq 'MyProductName'. Here are the different method calls:
- method
visitMember. The variablememberof typeUriInfoResourcecontains potentially several parts to support something like thatfield1/subField2. We can here simply extract the stringnameand returns it. - method
visitLiteral. The variableliteralcontains the value'MyProductName'. Since we are in the case of a string literal, we need to extract the string valueMyProductNameand returns it. If it was an integer, we could convert it to an integer and return it. - method
visitBinaryOperator. The variableoperatorcontains the type of operator,BinaryOperatorKind.EQin our case. The other parameters correspond to the values returned by the previous method.
Here is a sample implementation of methods visitMember and visitLiteral:
@Override
public Object visitLiteral(String literal)
throws ExpressionVisitException, ODataApplicationException {
return ODataQueryUtils.getRawValue(literal);
}
@Override
public Object visitMember(UriInfoResource member)
throws ExpressionVisitException, ODataApplicationException {
if (member.getUriResourceParts().size() == 1) {
UriResourcePrimitiveProperty property
= (UriResourcePrimitiveProperty)
member.getUriResourceParts().get(0);
return property.getProperty().getName();
} else {
List<String> propertyNames = new ArrayList<String>();
for (UriResource property : member.getUriResourceParts()) {
UriResourceProperty primitiveProperty
= (UriResourceProperty) property;
propertyNames.add(primitiveProperty.getProperty().getName());
}
return propertyNames;
}
}
Now we have described general principles to handle OData queries within Olingo, we can focus now on how to convert these queries to ElasticSearch ones.
Implementing the interaction with ElasticSearch
Now we have tackle generic concepts and have a look at Olingo classes to implement queries, we will now focus on the ElasticSearch specific stuff. We will use the official Java client to execute such queries from Olingo processors. We leverage the class SearchRequestBuilder and create it using the method prepareSearch of the client. The query can be configured within this request. The corresponding result data will be then convert to OData entities and send back to the client.
The following code shows a sample implementation of such processing within the processor previously described:
@Override
public EntitySet readEntitySet(EdmEntitySet edmEntitySet,
FilterOption filterOption, SelectOption selectOption,
ExpandOption expandOption, OrderByOption orderByOption,
SkipOption skipOption, TopOption topOption) {
EdmEntityType type = edmEntitySet.getEntityType();
FullQualifiedName fqName = type.getFullQualifiedName();
QueryBuilder queryBuilder = createQueryBuilder(
filterOption, expandOption);
SearchRequestBuilder requestBuilder = client
.prepareSearch(fqName.getNamespace())
.setTypes(fqName.getName())
.setQuery(queryBuilder);
configureSearchQuery(requestBuilder, selectOption,
orderByOption, skipOption, topOption);
SearchResponse response = requestBuilder.execute().actionGet();
EntitySet entitySet = new EntitySetImpl();
SearchHits hits = response.getHits();
for (SearchHit searchHit : hits) {
Entity entity = convertHitToEntity(
searchHit, type, edmProvider);
entity.setType(fqName.getName());
entitySet.getEntities().add(entity);
}
return entitySet;
}
We will now describe how to actually create ElasticSearch queries.
Creating ElasticSearch queries from OData requests
With OData, we can get all data for a particular type but also filter them using a query. If we want to get all data, we can use the query . In other case, the ElasticSearch query creation will be a bit more tricky. The latter will be created within a Olingo query expression visitor and can have serveral levels.
The following code describes the entry point method to create the ElasticSearch query:
public QueryBuilder createQueryBuilder(FilterOption filterOption) {
if (filterOption != null) {
Expression expression = filterOption.getExpression();
return expression.accept(
new ElasticSearchExpressionVisitor());
} else {
return QueryBuilders.matchAllQuery();
}
}
We dont describe here all possible cases but focus on two different operators. The first one is the equality one. Its implementation is pretty straightforward using a match query within the method visitBinaryOperator of our expression visito. We need however be careful to handle the case where the value is null.
@Override
public Object visitBinaryOperator(
BinaryOperatorKind operator, Object left, Object right)
throws ExpressionVisitException, ODataApplicationException {
if (BinaryOperatorKind.EQ.equals(operator)) {
String fieldName = left;
Object value = right;
if (value!=null) {
return QueryBuilders.matchQuery(fieldName, value);
} else {
return QueryBuilders.filteredQuery(QueryBuilders
.matchAllQuery(), FilterBuilders.missingFilter(fieldName));
}
}
(...)
}
We can notice that in the case where the field isnt indexed, a term query would be much relevant.
In the case of an operator, we only one level within the ElasticSearch query. The Olingo approach based on an expression visitor allows to compound more complex queries. We can take the sample of an operator that associates to sub queries, something like with OData query name eq 'MyProductName' and price eq 15. In this case, the following visitor methods will be called successfully:
- method
visitMemberwith membername. - method
visitLiteralwith value'MyProductName'. - method
visitBinaryOperatorwith operatoreqthat create the first sub query (query #1). - method
visitMemberwith memberprice. - method
visitLiteralwith value15. - method
visitBinaryOperatorwith operatoreqthat create the second sub query (query #2). - method
visitBinaryOperatorwith operatorand. The first parameter corresponds to query #1 and the second to query #2.
Having understand this, we can leverage an ElasticSeach filter to create our composite query within the method visitBinaryOperator, as describe below:
@Override
public Object visitBinaryOperator(
BinaryOperatorKind operator, Object left, Object right)
throws ExpressionVisitException, ODataApplicationException {
(...)
if (BinaryOperatorKind.AND.equals(operator)) {
return QueryBuilders.filteredQuery(QueryBuilders.matchAllQuery(),
FilterBuilders.andFilter(
FilterBuilders.queryFilter((QueryBuilder) left),
FilterBuilders.queryFilter((QueryBuilder) right)));
}
(...)
}
We describe here how to translate OData queries to ElasticSearch ones by leveraging the expression visitor of Olingo. We took concrete samples of an equals query and of a composite one.
In the next section, we will describe how to take into account nested fields within queries
Handling queries on nested fields
Within our equals operator support, we didnt take into account the fact that OData supports sub fields. As a matter of fact, we can have something like that: details/fullName eq 'My product details'. The field details would be an OData complex field and an ElasticSearch nested field. For such use case, we need to extend our support of the operator to handle both case:
- normal fields with match or term queries
- complex fields with nested queries.
The following code describes an adapted version of our method visitBinaryOperator to support this case:
@Override
public Object visitBinaryOperator(BinaryOperatorKind operator,
Object left, Object right)
throws ExpressionVisitException,
ODataApplicationException {
if (BinaryOperatorKind.EQ.equals(operator)) {
List<String> fieldNames = getFieldNamesAsList(left);
if (fieldNames.size() == 1) {
String fieldName = fieldNames.get(0);
Object value = right;
if (value!=null) {
return QueryBuilders.matchQuery(fieldName, value);
} else {
return QueryBuilders.filteredQuery(QueryBuilders
.matchAllQuery(), FilterBuilders.missingFilter(fieldName));
}
} else if (fieldNames.size() > 1) {
Object value = right;
if (value!=null) {
return QueryBuilders.nestedQuery(getRootFieldName(fieldNames),
QueryBuilders.matchQuery(
getTargetNestedFieldNames(fieldNames), value));
} else {
return QueryBuilders.nestedQuery(getRootFieldName(fieldNames),
QueryBuilders.filteredQuery(QueryBuilders
.matchAllQuery(), FilterBuilders.missingFilter(
getTargetNestedFieldNames(fieldNames))));
}
}
(...)
}
(...)
}
The last point will see here consists in the ability to parameterizing a subset of returned data.
Parameterizing the returned data
OData allows to specify a subset of data to return. This obviously applies to queries based on the following query parameters:
- $select to specify which fields will be included in returned entities
- $top to specify the maximum number of returned entities
- $skip to specify the index of the first entity of the returned subset
The two last parameters are particularly convenient to implement data pagination with OData.
Such parameters can be used to parameterized the ElasticSearch search request, as described below:
public void configureSearchQuery(
SearchRequestBuilder requestBuilder,
SelectOption selectOption, OrderByOption orderByOption,
SkipOption skipOption, TopOption topOption) {
requestBuilder.setSize(1000);
if (selectOption!=null) {
for (SelectItem selectItem : selectOption.getSelectItems()) {
requestBuilder.addField(selectItem.getResourcePath()
.getUriResourceParts().get(0).toString());
}
}
if (topOption!=null) {
requestBuilder.setSize(topOption.getValue());
} else {
requestBuilder.setSize(DEFAULT_QUERY_DATA_SIZE);
}
if (skipOption!=null) {
requestBuilder.setFrom(skipOption.getValue());
}
}
Sandbox for the Web stack 
Hi… Thanks for your good post.i read your post and it is very useful for me to implement filter in my project But i struck to implement now .I don’t know how to implement method present in above code snippet.You just call convertHitToEntity method in readEntitySet method but you don’t mention implementation of convertHitToEntity method.same thing is for different method you posted here ElasticSearchExpressionVisitor class.Kindly give full source code of the implementation.So that it is helpful for all as.