I was reading about logstash(http://logstash.net/), a tool to mine raw logs and make data useful. I got attracted to the elastic search module it incorporates. Elastic search(http://www.elasticsearch.org/) helps in indexing documents and thereby increase the performance of text searches. When you pass a document to elastic search, elastic search indexes the document and creates an inverted index. Inverted index is like an index at the back of the book. The terms are mapped to the documents they belong to. So text searching becomes easier.
I had a quick read through elastic search and tried a little bit of hands-on to use elastic search and create an autocomplete system. Elastic Search allows you to create an index which is similar to a database, an index can have types which is similar to tables and each type has mapping similar to schema.
Elastic search has a set of tokenizers and analyzers for text processing which it inherits from Lucene project.
Elastic search helps to scale with multiple shards across a cluster. During a search operation segments from shards are looked up for inverted index and the result is aggregated by the master and sends the response. With this horizontal scalability, elastic search outperforms its counterparts.
Download elastic search and start it by running elasticsearch in the bin directory.(Follow these steps
http://www.elasticsearch.org/guide/reference/setup/installation/)
Elastic search exposes a simple rest api to interact with it. Follow the steps to quickly create a dirty autocomplete feature
1)This creates an index trial and adds a shingle filter(http://www.elasticsearch.org/guide/reference/index-modules/analysis/shingle-tokenfilter/). Shingle filter splits a text "this is a test" as "this" "this is" "this is a" and "this is a test"
2)The following command adds a mapping, and creates a field 'description' which uses shingle analyzer
I had a quick read through elastic search and tried a little bit of hands-on to use elastic search and create an autocomplete system. Elastic Search allows you to create an index which is similar to a database, an index can have types which is similar to tables and each type has mapping similar to schema.
Elastic search has a set of tokenizers and analyzers for text processing which it inherits from Lucene project.
Elastic search helps to scale with multiple shards across a cluster. During a search operation segments from shards are looked up for inverted index and the result is aggregated by the master and sends the response. With this horizontal scalability, elastic search outperforms its counterparts.
Download elastic search and start it by running elasticsearch in the bin directory.(Follow these steps
http://www.elasticsearch.org/guide/reference/setup/installation/)
Elastic search exposes a simple rest api to interact with it. Follow the steps to quickly create a dirty autocomplete feature
1)This creates an index trial and adds a shingle filter(http://www.elasticsearch.org/guide/reference/index-modules/analysis/shingle-tokenfilter/). Shingle filter splits a text "this is a test" as "this" "this is" "this is a" and "this is a test"
curl -XPOST 'http://localhost:9200/trial?pretty=1' -d '
{
"settings": {
"analysis": {
"analyzer": {
"suggestions": {
"tokenizer": "standard",
"filter": ["suggestions_shingle"]
}
},
"filter": {
"suggestions_shingle": {
"type": "shingle",
"min_shingle_size": 2,
"max_shingle_size": 5
}
}
}
}
}'
2)The following command adds a mapping, and creates a field 'description' which uses shingle analyzer
curl -XPUT localhost:9200/trial/doc/_mapping -d '3)The following command adds few documents to elastic search so that it can index them and apply shingle filter
{
doc:
{
properties:
{
description:{
type:"multi_field",
fields:{
"description": { "type": "string", "analyzer": "standard", "include_in_all": true },
"suggestions": { "type": "string", "analyzer": "suggestions", "include_in_all": false }
}
}
}
}
}
'
curl -XPUT localhost:9200/trial/doc/1 -d'{description:"this is a shingles test. I am impressed with its usage"}'
4)Now facet(http://www.elasticsearch.org/guide/reference/api/search/facets/) is a feature of elastic search which gives possible combination to the regex we query, from the docs we have fed into the system.curl -XPUT localhost:9200/trial/doc/2 -d '{description:"elastic search uses lucene to index docs, it creates inverted index"}'curl -XPUT localhost:9200/trial/doc/3 -d '{description:"Last doc on elastic search. this will form examples in my blog"}'
curl -XGET 'http://localhost:9200/trial/doc/_seach?pretty=1' -d '{
"query":{
"prefix":{
"description.suggestions":"to"
}
},
"facets":{
"description_suggestions":{
"terms":{
"field":"description.suggestions",
"regex":"lucene.to.*",
"size": 10
}
}
}
}'An Ajax frontend will help us create a smooth auto complete feature with elastic search.
Comments
Post a Comment