How Elasticsearch works and basics .

Elastic Search Basic

 Introduction : Elasticsearch is a real-time distributed and open source full-text search and analytics engine. It is accessible from Restful web service interface and uses schema less JSON (JavaScript Object Notation) documents to store data. It is built on Java programming language and hence Elasticsearch can run on different platforms. It enables users to explore very large amount of data at very high speed.

Advantages of Elasticsearch 

1. BUILT ON TOP OF LUCIENNE – Being built on top of Lucienne, it offers the most powerful full-text search capabilities.It makes it very faster also.
2. DOCUMENT-ORIENTED – It stores complex entities as structured JSON documents and indexes all fields by default, providing a higher performance.
3. SCHEMA FREE – It stores a large quantity of semi-structured (JSON) data in a distributed fashion. It also attempts to detect the data structure, index the data present and makes it search-friendly.
4. FULL TEXT SEARCH – Elasticsearch performs linguistic searches against documents and returns the documents that matches the search condition. Result relevancy for the given query is calculated using TF/IDF algorithm.
5. FULL TEXT SEARCH – Elasticsearch performs linguistic searches against documents and returns the documents that matches the search condition. Result relevancy for the given query is calculated using TF/IDF algorithm.
6. RESTFUL API – Elasticsearch supports REST API which is light-weight protocol. We can query Elasticsearch using the REST API with Chrome plug-in Sense. Sense provides a simple user interface. Sense plugin has features like autocomplete Elasticsearch query syntax, copying the query as cURL command. 

Terminologies : 

1.  Cluster: A cluster is a collection of nodes that shares data.
2.  Node: A node is a single server that is part of the cluster, stores the data, and participates in the cluster’s indexing and search capabilities.
3. Index: An index is a collection of documents with similar characteristics. An index is more equivalent to a schema in RDBMS. 
4. Type: There can be multiple types within an index. For example, an ecommerce application can have used products in one type and new products in another type of the same index. One index can have multiple types as multiple tables in one database.
5. Document: A document is a basic unit of information that can be indexed. It is like a row in a table.
6. Shards and Replicas: Elastic Search indexes are divided into multiple pieces called shards, which allows the index to scale horizontally. Elastic Search also allows us to make copies of index shards, which are called replicas.

Usecases :

Ecommerce websites use elasticsearch to index their entire product catalog and inventory with all the product attributes with which the end user can search against.

So whenever a user search for a product in the website, the corresponding query will hit an index which has millions of products and it will retrieve the product in near real time.

You want to collect log or transaction data and want to analyze and mine this data to look for statistics, summarizations, or anomalies.

In this case, you can index this data into Elasticsearch. Once the data is in Elasticsearch, we can visualize the data in timelion/d3.js to better understand the collected logs.

Installation :

Let’s assume that you are in a Linux based environment. Assuming that you also have JDK 6 or above installed, let’s get on with downloading Elasticsearch using the command below:

wget https://artifacts.elastic.co/dow...

Then extract it.

tar -zxvf elasticsearch-5.4.0.tar.gz

Go to the folder where Elasticsearch has been installed.

cd elasticsearch-5.4.0

To start the Elasticsearch server,

bin/elasticsearch

You can access it at http://localhost:9200 on your web browser. Here, localhost denotes the host (server) and the default port of Elasticsearch is 9200.

To confirm everything is working fine, type http://localhost:9200 in your browser and you should see something like this.

{

“name” : “90AzDAw”,

“cluster_name” : “elasticsearch”,

“cluster_uuid” : “e6t_hv6eQCi280elcktrUQ”,

“version” : {

“number” : “5.4.0”,

“build_hash” : “780f8c4”,

“build_date” : “2017-04-28T17:43:27.229Z”,

“build_snapshot” : false,

“lucene_version” : “6.5.0”

},

“tagline” : “You Know, for Search”

}

Indexing Documents :

Elasticsearch tends to use Lucene indexes to store & retrieve data. Adding ‘data’ to Elasticsearch is known as “indexing.” While performing an indexing operation, Elasticsearch converts raw data into its internal documents. Each document is nothing but a mere set of correlating keys and values: Here, the keys are strings and the values would be one of the numerous data types such as strings, numbers, lists, and dates, etc.

We can query Elasticsearch using the methods mentioned below :

-cURL command

-Using an HTTP client

-Querying with the JSON DSL

ElasticSearch provides a REST API that we can interact with in a variety of ways through common HTTP methods like GET, POST, PUT, DELETE. Which does the same thing as the CRUD operations does.

Now, let’s try indexing some data in our Elasticsearch instance.
Insertin Documents :

curl -XPUT http://localhost:9200/patient/ou... -d’
{
“name” : “Ranjan”,
“City” : “Kumar”,
"age" : 30,
"Address":"Chennai"
}’

This command will insert the JSON document into an index named ‘patient‘ with the type named ‘outpatient‘. 1 is the ID here. If we didn’t provide any ID here, it will simply create one for you. Pretty is used to pretty print the JSON response. To replace an existing document with an updated data, we just PUT it again.

By using the above method, we can insert one document at a time.

In order to bulk load the data, we can use Bulk API of Elasticsearch.

curl -XPOST ‘localhost:9200/patient/outpatient/_bulk?pretty&refresh’ –data-binary “@/home/ubuntu/Ex.json”

The above command loads the data.json file into the patient index.

Retrieving a Document :

Retrieving a Document in a index can be done using GET request.

curl -XGET ‘localhost:9200/patient/outpatient/1?pretty’

The response of this command contains the resulting JSON document under the _source field.

{
“_index” : “patient”,
“_type” : “outpatient”,
“_id” : “1”,
“_version” : 1,
“found” : true,
“_source” : {
“name” : “John”,
“City” : “California”
}
}

It returns the document with the id 1 and some metadata about the document.

Deleting a Document:

This API allows us to delete a JSON document from an index.

curl -XDELETE ‘localhost:9200/patient/outpatient/1?pretty’

This command deletes the JSON document with the id 1.

In order to delete a document that matches a specific condition we can use _delete_by_query API.

curl -XPOST ‘localhost:9200/patient/_delete_by_query?pretty’ -H ‘Content-Type: application/json’ -d’

{

“query”: {

“match”: { “city”: “California” }

}

}’