-Xmx128gb -Xms128gb

adding more memory to my brain!

Elasticsearch Real Integration Tests With Security Enabled (Updated for GA)

| Comments

NOTE: This article is an updated version of Elasticsearch real integration tests with security enabled

In a recent post we have seen how to create real integration tests. Those tests launch a real elasticsearch cluster, then run some tests you write with JUnit or your favorite test framework then stop the cluster.

But sometimes, you may want to add existing plugins in your integration test cluster.

For example, you might want to use X-Pack to bring fantastic features such as:

  • Security
  • Alerting
  • Monitoring
  • Graph
  • Reporting

Let’s see how you can do that with Maven and Ant again…

Creating Elasticsearch Transport Action (Updated for GA)

| Comments

NOTE: This article is an updated version of Creating Elasticsearch Transport Action

This blog post is part of a series which will teach you:

In the previous article, we discovered how to add a REST plugin. It was a simple implementation as in HelloRestAction class we wrote something like:

1
2
3
4
5
6
7
8
9
10
11
12
@Override
protected RestChannelConsumer prepareRequest(RestRequest restRequest, NodeClient client) throws IOException {
    String name = restRequest.param("name");
    return channel -> {
        Message message = new Message(name);
        XContentBuilder builder = channel.newBuilder();
        builder.startObject();
        message.toXContent(builder, restRequest);
        builder.endObject();
        channel.sendResponse(new BytesRestResponse(RestStatus.OK, builder));
    };
}

But actually, you will probably like to execute some actions against a Node, call some internal services… So the implementation we wrote needs to be modified a bit.

Adding a New REST Endpoint to Elasticsearch (Updated for GA)

| Comments

NOTE: This article is an updated version of Adding a new REST endpoint to elasticsearch

This blog post is part of a series which will teach you:

Imagine that you wish to add a new REST endpoint so you can send requests like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
curl -XGET "http://localhost:9200/_hello?name=David&pretty"

# Or

curl -XGET "http://localhost:9200/_hello/David&pretty"

# Or

curl -XGET "http://localhost:9200/_hello?pretty" -d '{
    "name": "David"
}'

# Or

curl -XPOST "http://localhost:9200/_hello?pretty" -d '{
    "name": "David"
}'

And you want to get back something like:

1
2
3
{
  "message": "Hello David!"
}

Without any parameter:

1
curl -XGET "http://localhost:9200/_hello?pretty"

It should return:

1
2
3
{
    "message": "Hello World!"
}

Or get back a list of existing indices and the number of documents for a given type of indices with:

1
curl -XGET http://localhost:9200/_bano

Let’s see how to implement that!

Elasticsearch Real Integration Tests (Updated for GA)

| Comments

NOTE: This article is an updated version of Elasticsearch real integration tests

Integration tests… How do you run them?

Often, you are tempted to run services you want to test from JUnit for example. In elasticsearch, you can extend ESIntegTestCase class which will start a cluster of a given number of nodes.

1
2
3
4
5
public class BanoPluginIntegrationTest extends ESIntegTestCase {
    public void testPluginIsLoaded() throws Exception {
      // Your code here
    }
}

But to be honest, the test you are running does not guarantee that you will have the same result in production.

Why this? Because some services might be disabled, some others might be mocked.

Instead, running tests against a real cluster as the one you would install locally on your machine is the way to go.

Let’s see how you can do that with Maven…

Creating an Ingest Plugin for Elasticsearch (Updated for GA)

| Comments

NOTE: This article is an updated version of Creating an Ingest plugin for elasticsearch

This blog post is part of a series which will teach you:

Today, we will focus on writing an Ingest plugin for elasticsearch.

Hey! Wait! You wrote Ingest? What is that?

Ingest is a new feature coming in elasticsearch 5.0. It helps you to transform your data on the fly while injecting it into elasticsearch. Read more in elastic blog post.

If you know me and my work before I joined elastic, I have always been in love with data crawling and transformation as I wrote myself some plugins called rivers.

Creating a Plugin for Elasticsearch 5.0 Using Maven (Updated for GA)

| Comments

NOTE: This article is an updated version of Creating a plugin for elasticsearch 5.0 using Maven

Elasticsearch 5.0 switched to Gradle in October 2015.

You can obviously write a plugin using Gradle if you wish and you could benefit from all the goodies elasticsearch team wrote when it comes to integration tests and so on.

My colleague, Alexander Reelsen aka Spinscale on Twitter, wrote a super nice template if you wish to create an Ingest plugin for 5.0.

Hey! Wait! You wrote Ingest? What is that?

Ingest is a new feature coming in elasticsearch 5.0. It helps you to transform your data on the fly while injecting it into elasticsearch. Read more in elastic blog post.

If you know me and my work before I joined elastic, I have always been in love with data crawling and transformation as I wrote myself some plugins called rivers.

This blog post is part of a series which will teach you:

Elasticsearch Real Integration Tests With Security Enabled

| Comments

NOTE: This article is now outdated. Please read Elasticsearch real integration tests with security enabled (Updated for GA) instead!

In a recent post we have seen how to create real integration tests. Those tests launch a real elasticsearch cluster, then run some tests you write with JUnit or your favorite test framework then stop the cluster.

But sometimes, you may want to add existing plugins in your integration test cluster.

For example, you might want to use X-Pack to bring fantastic features such as:

  • Security
  • Alerting
  • Monitoring
  • Graph
  • Reporting

Let’s see how you can do that with Maven and Ant again…

Creating Elasticsearch Transport Action

| Comments

NOTE: This article is now outdated. Please read Creating Elasticsearch Transport Action (Updated for GA) instead!

This blog post is part of a series which will teach you:

In the previous article, we discovered how to add a REST plugin. It was a simple implementation as in RestHelloAction class we wrote something like:

1
2
3
4
5
public void handleRequest(RestRequest request, RestChannel channel, NodeClient client) throws Exception {
    RestToXContentListener<ToXContent> listener = new RestToXContentListener<>(channel);
    String name = request.param("name");
    listener.onResponse(new Message(name));
}

But actually, you will probably like to execute some actions against a Node, call some internal services… So the implementation we wrote needs to be modified a bit.

Adding a New REST Endpoint to Elasticsearch

| Comments

NOTE: This article is now outdated. Please read Adding a new REST endpoint to elasticsearch (Updated for GA) instead!

This blog post is part of a series which will teach you:

Imagine that you wish to add a new REST endpoint so you can send requests like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
curl -XGET "http://localhost:9200/_hello?name=David&pretty"

# Or

curl -XGET "http://localhost:9200/_hello/David&pretty"

# Or

curl -XGET "http://localhost:9200/_hello?pretty" -d '{
    "name": "David"
}'

# Or

curl -XPOST "http://localhost:9200/_hello?pretty" -d '{
    "name": "David"
}'

And you want to get back something like:

1
2
3
{
  "message": "Hello David!"
}

Without any parameter:

1
curl -XGET "http://localhost:9200/_hello?pretty"

It should return:

1
2
3
{
    "message": "Hello World!"
}

Or get back a list of existing indices and the number of documents for a given type of indices with:

1
curl -XGET http://localhost:9200/_bano

Let’s see how to implement that!

Elasticsearch Real Integration Tests

| Comments

NOTE: This article is now outdated. Please read Elasticsearch real integration tests (Updated for GA) instead!

Integration tests… How do you run them?

Often, you are tempted to run services you want to test from JUnit for example. In elasticsearch, you can extend ESIntegTestCase class which will start a cluster of a given number of nodes.

1
2
3
4
5
public class BanoPluginIntegrationTest extends ESIntegTestCase {
    public void testPluginIsLoaded() throws Exception {
      // Your code here
    }
}

But to be honest, the test you are running does not guarantee that you will have the same result in production.

Why this? Because some services might be disabled, some others might be mocked.

Instead, running tests against a real cluster as the one you would install locally on your machine is the way to go.

Let’s see how you can do that with Maven…