-Xmx128gb -Xms128gb

adding more memory to my brain!

Building a Directory Map With ELK

| Comments

I gave a BBL talk recently and while chatting with attendees, one of them told me a simple use case he covered with elasticsearch: indexing metadata files on a NAS with a simple ls -lR like command. His need is to be able to search on a NAS for files when a user wants to restore a deleted file.

As you can imagine a search engine is super helpful when you have hundreds of millions files!

I found this idea great and this is by the way why I love speaking at conferences or in companies: you always get great ideas when you listen to others!

I decided then to adapt this idea using the ELK stack.

Index Twitter on Found

| Comments

Some months ago, I published a recipe on how to index Twitter with Logstash and Elasticsearch.

I have the same need today as I want to monitor Twitter when we run the elastic FR meetup (join us by the way if you are in France!).

Well, this recipe can be really simplified and actually I don’t want to waste my time anymore on building and managing elasticsearch and Kibana clusters anymore.

Let’s use a Found by elastic cluster instead.

Importing From a Database Without a Database

| Comments

Recently, I got a database MySQL dump and I was thinking of importing it into elasticsearch.

The first idea which pops up was:

  • install MySQL
  • import the database
  • read the database with Logstash and import into elasticsearch
  • drop the database
  • uninstall MySQL

Well. I found that some of the steps are really not needed.

I can actually use ELK stack and create a simple recipe which can be used to import SQL dump scripts without needing to actually load the data to a database and then read it again from the database.

Let’s do it!

Reindex Elasticsearch With Logstash

| Comments

Sometimes, you would like to reindex your data to change your mapping or to change your index settings or to move from one server to another or to one cluster to another (think about multiple data centers for example).

For the later you can use Snapshot and Restore feature but if you need to change any index settings, you need something else.

With Logstash 1.5.0, you can now do it super easily using elasticsearch input and elasticsearch output.

Let’s do it!