Expect this post to get updated from time to time. You can come back here to check out what I’m using and why.
2012-11-05: I now map IP addresses (clientip field) as type IP to allow for range searches. I also map the fields in the geoip filter output to allow for non-analyzed terms facet output (allows full city names with spaces; proper capitalization, etc.)
curl -XPUT http://localhost:9200/_template/logstash_per_index -d '
{
"template" : "logstash*",
"settings" : {
"number_of_shards" : 4,
"index.cache.field.type" : "soft",
"index.refresh_interval" : "5s",
"index.store.compress.stored" : true,
"index.query.default_field" : "@message",
"index.routing.allocation.total_shards_per_node" : 4
},
"mappings" : {
"_default_" : {
"_all" : {"enabled" : false},
"properties" : {
"@fields" : {
"type" : "object",
"dynamic": true,
"path": "full",
"properties" : {
"clientip" : { "type": "ip" },
"geoip" : {
"type" : "object",
"dynamic": true,
"path": "full",
"properties" : {
"area_code" : { "type": "string", "index": "not_analyzed" },
"city_name" : { "type": "string", "index": "not_analyzed" },
"continent_code" : { "type": "string", "index": "not_analyzed" },
"country_code2" : { "type": "string", "index": "not_analyzed" },
"country_code3" : { "type": "string", "index": "not_analyzed" },
"country_name" : { "type": "string", "index": "not_analyzed" },
"dma_code" : { "type": "string", "index": "not_analyzed" },
"ip" : { "type": "string", "index": "not_analyzed" },
"latitude" : { "type": "float", "index": "not_analyzed" },
"longitude" : { "type": "float", "index": "not_analyzed" },
"metro_code" : { "type": "float", "index": "not_analyzed" },
"postal_code" : { "type": "string", "index": "not_analyzed" },
"region" : { "type": "string", "index": "not_analyzed" },
"region_name" : { "type": "string", "index": "not_analyzed" },
"timezone" : { "type": "string", "index": "not_analyzed" }
}
}
}
},
"@message": { "type": "string", "index": "analyzed" },
"@source": { "type": "string", "index": "not_analyzed" },
"@source_host": { "type": "string", "index": "not_analyzed" },
"@source_path": { "type": "string", "index": "not_analyzed" },
"@tags": { "type": "string", "index": "not_analyzed" },
"@timestamp": { "type": "date", "index": "not_analyzed" },
"@type": { "type": "string", "index": "not_analyzed" }
}
}
}
}
'
If you enjoyed this article, please consider sharing it!
Apologies for the potential unreadability of the code, here. It should still cut/paste properly.
Please add the following css to your blog:
pre {
overflow: scroll;
background-color: rgb(240, 240, 240);
}
Will make it much more readable.
I will try. The template and CSS hierarchy may not let me.
Cool! That worked.
[...] Home > Geek > Using elasticsearch mappings appropriately to map as type IP, int, float, etc. Using rsyslog to send pre-formatted JSON to logstash My current template/mapping [...]
[...] My Logstash Notes – Jan-Piet Mens Untergeek – Current Template/Mapping 08-2012 Modern Log Management and Monitoring 08-2011 Write Logs for Machines, use [...]