A lot of time has passed since I wrote much of anything here. Life happens, and that’s more where my focus has been. Most of what will go here will be my newer, long-format posts where I outline my (mis)adventures in my home lab and other tech exploits.
For anything short form, follow me on X at https://x.com/untergeek
I figured it was time to share my current template again, as much has changed since Logstash 1.2. Among the changes include:
If you couldn’t tell, I’m crazy about doc_values. Using doc_values (where permitted) prevents your elasticsearch java heap size from growing out of control when performing large aggregations—for example, a months worth of data with Kibana—with very little upfront cost in additional storage.
This is mostly generic, but it does have a few things which are specific to my use case (like the Nginx entry). Feel free to adapt to your needs.
{
"template" : "logstash-*",
"settings" : {
"index.refresh_interval" : "5s"
},
"mappings" : {
"_default_" : {
"_all" : {"enabled" : true, "omit_norms" : true},
"dynamic_templates" : [ {
"message_field" : {
"match" : "message",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string", "index" : "analyzed", "omit_norms" : true
}
}
}, {
"string_fields" : {
"match" : "*",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string", "index" : "analyzed", "omit_norms" : true,
"fields" : {
"raw" : {"type": "string", "index" : "not_analyzed", "doc_values" : true, "ignore_above" : 256}
}
}
}
}, {
"float_fields" : {
"match" : "*",
"match_mapping_type" : "float",
"mapping" : { "type" : "float", "doc_values" : true }
}
}, {
"double_fields" : {
"match" : "*",
"match_mapping_type" : "double",
"mapping" : { "type" : "double", "doc_values" : true }
}
}, {
"byte_fields" : {
"match" : "*",
"match_mapping_type" : "byte",
"mapping" : { "type" : "byte", "doc_values" : true }
}
}, {
"short_fields" : {
"match" : "*",
"match_mapping_type" : "short",
"mapping" : { "type" : "short", "doc_values" : true }
}
}, {
"integer_fields" : {
"match" : "*",
"match_mapping_type" : "integer",
"mapping" : { "type" : "integer", "doc_values" : true }
}
}, {
"long_fields" : {
"match" : "*",
"match_mapping_type" : "long",
"mapping" : { "type" : "long", "doc_values" : true }
}
}, {
"date_fields" : {
"match" : "*",
"match_mapping_type" : "date",
"mapping" : { "type" : "date", "doc_values" : true }
}
} ],
"properties" : {
"@timestamp": { "type": "date", "doc_values" : true },
"@version": { "type": "string", "index": "not_analyzed", "doc_values" : true },
"clientip": { "type": "ip", "doc_values" : true },
"geoip" : {
"type" : "object",
"dynamic": true,
"properties" : {
"ip": { "type": "ip", "doc_values" : true },
"location" : { "type" : "geo_point", "doc_values" : true },
"latitude" : { "type" : "float", "doc_values" : true },
"longitude" : { "type" : "float", "doc_values" : true }
}
}
}
},
"nginx_json" : {
"properties" : {
"duration" : { "type" : "float", "doc_values" : true },
"status" : { "type" : "short", "doc_values" : true }
}
}
}
}
You can also find this in a GitHub gist.
Feel free to add any suggestions, or adaptations you may have used in the comments below!
Update: Contrary to online job posting, you don’t need a college degree to work at Mass. McDonald’s
Looks like the story was bogus. Short version: Some other job posting service put the erroneous requirement in its listing.
However, my comments below remain as applicable as before. (Note the Iranian expat who flipped burgers until she could use her chemical engineering degree).
Headline: McDonald’s help-wanted ad: Cashier position available; college degree required
From the article: “experts argue the unusually high qualifications McDonald’s is demanding are a sign of the times.”
The job description says compensation is $10/hour. Did you go to college for 4 years with dreams of making $10 per hour? Is that a sign of the times, or is it a sign of something else?
Please don’t take offense. I am not trying to demean or belittle anyone’s efforts or education. Full disclosure: I do not have a college degree, though I truly wish I had completed my degree when I had the chance. I applaud the hard work and determination of those who succeeded where I did not. I am nearly 40, and I see a disconnect in the world around me and I’m trying to understand it. I feel as though many young people today are told that a college degree is their ticket to a better future. But for many, once they have that degree, jobs that feel equivalent to their education are hard to find. While it can be argued that having a degree is more valuable than not having one we seem to be in an economy where skills, projects and/or portfolio are more highly valued than education-level. What skills do you have? What projects have you worked on? What have you created? What have you done recently that you are proud of? When you are applying for a job and asked questions such as these, will you have an answer, or will you point to your degree and say, I have this?
A degree is not a ticket to a better job, it’s more like an upgrade from economy to economy plus, or first class. You still need to get that ticket or boarding pass yourself, and the price is hard work. Sometimes it means taking an entry-level position and working your way up from the bottom. You need to use whatever job you do take to show that you have skills, initiative, creativity, determination and can solve problems on your own. Once you start doing this effectively and establish a history of being able to do so you will have those skills, that portfolio of you, that will be your ticket to a better future.
It used to be that a high school diploma alone was an indicator to an employer that you had the necessary skills to be worth taking a chance on. In my lifetime I have seen that requirement climb from high school diploma to college degree. Now it seems that many employers don’t want to take chances employing people with even college degrees. People may be afraid of taking the wrong job for fear of being pigeon-holed or type-cast, which could tarnish your résumé or hurt your chances at obtaining a job later in the field of their choice. I don’t blame anyone for those fears. I have had to reinvent myself for being type-cast as a systems administrator when I wanted to grow to bigger and better things.
I will share my own experience here, take from it what you will: My success has been determined by three Ds: Desire, Drive & Determination. Desire is the beginning. If you don’t desire to succeed at something, you won’t. Once you have desire, drive kicks in. Your drive will motivate you to do what is necessary to achieve your desire. Determination is what keeps you working when your drive is flagging, and it seems like you may not be able to achieve your desire. These are gross simplifications, but they will suffice for now. Everyone wants to climb the ladder of success, but you have to take the rungs at the bottom first. A high school diploma used to “spot” you the first rung or two, and a college degree another rung or two. That may still be true in many cases. I thought, like many it seems, that the rungs leading to success are on the ladder already–all I had to do was climb them. My life has taught me, and continues to remind me, that there are no higher rungs on the ladder to success that aren’t achieved at least in part by myself by way of those three Ds. Education only counts for a few rungs. The rest is up to you.
So, that’s my rant. It’s also the message I want to send to my children. I read another article this morning from the WSJ that had a segment about a woman who’d escaped Iran before the Shah hit the fan in 1979. She’d had a college degree, too. She fled Iran, and eventually made it to Canada. One of her first jobs? Flipping burgers. She went on to become a successful chemical engineer. If I had to offer an explanation for her success, I’d probably say the three D‘s had something to do with it.
I finally created a repository on GitHub for all of my Zabbix scripts (or, if you want to go directly to the repository: https://github.com/untergeek/zabbix-grab-bag)
This is the culmination of a dream that started a few years ago. I wanted a way to share my scripts in a way that others would be able to both use and improve them. GitHub is the chosen vessel.
Rather than making this a true project, I envision it as more of a “grab-bag” of projects/scripts/templates from myself and others. And you should be able to license your own scripts however you want, too.
So check it out! Contribute! Let’s make Zabbix even more awesome!
It’s been such a long time since I wrote in earnest, anywhere. So much has changed for me. So many things are different in my life.
I live in Texas now, having spent all but 2 years of my life in Utah. I’ve been here for almost 4 years now and I love it! Texas is home, now. I can hardly imagine living somewhere else.
I work in an extremely challenging and engaging business. Having been a Unix Systems Admin for around 8 years in Utah, I took a position as a Java Application Admin at my current company. The similarities and the differences are striking. I use the standard set of Unix admin tools still on a daily basis. At the same time, I spend a lot more time poring over log files from Weblogic and JBoss JVMs, implementing and creating new ways to monitor and visualize data, and making sure our customers’ apps are running optimally.
I am older and more mature (hopefully). I have discovered that I spent most of my life looking at the world around me with the myopic view of one with symptoms of Asperger’s Syndrome. There has been a shift in me since then. At first, this discovery was like putting on corrective lenses which suddenly brought my past into focus. This was incredibly uncomfortable. They say that hindsight is 20/20, but this experience was more like 20/10 or better. All of my interpersonal mishaps and miscommunications were suddenly plain to me, and I still have painful recollections of something I did or said long ago. The pain is diminished, significantly, from what it initially was. Embarrassment can be a difficult thing for a perfectionist. I’m learning and growing still, however. That’s what really matters.
With that realization, however, come many new opportunities to see the world as it actually is or as it can be. This reawakening is ongoing for me. I have not forgotten my past, nor do I seek to hide from it. The old blog is still around at http://oldblog.untergeek.com with a disclaimer about its meaning as a part of my history. I do hope to convey, however, that my life has new meaning. This is my reawakening.
The rebirth of The Untergeek!
The Untergeek 