Sometimes having your system send you it's error logs via mail is the only feasible alternative when you physically can't get to your error logs in a production environment.
Here you are often trying to maintain a balance between sending enough information, as often as possible; and not flooding the mail server when your system decides to have a wobbly.
The solution that I've decided on in the end, should give me the best of both worlds, and has been working very nicely for a few months now.
Simply put, the idea is very simple: Send me everything I need to know, but don't spam me...
My solution was to write a very simple and purpose built "spam filter" within my application.
The basic logic is as follows:
- Store a key value pair in memory. The key is a hash, and the value is the time in milliseconds
- Have a configurable expiry period, whereby you can remove items from the key value pair after a specified time period.
- Before sending an mail, build a checksum of the mail's body (I use SHA-2 and then store a String Hex representation to save memory).
- Interrogate your key value pair using the hash that you've just generated
- If a match is not found, Insert the hash and the current time in milliseconds into the pair and release the mail
- If a match is found, discard the mail
- When the configurable expiry time is reached (Current Time in milliseconds - Time in the key value pair), remove the item from the key value pair
In the end it has worked extremely well for me. I don't get mailed about the exact same problem more than once, unless the problem persists past the expiry time.
I hope that this can be of use to you.
