Monitoring application log files is important. If something goes wrong, you want to be notified. This is even more critical for background jobs where no user is facing the error directly.
A state of the art solution for this problem would be ELK, combined with Alerting , but maybe your infrastructure is not at this point yet. For small systems or as a workaround solution, there’s an old Linux tool called
Logcheck was actually developed to monitor Linux system log files (e.g., detect a hacker attack), but it can be easily configured for your own application logs as well.
What logcheck can do:
- every x hours, it would scan a list of log files you specify
- each line in the logfile is matched against a list of regular expressions you specify
- all the lines which are matched would be sent out by e-mail
On Ubuntu, install it like
Logcheck gets configured in
/etc/logcheck. This directory looks like
. ├── cracking.d ├── cracking.ignore.d ├── ignore.d.paranoid ├── ignore.d.server ├── ignore.d.workstation ├── violations.d ├── violations.ignore.d ├── header.txt ├── logcheck.conf └── logcheck.logfiles
For our setup, only
. ├── ignore.d.server ├── violations.d ├── logcheck.conf └── logcheck.logfiles
logcheck.conf and enter the e-mail address for your alerts.
REPORTLEVEL="server" unchanged, so logcheck will run in
Therefore, we can ignore directories
Add all the logfiles you want to monitor to
logcheck.logfiles. Placeholders work:
# /etc/logcheck/logcheck.logfiles /var/log/my_app/*.log
Now, add a new file
violations.d/my_app and enter your regex, e.g.:
# /etc/logcheck/violations.d/my_app ^.*error.*$ ^.*Error.*$ ^.*exception.*$ ^.*Exception.*$
Finally, ignore everything else. Add
# /etc/logcheck/ignore.d.server/my_app ^.*$
Be aware of these caveats:
- it doesn’t scale: it can only act on logs from a single server
- it can only do regex, cannot detect patterns on multiple rows
- for low volume alerts only
- no UI
- no history (except the e-mail themselves)
However, until you have something like ELK, it might serve you well.