Tail the "in the last hour written lines from a log file" is it possible?

Question

I am trying to grep looking for a pattern in to a log file, but I need the last hour of the log file. A tail -n XX will not work. Does anybody know if this is possible? Some kind of tail the "in the last hour written lines from a log file"

If there is any command or procedure I appreciate that.

Thanks a lot

Answer 1

Let's say your log have the following structure:

219.369.42.449 - - [05/Mar/2020:11:05:17 +0200] "log line"
219.369.42.449 - - [05/Mar/2020:11:06:37 +0200] "log line"
219.369.42.449 - - [05/Mar/2020:12:01:14 +0200] "log line"
219.369.42.449 - - [05/Mar/2020:12:07:23 +0200] "log line"

We can get all lines from the first occurrence of 05/Mar/2020:11 to the end $ of the file by using sed in the following way:

sed -n '/05\/Mar\/2020:11/,$p' "/path/to/file.log"

• The option -n will suppress the normal output of sed, but the flag p will print the matched part of the file.

• Note, if there isn't presented any record that mach to 05/Mar/2020:11, sed wont provide any output.

We can automate the above by the help of the commands date and eval:

COMMAND="sed -n '/$(LANG=C date --date='1 hour ago' "+%d\/%b\/%Y:%H")/,\$p'"
eval $COMMAND \"/path/to/file.log\"

• Using sed with double quote marks and variable within the expression doesn't provide the desired output in this case.
• So we first constructing the command as string and convert it to a real command by eval.
• LANG=C (LANG=en_us_88591) stands in order to get the desired date format, because, for example, in my case the default value of this envvar is bg_BG.UTF-8.

You can create a script, based on the two lines above - examples of such script:

• apache2 : How to search a string from apache2 error logs in specific time range?

• modsecurity-whitelist-rule-generator.bash - that parse events within ModSecutity's modsec_audit.log by their unique-id, and then generates whitelist rules for ModSecutity.

Answer 2

There is no command or option to tail that will track changes in the past hour. You will have to grep the timestamps in the log or keep tail -f running and just scroll back when you need to check something. This has the advantage of also allowing you to catch events that happened 61 minutes ago.

Answer 3

When you run your command every 5 minutes, also make a copy of the log file. Then you can diff from the 12th-last copy you made to get the current changes.

Answer 4

The approach I normally take is:

tail -f {log}

This will have tail show messages -as they are written- to {log} and it will only end doing so when you stop the command. So what is shown is real time and logically also always within the current hour. The buffer size of the terminal session will be to where you can scroll back in time.

Answer 5

Make use of the date command that @pa4080 used to get exactly one hour ago. Pass that to an awk script that finds the first line containing the correct year, month, day, and either the matching hour and higher minute or higher hour. Output from the first line to the end of file.

My awk isn't that strong or I'd write it for you. Hopefully someone else will be the awk master for you.

Answer 6

Thanks a lot to everyone. I could handle the issue by using Zabbix, a monitoring tool.

@pa4080 that script would be the next opción. Thx

We can close this question.