python – Multiline Log message parsing by regex

admin

I’m trying to parse my log files from several different project folders to show my log messages on my website. But in somehow, I couldn’t manage to parse the second or more line messages.

Do you have any idea about it?

This is an example of my log messages in the file.

[31m2000-01-01 01:02:34,123[0m [36m123456@MainProcess[0m: KingArthur: [[33;1mINFO[0m] project_name: This is a log message
[31m2000-01-01 01:02:35,123[0m [36m123456@MainProcess[0m: KingArthur: [[33;1mINFO[0m] project_name: This is a log message
[31m2000-01-01 01:02:36,123[0m [36m123456@MainProcess[0m: KingArthur: [[33;1mINFO[0m] project_name: This is a log message
[31m2000-01-01 01:02:37,123[0m [36m123456@MainProcess[0m: KingArthur: [[33;1mWARNING[0m] py.warnings: /This/is/a/path.py:302: Warning: Warning: This is a log message.
  This is= a log message()

[31m2000-01-01 01:02:38,123[0m [36m123456@MainProcess[0m: KingArthur: [[33;1mWARNING[0m] py.warnings: /This/is/a/path.py:302: Warning: Warning: This is a log message.
  This is= a log message()

[31m2000-01-01 01:02:39,123[0m [36m123456@MainProcess[0m: KingArthur: [[33;1mINFO[0m] project_name: This is a log message
[31m2000-01-01 01:02:40,123[0m [36m123456@MainProcess[0m: KingArthur: [[33;1mWARNING[0m] py.warnings: /This/is/a/path.py:362: SettingWithCopyWarning: 
This is a log message
This is a log message

This is a log message
  This is a log message

This is my regex.

# This is my regex. 
log_regex = re.compile(
    r'^\x1b\[31m(?P<asctime>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3})\x1b\[0m\s\x1b\[36m(?P<process>\d+)@(?P<processName>.+)\x1b' +  # noqa
    r'\[0m:\s*(?P<threadName>.+):\s\[\x1b\[33;1m(?P<levelName>\w{3,7})\x1b\[0m\]\s(?P<name>.+):\s(?P<message>[\w|\s|\n|.].*)$',
    re.MULTILINE)

The expected output is as below.

2000-01-01 01:02:34,123 123456@MainProcess: KingArthur: [INFO] project_name: This is a log message
2000-01-01 01:02:35,123 123456@MainProcess: KingArthur: [INFO] project_name: This is a log message
2000-01-01 01:02:36,123 123456@MainProcess: KingArthur: [INFO] project_name: This is a log message
2000-01-01 01:02:37,123 123456@MainProcess: KingArthur: [WARNING] py.warnings: /This/is/a/path.py:302: 2000-01-01 01:02:37,123 123456@MainProcess: KingArthur: [WARNING] Warning: Warning: This is a log message.
2000-01-01 01:02:37,123 123456@MainProcess: KingArthur: [WARNING] This is= a log message()
2000-01-01 01:02:38,123 123456@MainProcess: KingArthur: [WARNING] py.warnings: /This/is/a/path.py:302: 2000-01-01 01:02:38,123 123456@MainProcess: KingArthur: [WARNING] Warning: Warning: This is a log message.
2000-01-01 01:02:38,123 123456@MainProcess: KingArthur: [WARNING] This is= a log message()
2000-01-01 01:02:39,123 123456@MainProcess: KingArthur: [INFO] project_name: This is a log message
2000-01-01 01:02:40,123 123456@MainProcess : KingArthur: [WARNING] py.warnings: /This/is/a/path.py:362: 2000-01-01 01:02:40,123 123456@MainProcess : KingArthur: [WARNING] SettingWithCopyWarning: 
2000-01-01 01:02:40,123 123456@MainProcess : KingArthur: [WARNING] This is a log message
2000-01-01 01:02:40,123 123456@MainProcess : KingArthur: [WARNING] This is a log message
2000-01-01 01:02:40,123 123456@MainProcess : KingArthur: [WARNING] This is a log message
2000-01-01 01:02:40,123 123456@MainProcess : KingArthur: [WARNING] This is a log message

enter image description here

Read more here: Source link