Syslog Server#
Syslog Server is central concentrator for syslog messages. Additionally, it provides web interface for real time monitoring and filtering of log messages.
Syslog listening socket can be configured as TCP, UDP or TCP+SSL socket. Communication is based on RFC 5425, RFC 5426, RFC 6587. Once message is received, server stores message in predefined database.
Running#
Syslog Server is implemented as python hat.syslog.server package which
can be run with hat-syslog-server
script with additional command line
arguments:
usage: hat-syslog-server [-h] [--conf PATH] [--log LEVEL] [--syslog-addr ADDR] [--syslog-pem PATH] [--ui-addr ADDR] [--db-path PATH] [--db-low-size N] [--db-high-size N] [--db-enable-archive] [--db-disable-journal] options: -h, --help show this help message and exit --conf PATH configuration defined by hat-syslog://server.yaml# (default $XDG_CONFIG_HOME/hat/syslog.{yaml|yml|json}) --log LEVEL console log level --syslog-addr ADDR syslog listening address (default tcp://0.0.0.0:6514) --syslog-pem PATH pem file path - mandatory for ssl communication --ui-addr ADDR UI listening address (default http://0.0.0.0:23020) --db-path PATH sqlite database file path (default $XDG_DATA_HOME/hat/syslog.db) --db-low-size N number of messages kept in database after database cleanup (default 1000000) --db-high-size N number of messages that will trigger database cleanup (default 10000000) --db-enable-archive should messages, deleted during database cleanup, be kept in archive files --db-disable-journal disable sqlite jurnaling
This application is part of hat-syslog python package.
Configuration#
Syslog Server configuration written in form of single YAML, JSON or TOML file
with structure defined by JSON Schema hat-syslog://server.yaml#
(see
JSON Schemas). Path to configuration file is provided as command line
argument during process startup. Additionally, configuration parameters
provided in configuration file can be overridden by command line arguments.
If configuration file could not be found, default values of configuration
parameters are used.
Example of configuration:
---
type: syslog
log:
version: 1
syslog_addr: 'tcp://0.0.0.0:6514'
ui_addr: 'http://0.0.0.0:23020'
dp_path: 'syslog.db'
db_low_size: 1_000_000
db_high_size: 10_000_000
db_enable_archive: false
db_disable_journal: false
...
Data backend#
All incoming syslog messages are stored in single sqlite database. Maximum
number of syslog messages stored in this database can be configured by
configuration parameter db_high_size
(value 0
represents unlimited
number of messages). Once number of messages exceed configured limit,
database cleanup procedure is triggered. During cleanup procedure, oldest
messages are removed from database until number of messages reaches
configuration parameter db_low_size
when cleanup procedure stops. Prior
to message deletion, if configuration parameter db_enable_archive
is set, new database with unique file name is created and all messages
scheduled for removal are inserted into newly created database. Archive
database has got same structure as original database and can be used in place
of original database for accessing archived syslog messages.
Web UI#
Together with acquiring and storing syslog messages, Syslog Server provides web-based user interface for querying messages from database and observing changes in real time. Communication between web server and browser is based on Juggler communication.

Server state#
Server state is used for providing continuously updated list of log entries to clients, based on applied filters.
State structure is defined by JSON schema
hat-syslog://juggler.yaml#/definitions/state
(see JSON Schemas).
Request/response#
Juggler request/response communication is used for changing filter parameters.
Request data structures are defined by JSON schema
hat-syslog://juggler.yaml#/definitions/request
(see JSON Schemas).
In case of successful request execution, response data is null
.
JSON Schemas#
Configuration#
---
"$schema": "http://json-schema.org/schema#"
id: "hat-syslog://server.yaml#"
title: Syslog server
description: Syslog server configuration
type: object
required:
- type
- log
- syslog_addr
- ui_addr
- db_path
- db_low_size
- db_high_size
- db_enable_archive
- db_disable_journal
properties:
type:
const: syslog
description: configuration type identification
version:
type: string
description: component version
log:
"$ref": "hat-json://logging.yaml#"
syslog_addr:
type: string
description: |
syslog listening address in form `<protocol>://<host>:<port>`
where `<protocol>` can be `tcp`, `udp` or `ssl`
syslog_pem:
type: string
description: |
path to PEM file is mandatory if syslog address is ssl
ui_addr:
type: string
description: |
web server listening address in form `http://<host>:<port>`
db_path:
type: string
description: |
path to sqlite database file
db_low_size:
type: integer
description: |
number of messages kept in database after database cleanup
db_high_size:
type: integer
description: |
number of messages that will trigger database cleanup
db_enable_archive:
type: boolean
description: |
should messages, deleted during database cleanup, be kept in
archive files
db_disable_journal:
type: boolean
description: |
disable sqlite jurnaling
...
Juggler#
---
"$schema": "http://json-schema.org/schema#"
id: "hat-syslog://juggler.yaml#"
definitions:
state:
type: object
required:
- filter
- entries
- first_id
- last_id
properties:
filter:
"$ref": "hat-syslog://juggler.yaml#/definitions/filter"
entries:
type: array
items:
"$ref": "hat-syslog://juggler.yaml#/definitions/entry"
first_id:
type:
- 'null'
- integer
last_id:
type:
- 'null'
- integer
request:
filter:
"$ref": "hat-syslog://juggler.yaml#/definitions/filter"
filter:
type: object
required:
- max_results
- last_id
- entry_timestamp_from
- entry_timestamp_to
- facility
- severity
- hostname
- app_name
- procid
- msgid
- msg
properties:
max_results:
type:
- 'null'
- integer
last_id:
type:
- 'null'
- integer
entry_timestamp_from:
type:
- 'null'
- number
entry_timestamp_to:
type:
- 'null'
- number
facility:
oneOf:
- type: 'null'
- "$ref": "hat-syslog://juggler.yaml#/definitions/facility"
severity:
oneOf:
- type: 'null'
- "$ref": "hat-syslog://juggler.yaml#/definitions/severity"
hostname:
type:
- 'null'
- string
app_name:
type:
- 'null'
- string
procid:
type:
- 'null'
- string
msgid:
type:
- 'null'
- string
msg:
type:
- 'null'
- string
entry:
type: object
required:
- id
- timestamp
- msg
properties:
id:
type: integer
timestamp:
type: number
msg:
"$ref": "hat-syslog://juggler.yaml#/definitions/msg"
msg:
type: object
required:
- facility
- severity
- version
- timestamp
- hostname
- app_name
- procid
- msgid
- data
- msg
properties:
facility:
oneOf:
- type: 'null'
- "$ref": "hat-syslog://juggler.yaml#/definitions/facility"
severity:
oneOf:
- type: 'null'
- "$ref": "hat-syslog://juggler.yaml#/definitions/severity"
version:
type: integer
timestamp:
type:
- 'null'
- number
hostname:
type:
- 'null'
- string
app_name:
type:
- 'null'
- string
procid:
type:
- 'null'
- string
msgid:
type:
- 'null'
- string
data:
type:
- 'null'
- string
msg:
type:
- 'null'
- string
facility:
enum:
- KERNEL
- USER
- MAIL
- SYSTEM
- AUTHORIZATION1
- INTERNAL
- PRINTER
- NETWORK
- UUCP
- CLOCK1
- AUTHORIZATION2
- FTP
- NTP
- AUDIT
- ALERT
- CLOCK2
- LOCAL0
- LOCAL1
- LOCAL2
- LOCAL3
- LOCAL4
- LOCAL5
- LOCAL6
- LOCAL7
severity:
enum:
- EMERGENCY
- ALERT
- CRITICAL
- ERROR
- WARNING
- NOTICE
- INFORMATIONAL
- DEBUG
...