Administration Guide
What does Roundup install?
There’s two “installations” that we talk about when using Roundup:
The installation of the software and its support files. This uses the standard Python mechanism called “setuptools” and thus Roundup’s core code, executable scripts and support data files are installed in Python’s directories. On Windows, this is typically:
- Scripts
<python dir>\scripts\...- Core code
<python dir>\lib\site-packages\roundup\...- Support files
<python dir>\share\roundup\...
and on Unix-like systems (eg. Linux):
- Scripts
<python root>/bin/...- Core code
<python root>/lib-<python version>/site-packages/roundup/...- Support files
<python root>/share/roundup/...
The installation of a specific tracker. When invoking the roundup-admin “inst” (and “init”) commands, you’re creating a new Roundup tracker. This installs configuration files, HTML templates, detector code and a new database. You have complete control over where this stuff goes through both choosing your “tracker home” and the
main->databasevariable in the tracker’s config.ini.
Configuring Roundup Message Logging
You can control how Roundup logs messages using your tracker’s
config.ini file. Roundup uses the standard Python logging
implementation. The config file and roundup-server provide
very basic control over logging.
roundup-server’s logging is controlled from the command
line. You can:
specify the location of a log file or
enable logging using the standard Python logging library under the tag/channel
roundup.http
Configuration for “BasicLogging” implementation for your tracker
is done using the settings in the tracker’s config.ini under
the logging section:
filenamesetting: specifies the location of a log filelevelsetting: specifies the minimum level to logdisable_loggerssetting: disable other loggers (e.g. when running under a wsgi framework)formatsetting: set the log format template. See Defining the Log Format for more info.
In either case, if logfile is not specified, logging is sent to sys.stderr. If level is not set, only ERROR or higher priority log messages will be reported.
You can get more control over logging by using the config
setting in the tracker’s config.ini. Using a logging config
file overrides all the rest of the other logging settings in
config.ini. You get more control over the logs by supplying
a log config file in ini or json (dictionary) format.
Using this, you can set different levels by channel. For example roundup.hyperdb can be set to WARNING while other Roundup log channels are set to INFO and the roundup.mailgw channel logs at the DEBUG level. You can also control the distribution of logs. For example roundup.mailgw logs to syslog, other channels log to an automatically rotating log file, or are submitted to your log aggregator over https.
Defining the Log Format
Starting with Roundup 2.6 you can specify the logging format in
config.ini. The logging -> format setting of config.ini
supports all of the standard logging LogRecord attributes
or Roundup logging attributes. However you must double any %
format markers. The default value is:
%%(asctime)s %%(trace_id)s %%(levelname)s %%(message)s
Roundup Logging Attributes
The logging package has a number of attributes that can be expanded in the format template. In addition to the ones supplied by Python’s logging module, Roundup defines additional attributes:
- trace_id
a unique string that is generated for each request. It is unique per thread.
- trace_reason
a string describing the reason for the trace/request.
the URL for a web triggered (http, rest, xmlrpc) request
the email message id for an email triggered request
the roundup-admin os user and start of command. Only first two words in command are printed so seting a password will not be leaked to the logs.
- sinfo
the stack traceback information at the time the log call id made.
This must be intentionally activated by using the extras parameter. For example calling:
logging.get_logger('roundup.something').warning( "I am here\n%(sinfo)s", extra={"sinfo": 2})in the function confirmid() of the file detectors/reauth.py in your demo tracker will print 2 items on the stack including the log call. It results in the following (5 lines total in the log file):
2025-09-14 23:07:58,668 Cm0ZPlBaklLZ3Mm6hAAgoC WARNING I am here File "[...]/roundup/hyperdb.py", line 1924, in fireAuditors audit(self.db, self, nodeid, newvalues) File "demo/detectors/reauth.py", line 7, in confirmid logging.getLogger('roundup.something').warning(Note that the output does not include the arguments to
warningbecause they are on the following line. If you want arguments to the log call included, they have to be on the same line.Setting
sinfoto an integer value N includes N lines up the stack ending with the logging call. Setting it to 0 includes all the lines in the stack ending with the logging call.If the value is less than 0, the stack dump doesn’t end at the logging call but continues to the function that generates the stack report. So it includes functions inside the logging module.
Setting it to a number larger than the stack trace will print the trace down to the log call. So using
-1000will print up to 1000 stack frames and start at the function that generates the stack report.Setting
sinfoto a non-integer value{"sinfo": None}will produce 5 lines of the stack trace ending at the logging call.- pct_char
produces a single
%sign in the log. The usual way of embedding a percent sign in a formatted string is to double it like:%%. However when the format string is specified in the config.ini file percent signs are manipulated. So%%(pct_char)scan be used in config.ini to print a percent sign.
The default logging template is defined in config.ini in the
logging -> format setting. It includes the trace_id.
When searching logs, you can use the trace_id to see all the log
messages associated with a request.
If you want to log from a detector, extension or other code, you
can use these tokens in your messages when calling the logging
functions. (Note that doubling % signs is only required when
defining the log format in a config file, not when defining a
msg.) For example:
logging.getLogger('roundup.myextension').error('problem with '
'%(trace_reason)s')
will include the url in the message when triggered from the
web. This also works with other log methods: warning(),
debug(), ….
Note you must not use positional arguments in your message. Using:
logging.getLogger('roundup.myextension').error(
'%s problem with %(trace_reason)s', "a")
will not properly substitute the argument. You must use mapping key based arguments and define the local values as part of the extra dictionary. For example:
logging.getLogger('roundup.myextension').error('%(article)s '
'problem with %(trace_reason)',
extra={"article": some_local_variable})
Also if you are logging any data supplied by a user, you must not
log it directly. If the variable url contains the url typed in
by the user, never use:
logger.info(url)
or
logger.info(“Url is %s” % url)
Use:
logger.info(“Url is %s”, url)
or
logger.info(“Url is %(url)s”, extra={“url”: url)
This prevents printf style tokens in url from being processed
where it can raise an exception. This could be used to prevent
the log message from being generated.
More on trace_id
The trace_id provides a unique token (a UUID4 encoded to make it shorter or a nanoid) for each transaction in the database. It is unique to each thread or transaction. A transaction:
- for the web interface is
each web, rest or xmlrpc request
- for the email interface is
each email request. Using pipe mode will generate one transaction. Using pop/imap etc can generate multiple transactions, one for each email. Logging that occurs prior to processing an email transaction has the default
not_setvalue for trace_id- for the roundup-admin interface is
each command in the interactive interface or on the command line. Plus one transaction when/if a commit happens on roundup-admin exit.
When creating scripts written using the roundup package the entry
point should use the @gen_trace_id decorator. For example to
decorate the entry point that performs one transaction:
from roundup.logcontext import gen_trace_id
# stuff ...
@gen_trace_id()
def main(...):
...
If your script does multiple processing operations, decorate the entry point for the processing operation:
from roundup.logcontext import gen_trace_id
@gen_trace_id()
def process_one(thing):
...
def main():
for thing in things:
process_one(thing)
You can change the format of the trace_id if required using the tracker’s interfaces.py file. See the module docs for the logcontext module for details.
Advanced Logging Setup
If the settings in config.ini are not sufficient for your logging requirements, you can specify a full logging configuration in one of two formats:
dictConfig format using json with comment support
fileConfig format in ini style
The dictConfig format allows more control over configuration
including loading your own log handlers and disabling existing
handlers. If you use the fileConfig format, the logging ->
disable_loggers flag in the tracker’s config is used to
enable/disable pre-existing loggers as there is no way to do this
in the logging config file.
dictConfig Based Logging Config
dictConfigs are specified in JSON format with support for
comments. The file name in the tracker’s config for the
logging -> config setting must end with .json to
choose the correct processing.
Comments have to be in one of two forms based on javascript line comments:
A
//possibly indented with whitespace on a line is considered a comment and is stripped from the file before being passed to the json parser. This is a “line comment”.A
//with at least three white space characters before it is stripped from the end of the line before being passed to the json parser. This is an “inline comment”.
Block style comments are not supported.
Other than this the file is a standard json file that matches the Configuration dictionary schema defined in the Python documentation.
Example dictConfig Logging Config
Note that this file is not actually JSON format as it include comments. However by using javascript style comments, some tools that treat JSON like javascript (editors, linters, formatters) might work with it. A command like:
sed -e 's#^\s*//.*##' -e 's#\s*\s\s\s//.*##' logging.json
can be used to strip comments for programs that need it.
The config below works with the Waitress wsgi server configured to use the roundup.wsgi channel. It also controls the TransLogger middleware configured to use roundup.wsgi.translogger, to produce httpd style combined logs.
The log file is specified relative to the current working directory not the tracker home. The tracker home is the subdirectory demo under the current working directory.
The config also expects the python-json-logger package to be
installed so that it can produce a jsonl (json line) formatted
output log file. This format is useful for sending to log
management/observability platforms like elasticsearch, splunk,
logly, or honeycomb.
The commented config is:
{
"version": 1, // only supported version
"disable_existing_loggers": false, // keep the wsgi loggers
"formatters": {
// standard format for Roundup messages
"standard": {
"format": "%(asctime)s %(trace_id)s %(levelname)s %(name)s:%(module)s %(message)s"
},
// Used to dump all log requests in jsonl format.
// Each json object is on one line. Can be pretty printed
// using:
// python -m json.tool --json-lines --sort-keys < roundup.json.log
// jq --slurp --sort-keys . < roundup.json.log
// requires that you pip install python-json-logger
// * does not report the fields in reserved_attrs
// * example to remap a field in the log to traceID in
// the output json. (note trace_id_eg is not defined by
// logging
// * also adds the env atribute to json with the value of demo
"json": {
"()": "pythonjsonlogger.json.JsonFormatter",
"reserved_attrs": ["ROUNDUP_CONTEXT_FILTER_CALLED",
"msg", "pct_char", "relativeCreated"],
"rename_fields": {
"trace_id_eg": "traceID"
},
"static_fields": {
"env": "demo"
}
},
// used for waitress wsgi server to produce httpd style logs
"http": {
"format": "%(message)s %(trace_id)"
}
},
"handlers": {
// create an access.log style http log file
"access": {
"level": "INFO",
"formatter": "http",
"class": "logging.FileHandler",
"filename": "demo/access.log"
},
// logging for roundup.* loggers
"roundup": {
"level": "DEBUG",
"formatter": "standard",
"class": "logging.FileHandler",
"filename": "demo/roundup.log"
},
// handler for json output log file
"roundup_json": {
"level": "DEBUG", // "DEBUG",
"formatter": "json",
"class": "logging.FileHandler",
"filename": "demo/roundup.json.log"
},
// print to stdout - fall through for other logging
"default": {
"level": "DEBUG",
"formatter": "standard",
"class": "logging.StreamHandler",
"stream": "ext://sys.stdout"
}
},
"loggers": {
"": {
"handlers": [
"default",
"roundup_json" // add json formatted logging
],
"level": "DEBUG",
"propagate": false
},
// used by roundup.* loggers
"roundup": {
"handlers": [
"roundup",
"roundup_json"
],
"level": "DEBUG",
"propagate": false // note pytest testing with caplog requires
// this to be true
},
"roundup.hyperdb": {
"handlers": [
"roundup"
],
"level": "INFO", // can be a little noisy use INFO for production
"propagate": false
},
"roundup.wsgi": { // using the waitress framework
"handlers": [
"roundup"
],
"level": "DEBUG",
"propagate": false
},
"roundup.wsgi.translogger": { // httpd style logging
"handlers": [
"access"
],
"level": "DEBUG",
"propagate": false
},
"root": {
"handlers": [
"default"
],
"level": "DEBUG",
"propagate": false
}
}
}
fileConfig Based Logging Config
The file config is an older and more limited method of
configuring logging. It is described by the Configuration file
format
in the Python documentation. The file name in the tracker’s
config for the logging -> config setting must end with
.ini to choose the correct processing.
Example fileConfig LoggingConfig
This is an example .ini used with roundup-server configured to use
roundup.http channel. It also includes some custom logging
qualnames/tags/channels for logging schema/permission detector and
extension output:
[loggers]
#other keys: roundup.hyperdb.backend
keys=root,roundup,roundup.http,roundup.hyperdb,actions,schema,extension,detector
[logger_root]
#also for root where channlel is not set (NOTSET) aka all
level=DEBUG
handlers=rotate
[logger_roundup]
# logger for all roundup.* not otherwise configured
level=DEBUG
handlers=rotate
qualname=roundup
propagate=0
[logger_roundup.http]
level=INFO
handlers=rotate_weblog
qualname=roundup.http
propagate=0
[logger_roundup.hyperdb]
level=WARNING
handlers=rotate
qualname=roundup.hyperdb
propagate=0
[logger_actions]
level=INFO
handlers=rotate
qualname=actions
propagate=0
[logger_detector]
level=INFO
handlers=rotate
qualname=detector
propagate=0
[logger_schema]
level=DEBUG
handlers=rotate
qualname=schema
propagate=0
[logger_extension]
level=INFO
handlers=rotate
qualname=extension
propagate=0
[handlers]
keys=basic,rotate,rotate_weblog
[handler_basic]
class=StreamHandler
args=(sys.stderr,)
formatter=basic
[handler_rotate]
class=logging.handlers.RotatingFileHandler
args=('roundup.log','a', 5120000, 2)
formatter=basic
[handler_rotate_weblog]
class=logging.handlers.RotatingFileHandler
args=('httpd.log','a', 1024000, 2)
formatter=plain
[formatters]
keys=basic,plain
[formatter_basic]
format=%(asctime)s %(trace_id)s %(process)d %(name)s:%(module)s.%(funcName)s,%(levelname)s: %(message)s
datefmt=%Y-%m-%d %H:%M:%S
[formatter_plain]
format=%(process)d %(message)s
Configuring roundup-server
The basic configuration file is as follows (taken from the
roundup-server.ini.example file in the “doc” directory):
[main]
# Host name of the Roundup web server instance.
# If left unconfigured (no 'host' setting) the default
# will be used.
# If empty, listen on all network interfaces.
# If you want to explicitly listen on all
# network interfaces, the address 0.0.0.0 is a more
# explicit way to achieve this, the use of an empty
# string for this purpose is deprecated and will go away
# in a future release.
# Default: localhost
host = localhost
# Port to listen on.
# Default: 8080
port = 8017
# Path to favicon.ico image file. If unset, built-in favicon.ico is used.
# The path may be either absolute or relative
# to the directory containing this config file.
# Default: favicon.ico
favicon = favicon.ico
# User ID as which the server will answer requests.
# In order to use this option, the server must be run initially as root.
# Availability: Unix.
# Default:
user = roundup
# Group ID as which the server will answer requests.
# In order to use this option, the server must be run initially as root.
# Availability: Unix.
# Default:
group =
# Maximum number of children to spawn using fork multiprocess mode.
# Default: 40
max_children = 40
# don't fork (this overrides the pidfile mechanism)'
# Allowed values: yes, no
# Default: no
nodaemon = no
# Log client machine names instead of IP addresses (much slower)
# Allowed values: yes, no
# Default: no
log_hostnames = no
# Have http(s) request logging done via python logger module.
# If set to yes the python logging module is used with qualname
# 'roundup.http'. Otherwise logging is done to stderr or the file
# specified using the -l/logfile option.
# Allowed values: yes, no
# Default: no
loghttpvialogger = no
# File to which the server records the process id of the daemon.
# If this option is not set, the server will run in foreground
#
# The path may be either absolute or relative
# to the directory containing this config file.
# Default:
pidfile =
# Log file path. If unset, log to stderr.
# The path may be either absolute or relative
# to the directory containing this config file.
# Default:
logfile =
# Set processing of each request in separate subprocess.
# Allowed values: debug, none, thread, fork.
# Default: fork
multiprocess = fork
# Tracker index template. If unset, built-in will be used.
# The path may be either absolute or relative
# to the directory containing this config file.
# Default:
template =
# Enable SSL support (requires pyopenssl)
# Allowed values: yes, no
# Default: no
ssl = no
# PEM file used for SSL. A temporary self-signed certificate
# will be used if left blank.
# The path may be either absolute or relative
# to the directory containing this config file.
# Default:
pem =
# Comma separated list of extra headers that should
# be copied into the CGI environment.
# E.G. if you want to access the REMOTE_USER and
# X-Proxy-User headers in the back end,
# set to the value REMOTE_USER,X-Proxy-User.
# Allowed values: comma-separated list of words
# Default:
include_headers =
# Change to HTTP/1.0 if needed. This disables keepalive.
# Default: HTTP/1.1
http_version = HTTP/1.1
# Roundup trackers to serve.
# Each option in this section defines single Roundup tracker.
# Option name identifies the tracker and will appear in the URL.
# Option value is tracker home directory path.
# The path may be either absolute or relative
# to the directory containing this config file.
[trackers]
demo = /trackers/demo
sysadmin = /trackers/sysadmin
Additional notes for each keyword:
- template
Specifies a template used for displaying the tracker index when multiple trackers are being used. It is processed by TAL and the variable “trackers” is available to the template and is a dict of all configured trackers.
- ssl
Enables use of SSL to secure the connection to the roundup-server. In most cases, you will want to run a real web server (Apache, Nginx) as a proxy to roundup-server running without SSL. The real web server can filter/rate limit/firewall requests to roundup-server. If you enable this, ensure that your tracker’s config.ini specifies an https URL. See roundup-server.1 man page for additional information.
- pem
If specified, the SSL PEM file containing the private key and certificate. The file must include both the private key and certificate with appropriate headers (e.g.
-----BEGIN PRIVATE KEY-----,-----END PRIVATE KEY-----and-----BEGIN CERTIFICATE-----,-----END CERTIFICATE-----. If not specified, roundup will generate a temporary, self-signed certificate for use.- loghttpvialogger section
If you:
have loghttpvialogger enabled
use pidfile
use a logging config file in the tracker’s config.ini
it is essential to specify absolute paths for log files in the tracker’s logging.config file. The use of pidfile causes the server to switch to the root directory (‘/’). As a result relative paths in the logging ini configuration file (as opposed to the tracker’s config.ini) will be written to the system’s root directory. The access error will cause the server to exit.
- trackers section
Each line denotes a mapping from a URL component to a tracker home. Make sure the name part doesn’t include any url-unsafe characters like spaces. Stick to alphanumeric characters and you’ll be ok.
To generate a config.ini in the current directory from the roundup-server command line use:
roundup_server -p 8017 -u roundup --save-config demo=/trackers/demo \
sysadmin=/trackers/sysadmin
Note it will save an old config.ini file to config.bak and create a new config.ini. The file is recreated from scratch ignoring the contents of the current config.ini. You may need to merge the backup and config files. save-config doesn’t attempt to load or verify an existing config.ini. Running this in a tracker home directory will move the exsiting config.ini to config.bak and replace it with the roundup-server’s config.ini. This will make the tracker in the directory fail to start util the original config.ini is restored.
Configuring Compression
Roundup will compress HTTP responses to clients on the fly. Dynamic, on the fly, compression is enabled by default, to disable it set:
[web]
...
dynamic_compression = No
in the tracker’s config.ini. You should disable compression if
your proxy (e.g. nginx or apache) is configured
to compress responses on the fly. The python standard library includes
gzip support. For brotli or zstd you will need to install packages. See
the installation documentation for details.
Some assets will not be compressed on the fly. Assets with mime types
of “image/png” or “image/jpeg” will not be compressed. You
can add mime types to the list by using interfaces.py as discussed
in the customisation documentation. As an example adding:
from roundup.cgi.client import Client
Client.precompressed_mime_types.append('application/zip`)
to interfaces.py will prevent zip files from being compressed.
Any content less than 100 bytes in size will not be compressed (e.g errors messages, short json responses).
Zstd will be used if the client can understand it, followed by brotli
then gzip encoding. Currently the preference order is hard coded into
the server and not parsed using q values from the client’s
Accept-Encoding header. This is an area for improvement.
In addition to dynamic compression, static files/assets accessed using
@@file can be pre-compressed. This reduces CPU load on the server
and reduces the time required to respond to the client. By default
searching for pre-compressed files is disabled. To enable it set:
[web]
...
use_precompressed_files = Yes
in the tracker’s config.ini file. Then you can create a
precompressed file and it will be served if the client is able to
accept it. For a file .../@@file/library.js you can create:
tracker_home/html/library.js.gzip
tracker_home/html/library.js.br
tracker_home/html/library.js.zstd
which should be created by using (respectively):
gzip --keep --suffix .gzip library.js
brotli library.js
zstd library.js && mv library.js.zst library.js.zstd
see the man pages for options that control compression level. Note that some levels require additional memory on the client side, so you may not always want to use the highest compression available.
A pre-compressed file will not be used if its modified date is earlier
than the uncompressed file. For example, if library.js.gzip is
older (has earlier modification date) than library.js,
library.js.gzip will be ignored. library.js will be
served instead. library.js will be dynamically compressed on the
fly and a warning message will be logged.
Precompressed files override dynamic compression. For example, assume the client can accept brotli and gzip. If there are no precompressed files, the data will be compressed dynamically (on the fly) using brotli. If there is a precompressed gzip file present the client will get the gzip version and not a brotli compressed version. This mechanism allows the admin to allow use of brotli and zstd for dynamic content, but not for static content.
Controlling Browser Handling of Attached Files
You may be aware of the allow_html_file config.ini setting. When set to yes, it permits
html files to be attached and displayed in the browser as html
files. The underlying mechanism used to enable/disable attaching HTML
is exposed using interfaces.py.
Similar to Client.precompressed_mime_types above, there is a
Client.mime_type_allowlist. If a mime type is present in this
list, an attachment with this mime type is served to the browser. If
the mime type is not present, the mime type is set to
application/octet-stream which causes the browser to download the
attachment to a file.
In release 2.4.0, the mime type application/pdf was removed from
the precompressed_mime_types list. This prevents the browser from
executing scripts that may be included in the PDF file. If you trust
the individuals uploading PDF files to your tracker and wish to allow
viewing PDF files from your tracker, you can do so by editing your
tracker’s “interfaces.py” file. Adding:
from roundup.cgi.client import Client
Client.mime_type_allowlist.append('application/pdf')
will permit the PDF files to be viewed in the browser rather than downloaded to a file.
Similarly, you can remove a mime type (e.g. audio/oog) using:
from roundup.cgi.client import Client
Client.mime_type_allowlist.remove('audio/oog')
which will force the browser to save the attachment to a file rather than playing the audio file.
Configuring REST Maximum Result Limit
To prevent denial of service (DOS) and limit user wait time for an
unbounded request, the REST endpoint has a maximum limit on the number
of rows that can be returned. By default, this is set to 10 million.
This setting applies to all users of the REST interface. If you want
to change this limit, you can add the following code to the
interfaces.py file in your tracker:
# change max response rows
from roundup.rest import RestfulInstance
RestfulInstance.max_response_row_size = 26
This code will set the maximum number of rows to 25 (one less than the
value). Note that this setting is rarely used and is not available in
the tracker’s config.ini file. Setting it through this mechanism
allows you to enter a string or number that may break Roundup, such as
“asdf” or 0. In general, it is recommended to keep the limit at its
default value. However, this option is available for cases when a
request requires more than 10 million rows and pagination using
@page_index and @page_size=9999999 is not possible.
Adding a Web Content Security Policy (CSP)
A Content Security Policy (CSP) adds a layer of security to Roundup’s web interface. It makes it more difficult for an attacker to compromise Roundup. By default Roundup does not add a CSP. If you need to implement a CSP, this section will help you understand how to add one and document the current level of support for CSP in Roundup.
Roundup’s web interface has remained mostly unchanged since it was created over a decade ago. Current releases have been slowly modernizing the HTML to improve security. There are still some improvements that need to happen before the tightest CSP configurations can be used.
Writing a CSP is complex. This section just touches on how to create and install a CSP to improve security. Some of it might break functionality.
There are two ways to add a CSP:
a fixed CSP added by a server
a dynamic CSP added by Roundup
Fixed CSP
If you are using a web server (Apache, Nginx) to run Roundup, you
can add a Content-Security-Policy header using that
server. WSGI middleware can be written to add headers. An example
header would look like:
Content-Security-Policy: default-src 'self' 'unsafe-inline' 'strict-dynamic';
One thing that may need to be included is the unsafe-inline.
The default templates use onload, onchange, onsubmit,
and onclick JavaScript handlers. Without unsafe-inline
these won’t work and popup helpers will not work. Sadly the use
of unsafe-inline is a pretty big hole in this CSP. You can
set the hashes for all the JavaScript handlers in the CSP. Then
replace unsafe-inline with unsafe-hashes to help close
this hole, but has its own issues. See remediating
unsafe-inline for another way to mitigate this.
The inclusion of strict-dynamic allows trusted JavaScript
files that are downloaded from Roundup to make changes to the web
interface. These changes are also trusted code that will be run
when invoked.
More secure CSPs can also be created. However because of the ability to customise the web interface, it is difficult to provide guidance.
Dynamic CSP
Roundup creates a cryptographic nonce for every client request. The
nonce is the value of the client.client_nonce property.
By changing the templates to use the nonce, we can better secure the Roundup instance. However the nonce has to be set in the CSP returned by Roundup.
One way to do this is to add a templating utility to the extensions directory that generates the CSP on the fly. For example:
default_security_headers = {
'Content-Security-Policy': (
"default-src 'self'; "
"base-uri 'self'; "
"script-src https: 'nonce-{nonce}' 'strict-dynamic'; "
"style-src 'self' 'nonce-{nonce}'; "
"img-src 'self' data:; "
"frame-ancestors 'self'; "
"object-src 'self' 'nonce-{nonce}'; "
),
}
def AddHtmlHeaders(self, header_dict=None):
''' Generate https headers from dict use default security headers
Setting the header with a value of None will not inject the
header and can override the default set.
Header values will be formatted with a dictionary including a
nonce. Use to set a nonce for inline scripts.
self is an instance of the TemplatingUtilities class, so
you have access to self.client as well as any functions added
using registerUtil.
'''
try:
if self.client.client_nonce is None:
# logger.warning("client_nonce is None")
self.client.client_nonce = self.client.session_api._gen_sid()
except AttributeError:
# self.client.client_nonce doesn't exist, create it
# logger.warning("client_nonce does not exist, creating")
self.client.client_nonce = client.session_api._gen_sid()
headers = default_security_headers.copy()
if isinstance(header_dict, dict):
headers.update(header_dict)
client_headers = self.client.additional_headers
for header, value in list(headers.items()):
if value is None:
continue
client_headers[header] = value.format(
nonce=self.client.client_nonce)
def init(instance):
# Note the use of the new (in version 2.5) registerUtilMethod
instance.registerUtilMethod('AddHtmlHeaders', AddHtmlHeaders)
Adding the following to page.html right after the opening
<html....`> tag:
<tal:code tal:content="python:utils.AddHtmlHeaders()" />
will invoke AddHtmlHeaders() to add the CSP header with the nonce.
With this set of CSP headers, all style, script and object tags will
need a nonce attribute. This can be added by changing:
<script src="javascript.js"></script>
to:
<script
tal:attributes="nonce request/client/client_nonce"
src="javascript.js"></script>
for each script, object or style tag.
If you are using a version of Roundup before version 2.5, you need to
replace instance.registerUtilMethod with
instance.registerUtil. For example:
def init(instance):
instance.registerUtil('AddHtmlHeaders', AddHtmlHeaders)
The AddHtmlHeaders function needs to be changed so that self.client
is replaced by client:
# replace self parameter with client
def AddHtmlHeaders(client, header_dict=None):
''' Generate https headers from dict use default security headers
Setting the header with a value of None will not inject the
header and can override the default set.
Header values will be formatted with a dictionary including a
nonce. Use to set a nonce for inline scripts.
'''
### Then change all references to self.client to client
try:
if client.client_nonce is None: # note self.client -> client
# logger.warning("client_nonce is None")
client.client_nonce = self.client.session_api._gen_sid()
...
Lastly the client must be passed explicitly when calling AddHtmlHeaders. The call looks like:
<tal:code tal:content="python:utils.AddHtmlHeaders(request.client)" />
Remediating unsafe-inline
Using a trusted script to set event handlers to replace the onX
handlers allows removal of the unsafe-inline handlers. If you
remove unsafe-inline the onX handlers will not run. However
you can use the label provided by the onX attribute to securely
enable a callback function.
This method is a work in progress. As an example proof of concept, adding this “decorator” script at the end of page.html:
<script tal:attributes="nonce request/client/client_nonce">
/* set submit event listener on forms that have an
onsubmit (case insensitive) attribute */
forms = document.querySelectorAll(form[onsubmit])
for (let form of f) {
form.addEventListener('submit',
() => submit_once());
};
</script>
will set callback for the submit even on any form that has an onsubmit
attribute to submit_once(). submit_once is defined in Roundup’s
base_javascript and is generated with a proper nonce.
By including the nonce in the dynamic CSP, we can use our trusted
“decorator” script to add event listeners. These listeners will call
the trusted function in base_javascript to replace the ignored onX
handlers.
Classhelper Web Component
Version 2.4.0 provides a new classhelper popup written as a web component. By installing 3 files and editing the tracker’s templates you can enable the new component.
The development of this component was done by team-03 of the Spring 2024 CS682 graduate software engineering SDL capstone class at the University of Massachusetts - Boston. Their documentation is copied/adapted below.
File Installation
There are three files to install in your tracker. You can
copy them from the template directory for the classic
tracker. The location of the template file can be obtained
by running: roundup-admin templates and looking for the
path value of the classic template. If the path value is
/path/to/template, copy:
/path/to/template/html/classhelper.js
/path/to/template/html/classhelper.css
/path/to/template/html/_generic.translation
to your tracker’s html directory.
Wrapping the Classic Classhelper
To allow your users to select items in the web interface
using the new classhelper, you should edit the template files
in the html/ subdirectory of your tracker. Where you see
code like:
<th i18n:translate="">Superseder</th>
<td>
<span tal:replace="structure
python:context.superseder.field(showid=1,size=20)" />
<span tal:condition="context/is_edit_ok"
tal:replace="structure
python:db.issue.classhelp('id,title',
property='superseder', pagesize=100)" />
[...]
</td>
change it to wrap the classhelp span like this:
<th i18n:translate="">Superseder</th>
<td>
<span tal:replace="structure
python:context.superseder.field(showid=1,size=20)" />
<roundup-classhelper
data-popup-title="Superseder Classhelper - {itemDesignator}"
data-search-with="title,status,keyword[]-name">
<span tal:condition="context/is_edit_ok"
tal:replace="structure
python:db.issue.classhelp('id,title',
property='superseder', pagesize=100)" />
</roundup-classhelper>
[...]
</td>
which displays a three part classhelper.
the search pane includes a text search box for the title and status properties and a dropdown for the keyword property, sorted by name in descending order.
the selection pane will show 100 search results per page. It also allows the user to move to the next or previous result page.
the accumulator at the bottom shows all the selected items. It also has buttons to accept the items or cancel the classhelper, leaving the original page unchanged.
Note that the user class is a little different because users without an Admin role can’t search for a user by Role. So we hide the Role search element for non admin users. Starting with:
<th i18n:translate="">Nosy List</th>
<td>
<span tal:replace="structure context/nosy/field" />
<span tal:condition="context/is_edit_ok" tal:replace="structure
python:db.user.classhelp('username,realname,address',
property='nosy', width='600'" />
</td>
wrap the classhelp span with <roundup-classhelper> like:
<th i18n:translate="">Nosy List</th>
<td>
<span tal:replace="structure context/nosy/field" />
<roundup-classhelper tal:define="search string:name,phone,roles[]"
tal:attributes="data-search-with python:search
if request.user.hasRole('Admin') else
','.join(search.split(',')[:-1])">
<span tal:condition="context/is_edit_ok" tal:replace="structure
python:db.user.classhelp('username,realname,address',
property='nosy', width='600'" />
</roundup-classhelper>
</td>
The ','.join(search.split(',')[:-1]) removes the last element of
the search string (roles[]) if the user does not have the Admin
role.
Loading the <roundup-classhelper> Script
To make the <roundup-classhelper> wrappers work, you
have to load the classhelper.js script. Add the following
html script tag in the head section of page.html:
<script type="text/javascript" src="@@file/classhelper.js">
</script>
You can place it anywhere before </head>. This will make
it available on all pages.
The script will also work if it is loaded at the end of the
body. It can also be added to the more-javascript slot
in one of the templates (see user.item.html for an
example) if you don’t want the overhead of loading the script
on every page.
You may want to minimize and precompress the file. Using dynamic compression, the file is 10k when compressed with gzip (8k with brotli). If you minimize the file first with rjsmin and then compress it, it’s about 6k. See the information about using precompressed files in the section on compression.
<roundup-classhelper> configuration
There are two attributes used to configure the classhelper.
- data-popup-title:
this attribute is optional. A reasonable default is provided if it is missing.
Adding
data-popup-titlechanges the title of the popup window with the value of the attribute.{itemDesignator}can be used inside the attribute value to replace it with the current classhelper usage context. E.G.data-popup-title="Nosy List Classhelper - {itemDesignator}"will display the popup window title asNosy List Classhelper - issue24
- data-search-with:
this attribute is optional. If it is not set, a search panel is not created to allow the user to search within the class.
Adding
data-search-withspecifies the fields that can be used for searching. For example when invoking the classhelper for the issue class, usingdata-search-with="title,status,keyword"wil enable three search fields.The search can be customized using the following syntax:
Adding
[]at then end of a field ("status[]") will displays a dropdown for the “status” field listing all the values the user can access. E.G.:<roundup-classhelper data-search-with="title,status[],keyword[]"> <span tal:condition="context/is_edit_ok" tal:replace="structure python:db.issue.classhelp('id,title', property='superseder', pagesize=100)" /> </roundup-classhelper>
will create a search pane with a text search for title and dropdowns for status and keyword.
Adding a sort key after the
[]allows you to select the order of the elements in the dropdown. For examplekeyword[]+namesorts the keyword dropdown in ascending order by name. Whilekeyword[]-namesorts the keyword dropdown in descending order by name. If the sort order is not specified, the default order for the class is used.
<roundup-classhelper> styling
The roundup-classhelper component uses minimal styling so it can blend in with most trackers. If you want to change the styling, you can modify the classhelper.css file in the html directory. Even though roundup-classhelper is a web component, it doesn’t use the shadow DOM. If you don’t know what this means, it just means that it’s easy to style.
Getting the web component to load changes to the css file is a bit tricky. The browser caches the old file and you have to resort to tricks to make it get a new copy of the file.
One way to do this is to open to the classhelper.css
file in your browser and force refresh it. To do this:
Open the home page for your Roundup issue tracker in a web browser.
In the address bar, append
@@file/classhelper.cssto the end of your Roundup URL. For example, if your Roundup URL ishttps://example.com/tracker/, the URL you should visit would behttps://example.com/tracker/@@file/classhelper.css.This will open the
classhelper.cssfile in your browser.Press
Ctrl+Shift+R(on Windows and Linux) orCmd+Shift+R(on macOS). This triggers a hard refresh of the page, which forces the browser to reload the file and associated resources from the server.
This should resolve any issues caused by cached or outdated files. It is possible that you have to open devtools and set the disable cache option in the network panel in extreme cases.
Also during development, you might want to set a very low cache time for classhelper.css using something like:
Client.Cache_Control['classhelper.css'] = "public, max-age=10"
Translations
To set up translations for the <roundup-classhelper> component, follow these steps.
Create a
messages.potfile by runningroundup-gettext <tracker_home_directory>. This createslocale/messages.potin your tracker’s home directory. It extracts all translatable strings from your tracker. We will use it as a base template for the new strings you want to translate.See if you already have a
.potranslation file for your language in the tracker’s locale/ directory. If you don’t, copymessages.potto a .po file for the language you want to translate. For example German would be atde.poEnglish would be aten.po(for example if you want to change theapplybutton to sayDo It.Edit the new .po file. After the header, add the translation entries for the <roundup-classhelper> component. For example next and submit are displayed in English when the rest of the interface is in German. Add:
msgid "submit" msgstr "gehen" msgid "next" msgstr "nächste" msgid "name" msgstr "name"Note: the value for msgid is case sensitive. You can see the msgid for static strings by looking for
CLASSHELPER_TRANSLATION_KEYWORDSin classhelper.js.Save the .po file.
Restart your Roundup instance.
This should display the missing translations, for more details refer to the translation (i18n) section of the developers documentation.
The default title used for read only popups can be changed by using the translation mechanism. Use the following:
msgid "Info on {className} - {itemDesignator} - Classhelper"
msgstr "{itemDesignator} - info on {className}"
in en.po to reformat the title for the English language. Note that
{classname} is only supported in the default title.
In addition to the default template you can translate a title set
using data-popup-title by matching the template as the msgid.
Using an example above:
msgid "Nosy List Classhelper - {itemDesignator}"
msgstr "Nosy List Klassenhelfer - {itemDesignator}"
placed in de.po will translate the title into German.
Troubleshooting
The roundup-classhelper will fallback to using the classic classhelper if:
the user doesn’t have REST access
the browser doesn’t support web components
It will display an alert modal dialog to the user before triggering
the classic classhelper as a fallback. A detailed error will be
printed to the browser console. The console is visible in devtools and
can be opened by pressing the F12 key.
You can disable the classhelper on a per URL basis by adding
#classhelper-wc-toggle to the end of the URL. This will prevent
the web component from starting up.
Also you can set DISABLE_CLASSHELP = true at the top of
classhelper.js to disable the classhelper without having to make any
changes to your templates.
Advanced Configuration
The classhelper.js file has a few tweakable options for use
by advanced users. The endpoint for the roles list requires
the user to have Admin rights. You can add your own roles
endpoint with a different authorization mechanism. The
following code can be added to your tracker’s interfaces.py.
You can create this file if it doesn’t exist. This code
creates a new REST endpoint at /rest/roles:
from roundup.rest import Routing, RestfulInstance, _data_decorator
class RestfulInstance:
@Routing.route("/roles", 'GET')
@_data_decorator
def get_roles(self, input):
"""Return all defined roles. The User class property
roles is a string but simulate it as a MultiLink
to an actual Roles class.
"""
return 200, {"collection":
[{"id": rolename,"name": rolename}
for rolename in list(self.db.security.role.keys())]}
See the REST documentation for details on
using interfaces.py to add new REST endpoints.
The code above allows any user with REST access to see all the roles defined in the tracker.
To make classhelper.js use this new endpoint, look for:
const ALTERNATIVE_DROPDOWN_PATHNAMES = {
"roles": "/rest/data/user/roles"
}
and change it to:
const ALTERNATIVE_DROPDOWN_PATHNAMES = {
"roles": "/rest/roles"
}
Users may have to perform a hard reload to cache this change on their system.
Configuring native-fts Full Text Search
Roundup release 2.2.0 supports database-native full text search. SQLite (minimum version 3.9.0) with FTS5 and PostgreSQL (minimum version 11.0) with websearch_to_tsvector are supported.
To enable this method, change the indexer setting in the tracker’s
config.ini to native-fts. Then reindex using roundup-admin -i
tracker_home reindex. The amount of time it takes to reindex
depends on the amount of data in your tracker, the speed of your
disks, etc. It can take hours.
SQLite details
The SQLite native-fts changes the full text search query a little bit. For the other search methods, the search terms are split on white space and each item in the index: a field (e.g. title), message content and file content is searched for all the terms. If any term is missing that item is ignored. Once the items are found they are mapped to an issue and the normal issue index is displayed.
When using FTS5, the search terms can use the full text search query language described at: https://www.sqlite.org/fts5.html#full_text_query_syntax. This supports:
plain word search (joined by ‘and’, similar to other search methods)
phrase search with terms enclosed in quotes (
")proximity search with varying distances using
NEAR()boolean operations by grouping with parentheses and using
ANDandORexclusion using
NOTprefix searching by prefixing the term with``^``
All of the data that is indexed is in a single column, so when column specifiers are used they usually result in an error which is detected and an enhanced error message is produced.
Unlike the native, xapian and whoosh indexers there is no limit to the length of terms that are indexed. Also stopwords are indexed but ignored when searching if they are the only word in the search. So a search for “the” will return no results but “the book” will return results. Pre-filtering the stopwords when indexing would break proximity and phrase searching. This may be helpful or problematic for your particular tracker.
To support the most languages available, the unicode61 tokenizer is
used without porter stemming. Using the indexer_language setting
to enable stemming for english is not available in this
implementation. Also ranking information is not used in this
implementation. These are areas for improvement.
PostgreSQL info
The PostgreSQL native-fts changes the full text search query a little bit. When using PostgreSQL full text search, two different query languages are supported.
websearch - described at the end of Parsing Queries under websearch_to_tsquery. This is the default.
tsquery - described at the beginning of Parsing Queries with to_tsquery. It is enabled by starting the search phrase with
ts:.
Websearch provides a more natural style of search and supports:
plain word search (stemmed in most cases)
phrase search with terms enclosed in quotes (
")exclusion by prefixing a term/phrase with
-alternative/or searching with
orbetween termsignores non-word characters including punctuation
Tsquery supports:
a strict query syntax
plain word search
phrase search with the
<->operator or enclosing the phrase in'single quotes (note this will use a stemmer on the terms in the phrase).proximity search with varying distances using
<N>boolean operations by grouping with parentheses and using
&and|exclusion using
!prefix searching using
:*at the end of the prefix
All of the data that is indexed is in a single column and input weighing is not used.
Depending on the FTS configuration (determined by the
indexer_language setting), stopwords are supported. PostgreSQL
takes the stopwords into account when calculating the data needed for
proximity/near searches. Like SQLite FTS, there is no limit to the
length of terms that are indexed. Again this may be helpful or
problematic for your particular tracker.
The config.ini indexer_language setting is used to define the
configuration used for indexing. For example with the default
english setting a snowball stemmer (english_stem) is used. So
words like ‘drive’ and ‘driving’ and ‘drive-in’ will all match a
search for ‘drive’ but will not match ‘driver’.
The indexer_language is used as the configuration name for every call to the text search functions (to_tsvector, to_tsquery). Changing this requires reindexing.
The configuration list can be obtained using using psql’s
\dF command.
Roundup includes a hardcoded list for all languages supported by
PostgreSQL 14.1. The list includes 5 custom “languages”
custom1 … custom5 to allow you to set up your own textsearch
configuration using one of the custom names. Depending on your
PostgreSQL version, we may allow an invalid language to be configured.
You will see an error about text search configuration ... does not
exist.
It may be possible to append to this list using the tracker’s
interfaces.py. For details, see test/test_indexer.py in the
roundup distribution and search for valid_langs. If you succeed
please email roundup-users AT lists.sourceforge.net with a description
of your success.
After changing the configuration language, you must reindex the tracker since the index must match the configuration language used for querying.
Also there are various dictionaries that allow you to:
add stopwords
override stemming for a term
add synonyms (e.g. a search for “pg” can also match ‘psql’ “postgresql”)
add terms that expand/contract the search space (Thesaurus dictionary)
additional transforms
Use of these is beyond this documentation. Please visit the appropriate PostgreSQL documents. The following my also be helpful:
Ranking information is not used in this implementation. Also stop words set in config.ini are ignored. These are areas for improvement.
Cleaning up old native indexes
If you are happy with the database fts indexing, you can save some space by
removing the data from the native text indexing tables. This requires
using the sqlite3 or psql commands to execute SQL to delete the
rows in the __textids and __words tables. You can do this with
the following SQL commands:
delete from __words;
delete from __textids;
Note this deletes data from the tables and does not delete
the table. This allows you to revert to Roundup’s native
full text indexing on SQLite or Postgres. If you were to
delete the tables, Roundup will not recreate the
tables. Under PostgreSQL, you can use the truncate
<tablename> command if you wish.
Configuring Session Databases
The session and OTK (one time key) databases store information about the operation of Roundup. This ephemeral data:
web login session keys,
CSRF tokens (if enabled),
email password recovery one time keys,
rate limiting data,
…
can be a performance bottleneck. It usually happens with anydbm or SQLite backends. PostgreSQL and MySQL are sufficiently powerful that they can handle the higher transaction rates.
If you are using sqlite, you can choose to use the anydbm database for session data. By default it will use additional sqlite databases for storing the session and otk data.
The following table shows which primary databases support different session database backends:
session db |
|||||
|---|---|---|---|---|---|
primary db |
anydbm |
sqlite |
redis |
mysql |
postgresql |
anydbm |
D |
+ |
|||
sqlite |
+ |
D |
+ |
||
mysql |
D |
||||
postgresql |
D |
||||
The backend setting is in the tracker’s config.ini
file under the sessiondb section.
Using Redis for Session Databases
Redis is an in memory key/value data structure store.
You need to install the redis-py module from pypi. Then install Redis using your package manager or by downloading it from the Redis website.
You need to secure your redis instance. The data that
Roundup stores includes session cookies and other
authentication tokens. At minimum you should require a
password to connect to your redis database. Set
requirepass in redis.conf. Then change the
redis_url in config.ini to use the password.
For example:
redis://:mypassword@localhost:7200/10
will connect to the redis instance running on localhost at
port 7200 using the password mypassword to open database
10. The redis_url setting can load a file to better
secure the url. If you are using redis 6.0 or newer, you can
specify a username/password and access control lists to
improve the security of your data. Another good alternative
is to talk to redis using a Unix domain socket.
If you are connecting to redis across the network rather
than on localhost, you should configure ssl/tls and use the
rediss scheme in the url along with the query
parameters:
ssl_cert_reqs=required&ssl_ca_certs=/path/to/custom/ca-cert
where you specify the file that can be used to validate the SSL certificate. Securing Redis has more details.
Users and Security
Roundup holds its own user database which primarily contains a username, password and email address for the user. Roundup must have its own user listing, in order to maintain internal consistency of its data. It is a relatively simple exercise to update this listing on a regular basis, or on demand, so that it matches an external listing (eg. unix passwd file, LDAP, etc.)
Roundup identifies users in a number of ways:
Through the web, users may be identified by either HTTP Basic Authentication or cookie authentication. If you are running the web server (roundup-server) through another HTTP server (eg. apache or IIS) then that server may require HTTP Basic Authentication, and it will pass the
REMOTE_USERvariable (or variable defined using http_auth_header) through to Roundup. If this variable is not present, then Roundup defaults to using its own cookie-based login mechanism.In email messages handled by roundup-mailgw, users are identified by the From address in the message.
In both cases, Roundup’s behaviour when dealing with unknown users is
controlled by Permissions defined in the “SECURITY SETTINGS” section of the
tracker’s schema.py module:
- Web Access and Register
If granted to the Anonymous Role, then anonymous users will be able to register through the web.
- Email Access and Register
If granted to the Anonymous Role, then email messages from unknown users will result in those users being registered with the tracker.
More information about how to customise your tracker’s security settings may be found in the reference documentation.
Configuring Authentication Header/Variable
The front end server running Roundup can perform the user
authentication. It pass the authenticated username to the backend in a
variable. By default roundup looks for the REMOTE_USER variable
This can be changed by setting the parameter http_auth_header in the
[web] section of the tracker’s config.ini file. If the value
is unset (the default) the REMOTE_USER variable is used.
If you are running roundup using roundup-server behind a proxy
that authenticates the user you need to configure roundup-server to
pass the proper header to the tracker. By default roundup-server
looks for the REMOTE_USER header for the authenticated user. You
can copy an arbitrary header variable to the tracker using the -I
option to roundup-server (or the equivalent option in the
roundup-server config file).
For example to use the uid_variable header, two configuration
changes are needed: First configure roundup-server to pass the
header to the tracker using:
roundup-server -I uid_variable ....
note that the header is passed exactly as supplied by the upstream
server. It is not prefixed with HTTP_ like other headers since
you are explicitly allowing the header. Multiple comma separated
headers can be passed to the -I option. These could be used in a
detector or other tracker extensions, but only one header can be used
by the tracker as an authentication header.
To make the tracker honor the new variable, change the tracker’s
config.ini to read:
[web]
...
http_auth_header = uid_variable
At the time this is written, support is experimental. If you use it you should notify the roundup maintainers using the roundup-users mailing list.
Securing Secrets
Roundup can read secrets from a file that is referenced from any of the config.ini files. If you use Docker, you can bind mount the files from a secure location, or store them in a subdirectory of the tracker home.
You can also use a secrets management tool like Docker Swarm’s secrets management. This example config.ini configuration gets the database password from a file populated by Swarm secrets:
[rdbms]
# Database user password.
# A string that starts with 'file://' is interpreted as a file
# path relative to the tracker home. Using 'file:///' defines
# an absolute path. The first line of the file will be used as
# the value. Any string that does not start with 'file://' is
# used as is. It removes any whitespace at the end of the
# line, so a newline can be put in the file.
#
# Default: roundup
password = file:///run/secrets/db_password
assuming that Docker Swarm secrets has the key db_password
and the --secret db_password option is used when starting the
Roundup service.
Because environment variables can be inadvertently exposed in logs or process listings, Roundup does not currently support loading secrets from environment variables.
Configuring PGP Email Support
Note
This section was written with the help of the Devin/DeepWiki AI.
You have to install the gpg module using pip (pip install gpg).
In your tracker’s config.ini configure the following settings in the
[pgp] section:
enable = yes
homedir = /path/to/pgp/configdir
roles = admin
This will allow any user with the admin role to send signed pgp
email. If roles is not set, all users will need to use signed
emails. If it is not signed it will be rejected. Note that homedir
must be an absolute path. Unlike other path settings, a relative path
is not interpreted relative to the tracker home. See the documentation
in config.ini for more information and other settings (e.g. to send
encrypted emails from the tracker).
When PGP is enabled and a message is signed with a valid signature,
the database transaction source (db.tx_Source) is set to
email-sig-openpgp instead of email. This allows you to
restrict certain operations (e.g. changing a private flag) to
authenticated/signed emails.
Creating GPG Keys for the Tracker
To generate a keypair use:
gpg --homedir /path/to/pgp/configdir --gen-key
where the homedir directory matches the one you set in config.ini. Note the gpg homedir must be created before you run the command. You will be prompted for the full name of your tracker and the email address for your tracker. You also need to do with as the user who runs roundup (aka the roundup user) and the roundup email gateway. Do not encrypt the key.
Roundup has no mechanism for reading the private key if it is encrypted. So make sure the permissions on the homedir only allow the roundup user to read the files.
You can export the public key for use by clients using:
gpg --homedir /path/to/pgp/configdir --export -a tracker@example.com > tracker-public.key
with homedir and email matching the values used to generate the key. This will allow users to import the public key and encrypt emails to the tracker.
The public gpg key for each user’s email address must be imported. To do this, obtain the user’s public key for their primary email address and import it using:
gpg --homedir /path/to/tracker/gpg --import user-public-key.asc
You may also be able to get it from a public keyserver using:
gpg --recv-keys KEYID
where the KEYID is supplied by the roundup user.
While Roundup supports multiple addresses for each user, only the primary address supports PGP signed or encrypted messages.
You should verify that the public key is sane and has few signatures attached. You can import a key into a throw away keystore:
mkdir throwaway
gpg --homedir throwaway -- import user-public-key.asc
gpg --homedir throwaway --list-sigs
and verify that the number of sig lines is small (under 10 or so). If it takes a long time to import you can kill the import without affecting your production keystore. Large numbers of sig lines can take a long time to import/access when compressed. See: https://nvd.nist.gov/vuln/detail/CVE-2022-3219.
Tasks
Maintenance of Roundup can involve one of the following:
Tracker Backup
The roundup-admin import and export commands are not recommended for performing backup.
Optionally stop the web and email frontends and to copy the contents of the tracker home directory to some other place using standard backup tools. This means using pg_dump to take a snapshot of your Postgres backend database, for example. A simple copy of the tracker home (and files storage area if you’ve configured it to be elsewhere) will then complete the backup.
Software Upgrade
Always make a backup of your tracker before upgrading software. Steps you may take:
Install pytest and ensure that the unit tests run on your system (using your preferred python version):
pip2 install pytest python2 -m pytest test/ pip3 install pytest python3 -m pytest test/
If you’re using an RDBMS backend, make a backup of its contents now.
Make a backup of the tracker home itself.
Stop the tracker web, email frontends and any scheduled (cron) jobs that access the tracker.
Install the new version of the software in a new virtual environment:
python3 -m venv /path/to/env . /path/to/env/bin/activate python3 pip install roundup
Follow the steps in the upgrading documentation for all the versions between your original version and the new version.
Usually you should run roundup_admin -i <tracker_home> migrate on your tracker(s) before you allow users to start accessing the tracker.
It’s safe to run this even if it’s not required, so just get into the habit.
Restart your tracker web and email frontends.
If something bad happens, you may reinstate your backup of the tracker and reinstall the older version of the sofware using the same install command:
python setup.py install
Migrating Backends
Note that the [rdbms] section label was called [database] in
earlier versions of Roundup.
Stop the existing tracker web and email frontends (preventing changes).
Use the roundup-admin tool “export” command to export the contents of your tracker to disk. (If you are running on windows see issue1441336 on how to use the command line rather than interactive mode to export data.)
Copy the tracker home to a new directory.
Delete the “db” directory from the new directory.
Set the value of the
backendkey under the[rdbms]section of the tracker’sconfig.inifile.Use the roundup-admin “import” command to import the previous export with the new tracker home. If non-interactively:
roundup-admin -i <tracker home> import <tracker export dir>
If interactively, enter ‘commit’ before exiting.
Test each of the admin tool, web interface and mail gateway using the new backend.
Move the old tracker home out of the way (rename to “tracker.old”) and move the new tracker home into its place.
Restart web and email frontends.
If you are importing into PostgreSQL, it autocommits the data every 10000 objects/rows by default. This can slow down importing, but it prevents an out of memory error caused by using a savepoint for each object. You can control the commit frequency by using:
pragma savepoint_limit=20000
to set a higher or lower number in roundup-admin. In this example a commit will be done every 20,000 objects/rows. The pragma can also be set on the roundup-admin command line as described below.
Migrating Only Database Data
The method above is well tested and works. However because it
exports/imports file content and works on a copy of the tracker home
it is slow. An alternate way is to use the exporttables and
importtables commands to export/import only the database entries
and do the migration in place.
The untested steps using this method are:
Backup your tracker.
Stop the existing tracker web and email frontends (preventing changes).
Use the roundup-admin tool’s “exporttable” command to export the database for your tracker to disk. (If you are running on windows see issue1441336 on how to use the command line rather than interactive mode to export data.)
Set the value of the
backendkey under the[rdbms]section of the tracker’sconfig.inifile to the new database backend.Use the roundup-admin “importtable” command to import the exporttable. The non-interactive command is:
roundup-admin -i <tracker home> importtable <tracker export dir>
If running the importtable interactively, enter ‘commit’ before exiting.
Test each of the admin tool, web interface and mail gateway using the new backend.
Restart web and email frontends.
Clean up the old database once you are confortable that the migration has suceeded.
If you are moving from an external database (postgres/mysql/mariadb) to an embedded db, drop your external db tables.
If you are migrating from anydbm to sqlite, you can move/delete files that match:
nodes.*_idsjournals.*lock
Keep the
db/dbfile as that is your new sqlite database.Also if you run for a few days and find the
otksandsessionsfile change times are the same, you can move/delete these as well.
Moving a Tracker
If you’re moving the tracker to a similar machine, you should:
install Roundup on the new machine and test that it works there,
stop the existing tracker web and email frontends (preventing changes),
copy the tracker home directory over to the new machine, and
start the tracker web and email frontends on the new machine.
Most of the backends are actually portable across platforms (ie. from Unix to Windows to Mac). If this isn’t the case (ie. the tracker doesn’t work when moved using the above steps) then you’ll need to:
install Roundup on the new machine and test that it works there,
stop the existing tracker web and email frontends (preventing changes),
use the roundup-admin tool “export” command to export the contents of the existing tracker,
copy the export to the new machine,
use the roundup-admin “import” command to import the tracker on the new machine, and
start the tracker web and email frontends on the new machine.
Migrating From Other Software
You have a couple of choices. You can either use a CSV import into Roundup, or you can write a simple Python script which uses the Roundup API directly. The latter is almost always simpler – see the “scripts” directory in the Roundup source for some example uses of the API.
“roundup-admin import” will import data into your tracker from a directory containing files with the following format:
one colon-separated-values file per Class with columns for each property, named <classname>.csv
one colon-separated-values file per Class with journal information, named <classname>-journals.csv (this is required, even if it’s empty)
if the Class is a FileClass, you may have the “content” property stored in separate files from the csv files. This goes in a directory structure:
<classname>-files/<N>/<designator>
where
<designator>is the item’s<classname><id>combination. The<N>value isint(<id> / 1000).
Adding A User From The Command-Line
The roundup-admin program can create any data you wish to in the
database. To create a new user, use:
roundup-admin create user
To figure out what good values might be for some of the fields (eg. Roles) you can just display another user:
roundup-admin list user
(or if you know their username, and it happens to be “richard”):
roundup-admin filter user username=richard
then using the user id (e.g. 5) you get from one of the above commands, you may display the user’s details:
roundup-admin display <designator>
where designator is user5.
Running the Servers
Unix
On Unix systems, use the scripts/server-ctl script to control the roundup-server server. Copy it somewhere and edit the variables at the top to reflect your specific installation.
If you use systemd look at scripts/systemd.gunicorn. It is configured for a wsgi deployment using gunicorn, but may be a good starting point for your setup.
Windows
On Windows, the roundup-server program runs as a Windows Service, and therefore may be controlled through the Services control panel. Note that you must install the pywin32 package to allow roundup to run as a service. The roundup-server program may also control the service directly:
- install the service
roundup-server -C /path/to/my/roundup-server.ini -c install- start the service
roundup-server -c start- stop the service
roundup-server -c stop
To bring up the services panel:
- Windows 2000 and later
Start/Control Panel/Administrative Tools/Services
- Windows NT4
Start/Control Panel/Services
You will need a server configuration file (as described in
Configuring roundup-server) for specifying tracker homes
and other roundup-server configuration. Specify the name of
this file using the -C switch when installing the service.
Running the Mail Gateway Script
The mail gateway script should be scheduled to run regularly on your Windows server. Normally this will result in a window popping up. The solution to this is to:
Create a new local account on the Roundup server
Set the scheduled task to run in the context of this user instead of your normal login
Mail gateway script command line
Usage:
usage: roundup_mailgw.py [-h] [-v] [-c DEFAULT_CLASS] [-I OAUTH_CLIENT_ID]
[-O OAUTH_DIRECTORY] [-S SET_VALUE]
[-T OAUTH_TOKEN_ENDPOINT]
[args ...]
The roundup mail gateway may be called in one of three ways:
without arguments. Then the env var ROUNDUP_INSTANCE will be tried.
with an instance home as the only argument,
with both an instance home and a mail spool file, or
with an instance home, a mail source type and its specification.
It also supports optional -S (or --set-value) arguments that allows you
to set fields for a class created by the roundup-mailgw. The format for
this option is [class.]property=value where class can be omitted and
defaults to msg. The -S options uses the same
property=value[;property=value] notation accepted by the command line
roundup command or the commands that can be given on the Subject line of
an email message (if you’re using multiple properties delimited with a
semicolon the class must be specified only once in the beginning).
It can let you set the type of the message on a per e-mail address basis by calling roundup-mailgw with different email addresses and other settings.
- PIPE:
If there is no mail source specified, the mail gateway reads a single message from the standard input and submits the message to the roundup.mailgw module.
- UNIX mailbox:
In this case, the gateway reads all messages from the UNIX mail spool file and submits each in turn to the roundup.mailgw module. The file is emptied once all messages have been successfully handled. The file is specified as:
mailbox /path/to/mailbox
In all of the following mail source types, the username and password
can be stored in a ~/.netrc file. If done so, only the server name
needs to be specified on the command-line.
The username and/or password will be prompted for if not supplied on
the command-line or in ~/.netrc.
- POP:
For the mail source “pop”, the gateway reads all messages from the POP server specified and submits each in turn to the roundup.mailgw module. The server is specified as:
pop username:password@server
The username and password may be omitted:
pop username@server pop server
are both valid.
- POPS:
Connect to a POP server over tls/ssl. This supports the same notation as POP:
pops username:password@server
- APOP:
Same as POP, but using Authenticated POP:
apop username:password@server
- IMAP:
Connect to an IMAP server. This supports the same notation as that of POP mail:
imap username:password@server
It also allows you to specify a specific mailbox other than INBOX using this format:
imap username:password@server mailbox
- IMAPS:
Connect to an IMAP server over tls/ssl. This supports the same notation as IMAP:
imaps username:password@server [mailbox]
- IMAPS_CRAM:
Connect to an IMAP server over tls/ssl using CRAM-MD5 authentication. This supports the same notation as IMAP:
imaps_cram username:password@server [mailbox]
- IMAPS_OAUTH:
Connect to an IMAP server over tls/ssl using OAUTH authentication. Note that this does not support a password in imaps URLs. Instead it uses only the user and server and a command-line option for the directory with the files
access_token,refresh_token,client_secret, andclient_id. By default this directory isoauthin your tracker home directory. The access token is tried first and, if expired, the refresh token together with the client secret is used to retrieve a new access token. Note that both token files need to be writeable, the access token is continuously replaced and some cloud providers may also renew the refresh token from time to time:imaps_oauth username@server [mailbox]
The refresh and access tokens (the latter can be left empty), the client id and the client secret need to be retrieved via cloud provider specific protocols or websites.
You need the requests library installed to ue the IMAPS_OAUTH method.
Using roundup-admin
Part of the installation includes a man page for roundup-admin. Ypu
should be able to read it using man roundup-admin. As shown above,
it is a generic tool for manipulating the underlying database for you
tracker.
Examples above show how to use it to:
install and initialize a new tracker
export/import tracker data for migrating between backends
creating a new user from the command line
list/find users in the tracker
The basic usage is:
Usage: roundup-admin [options] [<command> <arguments>]
Options:
-i instance home -- specify the issue tracker "home directory" to administer
-u -- the user[:password] to use for commands (default admin)
-d -- print full designators not just class id numbers
-c -- when outputting lists of data, comma-separate them.
Same as '-S ","'.
-S <string> -- when outputting lists of data, string-separate them
-s -- when outputting lists of data, space-separate them.
Same as '-S " "'.
-P pragma=value -- Set a pragma on command line rather than interactively.
Can be used multiple times.
-V -- be verbose when importing
-v -- report Roundup and Python versions (and quit)
Only one of -s, -c or -S can be specified.
Help:
roundup-admin -h
roundup-admin help -- this help
roundup-admin help <command> -- command-specific help
roundup-admin help all -- all available help
Commands:
commit
create classname property=value ...
display designator[,designator]*
export [[-]class[,class]] export_dir
exporttables [[-]class[,class]] export_dir
filter classname propname=value ...
find classname propname=value ...
genconfig <filename>
get property designator[,designator]*
help topic
history designator [skipquiet] [raw]
import import_dir
importtables export_dir
initialise [adminpw]
install [template [backend [key=val[,key=val]]]]
list classname [property]
migrate
pack period | date
perftest [mode] [arguments]*
pragma setting=value | 'list'
reindex [classname|classname:#-#|designator]*
restore designator[,designator]*
retire designator[,designator]*
rollback
security [Role name]
set items property=value [property=value ...]
specification classname
table classname [property[,property]*]
templates [trace_search]
updateconfig <filename>
Commands may be abbreviated as long as the abbreviation
matches only one command, e.g. l == li == lis == list.
In interactive mode entering: q, quit, or exit alone on a
line will exit the program.
One thing to note, The -u user setting does not currently operate
like a user logging in via the web. The user running roundup-admin
must have read access to the tracker home directory. As a result the
user has access to the files and the database info contained in
config.ini.
Using -u user sets the actor/user parameter in the
journal. Changes that are made are attributed to that
user. The password is ignored if provided. Any existing
username has full access to the data just like the admin
user. This is an area for further development so that
roundup-admin could be used with sudo to provide secure
command line access to a tracker.
In general you should forget that there is a -u parameter.
All commands (except help, genconfig, templates) require a tracker
specifier. This is just the path to the roundup tracker you’re working
with. A roundup tracker is where roundup keeps the database and
configuration file that defines an issue tracker. It may be thought of
as the issue tracker’s “home directory”. It may be specified in the
environment variable TRACKER_HOME or on the command line as “-i
tracker”.
A designator is a classname and an itemid concatenated, eg. bug1, user10, … Property values are represented as strings in command arguments and in the printed results:
Strings are, well, strings.
Password values will display as their encoded value.
Date values are printed in the full date format in the local time zone, and accepted in the full format or any of the partial formats explained below.:
Input of... Means... "2000-04-17.03:45" 2000-04-17.03:45:00 "2000-04-17" 2000-04-17.00:00:00 "01-25" yyyy-01-25.00:00:00 "08-13.22:13" yyyy-08-13.22:13:00 "11-07.09:32:43" yyyy-11-07.09:32:43 "14:25" yyyy-mm-dd.14:25:00 "8:47:11" yyyy-mm-dd.08:47:11 "2003" 2003-01-01.00:00:00 "2003-04" 2003-04-01.00:00:00 "." "right now"
Link values are printed as item designators. When given as an argument, item designators and key strings are both accepted.
Multilink values are printed as lists of item designators joined by commas. When given as an argument, item designators and key strings are both accepted; an empty string, a single item, or a list of items joined by commas is accepted.
When multiple items are specified to the roundup get or roundup set
commands, the specified properties are retrieved or set on all the
listed items. When multiple results are returned by the roundup get or
roundup find commands, they are printed one per line (default) or joined
by commas (with the “-c” option).
Where the command changes data, a login name/password is required. The
login may be specified as either “name” or “name:password”.
ROUNDUP_LOGINenvironment variablethe “
-u” command-line option
If either the name or password is not supplied, they are obtained from the command-line.
The -u user setting does not currently operate like a
user logging in via the web. The user running roundup-admin
must have read access to the tracker home directory. As a
result the user has access to the files and the database
info contained in config.ini.
Using -u user sets the actor/user parameter in the
journal. Changes that are made are attributed to that
user. The password is ignored if provided. Any existing
username has full access to the data just like the admin
user. This is an area for further development so that
roundup-admin could be used with sudo to provide secure
command line access to a tracker.
When you initialise a new tracker instance you are prompted for the admin password. If you want to initialise a tracker non-interactively you can put the initialise command and password on the command line. But this allows others on the host to see the password (using the ps command). To initialise a tracker non-interactively without exposing the password, create a file (e.g init_tracker) set to mode 600 (so only the owner can read it) with the contents:
initialise admin_password
and feed it to roundup-admin on standard input. E.G.:
cat init_tracker | roundup-admin -i tracker_dir -P history_features=2
setting the pragma history_features=2 prevents storing the command
in the user’s history file.
(for more details see https://issues.roundup-tracker.org/issue2550789.)
Using Interactively
You can edit the command line using left/right arrow keys to move the cursor. Using the up/down arrow keys moves among commands. It will save your commands between roundup-admin sessions. This is supported on Unix/Linux and Mac’s. On windows you should install the pyreadline3 package (see installation documentation).
Using the history_length pragma you can set the saved history size
for just one session.
You can add initialization commands to ~/.roundup_admin_rlrc. It
will be loaded when roundup-admin starts. This is the mechanism to
persistently set the number of history commands, change editing modes
(Vi vs. Emacs). Note that redefining the history file will not work.
If you are using GNU readline, set history-size 10. If your
installation uses libedit (macs), it should be possible to
persistently set the history size using history size
10. Pyreadline3 can set history length using
history_length(10). See the documentation for example syntax:
https://pythonhosted.org/pyreadline/usage.html#configuration-file.
History is saved to the file .roundup_admin_history in your home
directory (for windows usually \Users\<username>.
In Roundup 2.6.0 and newer, you can use the readline command to
make changes on the fly.
readline vi- change input mode to use vi key binding when editing. It starts in entry mode.readline emacs- change input mode to emacs key bindings when editing. This is also the default.readline reload- reloads the~/.roundup_admin_rlrcfile so you can test and use changes.readline history- dumps the history buffer and numbers all commands.readline .inputrc_command_linecan be used to make on the fly key and key sequence bindings to readline commands. It can also be used to change the internal readline settings using a set command. For example:readline set bell-style none
will turn off a
visibleoraudiblebell. Single character keybindings:readline Control-o: dump-variables
to list all the variables that can be set are supported. As are multi-character bindings:
readline "\C-o1": "commit"
will put “commit” on the input line when you type Control-o followed by 1. See the readline manual for details on the command lines that can be used.
Also a limited form of ! (bang) history reference was added. The
reference must be at the start of the line. Typing !23 will rerun
command number 23 from your history.
Typing !23:p will load command 23 into the buffer so you can edit
and submit it. Using the bang feature will append the command to the
end of the history list.
Pyreadline3 users can use readline history and the
bang commands (including :p). Single character bindings can be
done. For example:
readline Control-w: history-search-backward
The commands that are available are limited compared to Unix’s readline or libedit. Setting variables or entry mode (emacs, vi) switching do not work in testing.
Using with the shell
With version 0.6.0 or newer of roundup (which introduced support for multiple designators to display and the -d, -S and -s flags):
To find all messages regarding chatting issues that contain the word “spam”, for example, you could execute the following command from the directory where the database dumps its files:
shell% for issue in `roundup-admin -ds find issue status=chatting`; do
> grep -l spam `roundup-admin -ds ' ' get messages $issue`
> done
msg23
msg49
msg50
msg61
shell%
Or, using the -dc option, this can be written as a single command:
shell% grep -l spam `roundup get messages \
\`roundup -dc find issue status=chatting\``
msg23
msg49
msg50
msg61
shell%
You can also display issue contents:
shell% roundup-admin display `roundup-admin -dc get messages \
issue3,issue1`
files: []
inreplyto: None
recipients: []
author: 1
date: 2003-02-16.21:23:03
messageid: None
summary: jkdskldjf
files: []
inreplyto: None
recipients: []
author: 1
date: 2003-02-15.01:59:11
messageid: None
summary: jlkfjadsf
or status:
shell% roundup-admin get name `/tools/roundup/bin/roundup-admin \
-dc -i /var/roundup/sysadmin get status issue3,issue1`
unread
deferred
or status on a single line:
shell% echo `roundup-admin get name \`/tools/roundup/bin/roundup-admin \
-dc -i /var/roundup/sysadmin get status issue3,issue1\``
unread deferred
which is the same as:
shell% roundup-admin -s get name `/tools/roundup/bin/roundup-admin \
-dc -i /var/roundup/sysadmin get status issue3,issue1`
unread deferred
Also the tautological:
shell% roundup-admin get name \
`roundup-admin -dc get status \`roundup-admin -dc find issue \
status=chatting\``
chatting
chatting
Remember the roundup commands that accept multiple designators accept them ‘,’ separated so using ‘-dc’ is almost always required.
A Note on Import and Export
This is a little in the weeds, but I have noticed this and was asked about it so I am documenting it for the future.
Running roundup-admin with -V to get additional info when
importing/exporting the tracker generates three types of messages.
For example:
$ roundup-admin -i tracker -V export ./myExport
Exporting priority - 5
Exporting Journal for priority
Exporting status - 1
Exporting Journal for status
[...]
$ roundup-admin -i tracker -V import ./myExport
Importing priority - 7
setting priority 8
Importing status - 8
setting status 9
[...]
Note the numbers for status. Exported ends up at 1, Imported ends up at 8 and setting chooses 9. These numbers are derived differently and used differently. You can’t directly compare them.
Exporting issue - XXX:
XXXis the id number of the node being exported/processed from the database. The order is determined by sorting by the key of the class (as set by sortkey). If the class key is ‘id’, then it’s a string sort so ‘9’ comes before ‘1009’. You might notice if the export is slow the numbers jumping around.It does not usually end up as the total number of nodes exported. However if it crashes, you know what node it was processing at the time.
In the example above, the status node with id 1 was the last one when sorted alphabetically by name.
Importing <class> - XXX:
XXXis the number of the node (not the node id) being imported/currently processed at line XXX+1 in the file. It is an incrementing number starting at 0 and never jumps around. Value 0 is consumed when reading the header and not displayed. The final value is the same as the number of objects and one less then the number of lines in the file. If it crashes, you were processing the line at XXX+1.
setting <class> XXX:
XXXin the setting line should always be one more than the number of imported objects. The setting value is the id for the next created object of that type. So in theory the Importing number should be one less than the setting number.However under certain circumstances, Roundup can skip an id number. This can lead to a difference of more than 1 between the Importing and setting numbers. It’s not a problem. However setting can (and must) always be higher than the Importing number.
Interactive Help
The help command produces the following output for all the commands.
| commit |
Commit changes made to the database during an interactive session. The changes made during an interactive session are not automatically written to the database - they must be committed using this command. One-off commands on the command-line are automatically committed if they are successful.
|
| create | classname property=value ... Create a new entry of a given class. This creates a new entry of the given class using the property name=value arguments provided on the command line after the "create" command.
|
| display | designator[,designator]* Show the property values for the given node(s). A designator is a classname and a nodeid concatenated, eg. bug1, user10, ... This lists the properties and their associated values for the given node.
|
| export | [[-]class[,class]] export_dir Export the database and file content. Database content is exported to colon separated files. To exclude the files (e.g. for the msg or file class), use the exporttables command. Optionally limit the export to just the named classes or exclude the named classes, if the 1st argument starts with '-'. This action exports the current data from the database into colon-separated-value files that are placed in the nominated export_dir directory.
|
| exporttables | [[-]class[,class]] export_dir Export only the database to files, no file content. Database content is exported to colon separated files. The files below $TRACKER_HOME/db/files/ (which can be archived separately) are not part of the export. To include the files, use the export command. Optionally limit the export to just the named classes or exclude the named classes, if the 1st argument starts with '-'. This action exports the current data from the database into colon-separated-value files that are placed in the export_dir destination directory.
|
| filter | classname propname=value ... Find the nodes of the given class with a given property value. Find the nodes of the given class with a given property value. Multiple values can be specified by separating them with commas. If property is a string, all values must match. I.E. it's an 'and' operation. If the property is a link/multilink any value matches. I.E. an 'or' operation.
|
| find | classname propname=value ... Find the nodes of the given class with a given link property value. Find the nodes of the given class with a given link property value. The value may be either the nodeid of the linked node, or its key value.
|
| genconfig | filename Create a new tracker config file with default values in filename. See also updateconfig.
|
| get | property designator[,designator]* Get the given property of one or more designator(s). A designator is a classname and a nodeid concatenated, eg. bug1, user10, ... Retrieves the property value of the nodes specified by the designators.
|
| help | topic Give help about topic. commands -- list commands <command> -- help specific to a command initopts -- init command options all -- all available help
|
| history | designator [skipquiet] [raw] Show the history entries of a designator. A designator is a classname and a nodeid concatenated, eg. bug1, user10, ... Lists the journal entries viewable by the user for the node identified by the designator. If skipquiet is added, journal entries for quiet properties are not shown. If raw is added, the output is the raw representation of the journal entries.
|
| import | import_dir Import a database and file contents from the directory. The directory should have the same format as one containing the output of export. There are two files imported per class. The files used in the import are: <class>.csv This must define the same properties as the class (including having a "header" line with those property names.) <class>-journals.csv This defines the journals for the items being imported. The imported nodes will have the same nodeid as defined in the import file, thus replacing any existing content. The new nodes are added to the existing database - if you want to create a new database using the imported data, then create a new database (or, tediously, retire all the old data.)
|
| importtables | export_dir This imports the database tables exported using exporttables. It does not import the content of files like msgs and files.
|
| initialise | [adminpw] Initialise a new Roundup tracker. The administrator details will be set at this step. Execute the tracker's initialisation function dbinit.init()
|
| install | [template [backend [key=val[,key=val]]]] Install a new Roundup tracker. The command will prompt for the tracker home directory (if not supplied through TRACKER_HOME or the -i option). The template and backend may be specified on the command-line as arguments, in that order. Command line arguments following the backend allows you to pass initial values for config options. For example, passing "web_http_auth=no,rdbms_user=dinsdale" will override defaults for options http_auth in section [web] and user in section [rdbms]. Please be careful to not use spaces in this argument! (Enclose whole argument in quotes if you need spaces in option value). The initialise command must be called after this command in order to initialise the tracker's database. You may edit the tracker's initial database contents before running that command by editing the tracker's dbinit.py module init() function. See also initopts help.
|
| list | classname [property] List the instances of a class. Lists all instances of the given class. If the property is not specified, the "label" property is used. The label property is tried in order: the key, "name", "title" and then the first property, alphabetically. With -c, -S or -s print a list of item id's if no property specified. If property specified, print list of that property for every class instance.
|
| migrate |
Update a tracker's database to be compatible with the Roundup codebase. You should run the "migrate" command for your tracker once you've installed the latest codebase. Do this before you use the web, command-line or mail interface and before any users access the tracker. This command will respond with either "Tracker updated" (if you've not previously run it on an RDBMS backend) or "No migration action required" (if you have run it, or have used another interface to the tracker, or possibly because you are using anydbm). It's safe to run this even if it's not required, so just get into the habit.
|
| pack | period | date Remove journal entries older than the date/period. A period is specified using the suffixes "y", "m", and "d". The suffix "w" (for "week") means 7 days.
"3y" means three years "2y 1m" means two years and one month "1m 25d" means one month and 25 days "2w 3d" means two weeks and three days Date format is "YYYY-MM-DD" eg: 2001-01-01
|
| perftest | [mode] [arguments]* Time operations in Roundup. Supported arguments:
[password] [rounds=<integer>] [scheme=<scheme>] 'password' is the default mode. The tracker's config.ini setting for 'password_pbkdf2_default_rounds' is the default value for 'rounds'. On the command line, 'rounds' can include thousands separator of ',' or '.'. 'scheme' is the default coded into Roundup. List supported schemes by using 'scheme='.
|
| pragma | setting=value | 'list' Set internal admin settings to a value. For example:
pragma verbose=True pragma verbose=yes pragma verbose=on pragma verbose=1 will turn on verbose mode for roundup-admin.
pragma list will show all settings and their current values. If verbose is enabled hidden settings and descriptions will be shown.
|
| readline | initrc_line | 'emacs' | 'history' | 'reload' | 'vi'
Using 'reload' will reload the file ~/.roundup_admin_rlrc. 'history' will show (and number) all commands in the history.
You can change input mode using the 'emacs' or 'vi' parameters. The default is emacs. This is the same as using::
readline set editing-mode emacs
or::
readline set editing-mode vi
Any command that can be placed in a readline .inputrc file can be executed using the readline command. You can assign dump-variables to control O using::
readline Control-o: dump-variables
Assigning multi-key values also works.
pyreadline3 support on windows:
Mode switching doesn't work, emacs only.
Binding single key commands works with::
readline Control-w: history-search-backward
Multiple key sequences don't work.
Setting values may work. Difficult to tell because the library has no way to view the live settings.
|
| reindex | [classname|classname:#-#|designator]* Re-generate a tracker's search indexes. This will re-generate the search indexes for a tracker. This will typically happen automatically. You can incrementally reindex using an argument like:
reindex issue:23-1000 to reindex issue class items 23-1000. Missing items are reported but do not stop indexing of the range.
|
| restore | designator[,designator]* Restore the retired node specified by designator. A designator is a classname and a nodeid concatenated, eg. bug1, user10, ... The given nodes will become available for users again.
|
| retire | designator[,designator]* Retire the node specified by designator. A designator is a classname and a nodeid concatenated, eg. bug1, user10, ... This action indicates that a particular node is not to be retrieved by the list or find commands, and its key value may be re-used.
|
| rollback |
Undo all changes that are pending commit to the database. The changes made during an interactive session are not automatically written to the database - they must be committed manually. This command undoes all those changes, so a commit immediately after would make no changes to the database.
|
| security | [Role name] Display the Permissions available to one or all Roles. Also validates that any properties defined in a permission are valid. Run this after changing your permissions to catch typos.
|
| set | items property=value [property=value ...] Set the given properties of one or more items(s). The items are specified as a class or as a comma-separated list of item designators (ie "designator[,designator,...]"). A designator is a classname and a nodeid concatenated, eg. bug1, user10, ... This command sets the properties to the values for all designators given. If a class is used, the property will be set for all items in the class. If the value is missing (ie. "property=") then the property is un-set. If the property is a multilink, you specify the linked ids for the multilink as comma-separated numbers (ie "1,2,3").
|
| specification | classname Show the properties for a classname. This lists the properties for a given class.
|
| table | classname [property[,property]*] List the instances of a class in tabular form. Lists all instances of the given class. If the properties are not specified, all properties are displayed. By default, the column widths are the width of the largest value. The width may be explicitly defined by defining the property as "name:width". For example::
roundup> table priority id,name:10 Id Name 1 fatal-bug 2 bug 3 usability 4 feature Also to make the width of the column the width of the label, leave a trailing : without a width on the property. For example::
roundup> table priority id,name: Id Name 1 fata 2 bug 3 usab 4 feat will result in a the 4 character wide "Name" column.
|
| templates | [trace_search] List templates and their installed directories. With trace_search also list all directories that are searched for templates.
|
| updateconfig | <filename> Merge existing tracker config with new settings. Output the updated config file to <filename>. Use current settings from existing roundup tracker in tracker home.
|