Coverage for cc_modules/cc_cache.py : 100%

Hot-keys on this page
r m x p toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
1#!/usr/bin/env python
3# noinspection HttpUrlsUsage
4"""
5camcops_server/cc_modules/cc_cache.py
7===============================================================================
9 Copyright (C) 2012-2020 Rudolf Cardinal (rudolf@pobox.com).
11 This file is part of CamCOPS.
13 CamCOPS is free software: you can redistribute it and/or modify
14 it under the terms of the GNU General Public License as published by
15 the Free Software Foundation, either version 3 of the License, or
16 (at your option) any later version.
18 CamCOPS is distributed in the hope that it will be useful,
19 but WITHOUT ANY WARRANTY; without even the implied warranty of
20 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
21 GNU General Public License for more details.
23 You should have received a copy of the GNU General Public License
24 along with CamCOPS. If not, see <https://www.gnu.org/licenses/>.
26===============================================================================
28**Cache functions.**
301. The basic cache objects.
322. FIX FOR DOGPILE.CACHE FOR DECORATED FUNCTIONS, 2017-07-28 (PLUS SOME OTHER
33 IMPROVEMENTS). SEE
34 https://bitbucket.org/zzzeek/dogpile.cache/issues/96/error-in-python-35-with-use-of-deprecated
36Crash using type-hinted functions under Python 3.5 with dogpile.cache==0.6.4:
38.. code-block:: none
40 Traceback (most recent call last):
41 File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
42 "__main__", mod_spec)
43 File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
44 exec(code, run_globals)
45 File "/home/rudolf/Documents/code/camcops/server/camcops_server/cc_modules/cc_cache.py", line 64, in <module>
46 unit_test_cache()
47 File "/home/rudolf/Documents/code/camcops/server/camcops_server/cc_modules/cc_cache.py", line 50, in unit_test_cache
48 def testfunc() -> str:
49 File "/home/rudolf/dev/venvs/camcops/lib/python3.5/site-packages/dogpile/cache/region.py", line 1215, in decorator
50 key_generator = function_key_generator(namespace, fn)
51 File "/home/rudolf/dev/venvs/camcops/lib/python3.5/site-packages/dogpile/cache/util.py", line 31, in function_key_generator
52 args = inspect.getargspec(fn)
53 File "/usr/lib/python3.5/inspect.py", line 1045, in getargspec
54 raise ValueError("Function has keyword-only arguments or annotations"
55 ValueError: Function has keyword-only arguments or annotations, use getfullargspec() API which can support them
573. CACHING NOTES
59- We currently use 'dogpile.cache.memory' as the backend.
60 This means that for single-process (single-thread or multithreaded) servers,
61 the cache is unique, but that won't work for multi-process (e.g. Gunicorn)
62 servers.
64- That means that in a multiprocess environment it's fine to continue to use a
65 memory cache for file-based stuff (e.g. configs, XML strings), but not for
66 database-based stuff (e.g. which ID numbers are valid).
68- Correct solutions WITH a cache for those database-based things include:
70 - ignoring Python caching and relying on the MySQL query cache -- but this is
71 being removed because it's not all that great:
73 https://mysqlserverteam.com/mysql-8-0-retiring-support-for-the-query-cache/
75 - using memcached (via dogpile.cache.pylibmc)
77 https://www.ubergizmo.com/how-to/install-memcached-windows/
79 - using redis (via dogpile.cache.redis and
80 https://pypi.python.org/pypi/redis/ )
82 https://stackoverflow.com/questions/10558465/memcached-vs-redis
83 https://redis.io/
84 https://web.archive.org/web/20120118030804/http://simonwillison.net/static/2010/redis-tutorial/
85 http://oldblog.antirez.com/post/take-advantage-of-redis-adding-it-to-your-stack.html
86 https://redis.io/topics/security
88 redis unsupported under Windows:
89 https://redis.io/download
91- The other obvious alternative: don't cache such stuff! This may all be
92 premature optimization.
94 https://msol.io/blog/tech/youre-probably-wrong-about-caching/
96 The actual price is of the order of 0.6-1 ms per query, for the queries
97 "find me all the ID number definitions" and "fetch the server settings".
99- The answer is probably:
101 - continue to use dogpile.cache.memory for simple "fixed" stuff read from
102 disk;
103 - continue to use Pyramid's per-request caching mechanism (@reify);
104 - forget about database caching for now;
105 - if it becomes a problem later, move to Redis
107- Therefore:
109 - there should be no calls to cache_region_static.delete
111""" # noqa
114# =============================================================================
115# Imports; logging
116# =============================================================================
118from cardinal_pythonlib.dogpile_cache import kw_fkg_allowing_type_hints as fkg
119from dogpile.cache import make_region
121# =============================================================================
122# The main cache: static for the lifetime of this process.
123# =============================================================================
125cache_region_static = make_region()
126cache_region_static.configure(
127 backend='dogpile.cache.memory'
128 # Consider later: memcached via dogpile.cache.pylibmc
129)
131# Can now use:
132# @cache_region_static.cache_on_arguments(function_key_generator=fkg)
134# https://stackoverflow.com/questions/44834/can-someone-explain-all-in-python
135__all__ = ['cache_region_static', 'fkg'] # prevents "Unused import statement"