Mercurial > kallithea

annotate rhodecode/model/caching_query.py @ 1472:aaec08ad9daf beta

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Added pt_BR localization, added i18n wrappers on some places missing, fixed css in settings screen for longer labels.
author Augusto Herrmann <augusto.herrmann@planejamento.gov.br>
date Mon, 12 Sep 2011 00:51:53 -0300
parents cd865113423e
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
1 """caching_query.py
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
2
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
3 Represent persistence structures which allow the usage of
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
4 Beaker caching with SQLAlchemy.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
5
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
6 The three new concepts introduced here are:
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
7
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
8 * CachingQuery - a Query subclass that caches and
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
9 retrieves results in/from Beaker.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
10 * FromCache - a query option that establishes caching
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
11 parameters on a Query
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
12 * RelationshipCache - a variant of FromCache which is specific
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
13 to a query invoked during a lazy load.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
14 * _params_from_query - extracts value parameters from
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
15 a Query.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
16
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
17 The rest of what's here are standard SQLAlchemy and
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
18 Beaker constructs.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
19
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
20 """
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
21 import beaker
609
c1c1cf772337 moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents: 547
diff changeset
22 from beaker.exceptions import BeakerException
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
23
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
24 from sqlalchemy.orm.interfaces import MapperOption
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
25 from sqlalchemy.orm.query import Query
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
26 from sqlalchemy.sql import visitors
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
27
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
28
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
29 class CachingQuery(Query):
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
30 """A Query subclass which optionally loads full results from a Beaker
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
31 cache region.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
32
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
33 The CachingQuery stores additional state that allows it to consult
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
34 a Beaker cache before accessing the database:
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
35
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
36 * A "region", which is a cache region argument passed to a
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
37 Beaker CacheManager, specifies a particular cache configuration
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
38 (including backend implementation, expiration times, etc.)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
39 * A "namespace", which is a qualifying name that identifies a
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
40 group of keys within the cache. A query that filters on a name
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
41 might use the name "by_name", a query that filters on a date range
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
42 to a joined table might use the name "related_date_range".
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
43
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
44 When the above state is present, a Beaker cache is retrieved.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
45
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
46 The "namespace" name is first concatenated with
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
47 a string composed of the individual entities and columns the Query
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
48 requests, i.e. such as ``Query(User.id, User.name)``.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
49
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
50 The Beaker cache is then loaded from the cache manager based
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
51 on the region and composed namespace. The key within the cache
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
52 itself is then constructed against the bind parameters specified
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
53 by this query, which are usually literals defined in the
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
54 WHERE clause.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
55
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
56 The FromCache and RelationshipCache mapper options below represent
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
57 the "public" method of configuring this state upon the CachingQuery.
1046
9e93cad9357b fixed caching query to propagate data_dir default from beaker
Marcin Kuzminski <marcin@python-works.com>
parents: 784
diff changeset
58
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
59 """
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
60
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
61 def __init__(self, manager, *args, **kw):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
62 self.cache_manager = manager
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
63 Query.__init__(self, *args, **kw)
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
64
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
65 def __iter__(self):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
66 """override __iter__ to pull results from Beaker
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
67 if particular attributes have been configured.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
68
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
69 Note that this approach does *not* detach the loaded objects from
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
70 the current session. If the cache backend is an in-process cache
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
71 (like "memory") and lives beyond the scope of the current session's
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
72 transaction, those objects may be expired. The method here can be
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
73 modified to first expunge() each loaded item from the current
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
74 session before returning the list of items, so that the items
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
75 in the cache are not the same ones in the current Session.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
76
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
77 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
78 if hasattr(self, '_cache_parameters'):
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
79 return self.get_value(createfunc=lambda:
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
80 list(Query.__iter__(self)))
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
81 else:
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
82 return Query.__iter__(self)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
83
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
84 def invalidate(self):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
85 """Invalidate the value represented by this Query."""
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
86
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
87 cache, cache_key = _get_cache_parameters(self)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
88 cache.remove(cache_key)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
89
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
90 def get_value(self, merge=True, createfunc=None):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
91 """Return the value from the cache for this query.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
92
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
93 Raise KeyError if no value present and no
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
94 createfunc specified.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
95
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
96 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
97 cache, cache_key = _get_cache_parameters(self)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
98 ret = cache.get_value(cache_key, createfunc=createfunc)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
99 if merge:
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
100 ret = self.merge_result(ret, load=False)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
101 return ret
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
102
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
103 def set_value(self, value):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
104 """Set the value in the cache for this query."""
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
105
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
106 cache, cache_key = _get_cache_parameters(self)
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
107 cache.put(cache_key, value)
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
108
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
109
1339
e861eb1c4a2f update caching_query from latest sqlalchemy
Marcin Kuzminski <marcin@python-works.com>
parents: 1271
diff changeset
110 def query_callable(manager, query_cls=CachingQuery):
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
111 def query(*arg, **kw):
1339
e861eb1c4a2f update caching_query from latest sqlalchemy
Marcin Kuzminski <marcin@python-works.com>
parents: 1271
diff changeset
112 return query_cls(manager, *arg, **kw)
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
113 return query
609
c1c1cf772337 moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents: 547
diff changeset
114
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
115
609
c1c1cf772337 moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents: 547
diff changeset
116 def get_cache_region(name, region):
610
b0a411f5ec70 fixed error message about cache settings
Marcin Kuzminski <marcin@python-works.com>
parents: 609
diff changeset
117 if region not in beaker.cache.cache_regions:
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
118 raise BeakerException('Cache region `%s` not configured '
610
b0a411f5ec70 fixed error message about cache settings
Marcin Kuzminski <marcin@python-works.com>
parents: 609
diff changeset
119 'Check if proper cache settings are in the .ini files' % region)
b0a411f5ec70 fixed error message about cache settings
Marcin Kuzminski <marcin@python-works.com>
parents: 609
diff changeset
120 kw = beaker.cache.cache_regions[region]
b0a411f5ec70 fixed error message about cache settings
Marcin Kuzminski <marcin@python-works.com>
parents: 609
diff changeset
121 return beaker.cache.Cache._get_cache(name, kw)
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
122
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
123
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
124 def _get_cache_parameters(query):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
125 """For a query with cache_region and cache_namespace configured,
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
126 return the correspoinding Cache instance and cache key, based
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
127 on this query's current criterion and parameter values.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
128
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
129 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
130 if not hasattr(query, '_cache_parameters'):
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
131 raise ValueError("This Query does not have caching "
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
132 "parameters configured.")
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
133
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
134 region, namespace, cache_key = query._cache_parameters
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
135
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
136 namespace = _namespace_from_query(namespace, query)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
137
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
138 if cache_key is None:
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
139 # cache key - the value arguments from this query's parameters.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
140 args = _params_from_query(query)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
141 cache_key = " ".join([str(x) for x in args])
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
142
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
143 # get cache
609
c1c1cf772337 moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents: 547
diff changeset
144 #cache = query.cache_manager.get_cache_region(namespace, region)
c1c1cf772337 moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents: 547
diff changeset
145 cache = get_cache_region(namespace, region)
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
146 # optional - hash the cache_key too for consistent length
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
147 # import uuid
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
148 # cache_key= str(uuid.uuid5(uuid.NAMESPACE_DNS, cache_key))
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
149
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
150 return cache, cache_key
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
151
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
152
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
153 def _namespace_from_query(namespace, query):
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
154 # cache namespace - the token handed in by the
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
155 # option + class we're querying against
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
156 namespace = " ".join([namespace] + [str(x) for x in query._entities])
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
157
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
158 # memcached wants this
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
159 namespace = namespace.replace(' ', '_')
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
160
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
161 return namespace
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
162
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
163
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
164 def _set_cache_parameters(query, region, namespace, cache_key):
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
165
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
166 if hasattr(query, '_cache_parameters'):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
167 region, namespace, cache_key = query._cache_parameters
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
168 raise ValueError("This query is already configured "
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
169 "for region %r namespace %r" %
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
170 (region, namespace)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
171 )
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
172 query._cache_parameters = region, namespace, cache_key
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
173
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
174
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
175 class FromCache(MapperOption):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
176 """Specifies that a Query should load results from a cache."""
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
177
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
178 propagate_to_loaders = False
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
179
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
180 def __init__(self, region, namespace, cache_key=None):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
181 """Construct a new FromCache.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
182
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
183 :param region: the cache region. Should be a
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
184 region configured in the Beaker CacheManager.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
185
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
186 :param namespace: the cache namespace. Should
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
187 be a name uniquely describing the target Query's
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
188 lexical structure.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
189
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
190 :param cache_key: optional. A string cache key
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
191 that will serve as the key to the query. Use this
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
192 if your query has a huge amount of parameters (such
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
193 as when using in_()) which correspond more simply to
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
194 some other identifier.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
195
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
196 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
197 self.region = region
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
198 self.namespace = namespace
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
199 self.cache_key = cache_key
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
200
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
201 def process_query(self, query):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
202 """Process a Query during normal loading operation."""
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
203
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
204 _set_cache_parameters(query, self.region, self.namespace,
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
205 self.cache_key)
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
206
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
207
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
208 class RelationshipCache(MapperOption):
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
209 """Specifies that a Query as called within a "lazy load"
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
210 should load results from a cache."""
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
211
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
212 propagate_to_loaders = True
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
213
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
214 def __init__(self, region, namespace, attribute):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
215 """Construct a new RelationshipCache.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
216
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
217 :param region: the cache region. Should be a
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
218 region configured in the Beaker CacheManager.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
219
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
220 :param namespace: the cache namespace. Should
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
221 be a name uniquely describing the target Query's
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
222 lexical structure.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
223
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
224 :param attribute: A Class.attribute which
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
225 indicates a particular class relationship() whose
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
226 lazy loader should be pulled from the cache.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
227
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
228 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
229 self.region = region
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
230 self.namespace = namespace
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
231 self._relationship_options = {
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
232 (attribute.property.parent.class_, attribute.property.key): self
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
233 }
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
234
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
235 def process_query_conditionally(self, query):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
236 """Process a Query that is used within a lazy loader.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
237
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
238 (the process_query_conditionally() method is a SQLAlchemy
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
239 hook invoked only within lazyload.)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
240
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
241 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
242 if query._current_path:
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
243 mapper, key = query._current_path[-2:]
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
244
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
245 for cls in mapper.class_.__mro__:
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
246 if (cls, key) in self._relationship_options:
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
247 relationship_option = \
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
248 self._relationship_options[(cls, key)]
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
249 _set_cache_parameters(
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
250 query,
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
251 relationship_option.region,
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
252 relationship_option.namespace,
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
253 None)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
254
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
255 def and_(self, option):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
256 """Chain another RelationshipCache option to this one.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
257
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
258 While many RelationshipCache objects can be specified on a single
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
259 Query separately, chaining them together allows for a more efficient
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
260 lookup during load.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
261
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
262 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
263 self._relationship_options.update(option._relationship_options)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
264 return self
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
265
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
266
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
267 def _params_from_query(query):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
268 """Pull the bind parameter values from a query.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
269
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
270 This takes into account any scalar attribute bindparam set up.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
271
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
272 E.g. params_from_query(query.filter(Cls.foo==5).filter(Cls.bar==7)))
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
273 would return [5, 7].
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
274
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
275 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
276 v = []
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
277 def visit_bindparam(bind):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
278 value = query._params.get(bind.key, bind.value)
1365
cd865113423e applied caching query update from latest sqlalchemy
Marcin Kuzminski <marcin@python-works.com>
parents: 1339
diff changeset
279
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
280 # lazyloader may dig a callable in here, intended
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
281 # to late-evaluate params after autoflush is called.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
282 # convert to a scalar value.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
283 if callable(value):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
284 value = value()
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
285
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
286 v.append(value)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
287 if query._criterion is not None:
1339
e861eb1c4a2f update caching_query from latest sqlalchemy
Marcin Kuzminski <marcin@python-works.com>
parents: 1271
diff changeset
288 visitors.traverse(query._criterion, {}, {'bindparam':visit_bindparam})
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
289 return v