Mercurial > kallithea

annotate rhodecode/model/caching_query.py @ 1345:3bce31f026b8 beta

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
#47 implemented Adding of new repo_groups+forms+validators. Fixed sorting of repo groups by main names in multiple locations. Removed some unneeded calls to self.sa for exchange to .query() methods. Added new db unique key for Group
author Marcin Kuzminski <marcin@python-works.com>
date Mon, 23 May 2011 02:22:00 +0200
parents e861eb1c4a2f
children cd865113423e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
1 """caching_query.py
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
2
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
3 Represent persistence structures which allow the usage of
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
4 Beaker caching with SQLAlchemy.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
5
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
6 The three new concepts introduced here are:
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
7
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
8 * CachingQuery - a Query subclass that caches and
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
9 retrieves results in/from Beaker.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
10 * FromCache - a query option that establishes caching
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
11 parameters on a Query
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
12 * RelationshipCache - a variant of FromCache which is specific
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
13 to a query invoked during a lazy load.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
14 * _params_from_query - extracts value parameters from
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
15 a Query.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
16
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
17 The rest of what's here are standard SQLAlchemy and
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
18 Beaker constructs.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
19
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
20 """
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
21 import beaker
609
c1c1cf772337 moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents: 547
diff changeset
22 from beaker.exceptions import BeakerException
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
23
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
24 from sqlalchemy.orm.interfaces import MapperOption
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
25 from sqlalchemy.orm.query import Query
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
26 from sqlalchemy.sql import visitors
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
27
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
28
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
29 class CachingQuery(Query):
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
30 """A Query subclass which optionally loads full results from a Beaker
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
31 cache region.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
32
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
33 The CachingQuery stores additional state that allows it to consult
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
34 a Beaker cache before accessing the database:
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
35
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
36 * A "region", which is a cache region argument passed to a
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
37 Beaker CacheManager, specifies a particular cache configuration
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
38 (including backend implementation, expiration times, etc.)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
39 * A "namespace", which is a qualifying name that identifies a
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
40 group of keys within the cache. A query that filters on a name
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
41 might use the name "by_name", a query that filters on a date range
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
42 to a joined table might use the name "related_date_range".
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
43
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
44 When the above state is present, a Beaker cache is retrieved.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
45
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
46 The "namespace" name is first concatenated with
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
47 a string composed of the individual entities and columns the Query
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
48 requests, i.e. such as ``Query(User.id, User.name)``.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
49
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
50 The Beaker cache is then loaded from the cache manager based
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
51 on the region and composed namespace. The key within the cache
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
52 itself is then constructed against the bind parameters specified
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
53 by this query, which are usually literals defined in the
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
54 WHERE clause.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
55
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
56 The FromCache and RelationshipCache mapper options below represent
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
57 the "public" method of configuring this state upon the CachingQuery.
1046
9e93cad9357b fixed caching query to propagate data_dir default from beaker
Marcin Kuzminski <marcin@python-works.com>
parents: 784
diff changeset
58
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
59 """
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
60
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
61 def __init__(self, manager, *args, **kw):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
62 self.cache_manager = manager
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
63 Query.__init__(self, *args, **kw)
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
64
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
65 def __iter__(self):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
66 """override __iter__ to pull results from Beaker
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
67 if particular attributes have been configured.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
68
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
69 Note that this approach does *not* detach the loaded objects from
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
70 the current session. If the cache backend is an in-process cache
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
71 (like "memory") and lives beyond the scope of the current session's
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
72 transaction, those objects may be expired. The method here can be
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
73 modified to first expunge() each loaded item from the current
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
74 session before returning the list of items, so that the items
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
75 in the cache are not the same ones in the current Session.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
76
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
77 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
78 if hasattr(self, '_cache_parameters'):
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
79 return self.get_value(createfunc=lambda:
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
80 list(Query.__iter__(self)))
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
81 else:
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
82 return Query.__iter__(self)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
83
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
84 def invalidate(self):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
85 """Invalidate the value represented by this Query."""
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
86
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
87 cache, cache_key = _get_cache_parameters(self)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
88 cache.remove(cache_key)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
89
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
90 def get_value(self, merge=True, createfunc=None):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
91 """Return the value from the cache for this query.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
92
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
93 Raise KeyError if no value present and no
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
94 createfunc specified.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
95
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
96 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
97 cache, cache_key = _get_cache_parameters(self)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
98 ret = cache.get_value(cache_key, createfunc=createfunc)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
99 if merge:
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
100 ret = self.merge_result(ret, load=False)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
101 return ret
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
102
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
103 def set_value(self, value):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
104 """Set the value in the cache for this query."""
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
105
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
106 cache, cache_key = _get_cache_parameters(self)
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
107 cache.put(cache_key, value)
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
108
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
109
1339
e861eb1c4a2f update caching_query from latest sqlalchemy
Marcin Kuzminski <marcin@python-works.com>
parents: 1271
diff changeset
110 def query_callable(manager, query_cls=CachingQuery):
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
111 def query(*arg, **kw):
1339
e861eb1c4a2f update caching_query from latest sqlalchemy
Marcin Kuzminski <marcin@python-works.com>
parents: 1271
diff changeset
112 return query_cls(manager, *arg, **kw)
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
113 return query
609
c1c1cf772337 moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents: 547
diff changeset
114
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
115
609
c1c1cf772337 moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents: 547
diff changeset
116 def get_cache_region(name, region):
610
b0a411f5ec70 fixed error message about cache settings
Marcin Kuzminski <marcin@python-works.com>
parents: 609
diff changeset
117 if region not in beaker.cache.cache_regions:
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
118 raise BeakerException('Cache region `%s` not configured '
610
b0a411f5ec70 fixed error message about cache settings
Marcin Kuzminski <marcin@python-works.com>
parents: 609
diff changeset
119 'Check if proper cache settings are in the .ini files' % region)
b0a411f5ec70 fixed error message about cache settings
Marcin Kuzminski <marcin@python-works.com>
parents: 609
diff changeset
120 kw = beaker.cache.cache_regions[region]
b0a411f5ec70 fixed error message about cache settings
Marcin Kuzminski <marcin@python-works.com>
parents: 609
diff changeset
121 return beaker.cache.Cache._get_cache(name, kw)
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
122
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
123
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
124 def _get_cache_parameters(query):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
125 """For a query with cache_region and cache_namespace configured,
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
126 return the correspoinding Cache instance and cache key, based
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
127 on this query's current criterion and parameter values.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
128
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
129 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
130 if not hasattr(query, '_cache_parameters'):
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
131 raise ValueError("This Query does not have caching "
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
132 "parameters configured.")
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
133
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
134 region, namespace, cache_key = query._cache_parameters
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
135
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
136 namespace = _namespace_from_query(namespace, query)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
137
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
138 if cache_key is None:
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
139 # cache key - the value arguments from this query's parameters.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
140 args = _params_from_query(query)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
141 cache_key = " ".join([str(x) for x in args])
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
142
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
143 # get cache
609
c1c1cf772337 moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents: 547
diff changeset
144 #cache = query.cache_manager.get_cache_region(namespace, region)
c1c1cf772337 moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents: 547
diff changeset
145 cache = get_cache_region(namespace, region)
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
146 # optional - hash the cache_key too for consistent length
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
147 # import uuid
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
148 # cache_key= str(uuid.uuid5(uuid.NAMESPACE_DNS, cache_key))
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
149
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
150 return cache, cache_key
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
151
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
152
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
153 def _namespace_from_query(namespace, query):
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
154 # cache namespace - the token handed in by the
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
155 # option + class we're querying against
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
156 namespace = " ".join([namespace] + [str(x) for x in query._entities])
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
157
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
158 # memcached wants this
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
159 namespace = namespace.replace(' ', '_')
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
160
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
161 return namespace
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
162
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
163
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
164 def _set_cache_parameters(query, region, namespace, cache_key):
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
165
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
166 if hasattr(query, '_cache_parameters'):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
167 region, namespace, cache_key = query._cache_parameters
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
168 raise ValueError("This query is already configured "
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
169 "for region %r namespace %r" %
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
170 (region, namespace)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
171 )
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
172 query._cache_parameters = region, namespace, cache_key
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
173
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
174
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
175 class FromCache(MapperOption):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
176 """Specifies that a Query should load results from a cache."""
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
177
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
178 propagate_to_loaders = False
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
179
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
180 def __init__(self, region, namespace, cache_key=None):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
181 """Construct a new FromCache.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
182
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
183 :param region: the cache region. Should be a
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
184 region configured in the Beaker CacheManager.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
185
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
186 :param namespace: the cache namespace. Should
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
187 be a name uniquely describing the target Query's
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
188 lexical structure.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
189
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
190 :param cache_key: optional. A string cache key
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
191 that will serve as the key to the query. Use this
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
192 if your query has a huge amount of parameters (such
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
193 as when using in_()) which correspond more simply to
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
194 some other identifier.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
195
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
196 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
197 self.region = region
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
198 self.namespace = namespace
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
199 self.cache_key = cache_key
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
200
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
201 def process_query(self, query):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
202 """Process a Query during normal loading operation."""
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
203
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
204 _set_cache_parameters(query, self.region, self.namespace,
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
205 self.cache_key)
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
206
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
207
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
208 class RelationshipCache(MapperOption):
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
209 """Specifies that a Query as called within a "lazy load"
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
210 should load results from a cache."""
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
211
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
212 propagate_to_loaders = True
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
213
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
214 def __init__(self, region, namespace, attribute):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
215 """Construct a new RelationshipCache.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
216
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
217 :param region: the cache region. Should be a
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
218 region configured in the Beaker CacheManager.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
219
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
220 :param namespace: the cache namespace. Should
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
221 be a name uniquely describing the target Query's
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
222 lexical structure.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
223
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
224 :param attribute: A Class.attribute which
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
225 indicates a particular class relationship() whose
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
226 lazy loader should be pulled from the cache.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
227
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
228 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
229 self.region = region
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
230 self.namespace = namespace
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
231 self._relationship_options = {
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
232 (attribute.property.parent.class_, attribute.property.key): self
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
233 }
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
234
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
235 def process_query_conditionally(self, query):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
236 """Process a Query that is used within a lazy loader.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
237
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
238 (the process_query_conditionally() method is a SQLAlchemy
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
239 hook invoked only within lazyload.)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
240
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
241 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
242 if query._current_path:
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
243 mapper, key = query._current_path[-2:]
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
244
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
245 for cls in mapper.class_.__mro__:
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
246 if (cls, key) in self._relationship_options:
1271
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
247 relationship_option = \
aa7e45ad0cea Fixed permissions for users groups, group can have create repo permission now.
Marcin Kuzminski <marcin@python-works.com>
parents: 1203
diff changeset
248 self._relationship_options[(cls, key)]
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
249 _set_cache_parameters(
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
250 query,
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
251 relationship_option.region,
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
252 relationship_option.namespace,
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
253 None)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
254
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
255 def and_(self, option):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
256 """Chain another RelationshipCache option to this one.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
257
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
258 While many RelationshipCache objects can be specified on a single
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
259 Query separately, chaining them together allows for a more efficient
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
260 lookup during load.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
261
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
262 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
263 self._relationship_options.update(option._relationship_options)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
264 return self
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
265
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
266
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
267 def _params_from_query(query):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
268 """Pull the bind parameter values from a query.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
269
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
270 This takes into account any scalar attribute bindparam set up.
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
271
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
272 E.g. params_from_query(query.filter(Cls.foo==5).filter(Cls.bar==7)))
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
273 would return [5, 7].
1203
6832ef664673 source code cleanup: remove trailing white space, normalize file endings
Marcin Kuzminski <marcin@python-works.com>
parents: 1046
diff changeset
274
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
275 """
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
276 v = []
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
277 def visit_bindparam(bind):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
278 value = query._params.get(bind.key, bind.value)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
279 # lazyloader may dig a callable in here, intended
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
280 # to late-evaluate params after autoflush is called.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
281 # convert to a scalar value.
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
282 if callable(value):
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
283 value = value()
784
30d3161c6683 Implemented fancier top menu for logged and anonymous users
Marcin Kuzminski <marcin@python-works.com>
parents: 610
diff changeset
284
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
285 v.append(value)
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
286 if query._criterion is not None:
1339
e861eb1c4a2f update caching_query from latest sqlalchemy
Marcin Kuzminski <marcin@python-works.com>
parents: 1271
diff changeset
287 visitors.traverse(query._criterion, {}, {'bindparam':visit_bindparam})
e861eb1c4a2f update caching_query from latest sqlalchemy
Marcin Kuzminski <marcin@python-works.com>
parents: 1271
diff changeset
288 for f in query._from_obj:
e861eb1c4a2f update caching_query from latest sqlalchemy
Marcin Kuzminski <marcin@python-works.com>
parents: 1271
diff changeset
289 visitors.traverse(f, {}, {'bindparam':visit_bindparam})
482
7afbc45aab28 added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff changeset
290 return v