Mercurial > kallithea
annotate rhodecode/model/caching_query.py @ 723:bf26b46e82d6 beta
some tests update
author | Marcin Kuzminski <marcin@python-works.com> |
---|---|
date | Thu, 18 Nov 2010 23:11:51 +0100 |
parents | b0a411f5ec70 |
children | 30d3161c6683 |
rev | line source |
---|---|
482
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
1 """caching_query.py |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
2 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
3 Represent persistence structures which allow the usage of |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
4 Beaker caching with SQLAlchemy. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
5 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
6 The three new concepts introduced here are: |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
7 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
8 * CachingQuery - a Query subclass that caches and |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
9 retrieves results in/from Beaker. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
10 * FromCache - a query option that establishes caching |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
11 parameters on a Query |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
12 * RelationshipCache - a variant of FromCache which is specific |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
13 to a query invoked during a lazy load. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
14 * _params_from_query - extracts value parameters from |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
15 a Query. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
16 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
17 The rest of what's here are standard SQLAlchemy and |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
18 Beaker constructs. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
19 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
20 """ |
609
c1c1cf772337
moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
21 from beaker.exceptions import BeakerException |
482
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
22 from sqlalchemy.orm.interfaces import MapperOption |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
23 from sqlalchemy.orm.query import Query |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
24 from sqlalchemy.sql import visitors |
609
c1c1cf772337
moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
25 import beaker |
482
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
26 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
27 class CachingQuery(Query): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
28 """A Query subclass which optionally loads full results from a Beaker |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
29 cache region. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
30 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
31 The CachingQuery stores additional state that allows it to consult |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
32 a Beaker cache before accessing the database: |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
33 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
34 * A "region", which is a cache region argument passed to a |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
35 Beaker CacheManager, specifies a particular cache configuration |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
36 (including backend implementation, expiration times, etc.) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
37 * A "namespace", which is a qualifying name that identifies a |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
38 group of keys within the cache. A query that filters on a name |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
39 might use the name "by_name", a query that filters on a date range |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
40 to a joined table might use the name "related_date_range". |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
41 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
42 When the above state is present, a Beaker cache is retrieved. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
43 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
44 The "namespace" name is first concatenated with |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
45 a string composed of the individual entities and columns the Query |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
46 requests, i.e. such as ``Query(User.id, User.name)``. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
47 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
48 The Beaker cache is then loaded from the cache manager based |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
49 on the region and composed namespace. The key within the cache |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
50 itself is then constructed against the bind parameters specified |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
51 by this query, which are usually literals defined in the |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
52 WHERE clause. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
53 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
54 The FromCache and RelationshipCache mapper options below represent |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
55 the "public" method of configuring this state upon the CachingQuery. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
56 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
57 """ |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
58 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
59 def __init__(self, manager, *args, **kw): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
60 self.cache_manager = manager |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
61 Query.__init__(self, *args, **kw) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
62 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
63 def __iter__(self): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
64 """override __iter__ to pull results from Beaker |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
65 if particular attributes have been configured. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
66 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
67 Note that this approach does *not* detach the loaded objects from |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
68 the current session. If the cache backend is an in-process cache |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
69 (like "memory") and lives beyond the scope of the current session's |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
70 transaction, those objects may be expired. The method here can be |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
71 modified to first expunge() each loaded item from the current |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
72 session before returning the list of items, so that the items |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
73 in the cache are not the same ones in the current Session. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
74 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
75 """ |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
76 if hasattr(self, '_cache_parameters'): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
77 return self.get_value(createfunc=lambda: list(Query.__iter__(self))) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
78 else: |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
79 return Query.__iter__(self) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
80 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
81 def invalidate(self): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
82 """Invalidate the value represented by this Query.""" |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
83 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
84 cache, cache_key = _get_cache_parameters(self) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
85 cache.remove(cache_key) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
86 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
87 def get_value(self, merge=True, createfunc=None): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
88 """Return the value from the cache for this query. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
89 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
90 Raise KeyError if no value present and no |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
91 createfunc specified. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
92 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
93 """ |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
94 cache, cache_key = _get_cache_parameters(self) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
95 ret = cache.get_value(cache_key, createfunc=createfunc) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
96 if merge: |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
97 ret = self.merge_result(ret, load=False) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
98 return ret |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
99 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
100 def set_value(self, value): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
101 """Set the value in the cache for this query.""" |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
102 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
103 cache, cache_key = _get_cache_parameters(self) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
104 cache.put(cache_key, value) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
105 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
106 def query_callable(manager): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
107 def query(*arg, **kw): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
108 return CachingQuery(manager, *arg, **kw) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
109 return query |
609
c1c1cf772337
moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
110 |
c1c1cf772337
moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
111 def get_cache_region(name, region): |
610
b0a411f5ec70
fixed error message about cache settings
Marcin Kuzminski <marcin@python-works.com>
parents:
609
diff
changeset
|
112 if region not in beaker.cache.cache_regions: |
b0a411f5ec70
fixed error message about cache settings
Marcin Kuzminski <marcin@python-works.com>
parents:
609
diff
changeset
|
113 raise BeakerException('Cache region not configured: %s' |
b0a411f5ec70
fixed error message about cache settings
Marcin Kuzminski <marcin@python-works.com>
parents:
609
diff
changeset
|
114 'Check if proper cache settings are in the .ini files' % region) |
b0a411f5ec70
fixed error message about cache settings
Marcin Kuzminski <marcin@python-works.com>
parents:
609
diff
changeset
|
115 kw = beaker.cache.cache_regions[region] |
b0a411f5ec70
fixed error message about cache settings
Marcin Kuzminski <marcin@python-works.com>
parents:
609
diff
changeset
|
116 return beaker.cache.Cache._get_cache(name, kw) |
b0a411f5ec70
fixed error message about cache settings
Marcin Kuzminski <marcin@python-works.com>
parents:
609
diff
changeset
|
117 |
482
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
118 def _get_cache_parameters(query): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
119 """For a query with cache_region and cache_namespace configured, |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
120 return the correspoinding Cache instance and cache key, based |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
121 on this query's current criterion and parameter values. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
122 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
123 """ |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
124 if not hasattr(query, '_cache_parameters'): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
125 raise ValueError("This Query does not have caching parameters configured.") |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
126 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
127 region, namespace, cache_key = query._cache_parameters |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
128 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
129 namespace = _namespace_from_query(namespace, query) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
130 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
131 if cache_key is None: |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
132 # cache key - the value arguments from this query's parameters. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
133 args = _params_from_query(query) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
134 cache_key = " ".join([str(x) for x in args]) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
135 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
136 # get cache |
609
c1c1cf772337
moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
137 #cache = query.cache_manager.get_cache_region(namespace, region) |
c1c1cf772337
moved out sqlalchemy cache from meta to the config files.
Marcin Kuzminski <marcin@python-works.com>
parents:
547
diff
changeset
|
138 cache = get_cache_region(namespace, region) |
482
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
139 # optional - hash the cache_key too for consistent length |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
140 # import uuid |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
141 # cache_key= str(uuid.uuid5(uuid.NAMESPACE_DNS, cache_key)) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
142 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
143 return cache, cache_key |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
144 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
145 def _namespace_from_query(namespace, query): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
146 # cache namespace - the token handed in by the |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
147 # option + class we're querying against |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
148 namespace = " ".join([namespace] + [str(x) for x in query._entities]) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
149 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
150 # memcached wants this |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
151 namespace = namespace.replace(' ', '_') |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
152 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
153 return namespace |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
154 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
155 def _set_cache_parameters(query, region, namespace, cache_key): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
156 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
157 if hasattr(query, '_cache_parameters'): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
158 region, namespace, cache_key = query._cache_parameters |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
159 raise ValueError("This query is already configured " |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
160 "for region %r namespace %r" % |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
161 (region, namespace) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
162 ) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
163 query._cache_parameters = region, namespace, cache_key |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
164 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
165 class FromCache(MapperOption): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
166 """Specifies that a Query should load results from a cache.""" |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
167 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
168 propagate_to_loaders = False |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
169 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
170 def __init__(self, region, namespace, cache_key=None): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
171 """Construct a new FromCache. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
172 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
173 :param region: the cache region. Should be a |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
174 region configured in the Beaker CacheManager. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
175 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
176 :param namespace: the cache namespace. Should |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
177 be a name uniquely describing the target Query's |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
178 lexical structure. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
179 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
180 :param cache_key: optional. A string cache key |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
181 that will serve as the key to the query. Use this |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
182 if your query has a huge amount of parameters (such |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
183 as when using in_()) which correspond more simply to |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
184 some other identifier. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
185 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
186 """ |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
187 self.region = region |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
188 self.namespace = namespace |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
189 self.cache_key = cache_key |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
190 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
191 def process_query(self, query): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
192 """Process a Query during normal loading operation.""" |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
193 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
194 _set_cache_parameters(query, self.region, self.namespace, self.cache_key) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
195 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
196 class RelationshipCache(MapperOption): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
197 """Specifies that a Query as called within a "lazy load" |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
198 should load results from a cache.""" |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
199 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
200 propagate_to_loaders = True |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
201 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
202 def __init__(self, region, namespace, attribute): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
203 """Construct a new RelationshipCache. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
204 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
205 :param region: the cache region. Should be a |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
206 region configured in the Beaker CacheManager. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
207 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
208 :param namespace: the cache namespace. Should |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
209 be a name uniquely describing the target Query's |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
210 lexical structure. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
211 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
212 :param attribute: A Class.attribute which |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
213 indicates a particular class relationship() whose |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
214 lazy loader should be pulled from the cache. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
215 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
216 """ |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
217 self.region = region |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
218 self.namespace = namespace |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
219 self._relationship_options = { |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
220 (attribute.property.parent.class_, attribute.property.key) : self |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
221 } |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
222 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
223 def process_query_conditionally(self, query): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
224 """Process a Query that is used within a lazy loader. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
225 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
226 (the process_query_conditionally() method is a SQLAlchemy |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
227 hook invoked only within lazyload.) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
228 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
229 """ |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
230 if query._current_path: |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
231 mapper, key = query._current_path[-2:] |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
232 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
233 for cls in mapper.class_.__mro__: |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
234 if (cls, key) in self._relationship_options: |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
235 relationship_option = self._relationship_options[(cls, key)] |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
236 _set_cache_parameters( |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
237 query, |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
238 relationship_option.region, |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
239 relationship_option.namespace, |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
240 None) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
241 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
242 def and_(self, option): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
243 """Chain another RelationshipCache option to this one. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
244 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
245 While many RelationshipCache objects can be specified on a single |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
246 Query separately, chaining them together allows for a more efficient |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
247 lookup during load. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
248 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
249 """ |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
250 self._relationship_options.update(option._relationship_options) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
251 return self |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
252 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
253 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
254 def _params_from_query(query): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
255 """Pull the bind parameter values from a query. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
256 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
257 This takes into account any scalar attribute bindparam set up. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
258 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
259 E.g. params_from_query(query.filter(Cls.foo==5).filter(Cls.bar==7))) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
260 would return [5, 7]. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
261 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
262 """ |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
263 v = [] |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
264 def visit_bindparam(bind): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
265 value = query._params.get(bind.key, bind.value) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
266 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
267 # lazyloader may dig a callable in here, intended |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
268 # to late-evaluate params after autoflush is called. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
269 # convert to a scalar value. |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
270 if callable(value): |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
271 value = value() |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
272 |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
273 v.append(value) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
274 if query._criterion is not None: |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
275 visitors.traverse(query._criterion, {}, {'bindparam':visit_bindparam}) |
7afbc45aab28
added caching queries to hg-app
Marcin Kuzminski <marcin@python-works.com>
parents:
diff
changeset
|
276 return v |