Mercurial > gemma

annotate pkg/mesh/cache.go @ 5591:0011f50cf216 surveysperbottleneckid

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Removed no longer used alternative api for surveys/ endpoint. As bottlenecks in the summary for SR imports are now identified by their id and no longer by the (not guarantied to be unique!) name, there is no longer the need to request survey data by the name+date tuple (which isn't reliable anyway). So the workaround was now reversed.
author Sascha Wilde <wilde@sha-bang.de>
date Wed, 06 Apr 2022 13:30:29 +0200
parents ab6eb160cd29
children e54635c20d43
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1017
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
2 // without warranty, see README.md and license for details.
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
3 //
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
6 //
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
7 // Copyright (C) 2018 by via donau
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
9 // Software engineering by Intevation GmbH
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
10 //
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
11 // Author(s):
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
13
4827
f4abfd0ee8ad Renamed octree package to mesh as there is no octree any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4658
diff changeset
14 package mesh
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import (
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 "context"
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 "database/sql"
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 "sync"
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
20 "time"
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
21 )
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
22
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
23 type (
724
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
24 cacheKey struct {
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
25 date time.Time
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26 bottleneck string
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
27 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
28
724
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
29 cacheEntry struct {
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
30 checksum string
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
31 tree *STRTree
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
32 access time.Time
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
33 }
1692
f4dcbe8941a1 Octree: Resolved the remaing golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1333
diff changeset
34
f4dcbe8941a1 Octree: Resolved the remaing golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1333
diff changeset
35 // Cache holds Octrees for a defined amount of time in memory
f4dcbe8941a1 Octree: Resolved the remaing golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1333
diff changeset
36 // before they are released.
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
37 Cache struct {
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
38 sync.Mutex
724
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
39 entries map[cacheKey]*cacheEntry
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
40 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
41 )
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
42
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
43 const (
724
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
44 cleanupCacheSleep = 6 * time.Minute
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
45 maxCacheAge = 5 * time.Minute
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
46 maxCacheEntries = 4
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
47 )
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
48
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
49 const (
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
50 directMeshSQL = `
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
51 SELECT mesh_index FROM waterway.sounding_results
4214
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
52 WHERE id = $1
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
53 `
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
54 fetchMeshSQL = `
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
55 SELECT mesh_checksum, mesh_index
1131
2e6b47cdb2ca Store octrees along with sounding results
Tom Gottfried <tom@intevation.de>
parents: 1017
diff changeset
56 FROM waterway.sounding_results
2e6b47cdb2ca Store octrees along with sounding results
Tom Gottfried <tom@intevation.de>
parents: 1017
diff changeset
57 WHERE bottleneck_id = $1 AND date_info = $2::date
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
58 AND mesh_checksum IS NOT NULL AND mesh_index IS NOT NULL
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
59 `
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
60 checkMeshSQL = `
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
61 SELECT CASE
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
62 WHEN mesh_checksum = $3 THEN NULL
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
63 ELSE mesh_index
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
64 END
1131
2e6b47cdb2ca Store octrees along with sounding results
Tom Gottfried <tom@intevation.de>
parents: 1017
diff changeset
65 FROM waterway.sounding_results
2e6b47cdb2ca Store octrees along with sounding results
Tom Gottfried <tom@intevation.de>
parents: 1017
diff changeset
66 WHERE bottleneck_id = $1 AND date_info = $2::date
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
67 AND mesh_checksum IS NOT NULL AND mesh_index IS NOT NULL
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
68 `
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
69 )
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
70
4889
ab6eb160cd29 Sounding result cache: Start background cache cleanup lazily only when the cache is really used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4853
diff changeset
71 var cache Cache
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
72
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
73 func (c *Cache) background() {
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
74 for {
724
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
75 time.Sleep(cleanupCacheSleep)
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
76 c.cleanup()
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
77 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
78 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
79
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
80 func (c *Cache) cleanup() {
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
81 c.Lock()
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
82 defer c.Unlock()
724
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
83 good := time.Now().Add(-maxCacheAge)
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
84 for k, v := range c.entries {
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
85 if v.access.Before(good) {
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
86 delete(c.entries, k)
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
87 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
88 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
89 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
90
1692
f4dcbe8941a1 Octree: Resolved the remaing golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1333
diff changeset
91 // FromCache fetches an Octree from the global Octree cache.
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
92 func FromCache(
1328
d753ce6cf588 To make golint happier made context.Context to be the first argument in all calls.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1135
diff changeset
93 ctx context.Context,
d753ce6cf588 To make golint happier made context.Context to be the first argument in all calls.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1135
diff changeset
94 conn *sql.Conn,
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
95 bottleneck string, date time.Time,
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
96 ) (*STRTree, error) {
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
97 return cache.get(ctx, conn, bottleneck, date)
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
98 }
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
99
4853
181c2c05b12a More golint fixes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4827
diff changeset
100 // FetchMeshDirectly loads a mesh directly from the database.
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
101 func FetchMeshDirectly(
4214
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
102 ctx context.Context,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
103 tx *sql.Tx,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
104 id int64,
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
105 ) (*STRTree, error) {
4214
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
106 var data []byte
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
107 err := tx.QueryRowContext(ctx, directMeshSQL, id).Scan(&data)
4214
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
108 if err != nil {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
109 return nil, err
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
110 }
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
111 tree := new(STRTree)
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
112 if err := tree.FromBytes(data); err != nil {
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
113 return nil, err
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
114 }
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
115 return tree, nil
4214
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
116 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
117
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
118 func (c *Cache) get(
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
119 ctx context.Context,
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
120 conn *sql.Conn,
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
121 bottleneck string, date time.Time,
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
122 ) (*STRTree, error) {
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
123 c.Lock()
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
124 defer c.Unlock()
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
125
4889
ab6eb160cd29 Sounding result cache: Start background cache cleanup lazily only when the cache is really used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4853
diff changeset
126 // Start background cleanup lazily.
ab6eb160cd29 Sounding result cache: Start background cache cleanup lazily only when the cache is really used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4853
diff changeset
127 if c.entries == nil {
ab6eb160cd29 Sounding result cache: Start background cache cleanup lazily only when the cache is really used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4853
diff changeset
128 c.entries = map[cacheKey]*cacheEntry{}
ab6eb160cd29 Sounding result cache: Start background cache cleanup lazily only when the cache is really used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4853
diff changeset
129 go c.background()
ab6eb160cd29 Sounding result cache: Start background cache cleanup lazily only when the cache is really used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4853
diff changeset
130 }
ab6eb160cd29 Sounding result cache: Start background cache cleanup lazily only when the cache is really used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4853
diff changeset
131
724
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
132 key := cacheKey{date, bottleneck}
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
133 entry := c.entries[key]
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
134
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
135 var data []byte
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
136 var checksum string
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
137
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
138 if entry == nil {
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
139 // fetch from database
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
140 err := conn.QueryRowContext(
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
141 ctx, fetchMeshSQL, bottleneck, date).Scan(&checksum, &data)
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
142 switch {
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
143 case err == sql.ErrNoRows:
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
144 return nil, nil
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
145 case err != nil:
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
146 return nil, err
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
147 }
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
148 } else {
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
149 // check if we are not outdated.
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
150 err := conn.QueryRowContext(
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
151 ctx, checkMeshSQL, bottleneck, date, entry.checksum).Scan(&data)
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
152 switch {
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
153 case err == sql.ErrNoRows:
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
154 return nil, nil
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
155 case err != nil:
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
156 return nil, err
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
157 }
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
158 if data == nil { // we are still current
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
159 entry.access = time.Now()
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
160 return entry.tree, nil
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
161 }
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
162 }
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
163
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
164 tree := new(STRTree)
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
165
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
166 if err := tree.FromBytes(data); err != nil {
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
167 return nil, err
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
168 }
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
169
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
170 now := time.Now()
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
171
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
172 if entry != nil {
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
173 entry.tree = tree
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
174 entry.access = now
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
175 return tree, nil
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
176 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
177
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
178 for len(c.entries) >= maxCacheEntries {
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
179 // Evict the entry that is accessed the longest time ago.
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
180 var oldestKey cacheKey
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
181 oldest := now
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
182
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
183 for k, v := range c.entries {
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
184 if v.access.Before(oldest) {
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
185 oldest = v.access
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
186 oldestKey = k
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
187 }
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
188 }
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
189 delete(c.entries, oldestKey)
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
190 }
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
191
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
192 c.entries[key] = &cacheEntry{
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
193 checksum: checksum,
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
194 tree: tree,
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
195 access: now,
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
196 }
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
197
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
198 return tree, nil
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
199 }