Mercurial > gemma

annotate pkg/mesh/cache.go @ 5736:55892008ec96 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Fixed a bunch of corner cases in WG import.
author Sascha Wilde <wilde@sha-bang.de>
date Wed, 29 May 2024 19:02:42 +0200
parents 1ea1d3ef2258
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1017
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
2 // without warranty, see README.md and license for details.
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
3 //
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
6 //
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
7 // Copyright (C) 2018 by via donau
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
9 // Software engineering by Intevation GmbH
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
10 //
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
11 // Author(s):
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 727
diff changeset
13
4827
f4abfd0ee8ad Renamed octree package to mesh as there is no octree any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4658
diff changeset
14 package mesh
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import (
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 "context"
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 "database/sql"
5724
1ea1d3ef2258 Be tolerant in re-generating multi meshes iso areas if mesh data is empty.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5679
diff changeset
19 "errors"
1ea1d3ef2258 Be tolerant in re-generating multi meshes iso areas if mesh data is empty.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5679
diff changeset
20 "fmt"
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
21 "sync"
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
22 "time"
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
23 )
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
24
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
25 type (
724
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
26 cacheKey struct {
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
27 date time.Time
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
28 bottleneck string
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
29 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
30
724
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
31 cacheEntry struct {
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
32 checksum string
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
33 tree *STRTree
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
34 access time.Time
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
35 }
1692
f4dcbe8941a1 Octree: Resolved the remaing golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1333
diff changeset
36
f4dcbe8941a1 Octree: Resolved the remaing golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1333
diff changeset
37 // Cache holds Octrees for a defined amount of time in memory
f4dcbe8941a1 Octree: Resolved the remaing golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1333
diff changeset
38 // before they are released.
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
39 Cache struct {
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
40 sync.Mutex
724
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
41 entries map[cacheKey]*cacheEntry
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
42 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
43 )
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
44
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
45 const (
724
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
46 cleanupCacheSleep = 6 * time.Minute
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
47 maxCacheAge = 5 * time.Minute
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
48 maxCacheEntries = 4
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
49 )
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
50
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
51 const (
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
52 directMeshSQL = `
5679
03dfbe675842 Simplified version handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5674
diff changeset
53 SELECT mesh_index, coalesce(mesh_index_version, 1) AS mesh_index_version
03dfbe675842 Simplified version handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5674
diff changeset
54 FROM waterway.sounding_results
4214
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
55 WHERE id = $1
5724
1ea1d3ef2258 Be tolerant in re-generating multi meshes iso areas if mesh data is empty.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5679
diff changeset
56 AND mesh_checksum IS NOT NULL AND mesh_index IS NOT NULL
4214
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
57 `
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
58 fetchMeshSQL = `
5679
03dfbe675842 Simplified version handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5674
diff changeset
59 SELECT mesh_checksum, mesh_index, coalesce(mesh_index_version, 1) AS mesh_index_version
1131
2e6b47cdb2ca Store octrees along with sounding results
Tom Gottfried <tom@intevation.de>
parents: 1017
diff changeset
60 FROM waterway.sounding_results
2e6b47cdb2ca Store octrees along with sounding results
Tom Gottfried <tom@intevation.de>
parents: 1017
diff changeset
61 WHERE bottleneck_id = $1 AND date_info = $2::date
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
62 AND mesh_checksum IS NOT NULL AND mesh_index IS NOT NULL
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63 `
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
64 checkMeshSQL = `
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
65 SELECT CASE
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
66 WHEN mesh_checksum = $3 THEN NULL
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
67 ELSE mesh_index
5674
e54635c20d43 Load version fom database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4889
diff changeset
68 END,
5679
03dfbe675842 Simplified version handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5674
diff changeset
69 coalesce(mesh_index_version, 1) AS mesh_index_version
1131
2e6b47cdb2ca Store octrees along with sounding results
Tom Gottfried <tom@intevation.de>
parents: 1017
diff changeset
70 FROM waterway.sounding_results
2e6b47cdb2ca Store octrees along with sounding results
Tom Gottfried <tom@intevation.de>
parents: 1017
diff changeset
71 WHERE bottleneck_id = $1 AND date_info = $2::date
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
72 AND mesh_checksum IS NOT NULL AND mesh_index IS NOT NULL
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
73 `
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
74 )
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
75
4889
ab6eb160cd29 Sounding result cache: Start background cache cleanup lazily only when the cache is really used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4853
diff changeset
76 var cache Cache
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
77
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
78 func (c *Cache) background() {
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
79 for {
724
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
80 time.Sleep(cleanupCacheSleep)
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
81 c.cleanup()
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
82 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
83 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
84
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
85 func (c *Cache) cleanup() {
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
86 c.Lock()
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
87 defer c.Unlock()
724
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
88 good := time.Now().Add(-maxCacheAge)
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
89 for k, v := range c.entries {
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
90 if v.access.Before(good) {
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
91 delete(c.entries, k)
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
92 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
93 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
94 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
95
1692
f4dcbe8941a1 Octree: Resolved the remaing golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1333
diff changeset
96 // FromCache fetches an Octree from the global Octree cache.
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
97 func FromCache(
1328
d753ce6cf588 To make golint happier made context.Context to be the first argument in all calls.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1135
diff changeset
98 ctx context.Context,
d753ce6cf588 To make golint happier made context.Context to be the first argument in all calls.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1135
diff changeset
99 conn *sql.Conn,
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
100 bottleneck string, date time.Time,
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
101 ) (*STRTree, error) {
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
102 return cache.get(ctx, conn, bottleneck, date)
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
103 }
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
104
4853
181c2c05b12a More golint fixes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4827
diff changeset
105 // FetchMeshDirectly loads a mesh directly from the database.
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
106 func FetchMeshDirectly(
4214
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
107 ctx context.Context,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
108 tx *sql.Tx,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
109 id int64,
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
110 ) (*STRTree, error) {
4214
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
111 var data []byte
5679
03dfbe675842 Simplified version handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5674
diff changeset
112 var version int
5724
1ea1d3ef2258 Be tolerant in re-generating multi meshes iso areas if mesh data is empty.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5679
diff changeset
113 switch err := tx.QueryRowContext(ctx, directMeshSQL, id).Scan(&data, &version); {
1ea1d3ef2258 Be tolerant in re-generating multi meshes iso areas if mesh data is empty.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5679
diff changeset
114 case errors.Is(err, sql.ErrNoRows):
1ea1d3ef2258 Be tolerant in re-generating multi meshes iso areas if mesh data is empty.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5679
diff changeset
115 return nil, nil
1ea1d3ef2258 Be tolerant in re-generating multi meshes iso areas if mesh data is empty.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5679
diff changeset
116 case err != nil:
1ea1d3ef2258 Be tolerant in re-generating multi meshes iso areas if mesh data is empty.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5679
diff changeset
117 return nil, fmt.Errorf("fetching mesh from db failed: %w", err)
4214
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
118 }
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
119 tree := new(STRTree)
5679
03dfbe675842 Simplified version handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5674
diff changeset
120 if err := tree.FromBytes(data, version); err != nil {
5724
1ea1d3ef2258 Be tolerant in re-generating multi meshes iso areas if mesh data is empty.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5679
diff changeset
121 return nil, fmt.Errorf("deserializing mesh failed: %w (version %d)", err, version)
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
122 }
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
123 return tree, nil
4214
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
124 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
125
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
126 func (c *Cache) get(
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
127 ctx context.Context,
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
128 conn *sql.Conn,
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
129 bottleneck string, date time.Time,
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
130 ) (*STRTree, error) {
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
131 c.Lock()
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
132 defer c.Unlock()
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
133
4889
ab6eb160cd29 Sounding result cache: Start background cache cleanup lazily only when the cache is really used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4853
diff changeset
134 // Start background cleanup lazily.
ab6eb160cd29 Sounding result cache: Start background cache cleanup lazily only when the cache is really used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4853
diff changeset
135 if c.entries == nil {
ab6eb160cd29 Sounding result cache: Start background cache cleanup lazily only when the cache is really used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4853
diff changeset
136 c.entries = map[cacheKey]*cacheEntry{}
ab6eb160cd29 Sounding result cache: Start background cache cleanup lazily only when the cache is really used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4853
diff changeset
137 go c.background()
ab6eb160cd29 Sounding result cache: Start background cache cleanup lazily only when the cache is really used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4853
diff changeset
138 }
ab6eb160cd29 Sounding result cache: Start background cache cleanup lazily only when the cache is really used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4853
diff changeset
139
724
6ab0c170e5b8 Moved octree stuff to own package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 723
diff changeset
140 key := cacheKey{date, bottleneck}
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
141 entry := c.entries[key]
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
142
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
143 var data []byte
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
144 var checksum string
5679
03dfbe675842 Simplified version handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5674
diff changeset
145 var version int
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
146
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
147 if entry == nil {
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
148 // fetch from database
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
149 err := conn.QueryRowContext(
5674
e54635c20d43 Load version fom database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4889
diff changeset
150 ctx, fetchMeshSQL, bottleneck, date).Scan(&checksum, &data, &version)
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
151 switch {
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
152 case err == sql.ErrNoRows:
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
153 return nil, nil
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
154 case err != nil:
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
155 return nil, err
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
156 }
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
157 } else {
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
158 // check if we are not outdated.
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
159 err := conn.QueryRowContext(
5674
e54635c20d43 Load version fom database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4889
diff changeset
160 ctx, checkMeshSQL, bottleneck, date, entry.checksum).Scan(&data, &version)
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
161 switch {
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
162 case err == sql.ErrNoRows:
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
163 return nil, nil
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
164 case err != nil:
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
165 return nil, err
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
166 }
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
167 if data == nil { // we are still current
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
168 entry.access = time.Now()
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
169 return entry.tree, nil
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
170 }
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
171 }
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
172
4658
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
173 tree := new(STRTree)
4bbfe3dd2ab5 Completed usage of STRTrees.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4214
diff changeset
174
5679
03dfbe675842 Simplified version handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5674
diff changeset
175 if err := tree.FromBytes(data, version); err != nil {
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
176 return nil, err
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
177 }
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
178
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
179 now := time.Now()
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
180
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
181 if entry != nil {
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
182 entry.tree = tree
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
183 entry.access = now
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
184 return tree, nil
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
185 }
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
186
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
187 for len(c.entries) >= maxCacheEntries {
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
188 // Evict the entry that is accessed the longest time ago.
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
189 var oldestKey cacheKey
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
190 oldest := now
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
191
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
192 for k, v := range c.entries {
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
193 if v.access.Before(oldest) {
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
194 oldest = v.access
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
195 oldestKey = k
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
196 }
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
197 }
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
198 delete(c.entries, oldestKey)
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
199 }
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
200
1333
8c6b5d47a7ff Renamed octree.OctreeCache to octree.Cache to reduce stuttering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
201 c.entries[key] = &cacheEntry{
725
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
202 checksum: checksum,
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
203 tree: tree,
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
204 access: now,
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
205 }
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
206
e0437ec46798 Finished the octree cache. TODO: implement the deserialization.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 724
diff changeset
207 return tree, nil
723
7eed7ff3142d Started with model to load octrees from database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
208 }