Mercurial > gemma

annotate pkg/imports/wx.go @ 5688:6281c18b109f sr-v2

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Finsh serializing v2 meshes.
author Sascha L. Teichmann <sascha.teichmann@intevation.de>
date Mon, 12 Feb 2024 02:27:41 +0100
parents 6fe3662aafeb
children 6270951dda28
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1675
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
3 //
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
6 //
2649
9b6a02923bb4 Allow sys_admin to import waterway axis and area without clipping
Tom Gottfried <tom@intevation.de>
parents: 2622
diff changeset
7 // Copyright (C) 2018, 2019 by via donau
1675
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 //
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 // Author(s):
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
2649
9b6a02923bb4 Allow sys_admin to import waterway axis and area without clipping
Tom Gottfried <tom@intevation.de>
parents: 2622
diff changeset
13 // * Tom Gottfried <tom.gottfried@intevation.de>
1675
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15 package imports
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16
1696
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1683
diff changeset
17 // WXJobKind is the import queue type identifier.
1675
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 const WXJobKind JobKind = "wx"
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
20 func init() {
5016
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
21 RegisterJobCreator(WXJobKind,
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
22 &WFSFeatureJobCreator{
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
23 description: "waterway axis",
5020
e4ab338e7ba9 Removed unnecessary creatings of empty slices in expressing read dependencies in imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5019
diff changeset
24 depends: [2][]string{{"waterway_axis"}},
5016
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
25 newConsumer: newSQLConsumer(
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
26 prepareStmnts(insertWaterwayAxisSQL),
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
27 consume,
5018
d6873fa95ed6 ... and adjust the name.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5017
diff changeset
28 axisInvalidation,
5016
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
29 newMultiLineFeature(func() interface{} {
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
30 return new(waterwayAxisProperties)
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
31 }),
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
32 ),
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
33 })
1675
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
34 }
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
35
1680
de8089944b19 Waterway axis import: Started extracting properties from the features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1679
diff changeset
36 type waterwayAxisProperties struct {
de8089944b19 Waterway axis import: Started extracting properties from the features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1679
diff changeset
37 ObjNam string `json:"hydro_objnam"`
de8089944b19 Waterway axis import: Started extracting properties from the features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1679
diff changeset
38 NObjNnm *string `json:"hydro_nobjnm"`
de8089944b19 Waterway axis import: Started extracting properties from the features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1679
diff changeset
39 }
de8089944b19 Waterway axis import: Started extracting properties from the features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1679
diff changeset
40
1681
4d6ce621379e Waterway axis import: Completed, but the final commit unexpectedly resulted in rollback.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1680
diff changeset
41 const (
4d6ce621379e Waterway axis import: Completed, but the final commit unexpectedly resulted in rollback.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1680
diff changeset
42 insertWaterwayAxisSQL = `
1795
1333f96f18d0 Waterway axis: Clip features against responsibility area of importing user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
43 WITH resp AS (
2912
93fa55bce126 Add utility function to get users area of responsibility
Tom Gottfried <tom@intevation.de>
parents: 2903
diff changeset
44 SELECT users.current_user_area_utm() AS a
5016
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
45 ),
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
46 g AS (
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
47 SELECT
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
48 ST_Multi(ST_Node(ST_CollectionExtract(ST_Transform(new_ax, 4326), 2)))
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
49 AS new_ax
2855
8a8a929182f9 Enable import of axis and area independently from areas of responsibility
Tom Gottfried <tom@intevation.de>
parents: 2833
diff changeset
50 FROM ST_GeomFromWKB($1, $2::integer) AS new_line (new_line),
5009
e8b2dc771f9e Store axis as MultiLinestring
Tom Gottfried <tom@intevation.de>
parents: 4879
diff changeset
51 LATERAL (SELECT
2649
9b6a02923bb4 Allow sys_admin to import waterway axis and area without clipping
Tom Gottfried <tom@intevation.de>
parents: 2622
diff changeset
52 CASE WHEN pg_has_role('sys_admin', 'MEMBER')
5019
dc3d648bdf4a Avoid needless geometry processing
Tom Gottfried <tom@intevation.de>
parents: 5018
diff changeset
53 OR ST_Covers((SELECT a FROM resp),
dc3d648bdf4a Avoid needless geometry processing
Tom Gottfried <tom@intevation.de>
parents: 5018
diff changeset
54 ST_Transform(new_line, (SELECT ST_SRID(a) FROM resp)))
5009
e8b2dc771f9e Store axis as MultiLinestring
Tom Gottfried <tom@intevation.de>
parents: 4879
diff changeset
55 THEN new_line
5013
7dff1015283d Add row level security policies for waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5009
diff changeset
56 ELSE ST_Intersection((SELECT ST_Buffer(a, -0.0001) FROM resp),
2912
93fa55bce126 Add utility function to get users area of responsibility
Tom Gottfried <tom@intevation.de>
parents: 2903
diff changeset
57 ST_Node(ST_Transform(new_line, (SELECT ST_SRID(a) FROM resp))))
5009
e8b2dc771f9e Store axis as MultiLinestring
Tom Gottfried <tom@intevation.de>
parents: 4879
diff changeset
58 END) AS new_ax (new_ax)
e8b2dc771f9e Store axis as MultiLinestring
Tom Gottfried <tom@intevation.de>
parents: 4879
diff changeset
59 -- Do nothing if intersection is empty:
e8b2dc771f9e Store axis as MultiLinestring
Tom Gottfried <tom@intevation.de>
parents: 4879
diff changeset
60 WHERE NOT ST_IsEmpty(new_ax)
5016
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
61 ),
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
62 t AS (
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
63 UPDATE waterway.waterway_axis SET last_found = current_timestamp
5021
6fe3662aafeb Add explaining comment
Tom Gottfried <tom@intevation.de>
parents: 5020
diff changeset
64 -- The first condition is just to help the PostgreSQL query planner
6fe3662aafeb Add explaining comment
Tom Gottfried <tom@intevation.de>
parents: 5020
diff changeset
65 -- to avoid evaluating more costly conditions including those
6fe3662aafeb Add explaining comment
Tom Gottfried <tom@intevation.de>
parents: 5020
diff changeset
66 -- introduced by row level security policies:
5016
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
67 WHERE (SELECT new_ax FROM g) IS NOT NULL
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
68 AND validity @> current_timestamp
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
69 AND (
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
70 wtwaxs, objnam, nobjnam
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
71 ) IS NOT DISTINCT FROM (
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
72 (SELECT new_ax FROM g), $3, $4)
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
73 RETURNING 1
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
74 )
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
75 INSERT INTO waterway.waterway_axis (wtwaxs, objnam, nobjnam)
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
76 SELECT new_ax, $3, $4
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
77 FROM g
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
78 WHERE NOT EXISTS(SELECT 1 FROM t)
2726
dff749c07ae0 Warn if imported axis linestrings cross each other
Tom Gottfried <tom@intevation.de>
parents: 2723
diff changeset
79 RETURNING id
1795
1333f96f18d0 Waterway axis: Clip features against responsibility area of importing user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
80 `
5016
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
81 invalidateAxisSQL = `
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
82 UPDATE waterway.waterway_axis
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
83 SET validity = tstzrange(lower(validity), current_timestamp)
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
84 WHERE validity @> current_timestamp
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
85 AND last_found < current_timestamp
cf25b23e3eec Keep historic data of waterway axis
Tom Gottfried <tom@intevation.de>
parents: 5013
diff changeset
86 `
1681
4d6ce621379e Waterway axis import: Completed, but the final commit unexpectedly resulted in rollback.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1680
diff changeset
87 )
4d6ce621379e Waterway axis import: Completed, but the final commit unexpectedly resulted in rollback.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1680
diff changeset
88
5018
d6873fa95ed6 ... and adjust the name.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5017
diff changeset
89 func axisInvalidation(spc *SQLGeometryConsumer) error {
5017
557afcd9a131 Curried functions all the way down. In this case a level too much. ;-)
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5016
diff changeset
90 res, err := spc.tx.ExecContext(spc.ctx, invalidateAxisSQL)
557afcd9a131 Curried functions all the way down. In this case a level too much. ;-)
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5016
diff changeset
91 if err != nil {
557afcd9a131 Curried functions all the way down. In this case a level too much. ;-)
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5016
diff changeset
92 return err
1679
2dc7768be0e4 Waterway axis import: More on reading data from WFS. TODO: Parse to concrete features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1677
diff changeset
93 }
5017
557afcd9a131 Curried functions all the way down. In this case a level too much. ;-)
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5016
diff changeset
94 old, err := res.RowsAffected()
557afcd9a131 Curried functions all the way down. In this case a level too much. ;-)
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5016
diff changeset
95 if err != nil {
557afcd9a131 Curried functions all the way down. In this case a level too much. ;-)
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5016
diff changeset
96 return err
557afcd9a131 Curried functions all the way down. In this case a level too much. ;-)
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5016
diff changeset
97 }
557afcd9a131 Curried functions all the way down. In this case a level too much. ;-)
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5016
diff changeset
98 if old == 0 {
557afcd9a131 Curried functions all the way down. In this case a level too much. ;-)
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5016
diff changeset
99 return ErrFeaturesUnmodified
557afcd9a131 Curried functions all the way down. In this case a level too much. ;-)
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5016
diff changeset
100 }
557afcd9a131 Curried functions all the way down. In this case a level too much. ;-)
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5016
diff changeset
101 spc.feedback.Info(
557afcd9a131 Curried functions all the way down. In this case a level too much. ;-)
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5016
diff changeset
102 "Number of features removed from data source: %d", old)
557afcd9a131 Curried functions all the way down. In this case a level too much. ;-)
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5016
diff changeset
103 return nil
1675
8fec3887c7e5 Waterway axis import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
104 }