Mercurial > gemma
annotate pkg/imports/wg.go @ 5621:cf1e8ffe1ed5 erdms2
Fix missing zero point logging in wg import.
author | Sascha L. Teichmann <sascha.teichmann@intevation.de> |
---|---|
date | Mon, 28 Nov 2022 18:23:27 +0100 |
parents | 165e77c5736a |
children | b75d0b303328 |
rev | line source |
---|---|
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
1 // This is Free Software under GNU Affero General Public License v >= 3.0 |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
2 // without warranty, see README.md and license for details. |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
3 // |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
4 // SPDX-License-Identifier: AGPL-3.0-or-later |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
5 // License-Filename: LICENSES/AGPL-3.0.txt |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
6 // |
2318
06c4e57435f1
Warn on import of unknown reference level codes
Tom Gottfried <tom@intevation.de>
parents:
2317
diff
changeset
|
7 // Copyright (C) 2018, 2019 by via donau |
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
8 // – Österreichische Wasserstraßen-Gesellschaft mbH |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
9 // Software engineering by Intevation GmbH |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
10 // |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
11 // Author(s): |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de> |
2318
06c4e57435f1
Warn on import of unknown reference level codes
Tom Gottfried <tom@intevation.de>
parents:
2317
diff
changeset
|
13 // * Tom Gottfried <tom.gottfried@intevation.de> |
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
14 |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
15 package imports |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
16 |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
17 import ( |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
18 "context" |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
19 "database/sql" |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
20 "errors" |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
21 "strings" |
1835
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
22 "time" |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
23 |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
24 "github.com/jackc/pgx/pgtype" |
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
25 |
1829
b4b9089c2d79
Waterway gauges: Started with deleting old gauges to be overwritten.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1812
diff
changeset
|
26 "gemma.intevation.de/gemma/pkg/models" |
4062
6c760abcff0e
Move handling of PostgreSQL errors to own package
Tom Gottfried <tom@intevation.de>
parents:
4058
diff
changeset
|
27 "gemma.intevation.de/gemma/pkg/pgxutils" |
5609
e1936db6db8e
Adjust ta erdms data structure.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5565
diff
changeset
|
28 erdms "gemma.intevation.de/gemma/pkg/soap/erdms2" |
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
29 ) |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
30 |
4182
49012340336c
Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4147
diff
changeset
|
31 // WaterwayGauge is a Job to load gauge data from |
49012340336c
Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4147
diff
changeset
|
32 // a specified NTS service and stores them into the database. |
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
33 type WaterwayGauge struct { |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
34 // URL is the URL of the SOAP service. |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
35 URL string `json:"url"` |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
36 // Username is the username used to authenticate. |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
37 Username string `json:"username"` |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
38 // Passwort is the password to authenticate. |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
39 Password string `json:"password"` |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
40 // Insecure indicates if HTTPS traffic |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
41 // should validate certificates or not. |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
42 Insecure bool `json:"insecure"` |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
43 } |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
44 |
4852
046a07a33b19
Fixed the golint issues of the imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4799
diff
changeset
|
45 // Description gives a short info about relevant facts of this import. |
5565
ade07a3f2cfd
Forget to change signature of some imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5034
diff
changeset
|
46 func (wg *WaterwayGauge) Description([]string) (string, error) { |
4799
f32d086b5dbf
Removed the mechanical touch of the last commit.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4798
diff
changeset
|
47 return wg.URL, nil |
4798
ca6a5f722471
Added Description method to most imports.
Sascha Wilde <wilde@intevation.de>
parents:
4497
diff
changeset
|
48 } |
ca6a5f722471
Added Description method to most imports.
Sascha Wilde <wilde@intevation.de>
parents:
4497
diff
changeset
|
49 |
4182
49012340336c
Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4147
diff
changeset
|
50 // WGJobKind is the unique name of this import job type. |
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
51 const WGJobKind JobKind = "wg" |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
52 |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
53 type wgJobCreator struct{} |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
54 |
4182
49012340336c
Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4147
diff
changeset
|
55 func init() { RegisterJobCreator(WGJobKind, wgJobCreator{}) } |
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
56 |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
57 func (wgJobCreator) Description() string { return "waterway gauges" } |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
58 |
1829
b4b9089c2d79
Waterway gauges: Started with deleting old gauges to be overwritten.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1812
diff
changeset
|
59 func (wgJobCreator) AutoAccept() bool { return true } |
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
60 |
2187
7c83b5277c1c
Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
61 func (wgJobCreator) Create() Job { return new(WaterwayGauge) } |
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
62 |
3219
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3163
diff
changeset
|
63 func (wgJobCreator) Depends() [2][]string { |
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3163
diff
changeset
|
64 return [2][]string{ |
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3163
diff
changeset
|
65 {"gauges_reference_water_levels", "gauges"}, |
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3163
diff
changeset
|
66 {"depth_references"}, |
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
67 } |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
68 } |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
69 |
1829
b4b9089c2d79
Waterway gauges: Started with deleting old gauges to be overwritten.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1812
diff
changeset
|
70 // StageDone does nothing as there is no staging for gauges. |
5034
59a99655f34d
Added feedback support for StageDone.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4852
diff
changeset
|
71 func (wgJobCreator) StageDone(context.Context, *sql.Tx, int64, Feedback) error { return nil } |
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
72 |
1829
b4b9089c2d79
Waterway gauges: Started with deleting old gauges to be overwritten.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1812
diff
changeset
|
73 // CleanUp does nothing as there is nothing to cleanup with gauges. |
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
74 func (*WaterwayGauge) CleanUp() error { return nil } |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
75 |
1812
4910bcfab319
Waterway gauge import: Work around the expected (but nonetheless silly) namespace issues with the SOAP interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1810
diff
changeset
|
76 const ( |
3311
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
77 eraseObsoleteGaugesSQL = ` |
4028
5fed2f5bc104
End validity of gauge if it gets deleted during import.
Sascha Wilde <wilde@intevation.de>
parents:
3665
diff
changeset
|
78 UPDATE waterway.gauges SET erased = true, validity = validity - '[now,)' |
3551
a5448426e4e2
Only erase gauges that have been requested from ERDMS
Tom Gottfried <tom@intevation.de>
parents:
3529
diff
changeset
|
79 WHERE NOT erased |
a5448426e4e2
Only erase gauges that have been requested from ERDMS
Tom Gottfried <tom@intevation.de>
parents:
3529
diff
changeset
|
80 AND (location).country_code = ANY($1) |
a5448426e4e2
Only erase gauges that have been requested from ERDMS
Tom Gottfried <tom@intevation.de>
parents:
3529
diff
changeset
|
81 AND isrs_astext(location) <> ALL($2) |
3311
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
82 RETURNING isrs_astext(location) |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
83 ` |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
84 |
3302
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
85 eraseGaugeSQL = ` |
3402
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
86 WITH upd AS ( |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
87 UPDATE waterway.gauges SET |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
88 erased = true |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
89 WHERE isrs_astext(location) = $1 |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
90 AND NOT erased |
3648
0ec5c8ec1e44
Avoid empty validity time ranges
Tom Gottfried <tom@intevation.de>
parents:
3645
diff
changeset
|
91 -- Don't touch old entry if new validity contains old: will be updated |
0ec5c8ec1e44
Avoid empty validity time ranges
Tom Gottfried <tom@intevation.de>
parents:
3645
diff
changeset
|
92 AND NOT validity <@ $2 |
3402
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
93 RETURNING 1 |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
94 ) |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
95 -- Decide whether a new version will be INSERTed |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
96 SELECT EXISTS(SELECT 1 FROM upd) |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
97 OR NOT EXISTS(SELECT 1 FROM waterway.gauges WHERE isrs_astext(location) = $1) |
3163
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
98 ` |
1829
b4b9089c2d79
Waterway gauges: Started with deleting old gauges to be overwritten.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1812
diff
changeset
|
99 |
1835
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
100 insertGaugeSQL = ` |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
101 INSERT INTO waterway.gauges ( |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
102 location, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
103 objname, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
104 geom, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
105 applicability_from_km, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
106 applicability_to_km, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
107 validity, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
108 zero_point, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
109 geodref, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
110 date_info, |
3302
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
111 source_organization, |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
112 lastupdate |
1835
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
113 ) VALUES ( |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
114 ($1::char(2), $2::char(3), $3::char(5), $4::char(5), $5::int), |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
115 $6, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
116 ST_SetSRID(ST_MakePoint($7, $8), 4326)::geography, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
117 $9, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
118 $10, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
119 $11, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
120 $12, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
121 $13, |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
122 $14, |
3302
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
123 $15, |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
124 $16 |
3402
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
125 ) |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
126 ` |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
127 |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
128 fixValiditySQL = ` |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
129 UPDATE waterway.gauges SET |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
130 -- Set enddate of old entry to new startdate in case of overlap: |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
131 validity = validity - $2 |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
132 WHERE isrs_astext(location) = $1 |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
133 AND validity && $2 |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
134 AND erased |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
135 ` |
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
136 |
3302
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
137 updateGaugeSQL = ` |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
138 UPDATE waterway.gauges SET |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
139 objname = $6, |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
140 geom = ST_SetSRID(ST_MakePoint($7, $8), 4326), |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
141 applicability_from_km = $9, |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
142 applicability_to_km = $10, |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
143 zero_point = $11, |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
144 geodref = $12, |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
145 date_info = $13, |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
146 source_organization = $14, |
3648
0ec5c8ec1e44
Avoid empty validity time ranges
Tom Gottfried <tom@intevation.de>
parents:
3645
diff
changeset
|
147 lastupdate = $15, |
0ec5c8ec1e44
Avoid empty validity time ranges
Tom Gottfried <tom@intevation.de>
parents:
3645
diff
changeset
|
148 validity = $16 |
3302
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
149 WHERE location = ($1::char(2), $2::char(3), $3::char(5), $4::char(5), $5::int) |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
150 AND NOT erased |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
151 AND $15 > lastupdate |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
152 RETURNING 1 |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
153 ` |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
154 |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
155 deleteReferenceWaterLevelsSQL = ` |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
156 DELETE FROM waterway.gauges_reference_water_levels |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
157 WHERE isrs_astext(location) = $1 |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
158 AND validity = $2 |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
159 AND depth_reference <> ALL($3) |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
160 RETURNING depth_reference |
1836
4dcdd8891770
Waterway gauges import: Fixed insert/update of gauges. TODO: Re-insert reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1835
diff
changeset
|
161 ` |
3163
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
162 |
2318
06c4e57435f1
Warn on import of unknown reference level codes
Tom Gottfried <tom@intevation.de>
parents:
2317
diff
changeset
|
163 isNtSDepthRefSQL = ` |
06c4e57435f1
Warn on import of unknown reference level codes
Tom Gottfried <tom@intevation.de>
parents:
2317
diff
changeset
|
164 SELECT EXISTS(SELECT 1 FROM depth_references WHERE depth_reference = $1)` |
06c4e57435f1
Warn on import of unknown reference level codes
Tom Gottfried <tom@intevation.de>
parents:
2317
diff
changeset
|
165 |
1837
00d63eb9306a
Waterway gauges import: Finalized with re-inserting the reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1836
diff
changeset
|
166 insertReferenceWaterLevelsSQL = ` |
00d63eb9306a
Waterway gauges import: Finalized with re-inserting the reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1836
diff
changeset
|
167 INSERT INTO waterway.gauges_reference_water_levels ( |
3302
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
168 location, |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
169 validity, |
2317
8a8680e70d2e
Cleanup schema for reference water levels
Tom Gottfried <tom@intevation.de>
parents:
2187
diff
changeset
|
170 depth_reference, |
1837
00d63eb9306a
Waterway gauges import: Finalized with re-inserting the reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1836
diff
changeset
|
171 value |
00d63eb9306a
Waterway gauges import: Finalized with re-inserting the reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1836
diff
changeset
|
172 ) VALUES ( |
00d63eb9306a
Waterway gauges import: Finalized with re-inserting the reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1836
diff
changeset
|
173 ($1::char(2), $2::char(3), $3::char(5), $4::char(5), $5::int), |
00d63eb9306a
Waterway gauges import: Finalized with re-inserting the reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1836
diff
changeset
|
174 $6, |
3302
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
175 $7, |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
176 $8 |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
177 ) ON CONFLICT (location, validity, depth_reference) DO UPDATE SET |
3163
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
178 value = EXCLUDED.value |
1837
00d63eb9306a
Waterway gauges import: Finalized with re-inserting the reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1836
diff
changeset
|
179 ` |
1812
4910bcfab319
Waterway gauge import: Work around the expected (but nonetheless silly) namespace issues with the SOAP interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1810
diff
changeset
|
180 ) |
4910bcfab319
Waterway gauge import: Work around the expected (but nonetheless silly) namespace issues with the SOAP interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1810
diff
changeset
|
181 |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
182 var continueErr = errors.New("continue") |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
183 |
4182
49012340336c
Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4147
diff
changeset
|
184 // Do implements the actual import. |
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
185 func (wg *WaterwayGauge) Do( |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
186 ctx context.Context, |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
187 importID int64, |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
188 conn *sql.Conn, |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
189 feedback Feedback, |
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
190 ) (interface{}, error) { |
1810
7ee9bdaac336
Waterway gauge import: Added support for username/password.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1808
diff
changeset
|
191 |
1837
00d63eb9306a
Waterway gauges import: Finalized with re-inserting the reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1836
diff
changeset
|
192 start := time.Now() |
00d63eb9306a
Waterway gauges import: Finalized with re-inserting the reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1836
diff
changeset
|
193 |
3551
a5448426e4e2
Only erase gauges that have been requested from ERDMS
Tom Gottfried <tom@intevation.de>
parents:
3529
diff
changeset
|
194 responseData, countries, err := getRisData( |
2633
eb1d119f253f
Fetch data from ERDMS for all allowed countries
Tom Gottfried <tom@intevation.de>
parents:
2518
diff
changeset
|
195 ctx, |
3163
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
196 conn, |
2633
eb1d119f253f
Fetch data from ERDMS for all allowed countries
Tom Gottfried <tom@intevation.de>
parents:
2518
diff
changeset
|
197 feedback, |
eb1d119f253f
Fetch data from ERDMS for all allowed countries
Tom Gottfried <tom@intevation.de>
parents:
2518
diff
changeset
|
198 wg.Username, |
eb1d119f253f
Fetch data from ERDMS for all allowed countries
Tom Gottfried <tom@intevation.de>
parents:
2518
diff
changeset
|
199 wg.Password, |
eb1d119f253f
Fetch data from ERDMS for all allowed countries
Tom Gottfried <tom@intevation.de>
parents:
2518
diff
changeset
|
200 wg.URL, |
eb1d119f253f
Fetch data from ERDMS for all allowed countries
Tom Gottfried <tom@intevation.de>
parents:
2518
diff
changeset
|
201 wg.Insecure, |
eb1d119f253f
Fetch data from ERDMS for all allowed countries
Tom Gottfried <tom@intevation.de>
parents:
2518
diff
changeset
|
202 "wtwgag") |
eb1d119f253f
Fetch data from ERDMS for all allowed countries
Tom Gottfried <tom@intevation.de>
parents:
2518
diff
changeset
|
203 if err != nil { |
1812
4910bcfab319
Waterway gauge import: Work around the expected (but nonetheless silly) namespace issues with the SOAP interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1810
diff
changeset
|
204 return nil, err |
4910bcfab319
Waterway gauge import: Work around the expected (but nonetheless silly) namespace issues with the SOAP interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1810
diff
changeset
|
205 } |
4910bcfab319
Waterway gauge import: Work around the expected (but nonetheless silly) namespace issues with the SOAP interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1810
diff
changeset
|
206 |
3665
29ef6d41e4af
Use database triggers to move referencing objects to new versions
Tom Gottfried <tom@intevation.de>
parents:
3648
diff
changeset
|
207 var eraseGaugeStmt, insertStmt, |
3645
02951a62e8c6
'Historicise' bottlenecks on import
Tom Gottfried <tom@intevation.de>
parents:
3551
diff
changeset
|
208 fixValidityStmt, updateStmt, |
3302
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
209 deleteReferenceWaterLevelsStmt, |
3163
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
210 isNtSDepthRefStmt, insertWaterLevelStmt *sql.Stmt |
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
211 for _, x := range []struct { |
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
212 sql string |
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
213 stmt **sql.Stmt |
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
214 }{ |
3302
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
215 {eraseGaugeSQL, &eraseGaugeStmt}, |
3163
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
216 {insertGaugeSQL, &insertStmt}, |
3402
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
217 {fixValiditySQL, &fixValidityStmt}, |
3302
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
218 {updateGaugeSQL, &updateStmt}, |
3163
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
219 {deleteReferenceWaterLevelsSQL, &deleteReferenceWaterLevelsStmt}, |
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
220 {isNtSDepthRefSQL, &isNtSDepthRefStmt}, |
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
221 {insertReferenceWaterLevelsSQL, &insertWaterLevelStmt}, |
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
222 } { |
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
223 var err error |
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
224 if *x.stmt, err = conn.PrepareContext(ctx, x.sql); err != nil { |
1829
b4b9089c2d79
Waterway gauges: Started with deleting old gauges to be overwritten.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1812
diff
changeset
|
225 return nil, err |
b4b9089c2d79
Waterway gauges: Started with deleting old gauges to be overwritten.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1812
diff
changeset
|
226 } |
3163
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
227 defer (*x.stmt).Close() |
1812
4910bcfab319
Waterway gauge import: Work around the expected (but nonetheless silly) namespace issues with the SOAP interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1810
diff
changeset
|
228 } |
1810
7ee9bdaac336
Waterway gauge import: Added support for username/password.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1808
diff
changeset
|
229 |
3310 | 230 var gauges []string |
3302
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
231 var unchanged int |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
232 |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
233 var invalidISRS, startEndOrder, missingObjname, missingZeropoint []string |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
234 |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
235 databaseErrors := map[string][]string{} |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
236 |
3310 | 237 for _, data := range responseData { |
238 for _, dr := range data.RisdataReturn { | |
1836
4dcdd8891770
Waterway gauges import: Fixed insert/update of gauges. TODO: Re-insert reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1835
diff
changeset
|
239 |
3310 | 240 isrs := string(*dr.RisidxCode) |
241 code, err := models.IsrsFromString(isrs) | |
242 if err != nil { | |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
243 invalidISRS = append(invalidISRS, isrs) |
3310 | 244 continue |
1835
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
245 } |
3310 | 246 gauges = append(gauges, isrs) |
247 | |
3529
ba0339118d9c
Improve validation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3460
diff
changeset
|
248 // We need a valid, non-empty time range to identify gauge versions |
5609
e1936db6db8e
Adjust ta erdms data structure.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5565
diff
changeset
|
249 if dr.Enddate != nil && dr.Startdate != nil { |
5615
f6179b31e0a9
Use a type alias on date type to not loose XML serialisation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5609
diff
changeset
|
250 ed := dr.Enddate.ToGoTime() |
f6179b31e0a9
Use a type alias on date type to not loose XML serialisation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5609
diff
changeset
|
251 sd := dr.Startdate.ToGoTime() |
f6179b31e0a9
Use a type alias on date type to not loose XML serialisation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5609
diff
changeset
|
252 // log.Debugf("start date: %v end date: %v\n", sd, ed) |
f6179b31e0a9
Use a type alias on date type to not loose XML serialisation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5609
diff
changeset
|
253 if !ed.After(sd) { |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
254 startEndOrder = append(startEndOrder, isrs) |
5609
e1936db6db8e
Adjust ta erdms data structure.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5565
diff
changeset
|
255 unchanged++ |
e1936db6db8e
Adjust ta erdms data structure.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5565
diff
changeset
|
256 continue |
e1936db6db8e
Adjust ta erdms data structure.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5565
diff
changeset
|
257 } |
3458
aa3c83fb7018
Validate temporal validity of gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3402
diff
changeset
|
258 } |
aa3c83fb7018
Validate temporal validity of gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3402
diff
changeset
|
259 |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
260 if dr.Zeropoint == nil { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
261 missingZeropoint = append(missingZeropoint, isrs) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
262 unchanged++ |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
263 continue |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
264 } |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
265 |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
266 if dr.Objname.Loc == nil { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
267 missingObjname = append(missingObjname, isrs) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
268 unchanged++ |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
269 continue |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
270 } |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
271 |
3310 | 272 var from, to sql.NullInt64 |
1835
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
273 |
3310 | 274 if dr.Applicabilityfromkm != nil { |
275 from = sql.NullInt64{ | |
276 Int64: int64(*dr.Applicabilityfromkm), | |
277 Valid: true, | |
278 } | |
1836
4dcdd8891770
Waterway gauges import: Fixed insert/update of gauges. TODO: Re-insert reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1835
diff
changeset
|
279 } |
3310 | 280 if dr.Applicabilitytokm != nil { |
281 to = sql.NullInt64{ | |
282 Int64: int64(*dr.Applicabilitytokm), | |
283 Valid: true, | |
284 } | |
285 } | |
1835
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
286 |
3310 | 287 var tfrom, tto, dateInfo pgtype.Timestamptz |
1835
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
288 |
3310 | 289 if dr.Startdate != nil { |
290 tfrom = pgtype.Timestamptz{ | |
5615
f6179b31e0a9
Use a type alias on date type to not loose XML serialisation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5609
diff
changeset
|
291 Time: dr.Startdate.ToGoTime(), |
3310 | 292 Status: pgtype.Present, |
293 } | |
294 } else { | |
295 tfrom = pgtype.Timestamptz{ | |
296 Status: pgtype.Null, | |
297 } | |
298 } | |
1835
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
299 |
3310 | 300 if dr.Enddate != nil { |
301 tto = pgtype.Timestamptz{ | |
5615
f6179b31e0a9
Use a type alias on date type to not loose XML serialisation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5609
diff
changeset
|
302 Time: dr.Enddate.ToGoTime(), |
3310 | 303 Status: pgtype.Present, |
304 } | |
305 } else { | |
306 tto = pgtype.Timestamptz{ | |
307 Status: pgtype.Null, | |
308 } | |
1836
4dcdd8891770
Waterway gauges import: Fixed insert/update of gauges. TODO: Re-insert reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1835
diff
changeset
|
309 } |
1835
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
310 |
3310 | 311 validity := pgtype.Tstzrange{ |
312 Lower: tfrom, | |
313 Upper: tto, | |
314 LowerType: pgtype.Inclusive, | |
315 UpperType: pgtype.Exclusive, | |
316 Status: pgtype.Present, | |
1836
4dcdd8891770
Waterway gauges import: Fixed insert/update of gauges. TODO: Re-insert reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1835
diff
changeset
|
317 } |
3163
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
318 |
3310 | 319 if dr.Infodate != nil { |
320 dateInfo = pgtype.Timestamptz{ | |
5615
f6179b31e0a9
Use a type alias on date type to not loose XML serialisation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5609
diff
changeset
|
321 Time: dr.Infodate.ToGoTime(), |
3310 | 322 Status: pgtype.Present, |
323 } | |
324 } else { | |
325 dateInfo = pgtype.Timestamptz{ | |
326 Status: pgtype.Null, | |
327 } | |
328 } | |
329 | |
330 var geodref sql.NullString | |
331 if dr.Geodref != nil { | |
332 geodref = sql.NullString{ | |
333 String: string(*dr.Geodref), | |
334 Valid: true, | |
335 } | |
3302
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
336 } |
3310 | 337 |
338 var source sql.NullString | |
339 if dr.Source != nil { | |
340 source = sql.NullString{ | |
341 String: string(*dr.Source), | |
342 Valid: true, | |
343 } | |
344 } | |
345 | |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
346 err = func() error { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
347 tx, err := conn.BeginTx(ctx, nil) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
348 if err != nil { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
349 return err |
3310 | 350 } |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
351 defer tx.Rollback() |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
352 |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
353 // Mark old entry of gauge as erased, if applicable |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
354 var isNew bool |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
355 err = tx.StmtContext(ctx, eraseGaugeStmt).QueryRowContext(ctx, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
356 code.String(), |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
357 validity, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
358 ).Scan(&isNew) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
359 switch { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
360 case err != nil: |
4197
5d7ce7f926eb
Log fatal errors as errors, not warnings, in imports
Tom Gottfried <tom@intevation.de>
parents:
4182
diff
changeset
|
361 feedback.Error(pgxutils.ReadableError{Err: err}.Error()) |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
362 unchanged++ |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
363 return continueErr |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
364 case isNew: |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
365 var lu *time.Time |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
366 if dr.Lastupdate != nil { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
367 t := dr.Lastupdate.ToGoTime() |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
368 lu = &t |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
369 } |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
370 // insert gauge version entry |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
371 if _, err = tx.StmtContext(ctx, insertStmt).ExecContext(ctx, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
372 code.CountryCode, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
373 code.LoCode, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
374 code.FairwaySection, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
375 code.Orc, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
376 code.Hectometre, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
377 dr.Objname.Loc, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
378 dr.Lon, dr.Lat, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
379 from, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
380 to, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
381 &validity, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
382 dr.Zeropoint, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
383 geodref, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
384 &dateInfo, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
385 source, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
386 lu, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
387 ); err != nil { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
388 key := pgxutils.ReadableError{Err: err}.Error() |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
389 databaseErrors[key] = append(databaseErrors[key], isrs) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
390 return continueErr |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
391 } |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
392 //feedback.Info("insert new version") |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
393 case !isNew: |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
394 var lu *time.Time |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
395 if dr.Lastupdate != nil { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
396 t := dr.Lastupdate.ToGoTime() |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
397 lu = &t |
3402
c04b1409a596
Fix adaptation of gauge temporal validity
Tom Gottfried <tom@intevation.de>
parents:
3311
diff
changeset
|
398 } |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
399 // try to update |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
400 var dummy int |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
401 err2 := tx.StmtContext(ctx, updateStmt).QueryRowContext(ctx, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
402 code.CountryCode, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
403 code.LoCode, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
404 code.FairwaySection, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
405 code.Orc, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
406 code.Hectometre, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
407 dr.Objname.Loc, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
408 dr.Lon, dr.Lat, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
409 from, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
410 to, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
411 dr.Zeropoint, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
412 geodref, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
413 &dateInfo, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
414 source, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
415 lu, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
416 &validity, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
417 ).Scan(&dummy) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
418 switch { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
419 case err2 == sql.ErrNoRows: |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
420 //feedback.Info("unchanged") |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
421 unchanged++ |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
422 return continueErr |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
423 case err2 != nil: |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
424 key := pgxutils.ReadableError{Err: err}.Error() |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
425 databaseErrors[key] = append(databaseErrors[key], isrs) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
426 unchanged++ |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
427 return continueErr |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
428 default: |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
429 //feedback.Info("update") |
3310 | 430 } |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
431 |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
432 // Remove obsolete reference water levels |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
433 var currLevels pgtype.VarcharArray |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
434 currLevels.Set([]string{ |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
435 string(*dr.Reflevel1code), |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
436 string(*dr.Reflevel2code), |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
437 string(*dr.Reflevel3code), |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
438 }) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
439 rwls, err := tx.StmtContext(ctx, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
440 deleteReferenceWaterLevelsStmt).QueryContext(ctx, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
441 code.String(), |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
442 &validity, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
443 &currLevels, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
444 ) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
445 if err != nil { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
446 return err |
3310 | 447 } |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
448 defer rwls.Close() |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
449 for rwls.Next() { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
450 var delRef string |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
451 if err := rwls.Scan(&delRef); err != nil { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
452 return err |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
453 } |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
454 feedback.Warn("Removed reference water level %s from %s", |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
455 delRef, code) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
456 } |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
457 if err := rwls.Err(); err != nil { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
458 return err |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
459 } |
3302
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
460 } |
3310 | 461 |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
462 // Set end of validity of old version to start of new version |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
463 // in case of overlap |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
464 if _, err = tx.StmtContext(ctx, fixValidityStmt).ExecContext( |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
465 ctx, |
3310 | 466 code.String(), |
467 &validity, | |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
468 ); err != nil { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
469 key := pgxutils.ReadableError{Err: err}.Error() |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
470 databaseErrors[key] = append(databaseErrors[key], isrs) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
471 unchanged++ |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
472 return continueErr |
3310 | 473 } |
474 | |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
475 // "Upsert" reference water levels |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
476 for _, wl := range []struct { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
477 level **erdms.RisreflevelcodeType |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
478 value **erdms.RisreflevelvalueType |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
479 }{ |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
480 {&dr.Reflevel1code, &dr.Reflevel1value}, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
481 {&dr.Reflevel2code, &dr.Reflevel2value}, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
482 {&dr.Reflevel3code, &dr.Reflevel3value}, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
483 } { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
484 if *wl.level == nil || *wl.value == nil { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
485 continue |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
486 } |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
487 |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
488 var isNtSDepthRef bool |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
489 if err := tx.StmtContext( |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
490 ctx, isNtSDepthRefStmt).QueryRowContext(ctx, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
491 string(**wl.level), |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
492 ).Scan( |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
493 &isNtSDepthRef, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
494 ); err != nil { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
495 return err |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
496 } |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
497 if !isNtSDepthRef { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
498 feedback.Warn( |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
499 "Reference level code '%s' is not in line "+ |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
500 "with the NtS reference_code table", |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
501 string(**wl.level)) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
502 } |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
503 |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
504 if _, err := tx.StmtContext( |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
505 ctx, insertWaterLevelStmt).ExecContext(ctx, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
506 code.CountryCode, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
507 code.LoCode, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
508 code.FairwaySection, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
509 code.Orc, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
510 code.Hectometre, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
511 &validity, |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
512 string(**wl.level), |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
513 int64(**wl.value), |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
514 ); err != nil { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
515 key := pgxutils.ReadableError{Err: err}.Error() |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
516 databaseErrors[key] = append(databaseErrors[key], isrs) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
517 continue |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
518 } |
3310 | 519 } |
520 | |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
521 return tx.Commit() |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
522 }() |
3310 | 523 |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
524 if err != nil && err != continueErr { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
525 return err, nil |
3305
5932f9574493
Follow-up for 3350:e640f51b5a4e (Fix reporting of removed reference water levels).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3302
diff
changeset
|
526 } |
3163
d9903cb34842
Handle failing INSERTs gracefully during gauges import
Tom Gottfried <tom@intevation.de>
parents:
2633
diff
changeset
|
527 } |
1835
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
528 } |
f7b926440449
Waterway gauge import: More stuggling with inserting gauges. Not working, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1829
diff
changeset
|
529 |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
530 if len(invalidISRS) > 0 { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
531 feedback.Error("Invalid ISRS code: '%s'", strings.Join(invalidISRS, "', '")) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
532 } |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
533 |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
534 if len(startEndOrder) > 0 { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
535 feedback.Error("start date not before end date: %s", |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
536 strings.Join(startEndOrder, ", ")) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
537 } |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
538 |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
539 if len(databaseErrors) > 0 { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
540 for err, iris := range databaseErrors { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
541 feedback.Error("%s: %s", err, strings.Join(iris, ", ")) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
542 } |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
543 } |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
544 |
5621
cf1e8ffe1ed5
Fix missing zero point logging in wg import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5620
diff
changeset
|
545 if len(missingZeropoint) > 0 { |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
546 feedback.Error("Missing zeropoint: %s", strings.Join(missingZeropoint, ", ")) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
547 } |
5621
cf1e8ffe1ed5
Fix missing zero point logging in wg import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5620
diff
changeset
|
548 |
5620
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
549 if len(missingObjname) > 0 { |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
550 feedback.Error("Missing objname: %s", strings.Join(missingObjname, ", ")) |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
551 } |
165e77c5736a
Simplified WG import logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5615
diff
changeset
|
552 |
3310 | 553 if len(gauges) == 0 { |
554 return nil, UnchangedError("No gauges returned from ERDMS") | |
555 } | |
556 | |
3551
a5448426e4e2
Only erase gauges that have been requested from ERDMS
Tom Gottfried <tom@intevation.de>
parents:
3529
diff
changeset
|
557 var pgCountries, pgGauges pgtype.VarcharArray |
a5448426e4e2
Only erase gauges that have been requested from ERDMS
Tom Gottfried <tom@intevation.de>
parents:
3529
diff
changeset
|
558 pgCountries.Set(countries) |
3311
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
559 pgGauges.Set(gauges) |
3551
a5448426e4e2
Only erase gauges that have been requested from ERDMS
Tom Gottfried <tom@intevation.de>
parents:
3529
diff
changeset
|
560 obsGauges, err := conn.QueryContext(ctx, |
a5448426e4e2
Only erase gauges that have been requested from ERDMS
Tom Gottfried <tom@intevation.de>
parents:
3529
diff
changeset
|
561 eraseObsoleteGaugesSQL, |
a5448426e4e2
Only erase gauges that have been requested from ERDMS
Tom Gottfried <tom@intevation.de>
parents:
3529
diff
changeset
|
562 &pgCountries, |
a5448426e4e2
Only erase gauges that have been requested from ERDMS
Tom Gottfried <tom@intevation.de>
parents:
3529
diff
changeset
|
563 &pgGauges) |
3311
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
564 if err != nil { |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
565 return nil, err |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
566 } |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
567 defer obsGauges.Close() |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
568 for obsGauges.Next() { |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
569 var isrs string |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
570 if err := obsGauges.Scan(&isrs); err != nil { |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
571 return nil, err |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
572 } |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
573 feedback.Info("Erased %s", isrs) |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
574 unchanged-- |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
575 } |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
576 if err := obsGauges.Err(); err != nil { |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
577 return nil, err |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
578 } |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
579 |
3302
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
580 if unchanged == len(gauges) { |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
581 return nil, UnchangedError("All gauges unchanged") |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
582 } |
ec6163c6687d
'Historicise' gauges on import
Tom Gottfried <tom@intevation.de>
parents:
3286
diff
changeset
|
583 |
3311
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
584 feedback.Info("Importing gauges took %s", |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
585 time.Since(start)) |
0f6b156cff55
Mark gauges as erased if they are missing in ERDMS response
Tom Gottfried <tom@intevation.de>
parents:
3310
diff
changeset
|
586 |
1837
00d63eb9306a
Waterway gauges import: Finalized with re-inserting the reference water levels.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1836
diff
changeset
|
587 return nil, err |
1808
77582da3adb0
Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
588 } |