Mercurial > gemma
annotate pkg/imports/gm.go @ 2168:b868cb653c4d
Import queue: The job kind parameter when creating a new job from a string was never used.
author | Sascha L. Teichmann <sascha.teichmann@intevation.de> |
---|---|
date | Fri, 08 Feb 2019 15:39:17 +0100 |
parents | 8a62ce2a5c70 |
children | 7c83b5277c1c |
rev | line source |
---|---|
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
1 // This is Free Software under GNU Affero General Public License v >= 3.0 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
2 // without warranty, see README.md and license for details. |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
3 // |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
4 // SPDX-License-Identifier: AGPL-3.0-or-later |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
5 // License-Filename: LICENSES/AGPL-3.0.txt |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
6 // |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
7 // Copyright (C) 2018 by via donau |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
8 // – Österreichische Wasserstraßen-Gesellschaft mbH |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
9 // Software engineering by Intevation GmbH |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
10 // |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
11 // Author(s): |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
12 // * Raimund Renkert <raimund.renkert@intevation.de> |
1694
4a2fad8f57de
Imports: Resolved golint issues unrelated to exported symbols commenting.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
1688
diff
changeset
|
13 |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
14 package imports |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
15 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
16 import ( |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
17 "context" |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
18 "database/sql" |
1772
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
19 "fmt" |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
20 "strings" |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
21 "time" |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
22 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
23 "gemma.intevation.de/gemma/pkg/common" |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
24 "gemma.intevation.de/gemma/pkg/models" |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
25 "gemma.intevation.de/gemma/pkg/soap/nts" |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
26 ) |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
27 |
1696
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
28 // GaugeMeasurement is an import job to import |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
29 // gauges measurement data from a NtS SOAP service. |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
30 type GaugeMeasurement struct { |
1696
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
31 // URL is the URL of the SOAP service. |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
32 URL string `json:"url"` |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
33 // Insecure indicates if HTTPS traffic |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
34 // should validate certificates or not. |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
35 Insecure bool `json:"insecure"` |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
36 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
37 |
2018
8a62ce2a5c70
Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1981
diff
changeset
|
38 // gaugeMeasurement holds information about a gauge and the latest measurement |
8a62ce2a5c70
Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1981
diff
changeset
|
39 type gaugeMeasurement struct { |
8a62ce2a5c70
Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1981
diff
changeset
|
40 Gauge models.Isrs |
8a62ce2a5c70
Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1981
diff
changeset
|
41 LatestDateIssue time.Time |
8a62ce2a5c70
Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1981
diff
changeset
|
42 } |
8a62ce2a5c70
Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1981
diff
changeset
|
43 |
1696
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
44 // GMJobKind is the import queue type identifier. |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
45 const GMJobKind JobKind = "gm" |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
46 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
47 const ( |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
48 listGaugesSQL = ` |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
49 SELECT |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
50 (location).country_code, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
51 (location).locode, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
52 (location).fairway_section, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
53 (location).orc, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
54 (location).hectometre |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
55 FROM waterway.gauges |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
56 WHERE (location).country_code = users.current_user_country()` |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
57 |
1978
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
58 // TODO: Currently this statement updates existing data sets. In case we want |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
59 // 'historization' we need to develop an other mechanism to keep existing |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
60 // data. |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
61 insertGMSQL = ` |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
62 INSERT INTO waterway.gauge_measurements ( |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
63 fk_gauge_id, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
64 measure_date, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
65 sender, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
66 language_code, |
1766
e51db2a365a2
Insert country code in gm import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1761
diff
changeset
|
67 country_code, |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
68 date_issue, |
1775
fcb0106ec510
Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1772
diff
changeset
|
69 reference_code, |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
70 water_level, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
71 predicted, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
72 is_waterlevel, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
73 value_min, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
74 value_max, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
75 date_info, |
1780
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1776
diff
changeset
|
76 source_organization, |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1776
diff
changeset
|
77 staging_done |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
78 ) VALUES( |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
79 ($1, $2, $3, $4, $5), |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
80 $6, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
81 $7, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
82 $8, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
83 $9, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
84 $10, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
85 $11, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
86 $12, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
87 $13, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
88 $14, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
89 $15, |
1766
e51db2a365a2
Insert country code in gm import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1761
diff
changeset
|
90 $16, |
1775
fcb0106ec510
Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1772
diff
changeset
|
91 $17, |
1780
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1776
diff
changeset
|
92 $18, |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1776
diff
changeset
|
93 $19 |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
94 ) |
1978
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
95 ON CONFLICT ON CONSTRAINT gauge_measurements_fk_gauge_id_measure_date_staging_done_key |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
96 DO UPDATE SET |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
97 country_code = EXCLUDED.country_code, |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
98 sender = EXCLUDED.sender, |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
99 language_code = EXCLUDED.language_code, |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
100 date_issue = EXCLUDED.date_issue, |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
101 reference_code= EXCLUDED.reference_code, |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
102 water_level = EXCLUDED.water_level, |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
103 predicted = EXCLUDED.predicted, |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
104 is_waterlevel = EXCLUDED.is_waterlevel, |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
105 value_min = EXCLUDED.value_min, |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
106 value_max = EXCLUDED.value_max, |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
107 date_info = EXCLUDED.date_info, |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
108 source_organization = EXCLUDED.source_organization |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
109 RETURNING id |
0a8fa6893181
Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1954
diff
changeset
|
110 ` |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
111 ) |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
112 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
113 type gmJobCreator struct{} |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
114 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
115 func init() { |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
116 RegisterJobCreator(GMJobKind, gmJobCreator{}) |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
117 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
118 |
1646
a0982c38eac0
Import queue: Implemented email notifications.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1641
diff
changeset
|
119 func (gmJobCreator) Description() string { |
a0982c38eac0
Import queue: Implemented email notifications.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1641
diff
changeset
|
120 return "gauge measurements" |
a0982c38eac0
Import queue: Implemented email notifications.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1641
diff
changeset
|
121 } |
a0982c38eac0
Import queue: Implemented email notifications.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1641
diff
changeset
|
122 |
2168
b868cb653c4d
Import queue: The job kind parameter when creating a new job from a string was never used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2018
diff
changeset
|
123 func (gmJobCreator) Create(data string) (Job, error) { |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
124 gm := new(GaugeMeasurement) |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
125 if err := common.FromJSONString(data, gm); err != nil { |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
126 return nil, err |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
127 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
128 return gm, nil |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
129 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
130 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
131 func (gmJobCreator) Depends() []string { |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
132 return []string{ |
1664
819f67c31dfb
Imports: Stripped schema prefixes from table dependencies.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1646
diff
changeset
|
133 "gauges", |
819f67c31dfb
Imports: Stripped schema prefixes from table dependencies.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1646
diff
changeset
|
134 "gauge_measurements", |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
135 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
136 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
137 |
1754
807569b08513
Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1696
diff
changeset
|
138 func (gmJobCreator) AutoAccept() bool { return true } |
807569b08513
Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1696
diff
changeset
|
139 |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
140 // StageDone moves the imported gauge measurement out of the staging area. |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
141 // Currently doing nothing. |
1676
4407ecaa2192
Imports: Cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1664
diff
changeset
|
142 func (gmJobCreator) StageDone(context.Context, *sql.Tx, int64) error { |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
143 return nil |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
144 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
145 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
146 // CleanUp of a gauge measurement import is a NOP. |
1676
4407ecaa2192
Imports: Cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1664
diff
changeset
|
147 func (*GaugeMeasurement) CleanUp() error { return nil } |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
148 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
149 // Do executes the actual bottleneck import. |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
150 func (gm *GaugeMeasurement) Do( |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
151 ctx context.Context, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
152 importID int64, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
153 conn *sql.Conn, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
154 feedback Feedback, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
155 ) (interface{}, error) { |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
156 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
157 // Get available gauges from database for use as filter in SOAP request |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
158 var rows *sql.Rows |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
159 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
160 rows, err := conn.QueryContext(ctx, listGaugesSQL) |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
161 if err != nil { |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
162 return nil, err |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
163 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
164 defer rows.Close() |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
165 |
2018
8a62ce2a5c70
Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1981
diff
changeset
|
166 gauges := []gaugeMeasurement{} |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
167 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
168 for rows.Next() { |
2018
8a62ce2a5c70
Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1981
diff
changeset
|
169 var g gaugeMeasurement |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
170 if err = rows.Scan( |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
171 &g.Gauge.CountryCode, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
172 &g.Gauge.LoCode, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
173 &g.Gauge.FairwaySection, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
174 &g.Gauge.Orc, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
175 &g.Gauge.Hectometre, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
176 ); err != nil { |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
177 return nil, err |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
178 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
179 gauges = append(gauges, g) |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
180 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
181 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
182 if err = rows.Err(); err != nil { |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
183 return nil, err |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
184 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
185 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
186 // TODO get date_issue for selected gauges |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
187 gids, err := gm.doForGM(ctx, gauges, conn, feedback) |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
188 if err != nil { |
1761
0145537d5474
Gauge measument import: Die with an error if something went wrong.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1754
diff
changeset
|
189 feedback.Error("Error processing %d gauges: %v", len(gauges), err) |
0145537d5474
Gauge measument import: Die with an error if something went wrong.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1754
diff
changeset
|
190 return nil, err |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
191 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
192 if len(gids) == 0 { |
1638
2a35bbbb4d93
No new measurement are not an error. Fixed typos.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1637
diff
changeset
|
193 feedback.Info("No new gauge measurements found") |
2a35bbbb4d93
No new measurement are not an error. Fixed typos.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1637
diff
changeset
|
194 return nil, nil |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
195 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
196 // TODO: needs to be filled more useful. |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
197 summary := struct { |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
198 GaugeMeasuremets []string `json:"gaugeMeasurements"` |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
199 }{ |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
200 GaugeMeasuremets: gids, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
201 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
202 return &summary, err |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
203 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
204 |
1772
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
205 // rescale returns a scaling function to bring the unit all to cm. |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
206 func rescale(unit string) (func(float32) float32, error) { |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
207 |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
208 var scale float32 |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
209 |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
210 switch strings.ToLower(unit) { |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
211 case "mm": |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
212 scale = 0.1 |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
213 case "cm": |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
214 scale = 1.0 |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
215 case "dm": |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
216 scale = 10.0 |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
217 case "m": |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
218 scale = 100.0 |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
219 case "hm": |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
220 scale = 10000.0 |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
221 case "km": |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
222 scale = 100000.0 |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
223 default: |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
224 return nil, fmt.Errorf("unknown unit '%s'", unit) |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
225 } |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
226 |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
227 fn := func(x float32) float32 { return scale * x } |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
228 return fn, nil |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
229 } |
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
230 |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
231 func (gm *GaugeMeasurement) doForGM( |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
232 ctx context.Context, |
2018
8a62ce2a5c70
Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1981
diff
changeset
|
233 gauges []gaugeMeasurement, |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
234 conn *sql.Conn, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
235 feedback Feedback, |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
236 ) ([]string, error) { |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
237 start := time.Now() |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
238 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
239 client := nts.NewINtSMessageService(gm.URL, gm.Insecure, nil) |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
240 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
241 tx, err := conn.BeginTx(ctx, nil) |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
242 if err != nil { |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
243 return nil, err |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
244 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
245 defer tx.Rollback() |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
246 |
1981
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
247 mt := nts.Message_type_typeWRM |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
248 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
249 var gids []string |
1981
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
250 for _, g := range gauges { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
251 |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
252 var idPair []*nts.Id_pair |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
253 isrs := g.Gauge.String() |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
254 isrsID := nts.Isrs_code_type(isrs) |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
255 idPair = append(idPair, &nts.Id_pair{ |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
256 Id: &isrsID, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
257 }) |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
258 |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
259 req := &nts.Get_messages_query{ |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
260 Message_type: &mt, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
261 Ids: idPair, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
262 } |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
263 |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
264 resp, err := client.Get_messages(req) |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
265 if err != nil { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
266 feedback.Error("%v", err) |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
267 return nil, err |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
268 } |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
269 |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
270 if resp.Result_message == nil { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
271 for _, e := range resp.Result_error { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
272 if e != nil { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
273 feedback.Error("No gauge measurements found for %s", g.Gauge.String()) |
1772
70c4dc694d61
Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1766
diff
changeset
|
274 } else { |
1981
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
275 feedback.Error("unknown") |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
276 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
277 } |
1981
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
278 continue |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
279 } |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
280 result := resp.Result_message |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
281 |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
282 insertStmt, err := tx.PrepareContext(ctx, insertGMSQL) |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
283 if err != nil { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
284 return nil, err |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
285 } |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
286 defer insertStmt.Close() |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
287 |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
288 for _, msg := range result { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
289 var gid int64 |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
290 feedback.Info("Found measurements for %s", g.Gauge.String()) |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
291 for _, wrm := range msg.Wrm { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
292 currIsrs, err := models.IsrsFromString(string(*wrm.Geo_object.Id)) |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
293 if err != nil { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
294 feedback.Warn("Invalid ISRS code %v", err) |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
295 continue |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
296 } |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
297 var referenceCode string |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
298 if wrm.Reference_code == nil { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
299 feedback.Info("'Reference_code' not specified. Assuming 'ZPG'") |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
300 referenceCode = "ZPG" |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
301 } else { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
302 referenceCode = string(*wrm.Reference_code) |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
303 } |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
304 for _, measure := range wrm.Measure { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
305 var unit string |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
306 if measure.Unit == nil { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
307 feedback.Info("'Unit' not specified. Assuming 'cm'") |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
308 unit = "cm" |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
309 } else { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
310 unit = string(*measure.Unit) |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
311 } |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
312 convert, err := rescale(unit) |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
313 if err != nil { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
314 return nil, err |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
315 } |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
316 isWaterlevel := *measure.Measure_code == nts.Measure_code_enumWAL |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
317 err = insertStmt.QueryRowContext( |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
318 ctx, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
319 currIsrs.CountryCode, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
320 currIsrs.LoCode, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
321 currIsrs.FairwaySection, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
322 currIsrs.Orc, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
323 currIsrs.Hectometre, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
324 measure.Measuredate, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
325 msg.Identification.From, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
326 msg.Identification.Language_code, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
327 msg.Identification.Country_code, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
328 msg.Identification.Date_issue, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
329 referenceCode, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
330 convert(measure.Value), |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
331 measure.Predicted, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
332 isWaterlevel, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
333 convert(measure.Value_min), |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
334 convert(measure.Value_max), |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
335 msg.Identification.Date_issue, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
336 msg.Identification.Originator, |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
337 true, // staging_done |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
338 ).Scan(&gid) |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
339 if err != nil { |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
340 return nil, err |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
341 } |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
342 } |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
343 feedback.Info("Inserted %d measurements for %s", |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
344 len(wrm.Measure), currIsrs) |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
345 gids = append(gids, currIsrs.String()) |
e89368aec538
Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1978
diff
changeset
|
346 } |
1637
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
347 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
348 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
349 feedback.Info("Storing gauge measurements took %s", time.Since(start)) |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
350 if err = tx.Commit(); err == nil { |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
351 feedback.Info("Import of gauge measurements was successful") |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
352 } |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
353 |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
354 return gids, nil |
dd31be75ce6d
Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
355 } |