Mercurial > gemma

annotate pkg/imports/gm.go @ 2018:8a62ce2a5c70 unify_imports

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Moved gaugemeasurement to local package.
author Sascha L. Teichmann <sascha.teichmann@intevation.de>
date Thu, 24 Jan 2019 17:21:14 +0100
parents e89368aec538
children b868cb653c4d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
3 //
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
6 //
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
10 //
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
11 // Author(s):
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
12 // * Raimund Renkert <raimund.renkert@intevation.de>
1694
4a2fad8f57de Imports: Resolved golint issues unrelated to exported symbols commenting.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1688
diff changeset
13
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
14 package imports
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
15
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
16 import (
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
17 "context"
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
18 "database/sql"
1772
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
19 "fmt"
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
20 "strings"
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
21 "time"
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
22
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
23 "gemma.intevation.de/gemma/pkg/common"
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
24 "gemma.intevation.de/gemma/pkg/models"
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
25 "gemma.intevation.de/gemma/pkg/soap/nts"
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
26 )
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
27
1696
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
28 // GaugeMeasurement is an import job to import
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
29 // gauges measurement data from a NtS SOAP service.
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
30 type GaugeMeasurement struct {
1696
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
31 // URL is the URL of the SOAP service.
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
32 URL string `json:"url"`
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
33 // Insecure indicates if HTTPS traffic
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
34 // should validate certificates or not.
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
35 Insecure bool `json:"insecure"`
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
36 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
37
2018
8a62ce2a5c70 Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1981
diff changeset
38 // gaugeMeasurement holds information about a gauge and the latest measurement
8a62ce2a5c70 Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1981
diff changeset
39 type gaugeMeasurement struct {
8a62ce2a5c70 Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1981
diff changeset
40 Gauge models.Isrs
8a62ce2a5c70 Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1981
diff changeset
41 LatestDateIssue time.Time
8a62ce2a5c70 Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1981
diff changeset
42 }
8a62ce2a5c70 Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1981
diff changeset
43
1696
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
44 // GMJobKind is the import queue type identifier.
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
45 const GMJobKind JobKind = "gm"
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
46
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
47 const (
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
48 listGaugesSQL = `
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
49 SELECT
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
50 (location).country_code,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
51 (location).locode,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
52 (location).fairway_section,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
53 (location).orc,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
54 (location).hectometre
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
55 FROM waterway.gauges
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
56 WHERE (location).country_code = users.current_user_country()`
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
57
1978
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
58 // TODO: Currently this statement updates existing data sets. In case we want
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
59 // 'historization' we need to develop an other mechanism to keep existing
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
60 // data.
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
61 insertGMSQL = `
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
62 INSERT INTO waterway.gauge_measurements (
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
63 fk_gauge_id,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
64 measure_date,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
65 sender,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
66 language_code,
1766
e51db2a365a2 Insert country code in gm import.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1761
diff changeset
67 country_code,
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
68 date_issue,
1775
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
69 reference_code,
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
70 water_level,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
71 predicted,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
72 is_waterlevel,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
73 value_min,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
74 value_max,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
75 date_info,
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1776
diff changeset
76 source_organization,
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1776
diff changeset
77 staging_done
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
78 ) VALUES(
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
79 ($1, $2, $3, $4, $5),
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
80 $6,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
81 $7,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
82 $8,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
83 $9,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
84 $10,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
85 $11,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
86 $12,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
87 $13,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
88 $14,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
89 $15,
1766
e51db2a365a2 Insert country code in gm import.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1761
diff changeset
90 $16,
1775
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
91 $17,
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1776
diff changeset
92 $18,
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1776
diff changeset
93 $19
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
94 )
1978
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
95 ON CONFLICT ON CONSTRAINT gauge_measurements_fk_gauge_id_measure_date_staging_done_key
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
96 DO UPDATE SET
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
97 country_code = EXCLUDED.country_code,
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
98 sender = EXCLUDED.sender,
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
99 language_code = EXCLUDED.language_code,
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
100 date_issue = EXCLUDED.date_issue,
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
101 reference_code= EXCLUDED.reference_code,
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
102 water_level = EXCLUDED.water_level,
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
103 predicted = EXCLUDED.predicted,
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
104 is_waterlevel = EXCLUDED.is_waterlevel,
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
105 value_min = EXCLUDED.value_min,
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
106 value_max = EXCLUDED.value_max,
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
107 date_info = EXCLUDED.date_info,
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
108 source_organization = EXCLUDED.source_organization
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
109 RETURNING id
0a8fa6893181 Import: Upsert gauge measurement data to avoid unique contraint violation.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1954
diff changeset
110 `
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
111 )
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
112
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
113 type gmJobCreator struct{}
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
114
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
115 func init() {
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
116 RegisterJobCreator(GMJobKind, gmJobCreator{})
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
117 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
118
1646
a0982c38eac0 Import queue: Implemented email notifications.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1641
diff changeset
119 func (gmJobCreator) Description() string {
a0982c38eac0 Import queue: Implemented email notifications.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1641
diff changeset
120 return "gauge measurements"
a0982c38eac0 Import queue: Implemented email notifications.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1641
diff changeset
121 }
a0982c38eac0 Import queue: Implemented email notifications.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1641
diff changeset
122
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
123 func (gmJobCreator) Create(_ JobKind, data string) (Job, error) {
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
124 gm := new(GaugeMeasurement)
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
125 if err := common.FromJSONString(data, gm); err != nil {
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
126 return nil, err
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
127 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
128 return gm, nil
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
129 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
130
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
131 func (gmJobCreator) Depends() []string {
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
132 return []string{
1664
819f67c31dfb Imports: Stripped schema prefixes from table dependencies.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1646
diff changeset
133 "gauges",
819f67c31dfb Imports: Stripped schema prefixes from table dependencies.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1646
diff changeset
134 "gauge_measurements",
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
135 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
136 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
137
1754
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1696
diff changeset
138 func (gmJobCreator) AutoAccept() bool { return true }
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1696
diff changeset
139
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
140 // StageDone moves the imported gauge measurement out of the staging area.
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
141 // Currently doing nothing.
1676
4407ecaa2192 Imports: Cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1664
diff changeset
142 func (gmJobCreator) StageDone(context.Context, *sql.Tx, int64) error {
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
143 return nil
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
144 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
145
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
146 // CleanUp of a gauge measurement import is a NOP.
1676
4407ecaa2192 Imports: Cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1664
diff changeset
147 func (*GaugeMeasurement) CleanUp() error { return nil }
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
148
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
149 // Do executes the actual bottleneck import.
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
150 func (gm *GaugeMeasurement) Do(
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
151 ctx context.Context,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
152 importID int64,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
153 conn *sql.Conn,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
154 feedback Feedback,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
155 ) (interface{}, error) {
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
156
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
157 // Get available gauges from database for use as filter in SOAP request
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
158 var rows *sql.Rows
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
159
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
160 rows, err := conn.QueryContext(ctx, listGaugesSQL)
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
161 if err != nil {
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
162 return nil, err
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
163 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
164 defer rows.Close()
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
165
2018
8a62ce2a5c70 Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1981
diff changeset
166 gauges := []gaugeMeasurement{}
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
167
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
168 for rows.Next() {
2018
8a62ce2a5c70 Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1981
diff changeset
169 var g gaugeMeasurement
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
170 if err = rows.Scan(
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
171 &g.Gauge.CountryCode,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
172 &g.Gauge.LoCode,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
173 &g.Gauge.FairwaySection,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
174 &g.Gauge.Orc,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
175 &g.Gauge.Hectometre,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
176 ); err != nil {
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
177 return nil, err
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
178 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
179 gauges = append(gauges, g)
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
180 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
181
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
182 if err = rows.Err(); err != nil {
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
183 return nil, err
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
184 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
185
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
186 // TODO get date_issue for selected gauges
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
187 gids, err := gm.doForGM(ctx, gauges, conn, feedback)
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
188 if err != nil {
1761
0145537d5474 Gauge measument import: Die with an error if something went wrong.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1754
diff changeset
189 feedback.Error("Error processing %d gauges: %v", len(gauges), err)
0145537d5474 Gauge measument import: Die with an error if something went wrong.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1754
diff changeset
190 return nil, err
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
191 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
192 if len(gids) == 0 {
1638
2a35bbbb4d93 No new measurement are not an error. Fixed typos.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1637
diff changeset
193 feedback.Info("No new gauge measurements found")
2a35bbbb4d93 No new measurement are not an error. Fixed typos.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1637
diff changeset
194 return nil, nil
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
195 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
196 // TODO: needs to be filled more useful.
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
197 summary := struct {
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
198 GaugeMeasuremets []string `json:"gaugeMeasurements"`
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
199 }{
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
200 GaugeMeasuremets: gids,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
201 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
202 return &summary, err
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
203 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
204
1772
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
205 // rescale returns a scaling function to bring the unit all to cm.
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
206 func rescale(unit string) (func(float32) float32, error) {
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
207
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
208 var scale float32
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
209
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
210 switch strings.ToLower(unit) {
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
211 case "mm":
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
212 scale = 0.1
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
213 case "cm":
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
214 scale = 1.0
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
215 case "dm":
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
216 scale = 10.0
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
217 case "m":
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
218 scale = 100.0
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
219 case "hm":
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
220 scale = 10000.0
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
221 case "km":
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
222 scale = 100000.0
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
223 default:
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
224 return nil, fmt.Errorf("unknown unit '%s'", unit)
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
225 }
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
226
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
227 fn := func(x float32) float32 { return scale * x }
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
228 return fn, nil
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
229 }
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
230
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
231 func (gm *GaugeMeasurement) doForGM(
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
232 ctx context.Context,
2018
8a62ce2a5c70 Moved gaugemeasurement to local package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1981
diff changeset
233 gauges []gaugeMeasurement,
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
234 conn *sql.Conn,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
235 feedback Feedback,
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
236 ) ([]string, error) {
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
237 start := time.Now()
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
238
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
239 client := nts.NewINtSMessageService(gm.URL, gm.Insecure, nil)
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
240
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
241 tx, err := conn.BeginTx(ctx, nil)
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
242 if err != nil {
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
243 return nil, err
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
244 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
245 defer tx.Rollback()
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
246
1981
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
247 mt := nts.Message_type_typeWRM
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
248
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
249 var gids []string
1981
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
250 for _, g := range gauges {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
251
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
252 var idPair []*nts.Id_pair
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
253 isrs := g.Gauge.String()
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
254 isrsID := nts.Isrs_code_type(isrs)
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
255 idPair = append(idPair, &nts.Id_pair{
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
256 Id: &isrsID,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
257 })
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
258
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
259 req := &nts.Get_messages_query{
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
260 Message_type: &mt,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
261 Ids: idPair,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
262 }
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
263
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
264 resp, err := client.Get_messages(req)
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
265 if err != nil {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
266 feedback.Error("%v", err)
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
267 return nil, err
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
268 }
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
269
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
270 if resp.Result_message == nil {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
271 for _, e := range resp.Result_error {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
272 if e != nil {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
273 feedback.Error("No gauge measurements found for %s", g.Gauge.String())
1772
70c4dc694d61 Gauge measurement import: Added a rescale function that brings all numerical values into cm.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1766
diff changeset
274 } else {
1981
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
275 feedback.Error("unknown")
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
276 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
277 }
1981
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
278 continue
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
279 }
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
280 result := resp.Result_message
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
281
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
282 insertStmt, err := tx.PrepareContext(ctx, insertGMSQL)
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
283 if err != nil {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
284 return nil, err
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
285 }
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
286 defer insertStmt.Close()
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
287
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
288 for _, msg := range result {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
289 var gid int64
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
290 feedback.Info("Found measurements for %s", g.Gauge.String())
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
291 for _, wrm := range msg.Wrm {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
292 currIsrs, err := models.IsrsFromString(string(*wrm.Geo_object.Id))
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
293 if err != nil {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
294 feedback.Warn("Invalid ISRS code %v", err)
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
295 continue
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
296 }
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
297 var referenceCode string
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
298 if wrm.Reference_code == nil {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
299 feedback.Info("'Reference_code' not specified. Assuming 'ZPG'")
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
300 referenceCode = "ZPG"
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
301 } else {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
302 referenceCode = string(*wrm.Reference_code)
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
303 }
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
304 for _, measure := range wrm.Measure {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
305 var unit string
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
306 if measure.Unit == nil {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
307 feedback.Info("'Unit' not specified. Assuming 'cm'")
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
308 unit = "cm"
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
309 } else {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
310 unit = string(*measure.Unit)
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
311 }
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
312 convert, err := rescale(unit)
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
313 if err != nil {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
314 return nil, err
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
315 }
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
316 isWaterlevel := *measure.Measure_code == nts.Measure_code_enumWAL
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
317 err = insertStmt.QueryRowContext(
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
318 ctx,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
319 currIsrs.CountryCode,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
320 currIsrs.LoCode,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
321 currIsrs.FairwaySection,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
322 currIsrs.Orc,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
323 currIsrs.Hectometre,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
324 measure.Measuredate,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
325 msg.Identification.From,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
326 msg.Identification.Language_code,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
327 msg.Identification.Country_code,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
328 msg.Identification.Date_issue,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
329 referenceCode,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
330 convert(measure.Value),
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
331 measure.Predicted,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
332 isWaterlevel,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
333 convert(measure.Value_min),
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
334 convert(measure.Value_max),
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
335 msg.Identification.Date_issue,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
336 msg.Identification.Originator,
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
337 true, // staging_done
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
338 ).Scan(&gid)
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
339 if err != nil {
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
340 return nil, err
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
341 }
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
342 }
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
343 feedback.Info("Inserted %d measurements for %s",
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
344 len(wrm.Measure), currIsrs)
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
345 gids = append(gids, currIsrs.String())
e89368aec538 Import: Make one request per gauge for gauge measurement data.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1978
diff changeset
346 }
1637
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
347 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
348 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
349 feedback.Info("Storing gauge measurements took %s", time.Since(start))
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
350 if err = tx.Commit(); err == nil {
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
351 feedback.Info("Import of gauge measurements was successful")
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
352 }
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
353
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
354 return gids, nil
dd31be75ce6d Implemented gauge measurement import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
355 }