Mercurial > gemma
annotate pkg/imports/agm.go @ 2173:1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
author | Sascha L. Teichmann <sascha.teichmann@intevation.de> |
---|---|
date | Fri, 08 Feb 2019 17:59:24 +0100 |
parents | b868cb653c4d |
children | 7c83b5277c1c |
rev | line source |
---|---|
1741
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
1 // This is Free Software under GNU Affero General Public License v >= 3.0 |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
2 // without warranty, see README.md and license for details. |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
3 // |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
4 // SPDX-License-Identifier: AGPL-3.0-or-later |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
5 // License-Filename: LICENSES/AGPL-3.0.txt |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
6 // |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
7 // Copyright (C) 2018 by via donau |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
8 // – Österreichische Wasserstraßen-Gesellschaft mbH |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
9 // Software engineering by Intevation GmbH |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
10 // |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
11 // Author(s): |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de> |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
13 |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
14 package imports |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
15 |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
16 import ( |
1743
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
17 "bufio" |
1741
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
18 "context" |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
19 "database/sql" |
1743
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
20 "encoding/csv" |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
21 "encoding/json" |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
22 "fmt" |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
23 "io" |
2173
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
24 "math" |
1741
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
25 "os" |
1743
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
26 "path/filepath" |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
27 "strconv" |
1775
fcb0106ec510
Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1772
diff
changeset
|
28 "strings" |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
29 "time" |
1741
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
30 |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
31 "gemma.intevation.de/gemma/pkg/common" |
1938
206b1dd31112
Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1781
diff
changeset
|
32 "gemma.intevation.de/gemma/pkg/misc" |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
33 "gemma.intevation.de/gemma/pkg/models" |
1741
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
34 ) |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
35 |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
36 type ApprovedGaugeMeasurements struct { |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
37 Dir string `json:"dir"` |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
38 } |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
39 |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
40 // GMAPJobKind is the unique name of an approved gauge measurements import job. |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
41 const AGMJobKind JobKind = "agm" |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
42 |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
43 type agmJobCreator struct{} |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
44 |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
45 func init() { |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
46 RegisterJobCreator(AGMJobKind, agmJobCreator{}) |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
47 } |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
48 |
1754
807569b08513
Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1743
diff
changeset
|
49 func (agmJobCreator) AutoAccept() bool { return false } |
807569b08513
Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1743
diff
changeset
|
50 |
1741
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
51 func (agmJobCreator) Description() string { |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
52 return "approved gauge measurements" |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
53 } |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
54 |
2168
b868cb653c4d
Import queue: The job kind parameter when creating a new job from a string was never used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2120
diff
changeset
|
55 func (agmJobCreator) Create(data string) (Job, error) { |
1741
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
56 agm := new(ApprovedGaugeMeasurements) |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
57 if err := common.FromJSONString(data, agm); err != nil { |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
58 return nil, err |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
59 } |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
60 return agm, nil |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
61 } |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
62 |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
63 func (agmJobCreator) Depends() []string { |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
64 return []string{ |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
65 "gauges", |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
66 "gauge_measurements", |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
67 } |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
68 } |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
69 |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
70 const ( |
1780
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
71 // delete the old and keep the new measures. |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
72 agmStageDoneDeleteSQL = ` |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
73 WITH staged AS ( |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
74 SELECT key |
1995
59055c8301df
Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents:
1938
diff
changeset
|
75 FROM import.track_imports |
1780
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
76 WHERE import_id = $1 AND |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
77 relation = 'waterway.gauge_measurements'::regclass |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
78 ), |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
79 to_delete AS ( |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
80 SELECT o.id AS id |
1781
0a53c33bc7b2
Fixed broken SQL statement (once again).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1780
diff
changeset
|
81 FROM waterway.gauge_measurements o |
0a53c33bc7b2
Fixed broken SQL statement (once again).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1780
diff
changeset
|
82 JOIN waterway.gauge_measurements n |
1780
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
83 ON n.fk_gauge_id = o.fk_gauge_id AND n.measure_date = o.measure_date |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
84 WHERE n.id IN (SELECT key FROM staged) |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
85 AND o.id NOT IN (SELECT key FROM staged) |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
86 ) |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
87 DELETE FROM waterway.gauge_measurements WHERE id IN (SELECT id from to_delete)` |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
88 |
1741
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
89 agmStageDoneSQL = ` |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
90 UPDATE waterway.gauge_measurements SET staging_done = true |
1780
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
91 WHERE id IN ( |
1995
59055c8301df
Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents:
1938
diff
changeset
|
92 SELECT key FROM import.track_imports |
1741
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
93 WHERE import_id = $1 AND |
1780
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
94 relation = 'waterway.gauge_measurements'::regclass)` |
1741
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
95 ) |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
96 |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
97 func (agmJobCreator) StageDone( |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
98 ctx context.Context, |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
99 tx *sql.Tx, |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
100 id int64, |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
101 ) error { |
1780
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
102 _, err := tx.ExecContext(ctx, agmStageDoneDeleteSQL, id) |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
103 if err == nil { |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
104 _, err = tx.ExecContext(ctx, agmStageDoneSQL, id) |
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
105 } |
1741
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
106 return err |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
107 } |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
108 |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
109 // CleanUp removes the folder containing the CSV file with the |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
110 // the approved gauge measurements. |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
111 func (agm *ApprovedGaugeMeasurements) CleanUp() error { |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
112 return os.RemoveAll(agm.Dir) |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
113 } |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
114 |
1938
206b1dd31112
Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1781
diff
changeset
|
115 var guessDate = misc.TimeGuesser([]string{ |
206b1dd31112
Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1781
diff
changeset
|
116 "02.01.2006 15:04", |
206b1dd31112
Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1781
diff
changeset
|
117 "2006-01-02T15:04:05-07:00", |
206b1dd31112
Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1781
diff
changeset
|
118 }).Guess |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
119 |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
120 type timetz struct{ time.Time } |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
121 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
122 func (ttz *timetz) MarshalJSON() ([]byte, error) { |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
123 return json.Marshal(ttz.Time.Format("2006-01-02T15:04:05-07:00")) |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
124 } |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
125 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
126 type agmLine struct { |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
127 CountryCode string `json:"country-code"` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
128 Sender string `json:"sender"` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
129 LanguageCode string `json:"language-code"` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
130 DateIssue timetz `json:"date-issue"` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
131 ReferenceCode string `json:"reference-code"` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
132 WaterLevel float64 `json:"water-level"` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
133 Predicted bool `json:"predicted"` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
134 ValueMin *float64 `json:"value-min"` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
135 ValueMax *float64 `json:"value-max"` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
136 DateInfo timetz `json:"date-info"` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
137 SourceOrganization string `json:"source-organization"` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
138 } |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
139 |
2173
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
140 func (a *agmLine) hasDiff(b *agmLine) bool { |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
141 const eps = 0.00001 |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
142 fdiff := func(x, y *float64) bool { |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
143 if x == nil && y == nil { |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
144 return false |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
145 } |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
146 if (x == nil && y != nil) || (x != nil && y == nil) { |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
147 return true |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
148 } |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
149 return math.Abs(*x-*y) > eps |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
150 } |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
151 return a.CountryCode != b.CountryCode || |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
152 a.Sender != b.Sender || |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
153 a.LanguageCode != b.LanguageCode || |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
154 !a.DateIssue.Time.Equal(b.DateIssue.Time) || |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
155 a.ReferenceCode != b.ReferenceCode || |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
156 math.Abs(a.WaterLevel-b.WaterLevel) > eps || |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
157 a.Predicted != b.Predicted || |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
158 fdiff(a.ValueMin, b.ValueMin) || |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
159 fdiff(a.ValueMax, b.ValueMax) || |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
160 !a.DateInfo.Time.Equal(b.DateInfo.Time) || |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
161 a.SourceOrganization != b.SourceOrganization |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
162 } |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
163 |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
164 type agmSummaryEntry struct { |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
165 FKGaugeID models.Isrs `json:"fk-gauge-id"` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
166 MeasureDate timetz `json:"measure-date"` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
167 Versions []*agmLine `json:"versions"` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
168 } |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
169 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
170 const ( |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
171 agmSelectSQL = ` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
172 SELECT |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
173 id, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
174 country_code, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
175 sender, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
176 language_code, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
177 date_issue, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
178 reference_code, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
179 water_level, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
180 predicted, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
181 value_min, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
182 value_max, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
183 date_info, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
184 source_organization |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
185 FROM waterway.gauge_measurements |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
186 WHERE |
2120
817cd8b89a86
Approved gauge measuments import: Give type hints when insertinf/querying isrs location codes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2105
diff
changeset
|
187 fk_gauge_id = ($1::char(2), $2::char(3), $3::char(5), $4::char(5), $5::int) AND |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
188 measure_date = $6 AND staging_done` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
189 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
190 agmInsertSQL = ` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
191 INSERT INTO waterway.gauge_measurements ( |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
192 fk_gauge_id, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
193 measure_date, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
194 country_code, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
195 sender, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
196 language_code, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
197 date_issue, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
198 reference_code, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
199 water_level, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
200 predicted, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
201 value_min, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
202 value_max, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
203 date_info, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
204 source_organization, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
205 is_waterlevel, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
206 staging_done |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
207 ) VALUES( |
2120
817cd8b89a86
Approved gauge measuments import: Give type hints when insertinf/querying isrs location codes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2105
diff
changeset
|
208 ($1::char(2), $2::char(3), $3::char(5), $4::char(5), $5::int), |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
209 $6, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
210 $7, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
211 $8, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
212 $9, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
213 $10, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
214 $11, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
215 $12, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
216 $13, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
217 $14, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
218 $15, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
219 $16, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
220 $17, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
221 true, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
222 false |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
223 ) |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
224 RETURNING id` |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
225 ) |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
226 |
1741
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
227 // Do executes the actual approved gauge measurements import. |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
228 func (agm *ApprovedGaugeMeasurements) Do( |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
229 ctx context.Context, |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
230 importID int64, |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
231 conn *sql.Conn, |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
232 feedback Feedback, |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
233 ) (interface{}, error) { |
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
234 |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
235 start := time.Now() |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
236 |
1743
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
237 f, err := os.Open(filepath.Join(agm.Dir, "agm.csv")) |
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
238 if err != nil { |
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
239 return nil, err |
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
240 } |
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
241 defer f.Close() |
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
242 |
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
243 r := csv.NewReader(bufio.NewReader(f)) |
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
244 r.Comma = ';' |
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
245 r.ReuseRecord = true |
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
246 |
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
247 headers, err := r.Read() |
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
248 if err != nil { |
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
249 return nil, err |
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
250 } |
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
251 |
2098
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
252 var ( |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
253 fkGaugeIDIdx = -1 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
254 measureDateIdx = -1 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
255 fromIdx = -1 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
256 languageCodeIdx = -1 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
257 countryCodeIdx = -1 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
258 dateIssueIdx = -1 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
259 referenceCodeIdx = -1 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
260 valueIdx = -1 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
261 predictedIdx = -1 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
262 valueMinIdx = -1 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
263 valueMaxIdx = -1 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
264 dateInfoIdx = -1 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
265 originatorIdx = -1 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
266 unitIdx = -1 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
267 ) |
1775
fcb0106ec510
Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1772
diff
changeset
|
268 |
2098
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
269 headerFields := []struct { |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
270 idx *int |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
271 name string |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
272 }{ |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
273 {&fkGaugeIDIdx, "fk_gauge_id"}, |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
274 {&measureDateIdx, "measure_date"}, |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
275 {&fromIdx, "from"}, // "sender", |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
276 {&languageCodeIdx, "language_code"}, |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
277 {&countryCodeIdx, "country_code"}, |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
278 {&dateIssueIdx, "date_issue"}, |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
279 {&referenceCodeIdx, "reference_code"}, |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
280 {&valueIdx, "value"}, // "water_level", |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
281 {&predictedIdx, "predicted"}, |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
282 // "is_waterlevel", |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
283 {&valueMinIdx, "value_min"}, |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
284 {&valueMaxIdx, "value_max"}, |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
285 {&dateInfoIdx, "date_info"}, |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
286 {&originatorIdx, "originator"}, // "source_organization", |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
287 {&unitIdx, "unit"}, |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
288 } |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
289 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
290 nextHeader: |
1743
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
291 for i, f := range headers { |
2098
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
292 h := strings.Replace(strings.ToLower( |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
293 strings.TrimSpace(f)), " ", "_", -1) |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
294 |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
295 for j := range headerFields { |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
296 if headerFields[j].name == h { |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
297 if *headerFields[j].idx != -1 { |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
298 return nil, fmt.Errorf( |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
299 "There is more than one column namend '%s'", h) |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
300 } |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
301 *headerFields[j].idx = i |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
302 continue nextHeader |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
303 } |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
304 } |
1775
fcb0106ec510
Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1772
diff
changeset
|
305 } |
fcb0106ec510
Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1772
diff
changeset
|
306 |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
307 var missing []string |
2098
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
308 for i := range headerFields { |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
309 if headerFields[i].name != "unit" && *headerFields[i].idx == -1 { |
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
310 missing = append(missing, headerFields[i].name) |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
311 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
312 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
313 if len(missing) > 0 { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
314 return nil, fmt.Errorf("Missing columns: %s", strings.Join(missing, ", ")) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
315 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
316 |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
317 inCm, _ := rescale("cm") |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
318 scaler := func(row []string) (func(float32) float32, error) { |
2098
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
319 if unitIdx == -1 { |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
320 return inCm, nil |
1775
fcb0106ec510
Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1772
diff
changeset
|
321 } |
2098
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
322 unit := row[unitIdx] |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
323 if unit == "cm" { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
324 return inCm, nil |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
325 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
326 s, err := rescale(unit) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
327 return s, err |
1743
85d0f017fbee
Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1741
diff
changeset
|
328 } |
1741
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
329 |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
330 tx, err := conn.BeginTx(ctx, nil) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
331 if err != nil { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
332 return nil, err |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
333 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
334 defer tx.Rollback() |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
335 |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
336 selectStmt, err := tx.PrepareContext(ctx, agmSelectSQL) |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
337 if err != nil { |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
338 return nil, err |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
339 } |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
340 defer selectStmt.Close() |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
341 insertStmt, err := tx.PrepareContext(ctx, agmInsertSQL) |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
342 if err != nil { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
343 return nil, err |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
344 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
345 defer insertStmt.Close() |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
346 trackStmt, err := tx.PrepareContext(ctx, trackImportSQL) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
347 if err != nil { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
348 return nil, err |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
349 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
350 defer trackStmt.Close() |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
351 |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
352 entries := []*agmSummaryEntry{} |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
353 |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
354 lines: |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
355 for line := 1; ; line++ { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
356 |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
357 row, err := r.Read() |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
358 switch { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
359 case err == io.EOF || len(row) == 0: |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
360 break lines |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
361 case err != nil: |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
362 return nil, fmt.Errorf("CSV parsing failed: %v", err) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
363 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
364 convert, err := scaler(row) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
365 if err != nil { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
366 return nil, fmt.Errorf("line %d: %v", line, err) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
367 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
368 |
2098
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
369 gids := row[fkGaugeIDIdx] |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
370 gid, err := models.IsrsFromString(gids) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
371 if err != nil { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
372 return nil, fmt.Errorf("Invalid ISRS code line %d: %v", line, err) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
373 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
374 |
2098
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
375 md, err := guessDate(row[measureDateIdx]) |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
376 if err != nil { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
377 return nil, fmt.Errorf("Invalid 'measure_date' line %d: %v", line, err) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
378 } |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
379 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
380 var ( |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
381 oldID int64 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
382 oldCountryCode string |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
383 oldSender string |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
384 oldLanguageCode string |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
385 oldDateIssue time.Time |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
386 oldReferenceCode string |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
387 oldValue float64 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
388 oldPredicted bool |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
389 oldValueMin sql.NullFloat64 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
390 oldValueMax sql.NullFloat64 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
391 oldDateInfo time.Time |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
392 oldSourceOrganization string |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
393 ) |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
394 |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
395 err = selectStmt.QueryRowContext( |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
396 ctx, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
397 gid.CountryCode, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
398 gid.LoCode, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
399 gid.FairwaySection, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
400 gid.Orc, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
401 gid.Hectometre, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
402 md, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
403 ).Scan( |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
404 &oldID, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
405 &oldCountryCode, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
406 &oldSender, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
407 &oldLanguageCode, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
408 &oldDateIssue, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
409 &oldReferenceCode, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
410 &oldValue, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
411 &oldPredicted, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
412 &oldValueMin, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
413 &oldValueMax, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
414 &oldDateInfo, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
415 &oldSourceOrganization, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
416 ) |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
417 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
418 var newEntry bool |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
419 switch { |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
420 case err == sql.ErrNoRows: |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
421 // Complete new one |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
422 newEntry = true |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
423 case err != nil: |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
424 return nil, err |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
425 } |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
426 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
427 newSender := row[fromIdx] |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
428 newLanguageCode := row[languageCodeIdx] |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
429 newCountryCode := row[countryCodeIdx] |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
430 |
2098
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
431 dis, err := guessDate(row[dateIssueIdx]) |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
432 if err != nil { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
433 return nil, fmt.Errorf("Invalid 'date_issue' line %d: %v", line, err) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
434 } |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
435 newDateIssue := dis |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
436 |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
437 newReferenceCode := row[referenceCodeIdx] |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
438 |
2098
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
439 value, err := strconv.ParseFloat(row[valueIdx], 32) |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
440 if err != nil { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
441 return nil, fmt.Errorf("Invalid 'value' line %d: %v", line, err) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
442 } |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
443 newValue := float64(convert(float32(value))) |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
444 |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
445 newPredicted := strings.ToLower(row[predictedIdx]) == "true" |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
446 |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
447 var newValueMin sql.NullFloat64 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
448 if vm := row[valueMinIdx]; vm != "" { |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
449 valueMin, err := strconv.ParseFloat(vm, 32) |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
450 if err != nil { |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
451 return nil, fmt.Errorf("Invalid 'value_min' line %d: %v", line, err) |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
452 } |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
453 newValueMin = sql.NullFloat64{ |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
454 Float64: float64(convert(float32(valueMin))), |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
455 Valid: true, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
456 } |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
457 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
458 |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
459 var newValueMax sql.NullFloat64 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
460 if vm := row[valueMaxIdx]; vm != "" { |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
461 valueMax, err := strconv.ParseFloat(vm, 32) |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
462 if err != nil { |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
463 return nil, fmt.Errorf("Invalid 'value_max' line %d: %v", line, err) |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
464 } |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
465 newValueMax = sql.NullFloat64{ |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
466 Float64: float64(convert(float32(valueMax))), |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
467 Valid: true, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
468 } |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
469 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
470 |
2098
8a986d80e1c6
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
471 din, err := guessDate(row[dateInfoIdx]) |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
472 if err != nil { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
473 return nil, fmt.Errorf("Invalid 'date_info' line %d: %v", line, err) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
474 } |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
475 newDateInfo := din |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
476 |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
477 newSourceOrganization := row[originatorIdx] |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
478 |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
479 var newID int64 |
1780
48791416bea5
(Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1779
diff
changeset
|
480 |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
481 if err := insertStmt.QueryRowContext( |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
482 ctx, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
483 gid.CountryCode, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
484 gid.LoCode, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
485 gid.FairwaySection, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
486 gid.Orc, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
487 gid.Hectometre, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
488 md, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
489 newCountryCode, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
490 newSender, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
491 newLanguageCode, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
492 newDateIssue, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
493 newReferenceCode, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
494 newValue, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
495 newPredicted, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
496 newValueMin, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
497 newValueMax, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
498 newDateInfo, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
499 newSourceOrganization, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
500 ).Scan(&newID); err != nil { |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
501 return nil, err |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
502 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
503 if _, err := trackStmt.ExecContext( |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
504 ctx, importID, "waterway.gauge_measurements", newID, |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
505 ); err != nil { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
506 return nil, err |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
507 } |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
508 |
2173
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
509 n := newAGMLine( |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
510 newCountryCode, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
511 newSender, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
512 newLanguageCode, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
513 newDateIssue, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
514 newReferenceCode, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
515 newValue, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
516 newPredicted, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
517 newValueMin, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
518 newValueMax, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
519 newDateInfo, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
520 newSourceOrganization, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
521 ) |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
522 |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
523 ase := &agmSummaryEntry{ |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
524 FKGaugeID: *gid, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
525 MeasureDate: timetz{md}, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
526 } |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
527 |
2173
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
528 if newEntry { |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
529 ase.Versions = []*agmLine{n} |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
530 } else { |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
531 o := newAGMLine( |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
532 oldCountryCode, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
533 oldSender, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
534 oldLanguageCode, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
535 oldDateIssue, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
536 oldReferenceCode, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
537 oldValue, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
538 oldPredicted, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
539 oldValueMin, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
540 oldValueMax, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
541 oldDateInfo, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
542 oldSourceOrganization, |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
543 ) |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
544 // Ignore if there is no diff. |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
545 if !n.hasDiff(o) { |
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
546 continue |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
547 } |
2173
1b1cb6cbfbf0
Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
548 ase.Versions = []*agmLine{o, n} |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
549 } |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
550 entries = append(entries, ase) |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
551 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
552 |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
553 if err := tx.Commit(); err != nil { |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
554 return nil, fmt.Errorf("Commit failed: %v", err) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
555 } |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
556 |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
557 feedback.Info("Importing approved gauge measurements took %s", |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
558 time.Since(start)) |
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
559 |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
560 return entries, nil |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
561 } |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
562 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
563 func newAGMLine( |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
564 countryCode string, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
565 sender string, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
566 languageCode string, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
567 dateIssue time.Time, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
568 referenceCode string, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
569 waterLevel float64, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
570 predicted bool, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
571 valueMin sql.NullFloat64, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
572 valueMax sql.NullFloat64, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
573 dateInfo time.Time, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
574 sourceOrganization string, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
575 ) *agmLine { |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
576 nilFloat := func(v sql.NullFloat64) *float64 { |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
577 var p *float64 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
578 if v.Valid { |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
579 p = &v.Float64 |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
580 } |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
581 return p |
1778
164b46ebd60d
Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1775
diff
changeset
|
582 } |
2105
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
583 return &agmLine{ |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
584 CountryCode: countryCode, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
585 Sender: sender, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
586 LanguageCode: languageCode, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
587 DateIssue: timetz{dateIssue}, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
588 ReferenceCode: referenceCode, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
589 WaterLevel: waterLevel, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
590 Predicted: predicted, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
591 ValueMin: nilFloat(valueMin), |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
592 ValueMax: nilFloat(valueMax), |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
593 DateInfo: timetz{dateInfo}, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
594 SourceOrganization: sourceOrganization, |
58a28715e386
Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2098
diff
changeset
|
595 } |
1741
44398a8bdf94
Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
596 } |