Mercurial > gemma

annotate pkg/imports/agm.go @ 3220:56b297592c0a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Handle failing INSERTs gracefully during approved gauge measurements import
author Tom Gottfried <tom@intevation.de>
date Thu, 09 May 2019 14:21:50 +0200
parents 4acbee65275d
children 232fc90e6ee2
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
3 //
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
6 //
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 //
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 // Author(s):
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
2562
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
13 // * Sascha Wilde <wilde@intevation.de>
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15 package imports
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 import (
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
18 "bufio"
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 "context"
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
20 "database/sql"
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
21 "encoding/csv"
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
22 "encoding/json"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
23 "fmt"
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
24 "io"
2173
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
25 "math"
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26 "os"
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
27 "path/filepath"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
28 "strconv"
1775
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
29 "strings"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
30 "time"
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
31
1938
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
32 "gemma.intevation.de/gemma/pkg/misc"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
33 "gemma.intevation.de/gemma/pkg/models"
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
34 )
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
35
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
36 type ApprovedGaugeMeasurements struct {
2562
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
37 Dir string `json:"dir"`
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
38 Originator string `json:"originator"`
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
39 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
40
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
41 // GMAPJobKind is the unique name of an approved gauge measurements import job.
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
42 const AGMJobKind JobKind = "agm"
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
43
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
44 type agmJobCreator struct{}
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
45
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
46 func init() {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
47 RegisterJobCreator(AGMJobKind, agmJobCreator{})
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
48 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
49
1754
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1743
diff changeset
50 func (agmJobCreator) AutoAccept() bool { return false }
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1743
diff changeset
51
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
52 func (agmJobCreator) Description() string {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
53 return "approved gauge measurements"
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
54 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
55
2187
7c83b5277c1c Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2173
diff changeset
56 func (agmJobCreator) Create() Job { return new(ApprovedGaugeMeasurements) }
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
57
3219
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3218
diff changeset
58 func (agmJobCreator) Depends() [2][]string {
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3218
diff changeset
59 return [2][]string{
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3218
diff changeset
60 {"gauge_measurements"},
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3218
diff changeset
61 {"gauges"},
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
62 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
64
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
65 const (
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
66 // delete the old and keep the new measures.
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
67 agmStageDoneDeleteSQL = `
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
68 WITH staged AS (
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
69 SELECT key
1995
59055c8301df Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents: 1938
diff changeset
70 FROM import.track_imports
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
71 WHERE import_id = $1 AND
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
72 relation = 'waterway.gauge_measurements'::regclass
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
73 ),
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
74 to_delete AS (
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
75 SELECT o.id AS id
1781
0a53c33bc7b2 Fixed broken SQL statement (once again).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1780
diff changeset
76 FROM waterway.gauge_measurements o
0a53c33bc7b2 Fixed broken SQL statement (once again).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1780
diff changeset
77 JOIN waterway.gauge_measurements n
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
78 ON n.fk_gauge_id = o.fk_gauge_id AND n.measure_date = o.measure_date
2850
6d4f361c36e8 Do not delete predictions on import of agm.
Sascha Wilde <wilde@intevation.de>
parents: 2562
diff changeset
79 WHERE n.id IN (SELECT key FROM staged)
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
80 AND o.id NOT IN (SELECT key FROM staged)
2850
6d4f361c36e8 Do not delete predictions on import of agm.
Sascha Wilde <wilde@intevation.de>
parents: 2562
diff changeset
81 AND NOT o.predicted
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
82 )
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
83 DELETE FROM waterway.gauge_measurements WHERE id IN (SELECT id from to_delete)`
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
84
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
85 agmStageDoneSQL = `
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
86 UPDATE waterway.gauge_measurements SET staging_done = true
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
87 WHERE id IN (
1995
59055c8301df Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents: 1938
diff changeset
88 SELECT key FROM import.track_imports
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
89 WHERE import_id = $1 AND
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
90 relation = 'waterway.gauge_measurements'::regclass)`
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
91 )
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
92
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
93 func (agmJobCreator) StageDone(
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
94 ctx context.Context,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
95 tx *sql.Tx,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
96 id int64,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
97 ) error {
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
98 _, err := tx.ExecContext(ctx, agmStageDoneDeleteSQL, id)
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
99 if err == nil {
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
100 _, err = tx.ExecContext(ctx, agmStageDoneSQL, id)
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
101 }
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
102 return err
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
103 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
104
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
105 // CleanUp removes the folder containing the CSV file with the
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
106 // the approved gauge measurements.
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
107 func (agm *ApprovedGaugeMeasurements) CleanUp() error {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
108 return os.RemoveAll(agm.Dir)
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
109 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
110
1938
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
111 var guessDate = misc.TimeGuesser([]string{
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
112 "02.01.2006 15:04",
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
113 "2006-01-02T15:04:05-07:00",
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
114 }).Guess
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
115
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
116 type timetz struct{ time.Time }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
117
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
118 func (ttz *timetz) MarshalJSON() ([]byte, error) {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
119 return json.Marshal(ttz.Time.Format("2006-01-02T15:04:05-07:00"))
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
120 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
121
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
122 type agmLine struct {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
123 CountryCode string `json:"country-code"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
124 Sender string `json:"sender"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
125 LanguageCode string `json:"language-code"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
126 DateIssue timetz `json:"date-issue"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
127 ReferenceCode string `json:"reference-code"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
128 WaterLevel float64 `json:"water-level"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
129 Predicted bool `json:"predicted"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
130 ValueMin *float64 `json:"value-min"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
131 ValueMax *float64 `json:"value-max"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
132 DateInfo timetz `json:"date-info"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
133 SourceOrganization string `json:"source-organization"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
134 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
135
2173
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
136 func (a *agmLine) hasDiff(b *agmLine) bool {
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
137 const eps = 0.00001
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
138 fdiff := func(x, y *float64) bool {
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
139 if x == nil && y == nil {
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
140 return false
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
141 }
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
142 if (x == nil && y != nil) || (x != nil && y == nil) {
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
143 return true
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
144 }
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
145 return math.Abs(*x-*y) > eps
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
146 }
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
147 return a.CountryCode != b.CountryCode ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
148 a.Sender != b.Sender ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
149 a.LanguageCode != b.LanguageCode ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
150 a.ReferenceCode != b.ReferenceCode ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
151 math.Abs(a.WaterLevel-b.WaterLevel) > eps ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
152 a.Predicted != b.Predicted ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
153 fdiff(a.ValueMin, b.ValueMin) ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
154 fdiff(a.ValueMax, b.ValueMax) ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
155 a.SourceOrganization != b.SourceOrganization
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
156 }
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
157
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
158 type agmSummaryEntry struct {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
159 FKGaugeID models.Isrs `json:"fk-gauge-id"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
160 MeasureDate timetz `json:"measure-date"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
161 Versions []*agmLine `json:"versions"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
162 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
163
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
164 const (
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
165 agmSelectSQL = `
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
166 SELECT
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
167 id,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
168 country_code,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
169 sender,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
170 language_code,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
171 date_issue,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
172 reference_code,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
173 water_level,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
174 predicted,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
175 value_min,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
176 value_max,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
177 date_info,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
178 source_organization
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
179 FROM waterway.gauge_measurements
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
180 WHERE
2120
817cd8b89a86 Approved gauge measuments import: Give type hints when insertinf/querying isrs location codes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2105
diff changeset
181 fk_gauge_id = ($1::char(2), $2::char(3), $3::char(5), $4::char(5), $5::int) AND
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
182 measure_date = $6 AND staging_done`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
183
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
184 agmInsertSQL = `
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
185 INSERT INTO waterway.gauge_measurements (
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
186 fk_gauge_id,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
187 measure_date,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
188 country_code,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
189 sender,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
190 language_code,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
191 date_issue,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
192 reference_code,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
193 water_level,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
194 predicted,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
195 value_min,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
196 value_max,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
197 date_info,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
198 source_organization,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
199 is_waterlevel,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
200 staging_done
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
201 ) VALUES(
2120
817cd8b89a86 Approved gauge measuments import: Give type hints when insertinf/querying isrs location codes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2105
diff changeset
202 ($1::char(2), $2::char(3), $3::char(5), $4::char(5), $5::int),
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
203 $6,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
204 $7,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
205 $8,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
206 $9,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
207 $10,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
208 $11,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
209 $12,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
210 $13,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
211 $14,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
212 $15,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
213 $16,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
214 $17,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
215 true,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
216 false
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
217 )
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
218 RETURNING id`
2533
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
219
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
220 agmGaugeCheckSQL = `
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
221 SELECT EXISTS(
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
222 SELECT 1 FROM waterway.gauges
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
223 WHERE location = ($1::char(2), $2::char(3), $3::char(5), $4::char(5), $5::int))
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
224 `
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
225 )
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
226
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
227 // Do executes the actual approved gauge measurements import.
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
228 func (agm *ApprovedGaugeMeasurements) Do(
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
229 ctx context.Context,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
230 importID int64,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
231 conn *sql.Conn,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
232 feedback Feedback,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
233 ) (interface{}, error) {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
234
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
235 start := time.Now()
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
236
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
237 f, err := os.Open(filepath.Join(agm.Dir, "agm.csv"))
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
238 if err != nil {
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
239 return nil, err
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
240 }
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
241 defer f.Close()
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
242
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
243 r := csv.NewReader(bufio.NewReader(f))
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
244 r.Comma = ';'
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
245 r.ReuseRecord = true
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
246
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
247 headers, err := r.Read()
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
248 if err != nil {
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
249 return nil, err
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
250 }
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
251
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
252 var (
2562
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
253 fkGaugeIDIdx = -1
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
254 measureDateIdx = -1
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
255 valueIdx = -1
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
256 )
1775
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
257
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
258 headerFields := []struct {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
259 idx *int
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
260 name string
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
261 }{
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
262 {&fkGaugeIDIdx, "fk_gauge_id"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
263 {&measureDateIdx, "measure_date"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
264 {&valueIdx, "value"}, // "water_level",
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
265 }
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
266
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
267 nextHeader:
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
268 for i, f := range headers {
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
269 h := strings.Replace(strings.ToLower(
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
270 strings.TrimSpace(f)), " ", "_", -1)
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
271
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
272 for j := range headerFields {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
273 if headerFields[j].name == h {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
274 if *headerFields[j].idx != -1 {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
275 return nil, fmt.Errorf(
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
276 "There is more than one column namend '%s'", h)
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
277 }
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
278 *headerFields[j].idx = i
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
279 continue nextHeader
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
280 }
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
281 }
1775
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
282 }
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
283
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
284 var missing []string
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
285 for i := range headerFields {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
286 if headerFields[i].name != "unit" && *headerFields[i].idx == -1 {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
287 missing = append(missing, headerFields[i].name)
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
288 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
289 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
290 if len(missing) > 0 {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
291 return nil, fmt.Errorf("Missing columns: %s", strings.Join(missing, ", "))
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
292 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
293
3220
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
294 gaugeCheckStmt, err := conn.PrepareContext(ctx, agmGaugeCheckSQL)
2533
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
295 if err != nil {
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
296 return nil, err
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
297 }
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
298 defer gaugeCheckStmt.Close()
3186
928cdc06ff37 Warn only once if gauge is unknown
Tom Gottfried <tom@intevation.de>
parents: 2853
diff changeset
299
3220
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
300 selectStmt, err := conn.PrepareContext(ctx, agmSelectSQL)
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
301 if err != nil {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
302 return nil, err
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
303 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
304 defer selectStmt.Close()
3186
928cdc06ff37 Warn only once if gauge is unknown
Tom Gottfried <tom@intevation.de>
parents: 2853
diff changeset
305
3220
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
306 insertStmt, err := conn.PrepareContext(ctx, agmInsertSQL)
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
307 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
308 return nil, err
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
309 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
310 defer insertStmt.Close()
3186
928cdc06ff37 Warn only once if gauge is unknown
Tom Gottfried <tom@intevation.de>
parents: 2853
diff changeset
311
3220
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
312 trackStmt, err := conn.PrepareContext(ctx, trackImportSQL)
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
313 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
314 return nil, err
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
315 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
316 defer trackStmt.Close()
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
317
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
318 entries := []*agmSummaryEntry{}
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
319
2533
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
320 checkedGauges := map[models.Isrs]bool{}
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
321
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
322 lines:
3193
8329c6d3cf2a Do not offer an import for review that did not import anything
Tom Gottfried <tom@intevation.de>
parents: 3186
diff changeset
323 for line, ignored := 1, 0; ; line++ {
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
324
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
325 row, err := r.Read()
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
326 switch {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
327 case err == io.EOF || len(row) == 0:
3218
c2b65a549c6f Add minimal feedback in approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3193
diff changeset
328 feedback.Info("Read %d entries in CSV file", line-1)
c2b65a549c6f Add minimal feedback in approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3193
diff changeset
329 if ignored > 0 {
c2b65a549c6f Add minimal feedback in approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3193
diff changeset
330 feedback.Info("%d entries ignored", ignored)
c2b65a549c6f Add minimal feedback in approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3193
diff changeset
331 }
3193
8329c6d3cf2a Do not offer an import for review that did not import anything
Tom Gottfried <tom@intevation.de>
parents: 3186
diff changeset
332 if ignored == line-1 {
8329c6d3cf2a Do not offer an import for review that did not import anything
Tom Gottfried <tom@intevation.de>
parents: 3186
diff changeset
333 return nil, UnchangedError("No entries imported")
8329c6d3cf2a Do not offer an import for review that did not import anything
Tom Gottfried <tom@intevation.de>
parents: 3186
diff changeset
334 }
3218
c2b65a549c6f Add minimal feedback in approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3193
diff changeset
335 feedback.Info("Imported %d entries with changes", len(entries))
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
336 break lines
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
337 case err != nil:
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
338 return nil, fmt.Errorf("CSV parsing failed: %v", err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
339 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
340
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
341 gids := row[fkGaugeIDIdx]
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
342 gid, err := models.IsrsFromString(gids)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
343 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
344 return nil, fmt.Errorf("Invalid ISRS code line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
345 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
346
2533
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
347 if exists, found := checkedGauges[*gid]; found {
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
348 if !exists {
3186
928cdc06ff37 Warn only once if gauge is unknown
Tom Gottfried <tom@intevation.de>
parents: 2853
diff changeset
349 // Just ignore the line since we have already warned
3193
8329c6d3cf2a Do not offer an import for review that did not import anything
Tom Gottfried <tom@intevation.de>
parents: 3186
diff changeset
350 ignored++
2533
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
351 continue lines
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
352 }
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
353 } else { // not found in gauge cache
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
354 if err := gaugeCheckStmt.QueryRowContext(
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
355 ctx,
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
356 gid.CountryCode,
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
357 gid.LoCode,
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
358 gid.FairwaySection,
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
359 gid.Orc,
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
360 gid.Hectometre,
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
361 ).Scan(&exists); err != nil {
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
362 return nil, err
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
363 }
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
364 checkedGauges[*gid] = exists
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
365 if !exists {
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
366 feedback.Warn("Ignoring data for unknown gauge %s", gid.String())
3193
8329c6d3cf2a Do not offer an import for review that did not import anything
Tom Gottfried <tom@intevation.de>
parents: 3186
diff changeset
367 ignored++
2533
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
368 continue lines
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
369 }
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
370 }
de4dc3d16647 AGM import: Check if gauge exists for a meassurement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
371
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
372 md, err := guessDate(row[measureDateIdx])
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
373 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
374 return nil, fmt.Errorf("Invalid 'measure_date' line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
375 }
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
376
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
377 var (
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
378 oldID int64
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
379 oldCountryCode string
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
380 oldSender string
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
381 oldLanguageCode string
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
382 oldDateIssue time.Time
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
383 oldReferenceCode string
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
384 oldValue float64
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
385 oldPredicted bool
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
386 oldValueMin sql.NullFloat64
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
387 oldValueMax sql.NullFloat64
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
388 oldDateInfo time.Time
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
389 oldSourceOrganization string
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
390 )
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
391
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
392 err = selectStmt.QueryRowContext(
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
393 ctx,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
394 gid.CountryCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
395 gid.LoCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
396 gid.FairwaySection,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
397 gid.Orc,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
398 gid.Hectometre,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
399 md,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
400 ).Scan(
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
401 &oldID,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
402 &oldCountryCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
403 &oldSender,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
404 &oldLanguageCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
405 &oldDateIssue,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
406 &oldReferenceCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
407 &oldValue,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
408 &oldPredicted,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
409 &oldValueMin,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
410 &oldValueMax,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
411 &oldDateInfo,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
412 &oldSourceOrganization,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
413 )
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
414
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
415 var newEntry bool
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
416 switch {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
417 case err == sql.ErrNoRows:
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
418 // Complete new one
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
419 newEntry = true
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
420 case err != nil:
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
421 return nil, err
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
422 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
423
2562
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
424 newSender := agm.Originator
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
425 newCountryCode := gid.CountryCode
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
426 newLanguageCode := misc.CCtoLang[gid.CountryCode]
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
427 newDateIssue := time.Now()
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
428 newReferenceCode := "ZPG"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
429
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
430 value, err := strconv.ParseFloat(row[valueIdx], 32)
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
431 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
432 return nil, fmt.Errorf("Invalid 'value' line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
433 }
2562
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
434 newValue := value
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
435
2562
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
436 newPredicted := false
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
437
2562
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
438 newValueMin := sql.NullFloat64{
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
439 Float64: 0,
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
440 Valid: true,
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
441 }
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
442 newValueMax := sql.NullFloat64{
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
443 Float64: 0,
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
444 Valid: true,
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
445 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
446
2562
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
447 newDateInfo := newDateIssue
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
448
2562
ce39e9954e85 Make upload of AGM require only "fk_gauge_id" "measure_date" and "value"
Sascha Wilde <wilde@intevation.de>
parents: 2533
diff changeset
449 newSourceOrganization := newSender
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
450
3220
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
451 tx, err := conn.BeginTx(ctx, nil)
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
452 if err != nil {
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
453 return nil, err
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
454 }
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
455 defer tx.Rollback()
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
456
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
457 var newID int64
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
458
3220
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
459 if err := tx.StmtContext(ctx, insertStmt).QueryRowContext(
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
460 ctx,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
461 gid.CountryCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
462 gid.LoCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
463 gid.FairwaySection,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
464 gid.Orc,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
465 gid.Hectometre,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
466 md,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
467 newCountryCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
468 newSender,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
469 newLanguageCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
470 newDateIssue,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
471 newReferenceCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
472 newValue,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
473 newPredicted,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
474 newValueMin,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
475 newValueMax,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
476 newDateInfo,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
477 newSourceOrganization,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
478 ).Scan(&newID); err != nil {
3220
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
479 feedback.Warn(handleError(err).Error())
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
480 if err := tx.Rollback(); err != nil {
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
481 return nil, err
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
482 }
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
483 ignored++
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
484 continue
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
485 }
3220
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
486 if _, err := tx.StmtContext(ctx, trackStmt).ExecContext(
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
487 ctx, importID, "waterway.gauge_measurements", newID,
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
488 ); err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
489 return nil, err
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
490 }
3220
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
491 if err := tx.Commit(); err != nil {
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
492 return nil, fmt.Errorf("Commit failed: %v", err)
56b297592c0a Handle failing INSERTs gracefully during approved gauge measurements import
Tom Gottfried <tom@intevation.de>
parents: 3219
diff changeset
493 }
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
494
2173
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
495 n := newAGMLine(
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
496 newCountryCode,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
497 newSender,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
498 newLanguageCode,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
499 newDateIssue,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
500 newReferenceCode,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
501 newValue,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
502 newPredicted,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
503 newValueMin,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
504 newValueMax,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
505 newDateInfo,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
506 newSourceOrganization,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
507 )
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
508
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
509 ase := &agmSummaryEntry{
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
510 FKGaugeID: *gid,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
511 MeasureDate: timetz{md},
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
512 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
513
2173
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
514 if newEntry {
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
515 ase.Versions = []*agmLine{n}
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
516 } else {
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
517 o := newAGMLine(
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
518 oldCountryCode,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
519 oldSender,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
520 oldLanguageCode,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
521 oldDateIssue,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
522 oldReferenceCode,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
523 oldValue,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
524 oldPredicted,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
525 oldValueMin,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
526 oldValueMax,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
527 oldDateInfo,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
528 oldSourceOrganization,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
529 )
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
530 // Ignore if there is no diff.
2853
adf355aaef57 Ignore predicted values in AGM diff.
Sascha Wilde <wilde@intevation.de>
parents: 2850
diff changeset
531 if o.Predicted || !n.hasDiff(o) {
2173
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
532 continue
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
533 }
2173
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
534 ase.Versions = []*agmLine{o, n}
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
535 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
536 entries = append(entries, ase)
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
537 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
538
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
539 feedback.Info("Importing approved gauge measurements took %s",
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
540 time.Since(start))
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
541
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
542 return entries, nil
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
543 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
544
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
545 func newAGMLine(
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
546 countryCode string,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
547 sender string,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
548 languageCode string,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
549 dateIssue time.Time,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
550 referenceCode string,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
551 waterLevel float64,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
552 predicted bool,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
553 valueMin sql.NullFloat64,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
554 valueMax sql.NullFloat64,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
555 dateInfo time.Time,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
556 sourceOrganization string,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
557 ) *agmLine {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
558 nilFloat := func(v sql.NullFloat64) *float64 {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
559 var p *float64
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
560 if v.Valid {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
561 p = &v.Float64
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
562 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
563 return p
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
564 }
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
565 return &agmLine{
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
566 CountryCode: countryCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
567 Sender: sender,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
568 LanguageCode: languageCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
569 DateIssue: timetz{dateIssue},
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
570 ReferenceCode: referenceCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
571 WaterLevel: waterLevel,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
572 Predicted: predicted,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
573 ValueMin: nilFloat(valueMin),
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
574 ValueMax: nilFloat(valueMax),
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
575 DateInfo: timetz{dateInfo},
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
576 SourceOrganization: sourceOrganization,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
577 }
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
578 }