Mercurial > gemma

annotate pkg/imports/agm.go @ 2187:7c83b5277c1c

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue.
author Sascha L. Teichmann <sascha.teichmann@intevation.de>
date Mon, 11 Feb 2019 18:35:01 +0100
parents 1b1cb6cbfbf0
children de4dc3d16647
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
3 //
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
6 //
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 //
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 // Author(s):
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
13
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14 package imports
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import (
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
17 "bufio"
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 "context"
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 "database/sql"
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
20 "encoding/csv"
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
21 "encoding/json"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
22 "fmt"
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
23 "io"
2173
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
24 "math"
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
25 "os"
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
26 "path/filepath"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
27 "strconv"
1775
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
28 "strings"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
29 "time"
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
30
1938
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
31 "gemma.intevation.de/gemma/pkg/misc"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
32 "gemma.intevation.de/gemma/pkg/models"
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
33 )
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
34
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
35 type ApprovedGaugeMeasurements struct {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
36 Dir string `json:"dir"`
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
37 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
38
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
39 // GMAPJobKind is the unique name of an approved gauge measurements import job.
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
40 const AGMJobKind JobKind = "agm"
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
41
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
42 type agmJobCreator struct{}
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
43
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
44 func init() {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
45 RegisterJobCreator(AGMJobKind, agmJobCreator{})
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
46 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
47
1754
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1743
diff changeset
48 func (agmJobCreator) AutoAccept() bool { return false }
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1743
diff changeset
49
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
50 func (agmJobCreator) Description() string {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
51 return "approved gauge measurements"
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
52 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
53
2187
7c83b5277c1c Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2173
diff changeset
54 func (agmJobCreator) Create() Job { return new(ApprovedGaugeMeasurements) }
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
55
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
56 func (agmJobCreator) Depends() []string {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
57 return []string{
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
58 "gauges",
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
59 "gauge_measurements",
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
60 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
61 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
62
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63 const (
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
64 // delete the old and keep the new measures.
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
65 agmStageDoneDeleteSQL = `
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
66 WITH staged AS (
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
67 SELECT key
1995
59055c8301df Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents: 1938
diff changeset
68 FROM import.track_imports
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
69 WHERE import_id = $1 AND
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
70 relation = 'waterway.gauge_measurements'::regclass
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
71 ),
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
72 to_delete AS (
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
73 SELECT o.id AS id
1781
0a53c33bc7b2 Fixed broken SQL statement (once again).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1780
diff changeset
74 FROM waterway.gauge_measurements o
0a53c33bc7b2 Fixed broken SQL statement (once again).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1780
diff changeset
75 JOIN waterway.gauge_measurements n
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
76 ON n.fk_gauge_id = o.fk_gauge_id AND n.measure_date = o.measure_date
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
77 WHERE n.id IN (SELECT key FROM staged)
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
78 AND o.id NOT IN (SELECT key FROM staged)
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
79 )
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
80 DELETE FROM waterway.gauge_measurements WHERE id IN (SELECT id from to_delete)`
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
81
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
82 agmStageDoneSQL = `
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
83 UPDATE waterway.gauge_measurements SET staging_done = true
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
84 WHERE id IN (
1995
59055c8301df Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents: 1938
diff changeset
85 SELECT key FROM import.track_imports
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
86 WHERE import_id = $1 AND
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
87 relation = 'waterway.gauge_measurements'::regclass)`
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
88 )
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
89
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
90 func (agmJobCreator) StageDone(
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
91 ctx context.Context,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
92 tx *sql.Tx,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
93 id int64,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
94 ) error {
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
95 _, err := tx.ExecContext(ctx, agmStageDoneDeleteSQL, id)
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
96 if err == nil {
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
97 _, err = tx.ExecContext(ctx, agmStageDoneSQL, id)
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
98 }
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
99 return err
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
100 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
101
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
102 // CleanUp removes the folder containing the CSV file with the
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
103 // the approved gauge measurements.
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
104 func (agm *ApprovedGaugeMeasurements) CleanUp() error {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
105 return os.RemoveAll(agm.Dir)
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
106 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
107
1938
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
108 var guessDate = misc.TimeGuesser([]string{
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
109 "02.01.2006 15:04",
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
110 "2006-01-02T15:04:05-07:00",
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
111 }).Guess
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
112
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
113 type timetz struct{ time.Time }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
114
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
115 func (ttz *timetz) MarshalJSON() ([]byte, error) {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
116 return json.Marshal(ttz.Time.Format("2006-01-02T15:04:05-07:00"))
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
117 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
118
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
119 type agmLine struct {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
120 CountryCode string `json:"country-code"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
121 Sender string `json:"sender"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
122 LanguageCode string `json:"language-code"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
123 DateIssue timetz `json:"date-issue"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
124 ReferenceCode string `json:"reference-code"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
125 WaterLevel float64 `json:"water-level"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
126 Predicted bool `json:"predicted"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
127 ValueMin *float64 `json:"value-min"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
128 ValueMax *float64 `json:"value-max"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
129 DateInfo timetz `json:"date-info"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
130 SourceOrganization string `json:"source-organization"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
131 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
132
2173
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
133 func (a *agmLine) hasDiff(b *agmLine) bool {
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
134 const eps = 0.00001
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
135 fdiff := func(x, y *float64) bool {
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
136 if x == nil && y == nil {
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
137 return false
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
138 }
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
139 if (x == nil && y != nil) || (x != nil && y == nil) {
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
140 return true
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
141 }
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
142 return math.Abs(*x-*y) > eps
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
143 }
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
144 return a.CountryCode != b.CountryCode ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
145 a.Sender != b.Sender ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
146 a.LanguageCode != b.LanguageCode ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
147 !a.DateIssue.Time.Equal(b.DateIssue.Time) ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
148 a.ReferenceCode != b.ReferenceCode ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
149 math.Abs(a.WaterLevel-b.WaterLevel) > eps ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
150 a.Predicted != b.Predicted ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
151 fdiff(a.ValueMin, b.ValueMin) ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
152 fdiff(a.ValueMax, b.ValueMax) ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
153 !a.DateInfo.Time.Equal(b.DateInfo.Time) ||
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
154 a.SourceOrganization != b.SourceOrganization
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
155 }
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
156
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
157 type agmSummaryEntry struct {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
158 FKGaugeID models.Isrs `json:"fk-gauge-id"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
159 MeasureDate timetz `json:"measure-date"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
160 Versions []*agmLine `json:"versions"`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
161 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
162
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
163 const (
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
164 agmSelectSQL = `
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
165 SELECT
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
166 id,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
167 country_code,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
168 sender,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
169 language_code,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
170 date_issue,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
171 reference_code,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
172 water_level,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
173 predicted,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
174 value_min,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
175 value_max,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
176 date_info,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
177 source_organization
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
178 FROM waterway.gauge_measurements
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
179 WHERE
2120
817cd8b89a86 Approved gauge measuments import: Give type hints when insertinf/querying isrs location codes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2105
diff changeset
180 fk_gauge_id = ($1::char(2), $2::char(3), $3::char(5), $4::char(5), $5::int) AND
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
181 measure_date = $6 AND staging_done`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
182
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
183 agmInsertSQL = `
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
184 INSERT INTO waterway.gauge_measurements (
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
185 fk_gauge_id,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
186 measure_date,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
187 country_code,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
188 sender,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
189 language_code,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
190 date_issue,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
191 reference_code,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
192 water_level,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
193 predicted,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
194 value_min,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
195 value_max,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
196 date_info,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
197 source_organization,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
198 is_waterlevel,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
199 staging_done
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
200 ) VALUES(
2120
817cd8b89a86 Approved gauge measuments import: Give type hints when insertinf/querying isrs location codes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2105
diff changeset
201 ($1::char(2), $2::char(3), $3::char(5), $4::char(5), $5::int),
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
202 $6,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
203 $7,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
204 $8,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
205 $9,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
206 $10,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
207 $11,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
208 $12,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
209 $13,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
210 $14,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
211 $15,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
212 $16,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
213 $17,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
214 true,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
215 false
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
216 )
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
217 RETURNING id`
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
218 )
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
219
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
220 // Do executes the actual approved gauge measurements import.
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
221 func (agm *ApprovedGaugeMeasurements) Do(
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
222 ctx context.Context,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
223 importID int64,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
224 conn *sql.Conn,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
225 feedback Feedback,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
226 ) (interface{}, error) {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
227
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
228 start := time.Now()
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
229
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
230 f, err := os.Open(filepath.Join(agm.Dir, "agm.csv"))
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
231 if err != nil {
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
232 return nil, err
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
233 }
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
234 defer f.Close()
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
235
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
236 r := csv.NewReader(bufio.NewReader(f))
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
237 r.Comma = ';'
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
238 r.ReuseRecord = true
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
239
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
240 headers, err := r.Read()
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
241 if err != nil {
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
242 return nil, err
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
243 }
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
244
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
245 var (
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
246 fkGaugeIDIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
247 measureDateIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
248 fromIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
249 languageCodeIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
250 countryCodeIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
251 dateIssueIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
252 referenceCodeIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
253 valueIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
254 predictedIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
255 valueMinIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
256 valueMaxIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
257 dateInfoIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
258 originatorIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
259 unitIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
260 )
1775
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
261
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
262 headerFields := []struct {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
263 idx *int
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
264 name string
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
265 }{
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
266 {&fkGaugeIDIdx, "fk_gauge_id"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
267 {&measureDateIdx, "measure_date"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
268 {&fromIdx, "from"}, // "sender",
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
269 {&languageCodeIdx, "language_code"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
270 {&countryCodeIdx, "country_code"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
271 {&dateIssueIdx, "date_issue"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
272 {&referenceCodeIdx, "reference_code"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
273 {&valueIdx, "value"}, // "water_level",
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
274 {&predictedIdx, "predicted"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
275 // "is_waterlevel",
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
276 {&valueMinIdx, "value_min"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
277 {&valueMaxIdx, "value_max"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
278 {&dateInfoIdx, "date_info"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
279 {&originatorIdx, "originator"}, // "source_organization",
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
280 {&unitIdx, "unit"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
281 }
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
282
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
283 nextHeader:
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
284 for i, f := range headers {
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
285 h := strings.Replace(strings.ToLower(
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
286 strings.TrimSpace(f)), " ", "_", -1)
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
287
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
288 for j := range headerFields {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
289 if headerFields[j].name == h {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
290 if *headerFields[j].idx != -1 {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
291 return nil, fmt.Errorf(
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
292 "There is more than one column namend '%s'", h)
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
293 }
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
294 *headerFields[j].idx = i
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
295 continue nextHeader
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
296 }
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
297 }
1775
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
298 }
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
299
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
300 var missing []string
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
301 for i := range headerFields {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
302 if headerFields[i].name != "unit" && *headerFields[i].idx == -1 {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
303 missing = append(missing, headerFields[i].name)
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
304 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
305 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
306 if len(missing) > 0 {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
307 return nil, fmt.Errorf("Missing columns: %s", strings.Join(missing, ", "))
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
308 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
309
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
310 inCm, _ := rescale("cm")
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
311 scaler := func(row []string) (func(float32) float32, error) {
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
312 if unitIdx == -1 {
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
313 return inCm, nil
1775
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
314 }
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
315 unit := row[unitIdx]
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
316 if unit == "cm" {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
317 return inCm, nil
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
318 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
319 s, err := rescale(unit)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
320 return s, err
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
321 }
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
322
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
323 tx, err := conn.BeginTx(ctx, nil)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
324 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
325 return nil, err
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
326 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
327 defer tx.Rollback()
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
328
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
329 selectStmt, err := tx.PrepareContext(ctx, agmSelectSQL)
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
330 if err != nil {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
331 return nil, err
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
332 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
333 defer selectStmt.Close()
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
334 insertStmt, err := tx.PrepareContext(ctx, agmInsertSQL)
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
335 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
336 return nil, err
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
337 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
338 defer insertStmt.Close()
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
339 trackStmt, err := tx.PrepareContext(ctx, trackImportSQL)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
340 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
341 return nil, err
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
342 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
343 defer trackStmt.Close()
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
344
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
345 entries := []*agmSummaryEntry{}
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
346
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
347 lines:
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
348 for line := 1; ; line++ {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
349
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
350 row, err := r.Read()
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
351 switch {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
352 case err == io.EOF || len(row) == 0:
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
353 break lines
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
354 case err != nil:
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
355 return nil, fmt.Errorf("CSV parsing failed: %v", err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
356 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
357 convert, err := scaler(row)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
358 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
359 return nil, fmt.Errorf("line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
360 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
361
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
362 gids := row[fkGaugeIDIdx]
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
363 gid, err := models.IsrsFromString(gids)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
364 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
365 return nil, fmt.Errorf("Invalid ISRS code line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
366 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
367
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
368 md, err := guessDate(row[measureDateIdx])
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
369 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
370 return nil, fmt.Errorf("Invalid 'measure_date' line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
371 }
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
372
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
373 var (
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
374 oldID int64
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
375 oldCountryCode string
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
376 oldSender string
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
377 oldLanguageCode string
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
378 oldDateIssue time.Time
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
379 oldReferenceCode string
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
380 oldValue float64
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
381 oldPredicted bool
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
382 oldValueMin sql.NullFloat64
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
383 oldValueMax sql.NullFloat64
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
384 oldDateInfo time.Time
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
385 oldSourceOrganization string
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
386 )
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
387
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
388 err = selectStmt.QueryRowContext(
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
389 ctx,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
390 gid.CountryCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
391 gid.LoCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
392 gid.FairwaySection,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
393 gid.Orc,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
394 gid.Hectometre,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
395 md,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
396 ).Scan(
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
397 &oldID,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
398 &oldCountryCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
399 &oldSender,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
400 &oldLanguageCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
401 &oldDateIssue,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
402 &oldReferenceCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
403 &oldValue,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
404 &oldPredicted,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
405 &oldValueMin,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
406 &oldValueMax,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
407 &oldDateInfo,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
408 &oldSourceOrganization,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
409 )
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
410
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
411 var newEntry bool
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
412 switch {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
413 case err == sql.ErrNoRows:
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
414 // Complete new one
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
415 newEntry = true
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
416 case err != nil:
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
417 return nil, err
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
418 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
419
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
420 newSender := row[fromIdx]
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
421 newLanguageCode := row[languageCodeIdx]
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
422 newCountryCode := row[countryCodeIdx]
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
423
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
424 dis, err := guessDate(row[dateIssueIdx])
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
425 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
426 return nil, fmt.Errorf("Invalid 'date_issue' line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
427 }
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
428 newDateIssue := dis
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
429
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
430 newReferenceCode := row[referenceCodeIdx]
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
431
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
432 value, err := strconv.ParseFloat(row[valueIdx], 32)
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
433 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
434 return nil, fmt.Errorf("Invalid 'value' line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
435 }
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
436 newValue := float64(convert(float32(value)))
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
437
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
438 newPredicted := strings.ToLower(row[predictedIdx]) == "true"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
439
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
440 var newValueMin sql.NullFloat64
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
441 if vm := row[valueMinIdx]; vm != "" {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
442 valueMin, err := strconv.ParseFloat(vm, 32)
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
443 if err != nil {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
444 return nil, fmt.Errorf("Invalid 'value_min' line %d: %v", line, err)
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
445 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
446 newValueMin = sql.NullFloat64{
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
447 Float64: float64(convert(float32(valueMin))),
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
448 Valid: true,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
449 }
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
450 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
451
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
452 var newValueMax sql.NullFloat64
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
453 if vm := row[valueMaxIdx]; vm != "" {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
454 valueMax, err := strconv.ParseFloat(vm, 32)
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
455 if err != nil {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
456 return nil, fmt.Errorf("Invalid 'value_max' line %d: %v", line, err)
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
457 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
458 newValueMax = sql.NullFloat64{
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
459 Float64: float64(convert(float32(valueMax))),
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
460 Valid: true,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
461 }
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
462 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
463
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
464 din, err := guessDate(row[dateInfoIdx])
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
465 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
466 return nil, fmt.Errorf("Invalid 'date_info' line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
467 }
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
468 newDateInfo := din
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
469
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
470 newSourceOrganization := row[originatorIdx]
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
471
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
472 var newID int64
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
473
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
474 if err := insertStmt.QueryRowContext(
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
475 ctx,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
476 gid.CountryCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
477 gid.LoCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
478 gid.FairwaySection,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
479 gid.Orc,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
480 gid.Hectometre,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
481 md,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
482 newCountryCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
483 newSender,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
484 newLanguageCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
485 newDateIssue,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
486 newReferenceCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
487 newValue,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
488 newPredicted,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
489 newValueMin,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
490 newValueMax,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
491 newDateInfo,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
492 newSourceOrganization,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
493 ).Scan(&newID); err != nil {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
494 return nil, err
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
495 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
496 if _, err := trackStmt.ExecContext(
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
497 ctx, importID, "waterway.gauge_measurements", newID,
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
498 ); err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
499 return nil, err
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
500 }
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
501
2173
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
502 n := newAGMLine(
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
503 newCountryCode,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
504 newSender,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
505 newLanguageCode,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
506 newDateIssue,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
507 newReferenceCode,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
508 newValue,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
509 newPredicted,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
510 newValueMin,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
511 newValueMax,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
512 newDateInfo,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
513 newSourceOrganization,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
514 )
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
515
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
516 ase := &agmSummaryEntry{
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
517 FKGaugeID: *gid,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
518 MeasureDate: timetz{md},
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
519 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
520
2173
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
521 if newEntry {
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
522 ase.Versions = []*agmLine{n}
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
523 } else {
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
524 o := newAGMLine(
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
525 oldCountryCode,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
526 oldSender,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
527 oldLanguageCode,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
528 oldDateIssue,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
529 oldReferenceCode,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
530 oldValue,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
531 oldPredicted,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
532 oldValueMin,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
533 oldValueMax,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
534 oldDateInfo,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
535 oldSourceOrganization,
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
536 )
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
537 // Ignore if there is no diff.
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
538 if !n.hasDiff(o) {
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
539 continue
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
540 }
2173
1b1cb6cbfbf0 Approved gauge measurement imports: Suppress diff lines when old and new are identical.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
541 ase.Versions = []*agmLine{o, n}
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
542 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
543 entries = append(entries, ase)
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
544 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
545
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
546 if err := tx.Commit(); err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
547 return nil, fmt.Errorf("Commit failed: %v", err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
548 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
549
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
550 feedback.Info("Importing approved gauge measurements took %s",
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
551 time.Since(start))
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
552
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
553 return entries, nil
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
554 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
555
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
556 func newAGMLine(
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
557 countryCode string,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
558 sender string,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
559 languageCode string,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
560 dateIssue time.Time,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
561 referenceCode string,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
562 waterLevel float64,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
563 predicted bool,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
564 valueMin sql.NullFloat64,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
565 valueMax sql.NullFloat64,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
566 dateInfo time.Time,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
567 sourceOrganization string,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
568 ) *agmLine {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
569 nilFloat := func(v sql.NullFloat64) *float64 {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
570 var p *float64
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
571 if v.Valid {
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
572 p = &v.Float64
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
573 }
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
574 return p
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
575 }
2105
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
576 return &agmLine{
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
577 CountryCode: countryCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
578 Sender: sender,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
579 LanguageCode: languageCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
580 DateIssue: timetz{dateIssue},
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
581 ReferenceCode: referenceCode,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
582 WaterLevel: waterLevel,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
583 Predicted: predicted,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
584 ValueMin: nilFloat(valueMin),
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
585 ValueMax: nilFloat(valueMax),
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
586 DateInfo: timetz{dateInfo},
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
587 SourceOrganization: sourceOrganization,
58a28715e386 Approved gauge measurement import: Added diff-summary. XXX: May be broken!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2098
diff changeset
588 }
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
589 }