Mercurial > gemma

annotate pkg/imports/agm.go @ 2098:8a986d80e1c6

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
author Sascha L. Teichmann <sascha.teichmann@intevation.de>
date Mon, 04 Feb 2019 11:34:09 +0100
parents 59055c8301df
children 58a28715e386
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
3 //
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
6 //
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 //
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 // Author(s):
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
13
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14 package imports
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import (
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
17 "bufio"
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 "context"
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 "database/sql"
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
20 "encoding/csv"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
21 "fmt"
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
22 "io"
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
23 "os"
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
24 "path/filepath"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
25 "strconv"
1775
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
26 "strings"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
27 "time"
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
28
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
29 "gemma.intevation.de/gemma/pkg/common"
1938
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
30 "gemma.intevation.de/gemma/pkg/misc"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
31 "gemma.intevation.de/gemma/pkg/models"
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
32 )
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
33
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
34 type ApprovedGaugeMeasurements struct {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
35 Dir string `json:"dir"`
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
36 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
37
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
38 // GMAPJobKind is the unique name of an approved gauge measurements import job.
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
39 const AGMJobKind JobKind = "agm"
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
40
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
41 type agmJobCreator struct{}
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
42
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
43 func init() {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
44 RegisterJobCreator(AGMJobKind, agmJobCreator{})
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
45 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
46
1754
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1743
diff changeset
47 func (agmJobCreator) AutoAccept() bool { return false }
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1743
diff changeset
48
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
49 func (agmJobCreator) Description() string {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
50 return "approved gauge measurements"
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
51 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
52
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
53 func (agmJobCreator) Create(_ JobKind, data string) (Job, error) {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
54 agm := new(ApprovedGaugeMeasurements)
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
55 if err := common.FromJSONString(data, agm); err != nil {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
56 return nil, err
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
57 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
58 return agm, nil
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
59 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
60
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
61 func (agmJobCreator) Depends() []string {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
62 return []string{
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63 "gauges",
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
64 "gauge_measurements",
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
65 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
66 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
67
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
68 const (
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
69 // delete the old and keep the new measures.
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
70 agmStageDoneDeleteSQL = `
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
71 WITH staged AS (
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
72 SELECT key
1995
59055c8301df Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents: 1938
diff changeset
73 FROM import.track_imports
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
74 WHERE import_id = $1 AND
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
75 relation = 'waterway.gauge_measurements'::regclass
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
76 ),
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
77 to_delete AS (
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
78 SELECT o.id AS id
1781
0a53c33bc7b2 Fixed broken SQL statement (once again).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1780
diff changeset
79 FROM waterway.gauge_measurements o
0a53c33bc7b2 Fixed broken SQL statement (once again).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1780
diff changeset
80 JOIN waterway.gauge_measurements n
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
81 ON n.fk_gauge_id = o.fk_gauge_id AND n.measure_date = o.measure_date
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
82 WHERE n.id IN (SELECT key FROM staged)
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
83 AND o.id NOT IN (SELECT key FROM staged)
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
84 )
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
85 DELETE FROM waterway.gauge_measurements WHERE id IN (SELECT id from to_delete)`
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
86
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
87 agmStageDoneSQL = `
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
88 UPDATE waterway.gauge_measurements SET staging_done = true
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
89 WHERE id IN (
1995
59055c8301df Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents: 1938
diff changeset
90 SELECT key FROM import.track_imports
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
91 WHERE import_id = $1 AND
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
92 relation = 'waterway.gauge_measurements'::regclass)`
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
93 )
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
94
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
95 func (agmJobCreator) StageDone(
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
96 ctx context.Context,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
97 tx *sql.Tx,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
98 id int64,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
99 ) error {
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
100 _, err := tx.ExecContext(ctx, agmStageDoneDeleteSQL, id)
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
101 if err == nil {
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
102 _, err = tx.ExecContext(ctx, agmStageDoneSQL, id)
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
103 }
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
104 return err
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
105 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
106
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
107 // CleanUp removes the folder containing the CSV file with the
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
108 // the approved gauge measurements.
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
109 func (agm *ApprovedGaugeMeasurements) CleanUp() error {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
110 return os.RemoveAll(agm.Dir)
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
111 }
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
112
1938
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
113 var guessDate = misc.TimeGuesser([]string{
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
114 "02.01.2006 15:04",
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
115 "2006-01-02T15:04:05-07:00",
206b1dd31112 Moved time guessing to misc code to be recycled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1781
diff changeset
116 }).Guess
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
117
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
118 // Do executes the actual approved gauge measurements import.
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
119 func (agm *ApprovedGaugeMeasurements) Do(
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
120 ctx context.Context,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
121 importID int64,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
122 conn *sql.Conn,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
123 feedback Feedback,
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
124 ) (interface{}, error) {
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
125
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
126 start := time.Now()
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
127
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
128 f, err := os.Open(filepath.Join(agm.Dir, "agm.csv"))
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
129 if err != nil {
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
130 return nil, err
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
131 }
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
132 defer f.Close()
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
133
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
134 r := csv.NewReader(bufio.NewReader(f))
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
135 r.Comma = ';'
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
136 r.ReuseRecord = true
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
137
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
138 headers, err := r.Read()
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
139 if err != nil {
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
140 return nil, err
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
141 }
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
142
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
143 var (
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
144 fkGaugeIDIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
145 measureDateIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
146 fromIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
147 languageCodeIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
148 countryCodeIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
149 dateIssueIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
150 referenceCodeIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
151 valueIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
152 predictedIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
153 valueMinIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
154 valueMaxIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
155 dateInfoIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
156 originatorIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
157 unitIdx = -1
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
158 )
1775
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
159
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
160 headerFields := []struct {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
161 idx *int
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
162 name string
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
163 }{
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
164 {&fkGaugeIDIdx, "fk_gauge_id"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
165 {&measureDateIdx, "measure_date"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
166 {&fromIdx, "from"}, // "sender",
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
167 {&languageCodeIdx, "language_code"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
168 {&countryCodeIdx, "country_code"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
169 {&dateIssueIdx, "date_issue"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
170 {&referenceCodeIdx, "reference_code"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
171 {&valueIdx, "value"}, // "water_level",
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
172 {&predictedIdx, "predicted"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
173 // "is_waterlevel",
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
174 {&valueMinIdx, "value_min"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
175 {&valueMaxIdx, "value_max"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
176 {&dateInfoIdx, "date_info"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
177 {&originatorIdx, "originator"}, // "source_organization",
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
178 {&unitIdx, "unit"},
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
179 }
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
180
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
181 nextHeader:
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
182 for i, f := range headers {
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
183 h := strings.Replace(strings.ToLower(
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
184 strings.TrimSpace(f)), " ", "_", -1)
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
185
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
186 for j := range headerFields {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
187 if headerFields[j].name == h {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
188 if *headerFields[j].idx != -1 {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
189 return nil, fmt.Errorf(
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
190 "There is more than one column namend '%s'", h)
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
191 }
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
192 *headerFields[j].idx = i
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
193 continue nextHeader
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
194 }
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
195 }
1775
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
196 }
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
197
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
198 var missing []string
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
199
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
200 for i := range headerFields {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
201 if headerFields[i].name != "unit" && *headerFields[i].idx == -1 {
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
202 missing = append(missing, headerFields[i].name)
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
203 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
204 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
205
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
206 if len(missing) > 0 {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
207 return nil, fmt.Errorf("Missing columns: %s", strings.Join(missing, ", "))
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
208 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
209
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
210 inCm, _ := rescale("cm")
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
211 scaler := func(row []string) (func(float32) float32, error) {
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
212 if unitIdx == -1 {
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
213 return inCm, nil
1775
fcb0106ec510 Gauge measurement import: Added reference_code column.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1772
diff changeset
214 }
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
215 unit := row[unitIdx]
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
216 if unit == "cm" {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
217 return inCm, nil
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
218 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
219 s, err := rescale(unit)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
220 return s, err
1743
85d0f017fbee Approved gauges measurements: Open CSV and read headers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1741
diff changeset
221 }
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
222
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
223 tx, err := conn.BeginTx(ctx, nil)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
224 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
225 return nil, err
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
226 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
227 defer tx.Rollback()
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
228
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
229 insertStmt, err := tx.PrepareContext(ctx, insertGMSQL)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
230 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
231 return nil, err
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
232 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
233 defer insertStmt.Close()
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
234 trackStmt, err := tx.PrepareContext(ctx, trackImportSQL)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
235 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
236 return nil, err
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
237 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
238 defer trackStmt.Close()
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
239
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
240 ids := []int64{}
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
241
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
242 args := make([]interface{}, 19)
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
243
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
244 args[18] = false // staging_done
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
245
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
246 lines:
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
247 for line := 1; ; line++ {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
248
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
249 row, err := r.Read()
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
250 switch {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
251 case err == io.EOF || len(row) == 0:
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
252 break lines
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
253 case err != nil:
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
254 return nil, fmt.Errorf("CSV parsing failed: %v", err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
255 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
256 convert, err := scaler(row)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
257 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
258 return nil, fmt.Errorf("line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
259 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
260
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
261 gids := row[fkGaugeIDIdx]
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
262 gid, err := models.IsrsFromString(gids)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
263 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
264 return nil, fmt.Errorf("Invalid ISRS code line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
265 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
266
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
267 args[0] = gid.CountryCode
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
268 args[1] = gid.LoCode
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
269 args[2] = gid.FairwaySection
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
270 args[3] = gid.Orc
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
271 args[4] = gid.Hectometre
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
272
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
273 md, err := guessDate(row[measureDateIdx])
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
274 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
275 return nil, fmt.Errorf("Invalid 'measure_date' line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
276 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
277 args[5] = md
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
278
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
279 args[6] = row[fromIdx]
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
280 args[7] = row[languageCodeIdx]
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
281 args[8] = row[countryCodeIdx]
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
282
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
283 dis, err := guessDate(row[dateIssueIdx])
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
284 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
285 return nil, fmt.Errorf("Invalid 'date_issue' line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
286 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
287 args[9] = dis
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
288
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
289 args[10] = row[referenceCodeIdx]
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
290
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
291 value, err := strconv.ParseFloat(row[valueIdx], 32)
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
292 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
293 return nil, fmt.Errorf("Invalid 'value' line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
294 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
295 args[11] = convert(float32(value))
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
296
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
297 predicted := strings.ToLower(row[predictedIdx]) == "true"
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
298 args[12] = predicted
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
299
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
300 args[13] = true // is_waterlevel
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
301
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
302 valueMin, err := strconv.ParseFloat(row[valueMinIdx], 32)
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
303 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
304 return nil, fmt.Errorf("Invalid 'value_min' line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
305 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
306 args[14] = convert(float32(valueMin))
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
307
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
308 valueMax, err := strconv.ParseFloat(row[valueMaxIdx], 32)
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
309 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
310 return nil, fmt.Errorf("Invalid 'value_max' line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
311 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
312 args[15] = convert(float32(valueMax))
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
313
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
314 din, err := guessDate(row[dateInfoIdx])
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
315 if err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
316 return nil, fmt.Errorf("Invalid 'date_info' line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
317 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
318 args[16] = din
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
319
2098
8a986d80e1c6 Approved gauge measuremnet imports: Removed an indirection layer accessing the columns of the CSV file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1995
diff changeset
320 args[17] = row[originatorIdx]
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
321
1780
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
322 // args[18] (staging_done) is set to true outside the loop.
48791416bea5 (Approved) gauge measurement import: Fixed row level security.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1779
diff changeset
323
1778
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
324 var id int64
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
325 if err := insertStmt.QueryRowContext(ctx, args...).Scan(&id); err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
326 return nil, fmt.Errorf("Failed to insert line %d: %v", line, err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
327 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
328 ids = append(ids, id)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
329
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
330 if _, err := trackStmt.ExecContext(
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
331 ctx, importID, "waterway.gauge_measurements", id,
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
332 ); err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
333 return nil, err
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
334 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
335 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
336
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
337 if err := tx.Commit(); err != nil {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
338 return nil, fmt.Errorf("Commit failed: %v", err)
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
339 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
340
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
341 feedback.Info("Importing approved gauge measurements took %s",
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
342 time.Since(start))
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
343
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
344 summary := struct {
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
345 IDs []int64 `json:"ids"`
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
346 }{
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
347 IDs: ids,
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
348 }
164b46ebd60d Approved gauge measurement import: Implemented. TODO: Fix staging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1775
diff changeset
349 return &summary, nil
1741
44398a8bdf94 Approved gauge measurements: Added a stub to upload a CSV file for parsing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
350 }