Mercurial > gemma

annotate pkg/imports/fa.go @ 2198:4db1fa4f049c

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Fairway availabilty import: Fixed row query leak.
author Sascha L. Teichmann <sascha.teichmann@intevation.de>
date Tue, 12 Feb 2019 14:55:09 +0100
parents e57ba9585aaa
children 64147a137e0a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
3 //
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
6 //
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
10 //
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
11 // Author(s):
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
12 // * Raimund Renkert <raimund.renkert@intevation.de>
1694
4a2fad8f57de Imports: Resolved golint issues unrelated to exported symbols commenting.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1676
diff changeset
13
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
14 package imports
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
15
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
16 import (
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
17 "context"
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
18 "database/sql"
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
19 "errors"
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
20 "time"
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
21
1663
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
22 "github.com/jackc/pgx/pgtype"
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
23
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
24 "gemma.intevation.de/gemma/pkg/soap/ifaf"
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
25 )
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
26
1696
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
27 // FairwayAvailability imports fairway availability data
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
28 // from an IFAF SOAP service.
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
29 type FairwayAvailability struct {
1696
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
30 // URL is the URL of the SOAP service.
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
31 URL string `json:"url"`
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
32 // Insecure indicates if HTTPS traffic
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
33 // should validate certificates or not.
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
34 Insecure bool `json:"insecure"`
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
35 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
36
2019
aeff01c6edec Moved fairwayavailability to a local model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2016
diff changeset
37 type uniqueFairwayAvailability struct {
aeff01c6edec Moved fairwayavailability to a local model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2016
diff changeset
38 BottleneckId string
aeff01c6edec Moved fairwayavailability to a local model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2016
diff changeset
39 Surdat time.Time
aeff01c6edec Moved fairwayavailability to a local model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2016
diff changeset
40 }
aeff01c6edec Moved fairwayavailability to a local model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2016
diff changeset
41
1696
ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1694
diff changeset
42 // FAJobKind is import queue type identifier.
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
43 const FAJobKind JobKind = "fa"
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
44
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
45 const (
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
46 listBottlenecksSQL = `
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
47 SELECT
1663
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
48 bottleneck_id,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
49 responsible_country
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
50 FROM waterway.bottlenecks
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
51 WHERE responsible_country = users.current_user_country()
1663
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
52 AND staging_done = true
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
53 `
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
54 latestMeasureDateSQL = `
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
55 SELECT
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
56 measure_date
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
57 FROM waterway.effective_fairway_availability
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
58 ORDER BY measure_date DESC LIMIT 1
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
59 `
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
60 listFairwayAvailabilitySQL = `
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
61 SELECT
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
62 fa.id,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
63 bn.bottleneck_id,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
64 fa.surdat
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
65 FROM waterway.fairway_availability fa
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
66 JOIN waterway.bottlenecks bn ON bn.id = fa.bottleneck_id
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
67 `
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
68 insertFASQL = `
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
69 INSERT INTO waterway.fairway_availability (
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
70 position_code,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
71 bottleneck_id,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
72 surdat,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
73 critical,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
74 date_info,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
75 source_organization
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
76 ) VALUES (
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
77 $1,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
78 (SELECT id FROM waterway.bottlenecks WHERE bottleneck_id = $2),
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
79 $3,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
80 $4,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
81 $5,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
82 $6
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
83 )
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
84 RETURNING id`
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
85
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
86 insertBnPdfsSQL = `
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
87 INSERT INTO waterway.bottleneck_pdfs (
1663
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
88 fairway_availability_id,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
89 profile_pdf_filename,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
90 profile_pdf_url,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
91 pdf_generation_date,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
92 source_organization
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
93 ) VALUES (
1663
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
94 $1,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
95 $2,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
96 $3,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
97 $4,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
98 $5
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
99 ) ON CONFLICT ON CONSTRAINT bottleneck_pdfs_pkey DO NOTHING`
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
100 insertEFASQL = `
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
101 INSERT INTO waterway.effective_fairway_availability (
1663
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
102 fairway_availability_id,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
103 measure_date,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
104 level_of_service,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
105 available_depth_value,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
106 available_width_value,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
107 water_level_value,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
108 measure_type,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
109 source_organization,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
110 forecast_generation_time,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
111 value_lifetime
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
112 ) VALUES (
1663
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
113 $1,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
114 $2,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
115 (SELECT
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
116 level_of_service
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
117 FROM levels_of_service
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
118 WHERE name = $3),
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
119 $4,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
120 $5,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
121 $6,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
122 $7,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
123 $8,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
124 $9,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
125 $10
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
126 ) ON CONFLICT ON CONSTRAINT effective_fairway_availability_pkey DO NOTHING`
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
127 insertFAVSQL = `
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
128 INSERT INTO waterway.fa_reference_values (
1663
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
129 fairway_availability_id,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
130 level_of_service,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
131 fairway_depth,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
132 fairway_width,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
133 fairway_radius,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
134 shallowest_spot
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
135 ) VALUES (
1663
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
136 $1,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
137 (SELECT
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
138 level_of_service
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
139 FROM levels_of_service
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
140 WHERE name = $2),
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
141 $3,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
142 $4,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
143 $5,
10e3dd3b9363 Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1662
diff changeset
144 ST_MakePoint($6, $7)::geography
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
145 )ON CONFLICT ON CONSTRAINT fa_reference_values_pkey DO NOTHING`
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
146 )
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
147
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
148 type faJobCreator struct{}
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
149
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
150 func init() {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
151 RegisterJobCreator(FAJobKind, faJobCreator{})
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
152 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
153
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
154 func (faJobCreator) Description() string {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
155 return "fairway availability"
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
156 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
157
2187
7c83b5277c1c Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
158 func (faJobCreator) Create() Job { return new(FairwayAvailability) }
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
159
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
160 func (faJobCreator) Depends() []string {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
161 return []string{
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
162 "bottlenecks",
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
163 "fairway_availability",
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
164 "bottleneck_pdfs",
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
165 "effective_fairway_availability",
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
166 "fa_reference_values",
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
167 "levels_of_service",
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
168 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
169 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
170
1754
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1696
diff changeset
171 func (faJobCreator) AutoAccept() bool { return true }
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1696
diff changeset
172
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
173 // StageDone moves the imported fairway availablities out of the staging area.
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
174 // Currently doing nothing.
1676
4407ecaa2192 Imports: Cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1663
diff changeset
175 func (faJobCreator) StageDone(context.Context, *sql.Tx, int64) error {
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
176 return nil
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
177 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
178
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
179 // CleanUp of a fairway availablities import is a NOP.
1676
4407ecaa2192 Imports: Cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1663
diff changeset
180 func (*FairwayAvailability) CleanUp() error { return nil }
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
181
2016
25967829cf00 Started to simplify the import models.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1756
diff changeset
182 type bottleneckCountry struct {
25967829cf00 Started to simplify the import models.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1756
diff changeset
183 ID string
25967829cf00 Started to simplify the import models.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1756
diff changeset
184 ResponsibleCountry string
25967829cf00 Started to simplify the import models.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1756
diff changeset
185 }
25967829cf00 Started to simplify the import models.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1756
diff changeset
186
2198
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
187 func loadBottleneckCountries(ctx context.Context, conn *sql.Conn) ([]bottleneckCountry, error) {
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
188
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
189 // Get available bottlenecks from database for use as filter in SOAP request
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
190 rows, err := conn.QueryContext(ctx, listBottlenecksSQL)
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
191 if err != nil {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
192 return nil, err
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
193 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
194 defer rows.Close()
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
195
2198
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
196 var bottlenecks []bottleneckCountry
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
197
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
198 for rows.Next() {
2016
25967829cf00 Started to simplify the import models.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1756
diff changeset
199 var bn bottleneckCountry
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
200 if err = rows.Scan(
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
201 &bn.ID,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
202 &bn.ResponsibleCountry,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
203 ); err != nil {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
204 return nil, err
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
205 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
206 bottlenecks = append(bottlenecks, bn)
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
207 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
208 if err = rows.Err(); err != nil {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
209 return nil, err
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
210 }
2198
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
211 return bottlenecks, nil
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
212 }
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
213
2198
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
214 func loadFairwayAvailabilities(ctx context.Context, conn *sql.Conn) (map[uniqueFairwayAvailability]int64, error) {
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
215 rows, err := conn.QueryContext(ctx, listFairwayAvailabilitySQL)
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
216 if err != nil {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
217 return nil, err
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
218 }
2198
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
219 defer rows.Close()
2019
aeff01c6edec Moved fairwayavailability to a local model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2016
diff changeset
220 fairwayAvailabilities := map[uniqueFairwayAvailability]int64{}
2198
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
221 for rows.Next() {
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
222 var id int64
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
223 var bnId string
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
224 var sd time.Time
2198
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
225 if err = rows.Scan(
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
226 &id,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
227 &bnId,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
228 &sd,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
229 ); err != nil {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
230 return nil, err
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
231 }
2019
aeff01c6edec Moved fairwayavailability to a local model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2016
diff changeset
232 key := uniqueFairwayAvailability{
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
233 BottleneckId: bnId,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
234 Surdat: sd,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
235 }
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
236 fairwayAvailabilities[key] = id
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
237 }
2198
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
238 if err = rows.Err(); err != nil {
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
239 return nil, err
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
240 }
2198
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
241 return fairwayAvailabilities, nil
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
242 }
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
243
2198
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
244 func latestDate(ctx context.Context, conn *sql.Conn) (pgtype.Timestamp, error) {
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
245 var date pgtype.Timestamp
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
246 err := conn.QueryRowContext(ctx, latestMeasureDateSQL).Scan(&date)
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
247 switch {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
248 case err == sql.ErrNoRows:
2198
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
249 date = pgtype.Timestamp{
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
250 // Fill Database with data of the last 5 days. Change this to a more useful value.
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
251 Time: time.Now().AddDate(0, 0, -5),
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
252 }
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
253 case err != nil:
2198
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
254 return pgtype.Timestamp{}, err
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
255 }
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
256 return date, nil
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
257 }
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
258
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
259 // Do executes the actual fairway availability import.
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
260 func (fa *FairwayAvailability) Do(
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
261 ctx context.Context,
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
262 importID int64,
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
263 conn *sql.Conn,
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
264 feedback Feedback,
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
265 ) (interface{}, error) {
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
266
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
267 bottlenecks, err := loadBottleneckCountries(ctx, conn)
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
268 if err != nil {
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
269 return nil, err
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
270 }
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
271
2198
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
272 fairwayAvailabilities, err := loadFairwayAvailabilities(ctx, conn)
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
273 if err != nil {
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
274 return nil, err
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
275 }
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
276
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
277 latest, err := latestDate(ctx, conn)
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
278 if err != nil {
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
279 return nil, err
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
280 }
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
281
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
282 faids, err := fa.doForFAs(ctx, bottlenecks, fairwayAvailabilities, latest, conn, feedback)
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
283 if err != nil {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
284 feedback.Error("Error processing data: %s", err)
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
285 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
286 if len(faids) == 0 {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
287 feedback.Info("No new fairway availablity data found")
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
288 return nil, nil
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
289 }
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
290 feedback.Info("Processed %d fairway availabilities", len(faids))
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
291 // TODO: needs to be filled more useful.
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
292 summary := struct {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
293 FairwayAvailabilities []string `json:"fairwayAvailabilities"`
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
294 }{
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
295 FairwayAvailabilities: faids,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
296 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
297 return &summary, err
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
298 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
299
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
300 func (fa *FairwayAvailability) doForFAs(
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
301 ctx context.Context,
2016
25967829cf00 Started to simplify the import models.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1756
diff changeset
302 bottlenecks []bottleneckCountry,
2019
aeff01c6edec Moved fairwayavailability to a local model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2016
diff changeset
303 fairwayAvailabilities map[uniqueFairwayAvailability]int64,
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
304 latestDate pgtype.Timestamp,
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
305 conn *sql.Conn,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
306 feedback Feedback,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
307 ) ([]string, error) {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
308 start := time.Now()
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
309
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
310 client := ifaf.NewFairwayAvailabilityService(fa.URL, fa.Insecure, nil)
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
311
2196
e57ba9585aaa Fairway availability import: More idiomatic code / cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
312 bnIds := make([]string, len(bottlenecks))
e57ba9585aaa Fairway availability import: More idiomatic code / cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
313 for i := range bottlenecks {
e57ba9585aaa Fairway availability import: More idiomatic code / cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
314 bnIds[i] = bottlenecks[i].ID
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
315 }
2196
e57ba9585aaa Fairway availability import: More idiomatic code / cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2187
diff changeset
316
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
317 var period ifaf.RequestedPeriod
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
318 period.Date_start = latestDate.Time
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
319 period.Date_end = time.Now()
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
320
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
321 ids := ifaf.ArrayOfString{
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
322 String: bnIds,
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
323 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
324
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
325 req := &ifaf.Get_bottleneck_fa{
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
326 Bottleneck_id: &ids,
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
327 Period: &period,
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
328 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
329 resp, err := client.Get_bottleneck_fa(req)
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
330 if err != nil {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
331 return nil, err
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
332 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
333
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
334 if resp.Get_bottleneck_faResult == nil {
2198
4db1fa4f049c Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2196
diff changeset
335 return nil, errors.New("no fairway availabilities found")
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
336 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
337
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
338 result := resp.Get_bottleneck_faResult
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
339
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
340 tx, err := conn.BeginTx(ctx, nil)
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
341 if err != nil {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
342 return nil, err
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
343 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
344 defer tx.Rollback()
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
345
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
346 insertFAStmt, err := tx.PrepareContext(ctx, insertFASQL)
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
347 if err != nil {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
348 return nil, err
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
349 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
350 defer insertFAStmt.Close()
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
351 insertBnPdfsStmt, err := tx.PrepareContext(ctx, insertBnPdfsSQL)
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
352 if err != nil {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
353 return nil, err
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
354 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
355 defer insertBnPdfsStmt.Close()
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
356 insertEFAStmt, err := tx.PrepareContext(ctx, insertEFASQL)
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
357 if err != nil {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
358 return nil, err
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
359 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
360 defer insertEFAStmt.Close()
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
361 insertFAVStmt, err := tx.PrepareContext(ctx, insertFAVSQL)
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
362 if err != nil {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
363 return nil, err
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
364 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
365 defer insertFAVStmt.Close()
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
366
1694
4a2fad8f57de Imports: Resolved golint issues unrelated to exported symbols commenting.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1676
diff changeset
367 var faIDs []string
4a2fad8f57de Imports: Resolved golint issues unrelated to exported symbols commenting.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1676
diff changeset
368 var faID int64
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
369 feedback.Info("Found %d fairway availabilities", len(result.FairwayAvailability))
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
370 for _, faRes := range result.FairwayAvailability {
2019
aeff01c6edec Moved fairwayavailability to a local model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2016
diff changeset
371 uniqueFa := uniqueFairwayAvailability{
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
372 BottleneckId: faRes.Bottleneck_id,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
373 Surdat: faRes.SURDAT,
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
374 }
1756
295c82c5bc3e Fairway availability import: Cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1755
diff changeset
375 var found bool
295c82c5bc3e Fairway availability import: Cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1755
diff changeset
376 if faID, found = fairwayAvailabilities[uniqueFa]; !found {
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
377 err = insertFAStmt.QueryRowContext(
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
378 ctx,
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
379 faRes.POSITION,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
380 faRes.Bottleneck_id,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
381 faRes.SURDAT,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
382 faRes.Critical,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
383 faRes.Date_Info,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
384 faRes.Source,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
385 ).Scan(&faID)
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
386 if err != nil {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
387 return nil, err
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
388 }
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
389 fairwayAvailabilities[uniqueFa] = faID
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
390 }
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
391 feedback.Info("Processing for Bottleneck %s", faRes.Bottleneck_id)
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
392 faIDs = append(faIDs, faRes.Bottleneck_id)
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
393 if faRes.Bottleneck_PDFs != nil {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
394 bnPdfCount := 0
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
395 for _, bnPdfs := range faRes.Bottleneck_PDFs.PdfInfo {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
396 res, err := insertBnPdfsStmt.ExecContext(
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
397 ctx,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
398 faID,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
399 bnPdfs.ProfilePdfFilename,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
400 bnPdfs.ProfilePdfURL,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
401 bnPdfs.PDF_Generation_Date,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
402 bnPdfs.Source,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
403 )
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
404 if err != nil {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
405 return nil, err
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
406 }
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
407 affected, err := res.RowsAffected()
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
408 if err == nil {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
409 bnPdfCount += int(affected)
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
410 } else {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
411 bnPdfCount++
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
412 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
413 }
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
414 feedback.Info("Add %d Pdfs", bnPdfCount)
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
415 }
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
416 if faRes.Effective_fairway_availability != nil {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
417 efaCount := 0
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
418 for _, efa := range faRes.Effective_fairway_availability.EffectiveFairwayAvailability {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
419 los := efa.Level_of_Service
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
420 fgt := efa.Forecast_generation_time
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
421 if efa.Forecast_generation_time.Status == pgtype.Undefined {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
422 fgt = pgtype.Timestamp{
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
423 Status: pgtype.Null,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
424 }
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
425 }
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
426 res, err := insertEFAStmt.ExecContext(
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
427 ctx,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
428 faID,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
429 efa.Measure_date,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
430 string(*los),
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
431 efa.Available_depth_value,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
432 efa.Available_width_value,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
433 efa.Water_level_value,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
434 efa.Measure_type,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
435 efa.Source,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
436 fgt.Get(),
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
437 efa.Value_lifetime,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
438 )
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
439 if err != nil {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
440 return nil, err
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
441 }
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
442 affected, err := res.RowsAffected()
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
443 if err == nil {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
444 efaCount += int(affected)
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
445 } else {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
446 efaCount++
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
447 }
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
448 }
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
449 feedback.Info("Add %d Effective Fairway Availability", efaCount)
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
450 }
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
451
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
452 if faRes.Reference_values != nil {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
453 rvCount := 0
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
454 for _, fav := range faRes.Reference_values.ReferenceValue {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
455 res, err := insertFAVStmt.ExecContext(
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
456 ctx,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
457 faID,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
458 fav.Level_of_Service,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
459 fav.Fairway_depth,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
460 fav.Fairway_width,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
461 fav.Fairway_radius,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
462 fav.Shallowest_spot_Lat,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
463 fav.Shallowest_spot_Lon,
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
464 )
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
465 if err != nil {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
466 return nil, err
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
467 }
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
468 affected, err := res.RowsAffected()
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
469 if err == nil {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
470 rvCount += int(affected)
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
471 } else {
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
472 rvCount++
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
473 }
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
474 }
1755
d3fe20a13339 Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1754
diff changeset
475 feedback.Info("Add %d Reference Values", rvCount)
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
476 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
477 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
478 feedback.Info("Storing fairway availabilities took %s", time.Since(start))
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
479 if err = tx.Commit(); err == nil {
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
480 feedback.Info("Import of fairway availabilities was successful")
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
481 }
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
482
1694
4a2fad8f57de Imports: Resolved golint issues unrelated to exported symbols commenting.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1676
diff changeset
483 return faIDs, nil
1662
d8ca44615bfc Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff changeset
484 }