Mercurial > gemma
annotate pkg/imports/fa.go @ 4095:287dfd4dbdb6
Merged
author | Sascha L. Teichmann <sascha.teichmann@intevation.de> |
---|---|
date | Thu, 25 Jul 2019 18:12:50 +0200 |
parents | 7fb75deff16b |
children | c9bef8c27685 |
rev | line source |
---|---|
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
1 // This is Free Software under GNU Affero General Public License v >= 3.0 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
2 // without warranty, see README.md and license for details. |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
3 // |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
4 // SPDX-License-Identifier: AGPL-3.0-or-later |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
5 // License-Filename: LICENSES/AGPL-3.0.txt |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
6 // |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
7 // Copyright (C) 2018 by via donau |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
8 // – Österreichische Wasserstraßen-Gesellschaft mbH |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
9 // Software engineering by Intevation GmbH |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
10 // |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
11 // Author(s): |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
12 // * Raimund Renkert <raimund.renkert@intevation.de> |
2203
8d1a945d0c3b
Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2202
diff
changeset
|
13 // * Sascha L. Teichmann <sascha.teichmann@intevation.de> |
1694
4a2fad8f57de
Imports: Resolved golint issues unrelated to exported symbols commenting.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
1676
diff
changeset
|
14 |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
15 package imports |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
16 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
17 import ( |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
18 "context" |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
19 "database/sql" |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
20 "errors" |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
21 "fmt" |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
22 "sort" |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
23 "time" |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
24 |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
25 "github.com/jackc/pgx/pgtype" |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
26 |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
27 "gemma.intevation.de/gemma/pkg/soap/ifaf" |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
28 ) |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
29 |
1696
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
30 // FairwayAvailability imports fairway availability data |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
31 // from an IFAF SOAP service. |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
32 type FairwayAvailability struct { |
1696
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
33 // URL is the URL of the SOAP service. |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
34 URL string `json:"url"` |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
35 // Insecure indicates if HTTPS traffic |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
36 // should validate certificates or not. |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
37 Insecure bool `json:"insecure"` |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
38 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
39 |
1696
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
40 // FAJobKind is import queue type identifier. |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
41 const FAJobKind JobKind = "fa" |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
42 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
43 const ( |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
44 listBottlenecksSQL = ` |
3645
02951a62e8c6
'Historicise' bottlenecks on import
Tom Gottfried <tom@intevation.de>
parents:
3219
diff
changeset
|
45 SELECT DISTINCT |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
46 bottleneck_id |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
47 FROM waterway.bottlenecks |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
48 WHERE responsible_country = users.current_user_country() |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
49 AND staging_done = true |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
50 ORDER BY bottleneck_id |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
51 ` |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
52 |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
53 latestMeasureDateSQL = ` |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
54 SELECT |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
55 measure_date |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
56 FROM waterway.effective_fairway_availability |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
57 ORDER BY measure_date DESC LIMIT 1 |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
58 ` |
4079
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
59 |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
60 insertFASQL = ` |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
61 INSERT INTO waterway.fairway_availability ( |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
62 position_code, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
63 bottleneck_id, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
64 surdat, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
65 critical, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
66 date_info, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
67 source_organization |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
68 ) VALUES ( |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
69 $1, |
3666
db87f34805fb
Align bottleneck validity at gauges
Tom Gottfried <tom@intevation.de>
parents:
3645
diff
changeset
|
70 -- Always associate fairway availability data to newest bottleneck |
db87f34805fb
Align bottleneck validity at gauges
Tom Gottfried <tom@intevation.de>
parents:
3645
diff
changeset
|
71 -- version to prevent problems in analysis over longer time periods |
db87f34805fb
Align bottleneck validity at gauges
Tom Gottfried <tom@intevation.de>
parents:
3645
diff
changeset
|
72 (SELECT id FROM waterway.bottlenecks WHERE bottleneck_id = $2 |
db87f34805fb
Align bottleneck validity at gauges
Tom Gottfried <tom@intevation.de>
parents:
3645
diff
changeset
|
73 ORDER BY validity DESC FETCH FIRST ROW ONLY), |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
74 $3, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
75 $4, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
76 $5, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
77 $6 |
4079
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
78 ) ON CONFLICT (bottleneck_id, surdat) DO UPDATE SET |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
79 position_code = EXCLUDED.position_code, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
80 critical = EXCLUDED.critical, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
81 date_info = EXCLUDED.date_info, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
82 source_organization = EXCLUDED.source_organization |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
83 RETURNING id` |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
84 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
85 insertBnPdfsSQL = ` |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
86 INSERT INTO waterway.bottleneck_pdfs ( |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
87 fairway_availability_id, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
88 profile_pdf_filename, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
89 profile_pdf_url, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
90 pdf_generation_date, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
91 source_organization |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
92 ) VALUES ( |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
93 $1, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
94 $2, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
95 $3, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
96 $4, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
97 $5 |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
98 ) ON CONFLICT ON CONSTRAINT bottleneck_pdfs_pkey DO NOTHING` |
4079
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
99 |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
100 insertEFASQL = ` |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
101 INSERT INTO waterway.effective_fairway_availability ( |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
102 fairway_availability_id, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
103 measure_date, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
104 level_of_service, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
105 available_depth_value, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
106 available_width_value, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
107 water_level_value, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
108 measure_type, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
109 source_organization, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
110 forecast_generation_time, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
111 value_lifetime |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
112 ) VALUES ( |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
113 $1, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
114 $2, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
115 (SELECT |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
116 level_of_service |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
117 FROM levels_of_service |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
118 WHERE name = $3), |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
119 $4, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
120 $5, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
121 $6, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
122 $7, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
123 $8, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
124 $9, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
125 $10 |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
126 ) ON CONFLICT ON CONSTRAINT effective_fairway_availability_pkey DO NOTHING` |
4079
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
127 |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
128 insertFAVSQL = ` |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
129 INSERT INTO waterway.fa_reference_values ( |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
130 fairway_availability_id, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
131 level_of_service, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
132 fairway_depth, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
133 fairway_width, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
134 fairway_radius, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
135 shallowest_spot |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
136 ) VALUES ( |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
137 $1, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
138 (SELECT |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
139 level_of_service |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
140 FROM levels_of_service |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
141 WHERE name = $2), |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
142 $3, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
143 $4, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
144 $5, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
145 ST_MakePoint($6, $7)::geography |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
146 )ON CONFLICT ON CONSTRAINT fa_reference_values_pkey DO NOTHING` |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
147 ) |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
148 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
149 type faJobCreator struct{} |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
150 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
151 func init() { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
152 RegisterJobCreator(FAJobKind, faJobCreator{}) |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
153 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
154 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
155 func (faJobCreator) Description() string { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
156 return "fairway availability" |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
157 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
158 |
2187
7c83b5277c1c
Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
159 func (faJobCreator) Create() Job { return new(FairwayAvailability) } |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
160 |
3219
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2253
diff
changeset
|
161 func (faJobCreator) Depends() [2][]string { |
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2253
diff
changeset
|
162 return [2][]string{ |
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2253
diff
changeset
|
163 {"effective_fairway_availability", "fa_reference_values", |
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2253
diff
changeset
|
164 "bottleneck_pdfs", "fairway_availability"}, |
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2253
diff
changeset
|
165 {"bottlenecks", "levels_of_service"}, |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
166 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
167 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
168 |
1754
807569b08513
Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1696
diff
changeset
|
169 func (faJobCreator) AutoAccept() bool { return true } |
807569b08513
Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1696
diff
changeset
|
170 |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
171 // StageDone moves the imported fairway availablities out of the staging area. |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
172 // Currently doing nothing. |
1676
4407ecaa2192
Imports: Cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1663
diff
changeset
|
173 func (faJobCreator) StageDone(context.Context, *sql.Tx, int64) error { |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
174 return nil |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
175 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
176 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
177 // CleanUp of a fairway availablities import is a NOP. |
1676
4407ecaa2192
Imports: Cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1663
diff
changeset
|
178 func (*FairwayAvailability) CleanUp() error { return nil } |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
179 |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
180 type bottlenecks []string |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
181 |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
182 func (bns bottlenecks) contains(bn string) bool { |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
183 idx := sort.SearchStrings(bns, bn) |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
184 return idx < len(bns) && bns[idx] == bn |
2016
25967829cf00
Started to simplify the import models.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1756
diff
changeset
|
185 } |
25967829cf00
Started to simplify the import models.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1756
diff
changeset
|
186 |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
187 func loadBottleneckCountries(ctx context.Context, tx *sql.Tx) (bottlenecks, error) { |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
188 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
189 // Get available bottlenecks from database for use as filter in SOAP request |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
190 rows, err := tx.QueryContext(ctx, listBottlenecksSQL) |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
191 if err != nil { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
192 return nil, err |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
193 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
194 defer rows.Close() |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
195 |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
196 var bns bottlenecks |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
197 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
198 for rows.Next() { |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
199 var bn string |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
200 if err = rows.Scan(&bn); err != nil { |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
201 return nil, err |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
202 } |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
203 bns = append(bns, bn) |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
204 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
205 if err = rows.Err(); err != nil { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
206 return nil, err |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
207 } |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
208 |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
209 return bns, nil |
2198
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
210 } |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
211 |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
212 func latestDate(ctx context.Context, tx *sql.Tx) (pgtype.Timestamp, error) { |
2198
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
213 var date pgtype.Timestamp |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
214 err := tx.QueryRowContext(ctx, latestMeasureDateSQL).Scan(&date) |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
215 switch { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
216 case err == sql.ErrNoRows: |
2198
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
217 date = pgtype.Timestamp{ |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
218 // Fill Database with data of the last 5 days. Change this to a more useful value. |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
219 Time: time.Now().AddDate(0, 0, -5), |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
220 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
221 case err != nil: |
2198
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
222 return pgtype.Timestamp{}, err |
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
223 } |
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
224 return date, nil |
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
225 } |
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
226 |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
227 func storeFairwayAvailability( |
2198
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
228 ctx context.Context, |
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
229 conn *sql.Conn, |
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
230 feedback Feedback, |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
231 fetch func(context.Context, *sql.Tx, bottlenecks) ([]*ifaf.FairwayAvailability, error), |
2198
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
232 ) (interface{}, error) { |
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
233 |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
234 start := time.Now() |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
235 |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
236 tx, err := conn.BeginTx(ctx, nil) |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
237 if err != nil { |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
238 return nil, err |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
239 } |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
240 defer tx.Rollback() |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
241 |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
242 bns, err := loadBottleneckCountries(ctx, tx) |
2198
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
243 if err != nil { |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
244 return nil, err |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
245 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
246 |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
247 fas, err := fetch(ctx, tx, bns) |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
248 if err != nil { |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
249 return nil, err |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
250 } |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
251 |
4079
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
252 faids, err := doForFAs(ctx, bns, fas, tx, feedback) |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
253 if err != nil { |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
254 return nil, fmt.Errorf("Error processing data: %v", err) |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
255 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
256 if len(faids) == 0 { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
257 feedback.Info("No new fairway availablity data found") |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
258 return nil, UnchangedError("No new fairway availablity data found") |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
259 } |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
260 feedback.Info("Processed %d fairway availabilities", len(faids)) |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
261 |
2233
137addc77b1b
Gauge measurement imports: Do all database ops in one transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2203
diff
changeset
|
262 if err = tx.Commit(); err != nil { |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
263 feedback.Info( |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
264 "Importing fairway availabilities failed after %s", time.Since(start)) |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
265 return nil, err |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
266 } |
2233
137addc77b1b
Gauge measurement imports: Do all database ops in one transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2203
diff
changeset
|
267 feedback.Info( |
137addc77b1b
Gauge measurement imports: Do all database ops in one transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2203
diff
changeset
|
268 "Importing fairway availabilities successfully took %s", time.Since(start)) |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
269 |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
270 // TODO: needs to be filled more useful. |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
271 summary := struct { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
272 FairwayAvailabilities []string `json:"fairwayAvailabilities"` |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
273 }{ |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
274 FairwayAvailabilities: faids, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
275 } |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
276 return &summary, nil |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
277 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
278 |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
279 func doForFAs( |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
280 ctx context.Context, |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
281 bnIds bottlenecks, |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
282 fas []*ifaf.FairwayAvailability, |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
283 tx *sql.Tx, |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
284 feedback Feedback, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
285 ) ([]string, error) { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
286 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
287 insertFAStmt, err := tx.PrepareContext(ctx, insertFASQL) |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
288 if err != nil { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
289 return nil, err |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
290 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
291 defer insertFAStmt.Close() |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
292 insertBnPdfsStmt, err := tx.PrepareContext(ctx, insertBnPdfsSQL) |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
293 if err != nil { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
294 return nil, err |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
295 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
296 defer insertBnPdfsStmt.Close() |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
297 insertEFAStmt, err := tx.PrepareContext(ctx, insertEFASQL) |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
298 if err != nil { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
299 return nil, err |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
300 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
301 defer insertEFAStmt.Close() |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
302 insertFAVStmt, err := tx.PrepareContext(ctx, insertFAVSQL) |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
303 if err != nil { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
304 return nil, err |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
305 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
306 defer insertFAVStmt.Close() |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
307 |
1694
4a2fad8f57de
Imports: Resolved golint issues unrelated to exported symbols commenting.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
1676
diff
changeset
|
308 var faIDs []string |
4a2fad8f57de
Imports: Resolved golint issues unrelated to exported symbols commenting.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
1676
diff
changeset
|
309 var faID int64 |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
310 feedback.Info("Found %d fairway availabilities", len(fas)) |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
311 for _, faRes := range fas { |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
312 if !bnIds.contains(faRes.Bottleneck_id) { |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
313 feedback.Warn("Bottleneck %s not found in database.", faRes.Bottleneck_id) |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
314 continue |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
315 } |
4079
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
316 err = insertFAStmt.QueryRowContext( |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
317 ctx, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
318 faRes.POSITION, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
319 faRes.Bottleneck_id, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
320 faRes.SURDAT, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
321 faRes.Critical, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
322 faRes.Date_Info, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
323 faRes.Source, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
324 ).Scan(&faID) |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
325 if err != nil { |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
326 return nil, err |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
327 } |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
328 feedback.Info("Processing for Bottleneck %s", faRes.Bottleneck_id) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
329 faIDs = append(faIDs, faRes.Bottleneck_id) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
330 if faRes.Bottleneck_PDFs != nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
331 bnPdfCount := 0 |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
332 for _, bnPdfs := range faRes.Bottleneck_PDFs.PdfInfo { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
333 res, err := insertBnPdfsStmt.ExecContext( |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
334 ctx, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
335 faID, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
336 bnPdfs.ProfilePdfFilename, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
337 bnPdfs.ProfilePdfURL, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
338 bnPdfs.PDF_Generation_Date, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
339 bnPdfs.Source, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
340 ) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
341 if err != nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
342 return nil, err |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
343 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
344 affected, err := res.RowsAffected() |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
345 if err == nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
346 bnPdfCount += int(affected) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
347 } else { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
348 bnPdfCount++ |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
349 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
350 } |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
351 feedback.Info("Add %d Pdfs", bnPdfCount) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
352 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
353 if faRes.Effective_fairway_availability != nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
354 efaCount := 0 |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
355 for _, efa := range faRes.Effective_fairway_availability.EffectiveFairwayAvailability { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
356 los := efa.Level_of_Service |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
357 fgt := efa.Forecast_generation_time |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
358 if efa.Forecast_generation_time.Status == pgtype.Undefined { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
359 fgt = pgtype.Timestamp{ |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
360 Status: pgtype.Null, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
361 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
362 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
363 res, err := insertEFAStmt.ExecContext( |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
364 ctx, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
365 faID, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
366 efa.Measure_date, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
367 string(*los), |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
368 efa.Available_depth_value, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
369 efa.Available_width_value, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
370 efa.Water_level_value, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
371 efa.Measure_type, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
372 efa.Source, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
373 fgt.Get(), |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
374 efa.Value_lifetime, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
375 ) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
376 if err != nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
377 return nil, err |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
378 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
379 affected, err := res.RowsAffected() |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
380 if err == nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
381 efaCount += int(affected) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
382 } else { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
383 efaCount++ |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
384 } |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
385 } |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
386 feedback.Info("Add %d Effective Fairway Availability", efaCount) |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
387 } |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
388 |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
389 if faRes.Reference_values != nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
390 rvCount := 0 |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
391 for _, fav := range faRes.Reference_values.ReferenceValue { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
392 res, err := insertFAVStmt.ExecContext( |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
393 ctx, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
394 faID, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
395 fav.Level_of_Service, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
396 fav.Fairway_depth, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
397 fav.Fairway_width, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
398 fav.Fairway_radius, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
399 fav.Shallowest_spot_Lat, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
400 fav.Shallowest_spot_Lon, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
401 ) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
402 if err != nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
403 return nil, err |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
404 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
405 affected, err := res.RowsAffected() |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
406 if err == nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
407 rvCount += int(affected) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
408 } else { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
409 rvCount++ |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
410 } |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
411 } |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
412 feedback.Info("Add %d Reference Values", rvCount) |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
413 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
414 } |
1694
4a2fad8f57de
Imports: Resolved golint issues unrelated to exported symbols commenting.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
1676
diff
changeset
|
415 return faIDs, nil |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
416 } |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
417 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
418 // Do executes the actual fairway availability import. |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
419 func (fa *FairwayAvailability) Do( |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
420 ctx context.Context, |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
421 importID int64, |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
422 conn *sql.Conn, |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
423 feedback Feedback, |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
424 ) (interface{}, error) { |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
425 |
2203
8d1a945d0c3b
Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2202
diff
changeset
|
426 fetch := func( |
8d1a945d0c3b
Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2202
diff
changeset
|
427 ctx context.Context, |
8d1a945d0c3b
Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2202
diff
changeset
|
428 tx *sql.Tx, bns bottlenecks, |
8d1a945d0c3b
Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2202
diff
changeset
|
429 ) ([]*ifaf.FairwayAvailability, error) { |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
430 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
431 latest, err := latestDate(ctx, tx) |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
432 if err != nil { |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
433 return nil, err |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
434 } |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
435 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
436 client := ifaf.NewFairwayAvailabilityService(fa.URL, fa.Insecure, nil) |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
437 |
2253
cfc523c70e90
Uploaded imports: Small code clean-ups.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2233
diff
changeset
|
438 period := ifaf.RequestedPeriod{ |
cfc523c70e90
Uploaded imports: Small code clean-ups.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2233
diff
changeset
|
439 Date_start: latest.Time, |
cfc523c70e90
Uploaded imports: Small code clean-ups.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2233
diff
changeset
|
440 Date_end: time.Now(), |
cfc523c70e90
Uploaded imports: Small code clean-ups.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2233
diff
changeset
|
441 } |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
442 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
443 ids := ifaf.ArrayOfString{String: bns} |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
444 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
445 req := &ifaf.Get_bottleneck_fa{ |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
446 Bottleneck_id: &ids, |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
447 Period: &period, |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
448 } |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
449 resp, err := client.Get_bottleneck_fa(req) |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
450 if err != nil { |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
451 return nil, err |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
452 } |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
453 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
454 if resp.Get_bottleneck_faResult == nil { |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
455 return nil, errors.New("no fairway availabilities found") |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
456 } |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
457 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
458 result := resp.Get_bottleneck_faResult |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
459 return result.FairwayAvailability, nil |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
460 } |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
461 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
462 return storeFairwayAvailability(ctx, conn, feedback, fetch) |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
463 } |