Mercurial > gemma
annotate pkg/imports/fa.go @ 4687:f73868037365
morphology: do correct clipping of classbreak heights based on z range of model.
author | Sascha L. Teichmann <sascha.teichmann@intevation.de> |
---|---|
date | Wed, 16 Oct 2019 13:02:08 +0200 |
parents | 1e96ff2da1f3 |
children | ca6a5f722471 |
rev | line source |
---|---|
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
1 // This is Free Software under GNU Affero General Public License v >= 3.0 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
2 // without warranty, see README.md and license for details. |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
3 // |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
4 // SPDX-License-Identifier: AGPL-3.0-or-later |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
5 // License-Filename: LICENSES/AGPL-3.0.txt |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
6 // |
4072
1eb39e9e8ec2
Adapted fwa-import to new historization model.
Sascha Wilde <wilde@intevation.de>
parents:
3666
diff
changeset
|
7 // Copyright (C) 2018,2019 by via donau |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
8 // – Österreichische Wasserstraßen-Gesellschaft mbH |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
9 // Software engineering by Intevation GmbH |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
10 // |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
11 // Author(s): |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
12 // * Raimund Renkert <raimund.renkert@intevation.de> |
2203
8d1a945d0c3b
Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2202
diff
changeset
|
13 // * Sascha L. Teichmann <sascha.teichmann@intevation.de> |
4072
1eb39e9e8ec2
Adapted fwa-import to new historization model.
Sascha Wilde <wilde@intevation.de>
parents:
3666
diff
changeset
|
14 // * Sascha Wilde <wilde@intevation.de> |
1694
4a2fad8f57de
Imports: Resolved golint issues unrelated to exported symbols commenting.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
1676
diff
changeset
|
15 |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
16 package imports |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
17 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
18 import ( |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
19 "context" |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
20 "database/sql" |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
21 "errors" |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
22 "fmt" |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
23 "sort" |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
24 "time" |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
25 |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
26 "github.com/jackc/pgx/pgtype" |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
27 |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
28 "gemma.intevation.de/gemma/pkg/soap/ifaf" |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
29 ) |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
30 |
1696
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
31 // FairwayAvailability imports fairway availability data |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
32 // from an IFAF SOAP service. |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
33 type FairwayAvailability struct { |
1696
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
34 // URL is the URL of the SOAP service. |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
35 URL string `json:"url"` |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
36 // Insecure indicates if HTTPS traffic |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
37 // should validate certificates or not. |
ad5e1cddaa09
Imports: Resolved the remaining golint issues with this package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1694
diff
changeset
|
38 Insecure bool `json:"insecure"` |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
39 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
40 |
4281
c470d2202823
Limit time range requested from fa services.
Sascha Wilde <wilde@intevation.de>
parents:
4158
diff
changeset
|
41 const ( |
c470d2202823
Limit time range requested from fa services.
Sascha Wilde <wilde@intevation.de>
parents:
4158
diff
changeset
|
42 // FAJobKind is import queue type identifier. |
c470d2202823
Limit time range requested from fa services.
Sascha Wilde <wilde@intevation.de>
parents:
4158
diff
changeset
|
43 FAJobKind JobKind = "fa" |
4283
c811d5da69bd
Use untyped constant and let the compiler do the work.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
4281
diff
changeset
|
44 ) |
c811d5da69bd
Use untyped constant and let the compiler do the work.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
4281
diff
changeset
|
45 |
c811d5da69bd
Use untyped constant and let the compiler do the work.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
4281
diff
changeset
|
46 const ( |
c811d5da69bd
Use untyped constant and let the compiler do the work.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
4281
diff
changeset
|
47 // maxHistoryDays is the numbers of days to look back. |
c811d5da69bd
Use untyped constant and let the compiler do the work.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
4281
diff
changeset
|
48 maxHistoryDays = 7 |
4281
c470d2202823
Limit time range requested from fa services.
Sascha Wilde <wilde@intevation.de>
parents:
4158
diff
changeset
|
49 ) |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
50 |
4655
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
51 // Find bottlenecks of the current users country for a given time range |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
52 const ( |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
53 listBottlenecksSQL = ` |
3645
02951a62e8c6
'Historicise' bottlenecks on import
Tom Gottfried <tom@intevation.de>
parents:
3219
diff
changeset
|
54 SELECT DISTINCT |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
55 bottleneck_id |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
56 FROM waterway.bottlenecks |
4158
5466562cca60
Remove utility function with possibly bad performance impact
Tom Gottfried <tom@intevation.de>
parents:
4081
diff
changeset
|
57 WHERE responsible_country = ( |
5466562cca60
Remove utility function with possibly bad performance impact
Tom Gottfried <tom@intevation.de>
parents:
4081
diff
changeset
|
58 SELECT country FROM users.list_users WHERE username = current_user) |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
59 AND staging_done = true |
4655
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
60 AND validity && $1 |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
61 ORDER BY bottleneck_id |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
62 ` |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
63 |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
64 latestMeasureDateSQL = ` |
4655
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
65 SELECT CASE WHEN (SELECT array_agg(DISTINCT bottleneck_id) @> $1::varchar[] |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
66 FROM waterway.fairway_availability) |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
67 THEN (SELECT min(x.m) FROM |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
68 (SELECT max(efa.measure_date) AS m |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
69 FROM waterway.fairway_availability fa, |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
70 waterway.effective_fairway_availability efa |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
71 WHERE fa.bottleneck_id = ANY($1) |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
72 AND efa.fairway_availability_id = fa.id |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
73 GROUP BY fa.bottleneck_id) AS x) |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
74 END |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
75 ` |
4079
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
76 |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
77 insertFASQL = ` |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
78 INSERT INTO waterway.fairway_availability ( |
4461
7128741faeb9
Changed fa data to accept "position" as strings, not a 2char "position_code"
Sascha Wilde <wilde@intevation.de>
parents:
4283
diff
changeset
|
79 position, |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
80 bottleneck_id, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
81 surdat, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
82 critical, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
83 date_info, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
84 source_organization |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
85 ) VALUES ( |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
86 $1, |
4072
1eb39e9e8ec2
Adapted fwa-import to new historization model.
Sascha Wilde <wilde@intevation.de>
parents:
3666
diff
changeset
|
87 $2, |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
88 $3, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
89 $4, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
90 $5, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
91 $6 |
4079
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
92 ) ON CONFLICT (bottleneck_id, surdat) DO UPDATE SET |
4461
7128741faeb9
Changed fa data to accept "position" as strings, not a 2char "position_code"
Sascha Wilde <wilde@intevation.de>
parents:
4283
diff
changeset
|
93 position = EXCLUDED.position, |
4079
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
94 critical = EXCLUDED.critical, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
95 date_info = EXCLUDED.date_info, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
96 source_organization = EXCLUDED.source_organization |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
97 RETURNING id` |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
98 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
99 insertBnPdfsSQL = ` |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
100 INSERT INTO waterway.bottleneck_pdfs ( |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
101 fairway_availability_id, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
102 profile_pdf_filename, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
103 profile_pdf_url, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
104 pdf_generation_date, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
105 source_organization |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
106 ) VALUES ( |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
107 $1, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
108 $2, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
109 $3, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
110 $4, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
111 $5 |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
112 ) ON CONFLICT ON CONSTRAINT bottleneck_pdfs_pkey DO NOTHING` |
4079
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
113 |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
114 insertEFASQL = ` |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
115 INSERT INTO waterway.effective_fairway_availability ( |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
116 fairway_availability_id, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
117 measure_date, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
118 level_of_service, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
119 available_depth_value, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
120 available_width_value, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
121 water_level_value, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
122 measure_type, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
123 source_organization, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
124 forecast_generation_time, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
125 value_lifetime |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
126 ) VALUES ( |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
127 $1, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
128 $2, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
129 (SELECT |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
130 level_of_service |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
131 FROM levels_of_service |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
132 WHERE name = $3), |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
133 $4, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
134 $5, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
135 $6, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
136 $7, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
137 $8, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
138 $9, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
139 $10 |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
140 ) ON CONFLICT ON CONSTRAINT effective_fairway_availability_pkey DO NOTHING` |
4079
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
141 |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
142 insertFAVSQL = ` |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
143 INSERT INTO waterway.fa_reference_values ( |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
144 fairway_availability_id, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
145 level_of_service, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
146 fairway_depth, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
147 fairway_width, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
148 fairway_radius, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
149 shallowest_spot |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
150 ) VALUES ( |
1663
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
151 $1, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
152 (SELECT |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
153 level_of_service |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
154 FROM levels_of_service |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
155 WHERE name = $2), |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
156 $3, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
157 $4, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
158 $5, |
10e3dd3b9363
Fairway availability import: Don't use tabs in SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1662
diff
changeset
|
159 ST_MakePoint($6, $7)::geography |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
160 )ON CONFLICT ON CONSTRAINT fa_reference_values_pkey DO NOTHING` |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
161 ) |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
162 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
163 type faJobCreator struct{} |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
164 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
165 func init() { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
166 RegisterJobCreator(FAJobKind, faJobCreator{}) |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
167 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
168 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
169 func (faJobCreator) Description() string { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
170 return "fairway availability" |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
171 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
172 |
2187
7c83b5277c1c
Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2168
diff
changeset
|
173 func (faJobCreator) Create() Job { return new(FairwayAvailability) } |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
174 |
3219
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2253
diff
changeset
|
175 func (faJobCreator) Depends() [2][]string { |
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2253
diff
changeset
|
176 return [2][]string{ |
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2253
diff
changeset
|
177 {"effective_fairway_availability", "fa_reference_values", |
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2253
diff
changeset
|
178 "bottleneck_pdfs", "fairway_availability"}, |
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2253
diff
changeset
|
179 {"bottlenecks", "levels_of_service"}, |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
180 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
181 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
182 |
1754
807569b08513
Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1696
diff
changeset
|
183 func (faJobCreator) AutoAccept() bool { return true } |
807569b08513
Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1696
diff
changeset
|
184 |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
185 // StageDone moves the imported fairway availablities out of the staging area. |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
186 // Currently doing nothing. |
1676
4407ecaa2192
Imports: Cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1663
diff
changeset
|
187 func (faJobCreator) StageDone(context.Context, *sql.Tx, int64) error { |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
188 return nil |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
189 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
190 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
191 // CleanUp of a fairway availablities import is a NOP. |
1676
4407ecaa2192
Imports: Cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1663
diff
changeset
|
192 func (*FairwayAvailability) CleanUp() error { return nil } |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
193 |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
194 type bottlenecks []string |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
195 |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
196 func (bns bottlenecks) contains(bn string) bool { |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
197 idx := sort.SearchStrings(bns, bn) |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
198 return idx < len(bns) && bns[idx] == bn |
2016
25967829cf00
Started to simplify the import models.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1756
diff
changeset
|
199 } |
25967829cf00
Started to simplify the import models.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1756
diff
changeset
|
200 |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
201 func loadBottleneckCountries(ctx context.Context, tx *sql.Tx) (bottlenecks, error) { |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
202 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
203 // Get available bottlenecks from database for use as filter in SOAP request |
4655
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
204 // We are only interested in bottlenecks which were valid in the last |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
205 // maxHistoryDays |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
206 var tfrom pgtype.Timestamptz |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
207 tfrom.Set(time.Now().AddDate(0, 0, -maxHistoryDays)) |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
208 trange := pgtype.Tstzrange{ |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
209 Lower: tfrom, |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
210 LowerType: pgtype.Inclusive, |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
211 UpperType: pgtype.Unbounded, |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
212 Status: pgtype.Present, |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
213 } |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
214 |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
215 rows, err := tx.QueryContext(ctx, listBottlenecksSQL, trange) |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
216 if err != nil { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
217 return nil, err |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
218 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
219 defer rows.Close() |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
220 |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
221 var bns bottlenecks |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
222 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
223 for rows.Next() { |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
224 var bn string |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
225 if err = rows.Scan(&bn); err != nil { |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
226 return nil, err |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
227 } |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
228 bns = append(bns, bn) |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
229 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
230 if err = rows.Err(); err != nil { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
231 return nil, err |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
232 } |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
233 |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
234 return bns, nil |
2198
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
235 } |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
236 |
4655
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
237 // Get the earliest of all the latest measure_dates for a list of bottlenecks. |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
238 // We do not pick the latest of all dates, so that we don't miss data if one of |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
239 // the bottlenecks we are taking into account was not active for some time... |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
240 func latestDate( |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
241 ctx context.Context, |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
242 tx *sql.Tx, |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
243 bns bottlenecks, |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
244 ) (pgtype.Timestamp, error) { |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
245 var ( |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
246 date pgtype.Timestamp |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
247 pgbns pgtype.TextArray |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
248 ) |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
249 pgbns.Set(bns) |
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
250 err := tx.QueryRowContext(ctx, latestMeasureDateSQL, &pgbns).Scan(&date) |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
251 switch { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
252 case err == sql.ErrNoRows: |
2198
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
253 date = pgtype.Timestamp{ |
4283
c811d5da69bd
Use untyped constant and let the compiler do the work.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
4281
diff
changeset
|
254 Time: time.Now().AddDate(0, 0, -maxHistoryDays), |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
255 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
256 case err != nil: |
2198
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
257 return pgtype.Timestamp{}, err |
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
258 } |
4281
c470d2202823
Limit time range requested from fa services.
Sascha Wilde <wilde@intevation.de>
parents:
4158
diff
changeset
|
259 // Limit request range to MaxHistoryDays. Otherwise the Request might |
c470d2202823
Limit time range requested from fa services.
Sascha Wilde <wilde@intevation.de>
parents:
4158
diff
changeset
|
260 // fail, e.g. the AT service has an upper liimit of 10000 results. |
c470d2202823
Limit time range requested from fa services.
Sascha Wilde <wilde@intevation.de>
parents:
4158
diff
changeset
|
261 // |
c470d2202823
Limit time range requested from fa services.
Sascha Wilde <wilde@intevation.de>
parents:
4158
diff
changeset
|
262 // FIXME: the better solution would be to detect such errors and |
c470d2202823
Limit time range requested from fa services.
Sascha Wilde <wilde@intevation.de>
parents:
4158
diff
changeset
|
263 // dynamically and implement some kind of chunking... |
4283
c811d5da69bd
Use untyped constant and let the compiler do the work.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
4281
diff
changeset
|
264 if time.Since(date.Time).Hours() > maxHistoryDays*24 { |
4281
c470d2202823
Limit time range requested from fa services.
Sascha Wilde <wilde@intevation.de>
parents:
4158
diff
changeset
|
265 date = pgtype.Timestamp{ |
4283
c811d5da69bd
Use untyped constant and let the compiler do the work.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
4281
diff
changeset
|
266 Time: time.Now().AddDate(0, 0, -maxHistoryDays), |
4281
c470d2202823
Limit time range requested from fa services.
Sascha Wilde <wilde@intevation.de>
parents:
4158
diff
changeset
|
267 } |
c470d2202823
Limit time range requested from fa services.
Sascha Wilde <wilde@intevation.de>
parents:
4158
diff
changeset
|
268 } |
c470d2202823
Limit time range requested from fa services.
Sascha Wilde <wilde@intevation.de>
parents:
4158
diff
changeset
|
269 |
2198
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
270 return date, nil |
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
271 } |
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
272 |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
273 func storeFairwayAvailability( |
2198
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
274 ctx context.Context, |
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
275 conn *sql.Conn, |
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
276 feedback Feedback, |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
277 fetch func(context.Context, *sql.Tx, bottlenecks) ([]*ifaf.FairwayAvailability, error), |
2198
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
278 ) (interface{}, error) { |
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
279 |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
280 start := time.Now() |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
281 |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
282 tx, err := conn.BeginTx(ctx, nil) |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
283 if err != nil { |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
284 return nil, err |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
285 } |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
286 defer tx.Rollback() |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
287 |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
288 bns, err := loadBottleneckCountries(ctx, tx) |
2198
4db1fa4f049c
Fairway availabilty import: Fixed row query leak.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2196
diff
changeset
|
289 if err != nil { |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
290 return nil, err |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
291 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
292 |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
293 fas, err := fetch(ctx, tx, bns) |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
294 if err != nil { |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
295 return nil, err |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
296 } |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
297 |
4079
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
298 faids, err := doForFAs(ctx, bns, fas, tx, feedback) |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
299 if err != nil { |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
300 return nil, fmt.Errorf("Error processing data: %v", err) |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
301 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
302 if len(faids) == 0 { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
303 feedback.Info("No new fairway availablity data found") |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
304 return nil, UnchangedError("No new fairway availablity data found") |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
305 } |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
306 feedback.Info("Processed %d fairway availabilities", len(faids)) |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
307 |
2233
137addc77b1b
Gauge measurement imports: Do all database ops in one transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2203
diff
changeset
|
308 if err = tx.Commit(); err != nil { |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
309 feedback.Info( |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
310 "Importing fairway availabilities failed after %s", time.Since(start)) |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
311 return nil, err |
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
312 } |
2233
137addc77b1b
Gauge measurement imports: Do all database ops in one transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2203
diff
changeset
|
313 feedback.Info( |
137addc77b1b
Gauge measurement imports: Do all database ops in one transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2203
diff
changeset
|
314 "Importing fairway availabilities successfully took %s", time.Since(start)) |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
315 |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
316 // TODO: needs to be filled more useful. |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
317 summary := struct { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
318 FairwayAvailabilities []string `json:"fairwayAvailabilities"` |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
319 }{ |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
320 FairwayAvailabilities: faids, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
321 } |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
322 return &summary, nil |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
323 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
324 |
4515
7465b6244d5e
FA import: Code clean-up for handling default LOS.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4513
diff
changeset
|
325 // defaultLOS defaults to LOS3 when no expicit LOS is given. |
7465b6244d5e
FA import: Code clean-up for handling default LOS.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4513
diff
changeset
|
326 func defaultLOS(los *ifaf.LosEnum) ifaf.LosEnum { |
7465b6244d5e
FA import: Code clean-up for handling default LOS.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4513
diff
changeset
|
327 if los == nil { |
7465b6244d5e
FA import: Code clean-up for handling default LOS.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4513
diff
changeset
|
328 return ifaf.LosEnumLOS3 |
7465b6244d5e
FA import: Code clean-up for handling default LOS.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4513
diff
changeset
|
329 } |
7465b6244d5e
FA import: Code clean-up for handling default LOS.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4513
diff
changeset
|
330 return *los |
7465b6244d5e
FA import: Code clean-up for handling default LOS.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4513
diff
changeset
|
331 } |
7465b6244d5e
FA import: Code clean-up for handling default LOS.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4513
diff
changeset
|
332 |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
333 func doForFAs( |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
334 ctx context.Context, |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
335 bnIds bottlenecks, |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
336 fas []*ifaf.FairwayAvailability, |
2200
64147a137e0a
Fairway availability import: Run all db ops in same transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2198
diff
changeset
|
337 tx *sql.Tx, |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
338 feedback Feedback, |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
339 ) ([]string, error) { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
340 |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
341 insertFAStmt, err := tx.PrepareContext(ctx, insertFASQL) |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
342 if err != nil { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
343 return nil, err |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
344 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
345 defer insertFAStmt.Close() |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
346 insertBnPdfsStmt, err := tx.PrepareContext(ctx, insertBnPdfsSQL) |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
347 if err != nil { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
348 return nil, err |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
349 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
350 defer insertBnPdfsStmt.Close() |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
351 insertEFAStmt, err := tx.PrepareContext(ctx, insertEFASQL) |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
352 if err != nil { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
353 return nil, err |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
354 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
355 defer insertEFAStmt.Close() |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
356 insertFAVStmt, err := tx.PrepareContext(ctx, insertFAVSQL) |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
357 if err != nil { |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
358 return nil, err |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
359 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
360 defer insertFAVStmt.Close() |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
361 |
1694
4a2fad8f57de
Imports: Resolved golint issues unrelated to exported symbols commenting.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
1676
diff
changeset
|
362 var faIDs []string |
4a2fad8f57de
Imports: Resolved golint issues unrelated to exported symbols commenting.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
1676
diff
changeset
|
363 var faID int64 |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
364 feedback.Info("Found %d fairway availabilities", len(fas)) |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
365 for _, faRes := range fas { |
4072
1eb39e9e8ec2
Adapted fwa-import to new historization model.
Sascha Wilde <wilde@intevation.de>
parents:
3666
diff
changeset
|
366 // FIXME: The following test is propably unneccessary as already |
1eb39e9e8ec2
Adapted fwa-import to new historization model.
Sascha Wilde <wilde@intevation.de>
parents:
3666
diff
changeset
|
367 // done by DB constraints... [sw] |
2201
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
368 if !bnIds.contains(faRes.Bottleneck_id) { |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
369 feedback.Warn("Bottleneck %s not found in database.", faRes.Bottleneck_id) |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
370 continue |
cae0b597aefc
Fairway availabilty import: There is no need to fetch the responsibility country from the database. Only the bottleneck_ids are needed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2200
diff
changeset
|
371 } |
4079
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
372 err = insertFAStmt.QueryRowContext( |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
373 ctx, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
374 faRes.POSITION, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
375 faRes.Bottleneck_id, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
376 faRes.SURDAT, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
377 faRes.Critical, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
378 faRes.Date_Info, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
379 faRes.Source, |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
380 ).Scan(&faID) |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
381 if err != nil { |
7fb75deff16b
Allow fairway availability data to be updated
Tom Gottfried <tom@intevation.de>
parents:
3666
diff
changeset
|
382 return nil, err |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
383 } |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
384 feedback.Info("Processing for Bottleneck %s", faRes.Bottleneck_id) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
385 faIDs = append(faIDs, faRes.Bottleneck_id) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
386 if faRes.Bottleneck_PDFs != nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
387 bnPdfCount := 0 |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
388 for _, bnPdfs := range faRes.Bottleneck_PDFs.PdfInfo { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
389 res, err := insertBnPdfsStmt.ExecContext( |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
390 ctx, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
391 faID, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
392 bnPdfs.ProfilePdfFilename, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
393 bnPdfs.ProfilePdfURL, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
394 bnPdfs.PDF_Generation_Date, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
395 bnPdfs.Source, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
396 ) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
397 if err != nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
398 return nil, err |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
399 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
400 affected, err := res.RowsAffected() |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
401 if err == nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
402 bnPdfCount += int(affected) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
403 } else { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
404 bnPdfCount++ |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
405 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
406 } |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
407 feedback.Info("Add %d Pdfs", bnPdfCount) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
408 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
409 if faRes.Effective_fairway_availability != nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
410 efaCount := 0 |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
411 for _, efa := range faRes.Effective_fairway_availability.EffectiveFairwayAvailability { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
412 fgt := efa.Forecast_generation_time |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
413 if efa.Forecast_generation_time.Status == pgtype.Undefined { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
414 fgt = pgtype.Timestamp{ |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
415 Status: pgtype.Null, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
416 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
417 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
418 res, err := insertEFAStmt.ExecContext( |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
419 ctx, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
420 faID, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
421 efa.Measure_date, |
4515
7465b6244d5e
FA import: Code clean-up for handling default LOS.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4513
diff
changeset
|
422 string(defaultLOS(efa.Level_of_Service)), |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
423 efa.Available_depth_value, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
424 efa.Available_width_value, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
425 efa.Water_level_value, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
426 efa.Measure_type, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
427 efa.Source, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
428 fgt.Get(), |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
429 efa.Value_lifetime, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
430 ) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
431 if err != nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
432 return nil, err |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
433 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
434 affected, err := res.RowsAffected() |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
435 if err == nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
436 efaCount += int(affected) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
437 } else { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
438 efaCount++ |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
439 } |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
440 } |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
441 feedback.Info("Add %d Effective Fairway Availability", efaCount) |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
442 } |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
443 |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
444 if faRes.Reference_values != nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
445 rvCount := 0 |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
446 for _, fav := range faRes.Reference_values.ReferenceValue { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
447 res, err := insertFAVStmt.ExecContext( |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
448 ctx, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
449 faID, |
4515
7465b6244d5e
FA import: Code clean-up for handling default LOS.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4513
diff
changeset
|
450 string(defaultLOS(fav.Level_of_Service)), |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
451 fav.Fairway_depth, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
452 fav.Fairway_width, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
453 fav.Fairway_radius, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
454 fav.Shallowest_spot_Lat, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
455 fav.Shallowest_spot_Lon, |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
456 ) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
457 if err != nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
458 return nil, err |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
459 } |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
460 affected, err := res.RowsAffected() |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
461 if err == nil { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
462 rvCount += int(affected) |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
463 } else { |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
464 rvCount++ |
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
465 } |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
466 } |
1755
d3fe20a13339
Import: Filter fairway availabilities by period and resolve insert confilcts.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
1754
diff
changeset
|
467 feedback.Info("Add %d Reference Values", rvCount) |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
468 } |
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
469 } |
1694
4a2fad8f57de
Imports: Resolved golint issues unrelated to exported symbols commenting.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
1676
diff
changeset
|
470 return faIDs, nil |
1662
d8ca44615bfc
Implemented first version of fairway availability import.
Raimund Renkert <raimund.renkert@intevation.de>
parents:
diff
changeset
|
471 } |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
472 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
473 // Do executes the actual fairway availability import. |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
474 func (fa *FairwayAvailability) Do( |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
475 ctx context.Context, |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
476 importID int64, |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
477 conn *sql.Conn, |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
478 feedback Feedback, |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
479 ) (interface{}, error) { |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
480 |
2203
8d1a945d0c3b
Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2202
diff
changeset
|
481 fetch := func( |
8d1a945d0c3b
Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2202
diff
changeset
|
482 ctx context.Context, |
8d1a945d0c3b
Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2202
diff
changeset
|
483 tx *sql.Tx, bns bottlenecks, |
8d1a945d0c3b
Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2202
diff
changeset
|
484 ) ([]*ifaf.FairwayAvailability, error) { |
4655
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
485 feedback.Info("Requesting data for: %v", bns) |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
486 |
4655
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
487 latest, err := latestDate(ctx, tx, bns) |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
488 if err != nil { |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
489 return nil, err |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
490 } |
4655
1e96ff2da1f3
Fixed the time range for which data from FA service is requested.
Sascha Wilde <wilde@intevation.de>
parents:
4515
diff
changeset
|
491 feedback.Info("Requesting data starting from %s", latest.Time) |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
492 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
493 client := ifaf.NewFairwayAvailabilityService(fa.URL, fa.Insecure, nil) |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
494 |
2253
cfc523c70e90
Uploaded imports: Small code clean-ups.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2233
diff
changeset
|
495 period := ifaf.RequestedPeriod{ |
cfc523c70e90
Uploaded imports: Small code clean-ups.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2233
diff
changeset
|
496 Date_start: latest.Time, |
cfc523c70e90
Uploaded imports: Small code clean-ups.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2233
diff
changeset
|
497 Date_end: time.Now(), |
cfc523c70e90
Uploaded imports: Small code clean-ups.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2233
diff
changeset
|
498 } |
2202
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
499 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
500 ids := ifaf.ArrayOfString{String: bns} |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
501 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
502 req := &ifaf.Get_bottleneck_fa{ |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
503 Bottleneck_id: &ids, |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
504 Period: &period, |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
505 } |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
506 resp, err := client.Get_bottleneck_fa(req) |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
507 if err != nil { |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
508 return nil, err |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
509 } |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
510 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
511 if resp.Get_bottleneck_faResult == nil { |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
512 return nil, errors.New("no fairway availabilities found") |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
513 } |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
514 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
515 result := resp.Get_bottleneck_faResult |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
516 return result.FairwayAvailability, nil |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
517 } |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
518 |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
519 return storeFairwayAvailability(ctx, conn, feedback, fetch) |
0aee7d4954ae
Fairway availabilty import: Re-factored to be the base for the uploaded fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2201
diff
changeset
|
520 } |