gemma: pkg/imports/bn.go annotate

annotate pkg/imports/bn.go @ 3703:b07511ff859e

Don't include calculated area in unchanged bottleneck detection. This makes BN imports considerably faster. The only downside is, that when die waterway area changes there is no easy way to recalculate the areas of existing BN. But the semantics in that case are somewhat hard anyway (think of historization for the old area) so this should be ok.

author	Sascha Wilde <wilde@intevation.de>
date	Wed, 19 Jun 2019 12:34:48 +0200
parents	063a1883b5cb
children	7006b92c0334

rev	line source
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	1 // This is Free Software under GNU Affero General Public License v >= 3.0
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	2 // without warranty, see README.md and license for details.
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	3 //
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	4 // SPDX-License-Identifier: AGPL-3.0-or-later
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	5 // License-Filename: LICENSES/AGPL-3.0.txt
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	6 //
2532 452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	7 // Copyright (C) 2018, 2019 by via donau
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	8 // – Österreichische Wasserstraßen-Gesellschaft mbH
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	9 // Software engineering by Intevation GmbH
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	10 //
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	11 // Author(s):
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
2532 452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	13 // * Tom Gottfried <tom.gottfried@intevation.de>
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15 package imports
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	17 import (
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	18 "context"
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	19 "database/sql"
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	20 "errors"
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	21 "regexp"
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	22 "strconv"
3528 b9e331c1e616 Ignore zero length and whitespace-only values Tom Gottfried <tom@intevation.de> parents: 3423 diff changeset	23 "strings"
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	24 "time"
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	25
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	26 "gemma.intevation.de/gemma/pkg/soap/ifbn"
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	27 "github.com/jackc/pgx/pgtype"
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	28 )
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	29
1695 be78b429ef6e Bottleneck import: Resolved remaining golint issues. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1676 diff changeset	30 // Bottleneck is an import job to import
be78b429ef6e Bottleneck import: Resolved remaining golint issues. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1676 diff changeset	31 // bottlenecks from an IFBN SOAP service.
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	32 type Bottleneck struct {
1695 be78b429ef6e Bottleneck import: Resolved remaining golint issues. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1676 diff changeset	33 // URL is the URL of the SOAP service.
be78b429ef6e Bottleneck import: Resolved remaining golint issues. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1676 diff changeset	34 URL string `json:"url"`
2563 dc4fae4bdb8f Expose axis snapping tolerance to users Tom Gottfried <tom@intevation.de> parents: 2548 diff changeset	35 // Tolerance used for axis snapping
dc4fae4bdb8f Expose axis snapping tolerance to users Tom Gottfried <tom@intevation.de> parents: 2548 diff changeset	36 Tolerance float64 `json:"tolerance"`
1695 be78b429ef6e Bottleneck import: Resolved remaining golint issues. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1676 diff changeset	37 // Insecure indicates if HTTPS traffic
be78b429ef6e Bottleneck import: Resolved remaining golint issues. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1676 diff changeset	38 // should validate certificates or not.
be78b429ef6e Bottleneck import: Resolved remaining golint issues. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1676 diff changeset	39 Insecure bool `json:"insecure"`
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	40 }
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	41
1696 ad5e1cddaa09 Imports: Resolved the remaining golint issues with this package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1695 diff changeset	42 // BNJobKind is the import queue type identifier.
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	43 const BNJobKind JobKind = "bn"
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	44
1572 056a86b24be2 Made bottleneck primary key an int. Attention: This may break something! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1569 diff changeset	45 const (
1658 07f996b0e14a Bottleneck import: Cleaned up source a bit (removed outdated TODOs). Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1646 diff changeset	46 insertBottleneckSQL = `
3566 4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3562 diff changeset	47 WITH
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	48 bounds (b) AS (VALUES (isrs_fromText($6)), (isrs_fromText($7))),
3566 4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3562 diff changeset	49 r AS (SELECT isrsrange(
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3562 diff changeset	50 (SELECT b FROM bounds ORDER BY b USING <~ FETCH FIRST ROW ONLY),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3562 diff changeset	51 (SELECT b FROM bounds ORDER BY b USING >~ FETCH FIRST ROW ONLY)) AS r)
1572 056a86b24be2 Made bottleneck primary key an int. Attention: This may break something! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1569 diff changeset	52 INSERT INTO waterway.bottlenecks (
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	53 bottleneck_id,
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	54 validity,
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3224 diff changeset	55 gauge_location,
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3224 diff changeset	56 gauge_validity,
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	57 objnam,
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	58 nobjnm,
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	59 stretch,
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	60 area,
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	61 rb,
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	62 lb,
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	63 responsible_country,
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	64 revisiting_time,
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	65 limiting,
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	66 date_info,
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	67 source_organization
3666 db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	68 ) SELECT
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	69 $1,
3666 db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	70 validity * $2, -- intersections with gauge validity ranges
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	71 location,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	72 validity,
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	73 $4,
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	74 $5,
3566 4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3562 diff changeset	75 (SELECT r FROM r),
1631 6ee06eb37fa9 Use new function to calculate bn area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1611 diff changeset	76 ISRSrange_area(
3566 4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3562 diff changeset	77 ISRSrange_axis((SELECT r FROM r),
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	78 $15),
2248 cce158db02b0 Input area as multipolygons to generate area from stretch Tom Gottfried <tom@intevation.de> parents: 2195 diff changeset	79 (SELECT ST_Collect(CAST(area AS geometry))
1902 c4af342be999 Use waterway area as basis for bottleneck area generation. Sascha Wilde <wilde@intevation.de> parents: 1754 diff changeset	80 FROM waterway.waterway_area)),
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	81 $8,
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	82 $9,
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	83 $10,
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	84 $11,
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	85 $12,
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	86 $13,
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	87 $14
3666 db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	88 FROM waterway.gauges
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	89 WHERE location = isrs_fromText($3) AND validity && $2
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	90 ON CONFLICT (bottleneck_id, validity) DO UPDATE SET
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	91 gauge_location = EXCLUDED.gauge_location,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	92 gauge_validity = EXCLUDED.gauge_validity,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	93 objnam = EXCLUDED.objnam,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	94 nobjnm = EXCLUDED.nobjnm,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	95 stretch = EXCLUDED.stretch,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	96 area = EXCLUDED.area,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	97 rb = EXCLUDED.rb,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	98 lb = EXCLUDED.lb,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	99 responsible_country = EXCLUDED.responsible_country,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	100 revisiting_time = EXCLUDED.revisiting_time,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	101 limiting = EXCLUDED.limiting,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	102 date_info = EXCLUDED.date_info,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	103 source_organization = EXCLUDED.source_organization
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	104 RETURNING id
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	105 `
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	106
3698 063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	107 findExactMatchBottleneckSQL = `
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	108 WITH
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	109 bounds (b) AS (VALUES (isrs_fromText($6)), (isrs_fromText($7))),
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	110 r AS (SELECT isrsrange(
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	111 (SELECT b FROM bounds ORDER BY b USING <~ FETCH FIRST ROW ONLY),
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	112 (SELECT b FROM bounds ORDER BY b USING >~ FETCH FIRST ROW ONLY)) AS r)
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	113 SELECT id FROM waterway.bottlenecks
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	114 WHERE (
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	115 bottleneck_id,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	116 validity,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	117 gauge_location,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	118 gauge_validity,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	119 objnam,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	120 nobjnm,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	121 stretch,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	122 rb,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	123 lb,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	124 responsible_country,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	125 revisiting_time,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	126 limiting,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	127 date_info,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	128 source_organization,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	129 staging_done
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	130 ) = ( SELECT
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	131 $1,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	132 validity * $2, -- intersections with gauge validity ranges
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	133 location,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	134 validity,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	135 $4,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	136 $5,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	137 (SELECT r FROM r),
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	138 $8,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	139 $9,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	140 $10,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	141 $11::smallint,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	142 $12,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	143 $13::timestamptz,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	144 $14,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	145 true
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	146 FROM waterway.gauges
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	147 WHERE location = isrs_fromText($3) AND validity && $2
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	148 )
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	149 `
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	150
3666 db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	151 // Alignment with gauge validity might have generated new entries
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	152 // for the same time range. Thus, remove the old ones
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	153 deleteObsoleteBNSQL = `
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	154 DELETE FROM waterway.bottlenecks
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	155 WHERE bottleneck_id = $1 AND validity <@ $2 AND id <> ALL($3)
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	156 `
2999 b3c3c5b5b7c1 Bottleneck import: Import riverbed materials, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2765 diff changeset	157
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	158 fixBNValiditySQL = `
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	159 UPDATE waterway.bottlenecks SET
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	160 -- Set enddate of old entry to new startdate in case of overlap:
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	161 validity = validity - $2
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	162 WHERE bottleneck_id = $1
3666 db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	163 AND validity && $2 AND NOT validity <@ $2
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	164 `
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	165
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	166 deleteBottleneckMaterialSQL = `
3666 db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	167 WITH del AS (
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	168 DELETE FROM waterway.bottlenecks_riverbed_materials
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	169 WHERE bottleneck_id = ANY($1)
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	170 AND riverbed <> ALL($2)
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	171 RETURNING riverbed)
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	172 SELECT DISTINCT riverbed FROM del
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	173 `
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	174
2999 b3c3c5b5b7c1 Bottleneck import: Import riverbed materials, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2765 diff changeset	175 insertBottleneckMaterialSQL = `
b3c3c5b5b7c1 Bottleneck import: Import riverbed materials, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2765 diff changeset	176 INSERT INTO waterway.bottlenecks_riverbed_materials (
b3c3c5b5b7c1 Bottleneck import: Import riverbed materials, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2765 diff changeset	177 bottleneck_id,
b3c3c5b5b7c1 Bottleneck import: Import riverbed materials, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2765 diff changeset	178 riverbed
3666 db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	179 ) SELECT *
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	180 FROM unnest(CAST($1 AS int[])) AS bns,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	181 unnest(CAST($2 AS varchar[])) AS materials
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	182 ON CONFLICT (bottleneck_id, riverbed) DO NOTHING
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	183 `
1572 056a86b24be2 Made bottleneck primary key an int. Attention: This may break something! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1569 diff changeset	184 )
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	185
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	186 type bnJobCreator struct{}
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	187
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	188 func init() {
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	189 RegisterJobCreator(BNJobKind, bnJobCreator{})
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	190 }
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	191
1754 807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1696 diff changeset	192 func (bnJobCreator) Description() string { return "bottlenecks" }
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1696 diff changeset	193
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1696 diff changeset	194 func (bnJobCreator) AutoAccept() bool { return false }
1646 a0982c38eac0 Import queue: Implemented email notifications. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1631 diff changeset	195
2187 7c83b5277c1c Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2168 diff changeset	196 func (bnJobCreator) Create() Job { return new(Bottleneck) }
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	197
3219 4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3164 diff changeset	198 func (bnJobCreator) Depends() [2][]string {
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3164 diff changeset	199 return [2][]string{
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3164 diff changeset	200 {"bottlenecks", "bottlenecks_riverbed_materials"},
3224 bfde4f8dd323 Add missing table dependencies of imports with area generation Tom Gottfried <tom@intevation.de> parents: 3219 diff changeset	201 {"gauges", "distance_marks_virtual", "waterway_axis", "waterway_area"},
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	202 }
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	203 }
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	204
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	205 const (
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	206 bnStageDoneSQL = `
1533 8fc1a2298acb Pseudo fixed unstaging of bottlenecks. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1531 diff changeset	207 UPDATE waterway.bottlenecks SET staging_done = true
8fc1a2298acb Pseudo fixed unstaging of bottlenecks. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1531 diff changeset	208 WHERE id IN (
1995 59055c8301df Move import queue to its own database namespace Tom Gottfried <tom@intevation.de> parents: 1975 diff changeset	209 SELECT key from import.track_imports
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	210 WHERE import_id = $1 AND
1533 8fc1a2298acb Pseudo fixed unstaging of bottlenecks. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1531 diff changeset	211 relation = 'waterway.bottlenecks'::regclass)`
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	212 )
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	213
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	214 // StageDone moves the imported bottleneck out of the staging area.
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	215 func (bnJobCreator) StageDone(
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	216 ctx context.Context,
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	217 tx *sql.Tx,
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	218 id int64,
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	219 ) error {
1594 e3abd3b3ebc6 Fixed stage done SQL for bottleneck imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1580 diff changeset	220 _, err := tx.ExecContext(ctx, bnStageDoneSQL, id)
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	221 return err
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	222 }
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	223
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	224 // CleanUp of a bottleneck import is a NOP.
1676 4407ecaa2192 Imports: Cosmetics. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1664 diff changeset	225 func (*Bottleneck) CleanUp() error { return nil }
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	226
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	227 var rblbRe = regexp.MustCompile(`(..)_(..)`)
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	228
2765 41f5ac76d642 BT import: allow for rb and lb to be null. Sascha Wilde <wilde@intevation.de> parents: 2752 diff changeset	229 func splitRBLB(s string) (string, string) {
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	230 m := rblbRe.FindStringSubmatch(s)
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	231 if len(m) == 0 {
2765 41f5ac76d642 BT import: allow for rb and lb to be null. Sascha Wilde <wilde@intevation.de> parents: 2752 diff changeset	232 return nil, nil
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	233 }
2765 41f5ac76d642 BT import: allow for rb and lb to be null. Sascha Wilde <wilde@intevation.de> parents: 2752 diff changeset	234 return &m[1], &m[2]
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	235 }
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	236
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	237 // Do executes the actual bottleneck import.
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	238 func (bn *Bottleneck) Do(
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	239 ctx context.Context,
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	240 importID int64,
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	241 conn *sql.Conn,
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	242 feedback Feedback,
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	243 ) (interface{}, error) {
2195 3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	244
2253 cfc523c70e90 Uploaded imports: Small code clean-ups. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2248 diff changeset	245 fetch := func() ([]*ifbn.BottleNeckType, error) {
2195 3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	246 client := ifbn.NewIBottleneckService(bn.URL, bn.Insecure, nil)
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	247
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	248 req := &ifbn.Export_bn_by_isrs{}
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	249
2195 3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	250 resp, err := client.Export_bn_by_isrs(req)
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	251 if err != nil {
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	252 return nil, err
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	253 }
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	254
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	255 if resp.Export_bn_by_isrsResult == nil {
3562 c03224d33e78 Improve error message Tom Gottfried <tom@intevation.de> parents: 3533 diff changeset	256 return nil, errors.New(
c03224d33e78 Improve error message Tom Gottfried <tom@intevation.de> parents: 3533 diff changeset	257 "The requested service returned no bottlenecks")
2195 3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	258 }
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	259
2195 3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	260 return resp.Export_bn_by_isrsResult.BottleNeckType, nil
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	261 }
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	262
2563 dc4fae4bdb8f Expose axis snapping tolerance to users Tom Gottfried <tom@intevation.de> parents: 2548 diff changeset	263 return storeBottlenecks(ctx, fetch, importID, conn, feedback, bn.Tolerance)
2195 3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	264 }
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	265
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	266 func storeBottlenecks(
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	267 ctx context.Context,
2253 cfc523c70e90 Uploaded imports: Small code clean-ups. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2248 diff changeset	268 fetch func() ([]*ifbn.BottleNeckType, error),
2195 3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	269 importID int64,
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	270 conn *sql.Conn,
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	271 feedback Feedback,
2563 dc4fae4bdb8f Expose axis snapping tolerance to users Tom Gottfried <tom@intevation.de> parents: 2548 diff changeset	272 tolerance float64,
2195 3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	273 ) (interface{}, error) {
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	274 start := time.Now()
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	275
2253 cfc523c70e90 Uploaded imports: Small code clean-ups. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2248 diff changeset	276 bns, err := fetch()
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	277 if err != nil {
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	278 return nil, err
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	279 }
24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	280
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	281 feedback.Info("Found %d bottlenecks for import", len(bns))
cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	282
3698 063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	283 var insertStmt, findExactMatchingBNStmt, deleteObsoleteBNStmt,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	284 fixValidityStmt, deleteMaterialStmt, insertMaterialStmt,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	285 trackStmt *sql.Stmt
1580 c503d2fa9776 Bottleneck import: Use a prepared statement for import tracking, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1572 diff changeset	286
c503d2fa9776 Bottleneck import: Use a prepared statement for import tracking, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1572 diff changeset	287 for _, x := range []struct {
c503d2fa9776 Bottleneck import: Use a prepared statement for import tracking, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1572 diff changeset	288 sql string
c503d2fa9776 Bottleneck import: Use a prepared statement for import tracking, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1572 diff changeset	289 stmt **sql.Stmt
c503d2fa9776 Bottleneck import: Use a prepared statement for import tracking, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1572 diff changeset	290 }{
1658 07f996b0e14a Bottleneck import: Cleaned up source a bit (removed outdated TODOs). Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1646 diff changeset	291 {insertBottleneckSQL, &insertStmt},
3698 063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	292 {findExactMatchBottleneckSQL, &findExactMatchingBNStmt},
3666 db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	293 {deleteObsoleteBNSQL, &deleteObsoleteBNStmt},
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	294 {fixBNValiditySQL, &fixValidityStmt},
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	295 {deleteBottleneckMaterialSQL, &deleteMaterialStmt},
2999 b3c3c5b5b7c1 Bottleneck import: Import riverbed materials, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2765 diff changeset	296 {insertBottleneckMaterialSQL, &insertMaterialStmt},
1580 c503d2fa9776 Bottleneck import: Use a prepared statement for import tracking, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1572 diff changeset	297 {trackImportSQL, &trackStmt},
c503d2fa9776 Bottleneck import: Use a prepared statement for import tracking, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1572 diff changeset	298 } {
c503d2fa9776 Bottleneck import: Use a prepared statement for import tracking, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1572 diff changeset	299 var err error
2532 452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	300 if *x.stmt, err = conn.PrepareContext(ctx, x.sql); err != nil {
1580 c503d2fa9776 Bottleneck import: Use a prepared statement for import tracking, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1572 diff changeset	301 return nil, err
c503d2fa9776 Bottleneck import: Use a prepared statement for import tracking, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1572 diff changeset	302 }
c503d2fa9776 Bottleneck import: Use a prepared statement for import tracking, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1572 diff changeset	303 defer (*x.stmt).Close()
1572 056a86b24be2 Made bottleneck primary key an int. Attention: This may break something! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1569 diff changeset	304 }
056a86b24be2 Made bottleneck primary key an int. Attention: This may break something! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1569 diff changeset	305
1611 1835d1c9eb9b Store bottleneck_id's instead of id's in bottleneck import summary. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1595 diff changeset	306 var nids []string
1572 056a86b24be2 Made bottleneck primary key an int. Attention: This may break something! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1569 diff changeset	307
2693 02d012272dd4 Log snapping tolerance Tom Gottfried <tom@intevation.de> parents: 2563 diff changeset	308 feedback.Info("Tolerance used to snap waterway axis: %g", tolerance)
02d012272dd4 Log snapping tolerance Tom Gottfried <tom@intevation.de> parents: 2563 diff changeset	309
1572 056a86b24be2 Made bottleneck primary key an int. Attention: This may break something! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1569 diff changeset	310 for _, bn := range bns {
2532 452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	311 if err := storeBottleneck(
2563 dc4fae4bdb8f Expose axis snapping tolerance to users Tom Gottfried <tom@intevation.de> parents: 2548 diff changeset	312 ctx, importID, conn, feedback, bn, &nids, tolerance,
3698 063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	313 insertStmt, findExactMatchingBNStmt, deleteObsoleteBNStmt,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	314 fixValidityStmt, deleteMaterialStmt, insertMaterialStmt,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	315 trackStmt); err != nil {
1572 056a86b24be2 Made bottleneck primary key an int. Attention: This may break something! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1569 diff changeset	316 return nil, err
056a86b24be2 Made bottleneck primary key an int. Attention: This may break something! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1569 diff changeset	317 }
056a86b24be2 Made bottleneck primary key an int. Attention: This may break something! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1569 diff changeset	318 }
056a86b24be2 Made bottleneck primary key an int. Attention: This may break something! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1569 diff changeset	319 if len(nids) == 0 {
2532 452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	320 return nil, UnchangedError("No new bottlenecks inserted")
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	321 }
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	322
1572 056a86b24be2 Made bottleneck primary key an int. Attention: This may break something! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1569 diff changeset	323 feedback.Info("Storing %d bottlenecks took %s", len(nids), time.Since(start))
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1902 diff changeset	324 feedback.Info("Import of bottlenecks was successful")
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	325 summary := struct {
1611 1835d1c9eb9b Store bottleneck_id's instead of id's in bottleneck import summary. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1595 diff changeset	326 Bottlenecks []string `json:"bottlenecks"`
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	327 }{
1572 056a86b24be2 Made bottleneck primary key an int. Attention: This may break something! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1569 diff changeset	328 Bottlenecks: nids,
1569 cf0ca4a9812a Bottleneck import: Implemented backend to store bottlenecks in database. Raimund Renkert <raimund.renkert@intevation.de> parents: 1533 diff changeset	329 }
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1902 diff changeset	330 return &summary, nil
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	331 }
2532 452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	332
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	333 func storeBottleneck(
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	334 ctx context.Context,
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	335 importID int64,
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	336 conn *sql.Conn,
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	337 feedback Feedback,
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	338 bn *ifbn.BottleNeckType,
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	339 nids *[]string,
2563 dc4fae4bdb8f Expose axis snapping tolerance to users Tom Gottfried <tom@intevation.de> parents: 2548 diff changeset	340 tolerance float64,
3698 063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	341 insertStmt, findExactMatchingBNStmt, deleteObsoleteBNStmt,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	342 fixValidityStmt, deleteMaterialStmt, insertMaterialStmt,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	343 trackStmt *sql.Stmt,
2532 452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	344 ) error {
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	345 feedback.Info("Processing %s (%s)", bn.OBJNAM, bn.Bottleneck_id)
2532 452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	346
3679 0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	347 var tfrom, tto pgtype.Timestamptz
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	348 var uBound pgtype.BoundType
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	349
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	350 if bn.AdditionalData == nil \|\| bn.AdditionalData.KeyValuePair == nil {
3679 0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	351 // This is a workaround for the fact that most BN data does not
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	352 // contain the optional validity information.
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	353 //
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	354 // The current solution makes every change in the data an
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	355 // update, which might be not the best solution. Alternative
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	356 // approaches could include usingthe Date Info of the data as
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	357 // start date, and maybe even inspecting the details of changed
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	358 // data for hints wether an update or historisation with a new
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	359 // version is advisable.
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	360 //
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	361 // Never the less, the current solution "just works" for the
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	362 // rtime being... -- sw
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	363 feedback.Warn("No validity information, assuming infinite validity.")
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	364 tfrom.Set(time.Date(1970, time.January, 1, 0, 0, 0, 0, time.UTC))
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	365 uBound = pgtype.Unbounded
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	366 } else {
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	367 const (
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	368 fromKey = "Valid_from_date"
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	369 toKey = "Valid_to_date"
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	370 )
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	371 fromTo := make(map[string]time.Time)
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	372 for _, kv := range bn.AdditionalData.KeyValuePair {
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	373 k := string(kv.Key)
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	374 if k == fromKey \|\| k == toKey {
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	375 if t, err := time.Parse(time.RFC3339, kv.Value); err != nil {
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	376 return err
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	377 } else {
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	378 fromTo[k] = t
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	379 }
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	380 }
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	381 }
3679 0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	382
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	383 if t, ok := fromTo[fromKey]; ok {
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	384 tfrom.Set(t)
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	385 } else {
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	386 feedback.Warn("Missing start date")
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	387 return nil
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	388 }
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	389
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	390 if t, ok := fromTo[toKey]; ok {
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	391 tto.Set(t)
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	392 uBound = pgtype.Exclusive
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	393 } else {
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	394 uBound = pgtype.Unbounded
0227670dedd5 Use infinite validity for bottlenecks with no validity information. Sascha Wilde <wilde@intevation.de> parents: 3666 diff changeset	395 }
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	396 }
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	397
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	398 validity := pgtype.Tstzrange{
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	399 Lower: tfrom,
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	400 Upper: tto,
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	401 LowerType: pgtype.Inclusive,
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	402 UpperType: uBound,
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	403 Status: pgtype.Present,
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	404 }
2532 452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	405
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	406 rb, lb := splitRBLB(bn.Rb_lb)
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	407
3423 6592396f5061 Make revisiting time of a bottleneck optional Tom Gottfried <tom@intevation.de> parents: 3302 diff changeset	408 var revisitingTime *int
3528 b9e331c1e616 Ignore zero length and whitespace-only values Tom Gottfried <tom@intevation.de> parents: 3423 diff changeset	409 if bn.Revisiting_time != nil &&
b9e331c1e616 Ignore zero length and whitespace-only values Tom Gottfried <tom@intevation.de> parents: 3423 diff changeset	410 len(strings.TrimSpace(*bn.Revisiting_time)) > 0 {
3423 6592396f5061 Make revisiting time of a bottleneck optional Tom Gottfried <tom@intevation.de> parents: 3302 diff changeset	411 i, err := strconv.Atoi(*bn.Revisiting_time)
6592396f5061 Make revisiting time of a bottleneck optional Tom Gottfried <tom@intevation.de> parents: 3302 diff changeset	412 if err != nil {
3530 e98ea184538e Improve warning message Tom Gottfried <tom@intevation.de> parents: 3528 diff changeset	413 feedback.Warn(
e98ea184538e Improve warning message Tom Gottfried <tom@intevation.de> parents: 3528 diff changeset	414 "Cannot convert given revisiting time '%s' to number of months",
3423 6592396f5061 Make revisiting time of a bottleneck optional Tom Gottfried <tom@intevation.de> parents: 3302 diff changeset	415 *bn.Revisiting_time)
6592396f5061 Make revisiting time of a bottleneck optional Tom Gottfried <tom@intevation.de> parents: 3302 diff changeset	416 } else {
6592396f5061 Make revisiting time of a bottleneck optional Tom Gottfried <tom@intevation.de> parents: 3302 diff changeset	417 revisitingTime = &i
6592396f5061 Make revisiting time of a bottleneck optional Tom Gottfried <tom@intevation.de> parents: 3302 diff changeset	418 }
6592396f5061 Make revisiting time of a bottleneck optional Tom Gottfried <tom@intevation.de> parents: 3302 diff changeset	419 }
6592396f5061 Make revisiting time of a bottleneck optional Tom Gottfried <tom@intevation.de> parents: 3302 diff changeset	420
2532 452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	421 var limiting, country string
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	422
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	423 if bn.Limiting_factor != nil {
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	424 limiting = string(*bn.Limiting_factor)
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	425 }
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	426
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	427 if bn.Responsible_country != nil {
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	428 country = string(*bn.Responsible_country)
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	429 }
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	430
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	431 var materials []string
2999 b3c3c5b5b7c1 Bottleneck import: Import riverbed materials, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2765 diff changeset	432 if bn.Riverbed != nil {
b3c3c5b5b7c1 Bottleneck import: Import riverbed materials, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2765 diff changeset	433 for _, material := range bn.Riverbed.Material {
b3c3c5b5b7c1 Bottleneck import: Import riverbed materials, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2765 diff changeset	434 if material != nil {
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	435 materials = append(materials, string(*material))
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	436 }
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	437 }
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	438 }
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	439
3698 063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	440 // Check if an bottleneck identical to the one we would insert already
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	441 // exists:
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	442 bns, err := findExactMatchingBNStmt.QueryContext(ctx,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	443 bn.Bottleneck_id,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	444 &validity,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	445 bn.Fk_g_fid,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	446 bn.OBJNAM,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	447 bn.NOBJNM,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	448 bn.From_ISRS, bn.To_ISRS,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	449 rb,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	450 lb,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	451 country,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	452 revisitingTime,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	453 limiting,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	454 bn.Date_Info,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	455 bn.Source,
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	456 )
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	457
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	458 if err != nil {
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	459 return err
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	460 }
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	461 defer bns.Close()
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	462 if bns.Next() {
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	463 feedback.Info("unchanged")
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	464 return nil
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	465 }
063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	466
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	467 tx, err := conn.BeginTx(ctx, nil)
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	468 if err != nil {
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	469 return err
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	470 }
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	471 defer tx.Rollback()
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	472
3666 db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	473 var bnIds []int64
3698 063a1883b5cb Detect and handle unchanged BN during import. Sascha Wilde <wilde@intevation.de> parents: 3681 diff changeset	474 bns, err = tx.StmtContext(ctx, insertStmt).QueryContext(ctx,
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	475 bn.Bottleneck_id,
3666 db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	476 &validity,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	477 bn.Fk_g_fid,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	478 bn.OBJNAM,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	479 bn.NOBJNM,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	480 bn.From_ISRS, bn.To_ISRS,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	481 rb,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	482 lb,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	483 country,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	484 revisitingTime,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	485 limiting,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	486 bn.Date_Info,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	487 bn.Source,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	488 tolerance,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	489 )
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	490 if err != nil {
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	491 feedback.Warn(handleError(err).Error())
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	492 return nil
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	493 }
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	494 defer bns.Close()
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	495 for bns.Next() {
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	496 var nid int64
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	497 if err := bns.Scan(&nid); err != nil {
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	498 return err
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	499 }
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	500 bnIds = append(bnIds, nid)
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	501 }
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	502 if err := bns.Err(); err != nil {
3681 c9e1848a516a Handle violation of responsibility areas graceful for BN import. Sascha Wilde <wilde@intevation.de> parents: 3679 diff changeset	503 feedback.Warn(handleError(err).Error())
c9e1848a516a Handle violation of responsibility areas graceful for BN import. Sascha Wilde <wilde@intevation.de> parents: 3679 diff changeset	504 return nil
3666 db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	505 }
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	506 if len(bnIds) == 0 {
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	507 feedback.Warn(
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	508 "No gauge matching '%s' or given time available", bn.Fk_g_fid)
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	509 return nil
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	510 }
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	511
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	512 // Remove obsolete bottleneck version entries
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	513 var pgBnIds pgtype.Int8Array
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	514 pgBnIds.Set(bnIds)
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	515 if _, err = tx.StmtContext(ctx, deleteObsoleteBNStmt).ExecContext(ctx,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	516 bn.Bottleneck_id,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	517 &validity,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	518 &pgBnIds,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	519 ); err != nil {
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	520 feedback.Warn(handleError(err).Error())
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	521 if err2 := tx.Rollback(); err2 != nil {
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	522 return err2
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	523 }
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	524 return nil
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	525 }
02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	526
3648 0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	527 // Set end of validity of old version to start of new version
0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	528 // in case of overlap
0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	529 if _, err = tx.StmtContext(ctx, fixValidityStmt).ExecContext(ctx,
0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	530 bn.Bottleneck_id,
0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	531 validity,
0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	532 ); err != nil {
0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	533 feedback.Warn(handleError(err).Error())
0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	534 if err2 := tx.Rollback(); err2 != nil {
0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	535 return err2
0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	536 }
0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	537 return nil
0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	538 }
0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	539
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3566 diff changeset	540 if materials != nil {
3666 db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	541 // Remove obsolete riverbed materials
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	542 var pgMaterials pgtype.VarcharArray
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	543 pgMaterials.Set(materials)
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	544 mtls, err := tx.StmtContext(ctx,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	545 deleteMaterialStmt).QueryContext(ctx,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	546 &pgBnIds,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	547 &pgMaterials,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	548 )
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	549 if err != nil {
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	550 return err
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	551 }
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	552 defer mtls.Close()
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	553 for mtls.Next() {
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	554 var delMat string
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	555 if err := mtls.Scan(&delMat); err != nil {
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	556 return err
2999 b3c3c5b5b7c1 Bottleneck import: Import riverbed materials, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2765 diff changeset	557 }
3666 db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	558 feedback.Warn("Removed riverbed material %s", delMat)
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	559 }
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	560 if err := mtls.Err(); err != nil {
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	561 return err
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	562 }
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	563
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	564 // Insert riverbed materials
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	565 if _, err := tx.StmtContext(ctx, insertMaterialStmt).ExecContext(ctx,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	566 &pgBnIds,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	567 &pgMaterials,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	568 ); err != nil {
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	569 feedback.Warn("Failed to insert riverbed materials")
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	570 feedback.Warn(handleError(err).Error())
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	571 return nil
2999 b3c3c5b5b7c1 Bottleneck import: Import riverbed materials, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2765 diff changeset	572 }
b3c3c5b5b7c1 Bottleneck import: Import riverbed materials, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2765 diff changeset	573 }
b3c3c5b5b7c1 Bottleneck import: Import riverbed materials, too. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2765 diff changeset	574
3666 db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	575 for _, nid := range bnIds {
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	576 if _, err := tx.StmtContext(ctx, trackStmt).ExecContext(
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	577 ctx, importID, "waterway.bottlenecks", nid,
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	578 ); err != nil {
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	579 return err
db87f34805fb Align bottleneck validity at gauges Tom Gottfried <tom@intevation.de> parents: 3665 diff changeset	580 }
2532 452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	581 }
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	582 if err = tx.Commit(); err != nil {
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	583 return err
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	584 }
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	585 nids = append(nids, bn.Bottleneck_id)
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	586 return nil
452bc714bfd9 Continue importing bottlenecks if one item fails Tom Gottfried <tom@intevation.de> parents: 2253 diff changeset	587 }

Mercurial > gemma

annotate pkg/imports/bn.go @ 3703:b07511ff859e