gemma: pkg/imports/wg.go annotate

annotate pkg/imports/wg.go @ 5711:2dd155cc95ec revive-cleanup

Fix all revive issue (w/o machine generated stuff).

author	Sascha L. Teichmann <sascha.teichmann@intevation.de>
date	Tue, 20 Feb 2024 22:22:57 +0100
parents	31973f6f5cca
children	6270951dda28

rev	line source
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	1 // This is Free Software under GNU Affero General Public License v >= 3.0
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	2 // without warranty, see README.md and license for details.
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	3 //
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	4 // SPDX-License-Identifier: AGPL-3.0-or-later
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	5 // License-Filename: LICENSES/AGPL-3.0.txt
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	6 //
2318 06c4e57435f1 Warn on import of unknown reference level codes Tom Gottfried <tom@intevation.de> parents: 2317 diff changeset	7 // Copyright (C) 2018, 2019 by via donau
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	8 // – Österreichische Wasserstraßen-Gesellschaft mbH
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	9 // Software engineering by Intevation GmbH
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	10 //
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	11 // Author(s):
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
2318 06c4e57435f1 Warn on import of unknown reference level codes Tom Gottfried <tom@intevation.de> parents: 2317 diff changeset	13 // * Tom Gottfried <tom.gottfried@intevation.de>
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15 package imports
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	17 import (
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	18 "context"
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	19 "database/sql"
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	20 "errors"
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	21 "strings"
1835 f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	22 "time"
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	23
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	24 "github.com/jackc/pgx/pgtype"
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	25
1829 b4b9089c2d79 Waterway gauges: Started with deleting old gauges to be overwritten. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1812 diff changeset	26 "gemma.intevation.de/gemma/pkg/models"
4062 6c760abcff0e Move handling of PostgreSQL errors to own package Tom Gottfried <tom@intevation.de> parents: 4058 diff changeset	27 "gemma.intevation.de/gemma/pkg/pgxutils"
5609 e1936db6db8e Adjust ta erdms data structure. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5565 diff changeset	28 erdms "gemma.intevation.de/gemma/pkg/soap/erdms2"
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	29 )
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	30
4182 49012340336c Made 'golint' finally happy with imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4147 diff changeset	31 // WaterwayGauge is a Job to load gauge data from
49012340336c Made 'golint' finally happy with imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4147 diff changeset	32 // a specified NTS service and stores them into the database.
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	33 type WaterwayGauge struct {
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	34 // URL is the URL of the SOAP service.
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	35 URL string `json:"url"`
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	36 // Username is the username used to authenticate.
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	37 Username string `json:"username"`
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	38 // Passwort is the password to authenticate.
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	39 Password string `json:"password"`
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	40 // Insecure indicates if HTTPS traffic
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	41 // should validate certificates or not.
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	42 Insecure bool `json:"insecure"`
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	43 }
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	44
4852 046a07a33b19 Fixed the golint issues of the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4799 diff changeset	45 // Description gives a short info about relevant facts of this import.
5565 ade07a3f2cfd Forget to change signature of some imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5034 diff changeset	46 func (wg *WaterwayGauge) Description([]string) (string, error) {
4799 f32d086b5dbf Removed the mechanical touch of the last commit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4798 diff changeset	47 return wg.URL, nil
4798 ca6a5f722471 Added Description method to most imports. Sascha Wilde <wilde@intevation.de> parents: 4497 diff changeset	48 }
ca6a5f722471 Added Description method to most imports. Sascha Wilde <wilde@intevation.de> parents: 4497 diff changeset	49
4182 49012340336c Made 'golint' finally happy with imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4147 diff changeset	50 // WGJobKind is the unique name of this import job type.
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	51 const WGJobKind JobKind = "wg"
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	52
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	53 type wgJobCreator struct{}
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	54
4182 49012340336c Made 'golint' finally happy with imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4147 diff changeset	55 func init() { RegisterJobCreator(WGJobKind, wgJobCreator{}) }
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	56
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	57 func (wgJobCreator) Description() string { return "waterway gauges" }
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	58
1829 b4b9089c2d79 Waterway gauges: Started with deleting old gauges to be overwritten. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1812 diff changeset	59 func (wgJobCreator) AutoAccept() bool { return true }
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	60
2187 7c83b5277c1c Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2168 diff changeset	61 func (wgJobCreator) Create() Job { return new(WaterwayGauge) }
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	62
3219 4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3163 diff changeset	63 func (wgJobCreator) Depends() [2][]string {
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3163 diff changeset	64 return [2][]string{
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3163 diff changeset	65 {"gauges_reference_water_levels", "gauges"},
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3163 diff changeset	66 {"depth_references"},
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	67 }
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	68 }
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	69
1829 b4b9089c2d79 Waterway gauges: Started with deleting old gauges to be overwritten. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1812 diff changeset	70 // StageDone does nothing as there is no staging for gauges.
5034 59a99655f34d Added feedback support for StageDone. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4852 diff changeset	71 func (wgJobCreator) StageDone(context.Context, *sql.Tx, int64, Feedback) error { return nil }
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	72
1829 b4b9089c2d79 Waterway gauges: Started with deleting old gauges to be overwritten. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1812 diff changeset	73 // CleanUp does nothing as there is nothing to cleanup with gauges.
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	74 func (*WaterwayGauge) CleanUp() error { return nil }
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	75
1812 4910bcfab319 Waterway gauge import: Work around the expected (but nonetheless silly) namespace issues with the SOAP interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1810 diff changeset	76 const (
3311 0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	77 eraseObsoleteGaugesSQL = `
4028 5fed2f5bc104 End validity of gauge if it gets deleted during import. Sascha Wilde <wilde@intevation.de> parents: 3665 diff changeset	78 UPDATE waterway.gauges SET erased = true, validity = validity - '[now,)'
3551 a5448426e4e2 Only erase gauges that have been requested from ERDMS Tom Gottfried <tom@intevation.de> parents: 3529 diff changeset	79 WHERE NOT erased
a5448426e4e2 Only erase gauges that have been requested from ERDMS Tom Gottfried <tom@intevation.de> parents: 3529 diff changeset	80 AND (location).country_code = ANY($1)
a5448426e4e2 Only erase gauges that have been requested from ERDMS Tom Gottfried <tom@intevation.de> parents: 3529 diff changeset	81 AND isrs_astext(location) <> ALL($2)
3311 0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	82 RETURNING isrs_astext(location)
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	83 `
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	84
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	85 eraseGaugeSQL = `
3402 c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	86 WITH upd AS (
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	87 UPDATE waterway.gauges SET
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	88 erased = true
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	89 WHERE isrs_astext(location) = $1
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	90 AND NOT erased
3648 0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	91 -- Don't touch old entry if new validity contains old: will be updated
0ec5c8ec1e44 Avoid empty validity time ranges Tom Gottfried <tom@intevation.de> parents: 3645 diff changeset	92 AND NOT validity <@ $2
3402 c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	93 RETURNING 1
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	94 )
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	95 -- Decide whether a new version will be INSERTed
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	96 SELECT EXISTS(SELECT 1 FROM upd)
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	97 OR NOT EXISTS(SELECT 1 FROM waterway.gauges WHERE isrs_astext(location) = $1)
3163 d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	98 `
1829 b4b9089c2d79 Waterway gauges: Started with deleting old gauges to be overwritten. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1812 diff changeset	99
1835 f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	100 insertGaugeSQL = `
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	101 INSERT INTO waterway.gauges (
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	102 location,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	103 objname,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	104 geom,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	105 applicability_from_km,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	106 applicability_to_km,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	107 validity,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	108 zero_point,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	109 geodref,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	110 date_info,
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	111 source_organization,
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	112 lastupdate
1835 f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	113 ) VALUES (
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	114 ($1::char(2), $2::char(3), $3::char(5), $4::char(5), $5::int),
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	115 $6,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	116 ST_SetSRID(ST_MakePoint($7, $8), 4326)::geography,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	117 $9,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	118 $10,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	119 $11,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	120 $12,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	121 $13,
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	122 $14,
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	123 $15,
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	124 $16
3402 c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	125 )
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	126 `
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	127
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	128 fixValiditySQL = `
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	129 UPDATE waterway.gauges SET
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	130 -- Set enddate of old entry to new startdate in case of overlap:
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	131 validity = validity - $2
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	132 WHERE isrs_astext(location) = $1
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	133 AND validity && $2
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	134 AND erased
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	135 `
c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	136
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	137 updateGaugeSQL = `
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	138 UPDATE waterway.gauges SET
5670 b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	139 objname = $2,
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	140 geom = ST_SetSRID(ST_MakePoint($3, $4), 4326),
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	141 applicability_from_km = $5,
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	142 applicability_to_km = $6,
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	143 zero_point = $7,
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	144 geodref = $8,
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	145 date_info = $9,
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	146 source_organization = $10,
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	147 lastupdate = $11,
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	148 validity = $12,
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	149 erased = false
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	150 WHERE isrs_astext(location) = $1
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	151 AND validity <@ $12
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	152 AND $11 > lastupdate
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	153 RETURNING 1
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	154 `
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	155
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	156 deleteReferenceWaterLevelsSQL = `
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	157 DELETE FROM waterway.gauges_reference_water_levels
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	158 WHERE isrs_astext(location) = $1
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	159 AND validity = $2
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	160 AND depth_reference <> ALL($3)
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	161 RETURNING depth_reference
1836 4dcdd8891770 Waterway gauges import: Fixed insert/update of gauges. TODO: Re-insert reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1835 diff changeset	162 `
3163 d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	163
2318 06c4e57435f1 Warn on import of unknown reference level codes Tom Gottfried <tom@intevation.de> parents: 2317 diff changeset	164 isNtSDepthRefSQL = `
06c4e57435f1 Warn on import of unknown reference level codes Tom Gottfried <tom@intevation.de> parents: 2317 diff changeset	165 SELECT EXISTS(SELECT 1 FROM depth_references WHERE depth_reference = $1)`
06c4e57435f1 Warn on import of unknown reference level codes Tom Gottfried <tom@intevation.de> parents: 2317 diff changeset	166
1837 00d63eb9306a Waterway gauges import: Finalized with re-inserting the reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1836 diff changeset	167 insertReferenceWaterLevelsSQL = `
00d63eb9306a Waterway gauges import: Finalized with re-inserting the reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1836 diff changeset	168 INSERT INTO waterway.gauges_reference_water_levels (
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	169 location,
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	170 validity,
2317 8a8680e70d2e Cleanup schema for reference water levels Tom Gottfried <tom@intevation.de> parents: 2187 diff changeset	171 depth_reference,
1837 00d63eb9306a Waterway gauges import: Finalized with re-inserting the reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1836 diff changeset	172 value
00d63eb9306a Waterway gauges import: Finalized with re-inserting the reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1836 diff changeset	173 ) VALUES (
00d63eb9306a Waterway gauges import: Finalized with re-inserting the reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1836 diff changeset	174 ($1::char(2), $2::char(3), $3::char(5), $4::char(5), $5::int),
00d63eb9306a Waterway gauges import: Finalized with re-inserting the reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1836 diff changeset	175 $6,
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	176 $7,
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	177 $8
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	178 ) ON CONFLICT (location, validity, depth_reference) DO UPDATE SET
3163 d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	179 value = EXCLUDED.value
1837 00d63eb9306a Waterway gauges import: Finalized with re-inserting the reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1836 diff changeset	180 `
1812 4910bcfab319 Waterway gauge import: Work around the expected (but nonetheless silly) namespace issues with the SOAP interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1810 diff changeset	181 )
4910bcfab319 Waterway gauge import: Work around the expected (but nonetheless silly) namespace issues with the SOAP interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1810 diff changeset	182
5683 31973f6f5cca Get rid of deprecation warnings from staticcheck. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5670 diff changeset	183 var errContinue = errors.New("continue")
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	184
4182 49012340336c Made 'golint' finally happy with imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4147 diff changeset	185 // Do implements the actual import.
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	186 func (wg *WaterwayGauge) Do(
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	187 ctx context.Context,
5711 2dd155cc95ec Fix all revive issue (w/o machine generated stuff). Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5683 diff changeset	188 _ int64,
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	189 conn *sql.Conn,
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	190 feedback Feedback,
77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	191 ) (interface{}, error) {
1810 7ee9bdaac336 Waterway gauge import: Added support for username/password. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1808 diff changeset	192
1837 00d63eb9306a Waterway gauges import: Finalized with re-inserting the reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1836 diff changeset	193 start := time.Now()
00d63eb9306a Waterway gauges import: Finalized with re-inserting the reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1836 diff changeset	194
3551 a5448426e4e2 Only erase gauges that have been requested from ERDMS Tom Gottfried <tom@intevation.de> parents: 3529 diff changeset	195 responseData, countries, err := getRisData(
2633 eb1d119f253f Fetch data from ERDMS for all allowed countries Tom Gottfried <tom@intevation.de> parents: 2518 diff changeset	196 ctx,
3163 d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	197 conn,
2633 eb1d119f253f Fetch data from ERDMS for all allowed countries Tom Gottfried <tom@intevation.de> parents: 2518 diff changeset	198 feedback,
eb1d119f253f Fetch data from ERDMS for all allowed countries Tom Gottfried <tom@intevation.de> parents: 2518 diff changeset	199 wg.Username,
eb1d119f253f Fetch data from ERDMS for all allowed countries Tom Gottfried <tom@intevation.de> parents: 2518 diff changeset	200 wg.Password,
eb1d119f253f Fetch data from ERDMS for all allowed countries Tom Gottfried <tom@intevation.de> parents: 2518 diff changeset	201 wg.URL,
eb1d119f253f Fetch data from ERDMS for all allowed countries Tom Gottfried <tom@intevation.de> parents: 2518 diff changeset	202 wg.Insecure,
eb1d119f253f Fetch data from ERDMS for all allowed countries Tom Gottfried <tom@intevation.de> parents: 2518 diff changeset	203 "wtwgag")
eb1d119f253f Fetch data from ERDMS for all allowed countries Tom Gottfried <tom@intevation.de> parents: 2518 diff changeset	204 if err != nil {
1812 4910bcfab319 Waterway gauge import: Work around the expected (but nonetheless silly) namespace issues with the SOAP interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1810 diff changeset	205 return nil, err
4910bcfab319 Waterway gauge import: Work around the expected (but nonetheless silly) namespace issues with the SOAP interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1810 diff changeset	206 }
4910bcfab319 Waterway gauge import: Work around the expected (but nonetheless silly) namespace issues with the SOAP interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1810 diff changeset	207
3665 29ef6d41e4af Use database triggers to move referencing objects to new versions Tom Gottfried <tom@intevation.de> parents: 3648 diff changeset	208 var eraseGaugeStmt, insertStmt,
3645 02951a62e8c6 'Historicise' bottlenecks on import Tom Gottfried <tom@intevation.de> parents: 3551 diff changeset	209 fixValidityStmt, updateStmt,
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	210 deleteReferenceWaterLevelsStmt,
3163 d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	211 isNtSDepthRefStmt, insertWaterLevelStmt *sql.Stmt
d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	212 for _, x := range []struct {
d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	213 sql string
d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	214 stmt **sql.Stmt
d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	215 }{
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	216 {eraseGaugeSQL, &eraseGaugeStmt},
3163 d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	217 {insertGaugeSQL, &insertStmt},
3402 c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	218 {fixValiditySQL, &fixValidityStmt},
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	219 {updateGaugeSQL, &updateStmt},
3163 d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	220 {deleteReferenceWaterLevelsSQL, &deleteReferenceWaterLevelsStmt},
d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	221 {isNtSDepthRefSQL, &isNtSDepthRefStmt},
d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	222 {insertReferenceWaterLevelsSQL, &insertWaterLevelStmt},
d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	223 } {
d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	224 var err error
d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	225 if *x.stmt, err = conn.PrepareContext(ctx, x.sql); err != nil {
1829 b4b9089c2d79 Waterway gauges: Started with deleting old gauges to be overwritten. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1812 diff changeset	226 return nil, err
b4b9089c2d79 Waterway gauges: Started with deleting old gauges to be overwritten. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1812 diff changeset	227 }
3163 d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	228 defer (*x.stmt).Close()
1812 4910bcfab319 Waterway gauge import: Work around the expected (but nonetheless silly) namespace issues with the SOAP interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1810 diff changeset	229 }
1810 7ee9bdaac336 Waterway gauge import: Added support for username/password. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1808 diff changeset	230
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	231 var gauges []string
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	232 var unchanged int
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	233
5670 b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	234 var invalidISRS, startEndOrder, missingObjname, missingZeropoint, updatedGauges, newGauges []string
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	235
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	236 databaseErrors := map[string][]string{}
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	237
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	238 for _, data := range responseData {
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	239 for _, dr := range data.RisdataReturn {
1836 4dcdd8891770 Waterway gauges import: Fixed insert/update of gauges. TODO: Re-insert reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1835 diff changeset	240
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	241 isrs := string(*dr.RisidxCode)
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	242 code, err := models.IsrsFromString(isrs)
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	243 if err != nil {
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	244 invalidISRS = append(invalidISRS, isrs)
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	245 continue
1835 f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	246 }
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	247 gauges = append(gauges, isrs)
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	248
3529 ba0339118d9c Improve validation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3460 diff changeset	249 // We need a valid, non-empty time range to identify gauge versions
5609 e1936db6db8e Adjust ta erdms data structure. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5565 diff changeset	250 if dr.Enddate != nil && dr.Startdate != nil {
5615 f6179b31e0a9 Use a type alias on date type to not loose XML serialisation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5609 diff changeset	251 ed := dr.Enddate.ToGoTime()
f6179b31e0a9 Use a type alias on date type to not loose XML serialisation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5609 diff changeset	252 sd := dr.Startdate.ToGoTime()
f6179b31e0a9 Use a type alias on date type to not loose XML serialisation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5609 diff changeset	253 // log.Debugf("start date: %v end date: %v\n", sd, ed)
f6179b31e0a9 Use a type alias on date type to not loose XML serialisation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5609 diff changeset	254 if !ed.After(sd) {
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	255 startEndOrder = append(startEndOrder, isrs)
5609 e1936db6db8e Adjust ta erdms data structure. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5565 diff changeset	256 unchanged++
e1936db6db8e Adjust ta erdms data structure. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5565 diff changeset	257 continue
e1936db6db8e Adjust ta erdms data structure. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5565 diff changeset	258 }
3458 aa3c83fb7018 Validate temporal validity of gauges on import Tom Gottfried <tom@intevation.de> parents: 3402 diff changeset	259 }
aa3c83fb7018 Validate temporal validity of gauges on import Tom Gottfried <tom@intevation.de> parents: 3402 diff changeset	260
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	261 if dr.Zeropoint == nil {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	262 missingZeropoint = append(missingZeropoint, isrs)
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	263 unchanged++
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	264 continue
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	265 }
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	266
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	267 if dr.Objname.Loc == nil {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	268 missingObjname = append(missingObjname, isrs)
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	269 unchanged++
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	270 continue
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	271 }
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	272
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	273 var from, to sql.NullInt64
1835 f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	274
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	275 if dr.Applicabilityfromkm != nil {
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	276 from = sql.NullInt64{
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	277 Int64: int64(*dr.Applicabilityfromkm),
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	278 Valid: true,
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	279 }
1836 4dcdd8891770 Waterway gauges import: Fixed insert/update of gauges. TODO: Re-insert reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1835 diff changeset	280 }
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	281 if dr.Applicabilitytokm != nil {
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	282 to = sql.NullInt64{
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	283 Int64: int64(*dr.Applicabilitytokm),
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	284 Valid: true,
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	285 }
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	286 }
1835 f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	287
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	288 var tfrom, tto, dateInfo pgtype.Timestamptz
1835 f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	289
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	290 if dr.Startdate != nil {
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	291 tfrom = pgtype.Timestamptz{
5615 f6179b31e0a9 Use a type alias on date type to not loose XML serialisation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5609 diff changeset	292 Time: dr.Startdate.ToGoTime(),
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	293 Status: pgtype.Present,
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	294 }
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	295 } else {
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	296 tfrom = pgtype.Timestamptz{
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	297 Status: pgtype.Null,
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	298 }
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	299 }
1835 f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	300
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	301 if dr.Enddate != nil {
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	302 tto = pgtype.Timestamptz{
5615 f6179b31e0a9 Use a type alias on date type to not loose XML serialisation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5609 diff changeset	303 Time: dr.Enddate.ToGoTime(),
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	304 Status: pgtype.Present,
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	305 }
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	306 } else {
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	307 tto = pgtype.Timestamptz{
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	308 Status: pgtype.Null,
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	309 }
1836 4dcdd8891770 Waterway gauges import: Fixed insert/update of gauges. TODO: Re-insert reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1835 diff changeset	310 }
1835 f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	311
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	312 validity := pgtype.Tstzrange{
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	313 Lower: tfrom,
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	314 Upper: tto,
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	315 LowerType: pgtype.Inclusive,
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	316 UpperType: pgtype.Exclusive,
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	317 Status: pgtype.Present,
1836 4dcdd8891770 Waterway gauges import: Fixed insert/update of gauges. TODO: Re-insert reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1835 diff changeset	318 }
3163 d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	319
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	320 if dr.Infodate != nil {
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	321 dateInfo = pgtype.Timestamptz{
5615 f6179b31e0a9 Use a type alias on date type to not loose XML serialisation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5609 diff changeset	322 Time: dr.Infodate.ToGoTime(),
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	323 Status: pgtype.Present,
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	324 }
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	325 } else {
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	326 dateInfo = pgtype.Timestamptz{
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	327 Status: pgtype.Null,
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	328 }
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	329 }
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	330
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	331 var geodref sql.NullString
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	332 if dr.Geodref != nil {
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	333 geodref = sql.NullString{
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	334 String: string(*dr.Geodref),
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	335 Valid: true,
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	336 }
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	337 }
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	338
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	339 var source sql.NullString
5670 b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	340 if dr.RisidxSource != nil {
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	341 source = sql.NullString{
5670 b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	342 String: string(*dr.RisidxSource),
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	343 Valid: true,
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	344 }
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	345 }
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	346
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	347 err = func() error {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	348 tx, err := conn.BeginTx(ctx, nil)
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	349 if err != nil {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	350 return err
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	351 }
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	352 defer tx.Rollback()
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	353
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	354 // Mark old entry of gauge as erased, if applicable
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	355 var isNew bool
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	356 err = tx.StmtContext(ctx, eraseGaugeStmt).QueryRowContext(ctx,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	357 code.String(),
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	358 validity,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	359 ).Scan(&isNew)
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	360 switch {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	361 case err != nil:
4197 5d7ce7f926eb Log fatal errors as errors, not warnings, in imports Tom Gottfried <tom@intevation.de> parents: 4182 diff changeset	362 feedback.Error(pgxutils.ReadableError{Err: err}.Error())
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	363 unchanged++
5683 31973f6f5cca Get rid of deprecation warnings from staticcheck. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5670 diff changeset	364 return errContinue
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	365 case isNew:
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	366 var lu *time.Time
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	367 if dr.Lastupdate != nil {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	368 t := dr.Lastupdate.ToGoTime()
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	369 lu = &t
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	370 }
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	371 // insert gauge version entry
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	372 if _, err = tx.StmtContext(ctx, insertStmt).ExecContext(ctx,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	373 code.CountryCode,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	374 code.LoCode,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	375 code.FairwaySection,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	376 code.Orc,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	377 code.Hectometre,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	378 dr.Objname.Loc,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	379 dr.Lon, dr.Lat,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	380 from,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	381 to,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	382 &validity,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	383 dr.Zeropoint,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	384 geodref,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	385 &dateInfo,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	386 source,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	387 lu,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	388 ); err != nil {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	389 key := pgxutils.ReadableError{Err: err}.Error()
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	390 databaseErrors[key] = append(databaseErrors[key], isrs)
5683 31973f6f5cca Get rid of deprecation warnings from staticcheck. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5670 diff changeset	391 return errContinue
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	392 }
5670 b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	393 newGauges = append(newGauges, isrs)
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	394 //feedback.Info("insert new version")
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	395 case !isNew:
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	396 var lu *time.Time
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	397 if dr.Lastupdate != nil {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	398 t := dr.Lastupdate.ToGoTime()
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	399 lu = &t
3402 c04b1409a596 Fix adaptation of gauge temporal validity Tom Gottfried <tom@intevation.de> parents: 3311 diff changeset	400 }
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	401 // try to update
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	402 var dummy int
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	403 err2 := tx.StmtContext(ctx, updateStmt).QueryRowContext(ctx,
5670 b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	404 code.String(),
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	405 dr.Objname.Loc,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	406 dr.Lon, dr.Lat,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	407 from,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	408 to,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	409 dr.Zeropoint,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	410 geodref,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	411 &dateInfo,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	412 source,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	413 lu,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	414 &validity,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	415 ).Scan(&dummy)
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	416 switch {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	417 case err2 == sql.ErrNoRows:
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	418 //feedback.Info("unchanged")
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	419 unchanged++
5683 31973f6f5cca Get rid of deprecation warnings from staticcheck. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5670 diff changeset	420 return errContinue
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	421 case err2 != nil:
5670 b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	422 key := pgxutils.ReadableError{Err: err2}.Error()
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	423 databaseErrors[key] = append(databaseErrors[key], isrs)
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	424 unchanged++
5683 31973f6f5cca Get rid of deprecation warnings from staticcheck. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5670 diff changeset	425 return errContinue
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	426 default:
5670 b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	427 updatedGauges = append(updatedGauges, isrs)
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	428 //feedback.Info("update")
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	429 }
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	430
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	431 // Remove obsolete reference water levels
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	432 var currLevels pgtype.VarcharArray
5670 b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	433 var reflevels []string
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	434 if dr.Reflevel1code != nil {
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	435 reflevels = append(reflevels,
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	436 string(*dr.Reflevel1code))
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	437 }
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	438 if dr.Reflevel2code != nil {
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	439 reflevels = append(reflevels,
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	440 string(*dr.Reflevel2code))
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	441 }
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	442 if dr.Reflevel3code != nil {
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	443 reflevels = append(reflevels,
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	444 string(*dr.Reflevel3code))
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	445 }
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	446 currLevels.Set(reflevels)
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	447 rwls, err := tx.StmtContext(ctx,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	448 deleteReferenceWaterLevelsStmt).QueryContext(ctx,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	449 code.String(),
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	450 &validity,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	451 &currLevels,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	452 )
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	453 if err != nil {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	454 return err
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	455 }
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	456 defer rwls.Close()
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	457 for rwls.Next() {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	458 var delRef string
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	459 if err := rwls.Scan(&delRef); err != nil {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	460 return err
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	461 }
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	462 feedback.Warn("Removed reference water level %s from %s",
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	463 delRef, code)
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	464 }
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	465 if err := rwls.Err(); err != nil {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	466 return err
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	467 }
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	468 }
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	469
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	470 // Set end of validity of old version to start of new version
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	471 // in case of overlap
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	472 if _, err = tx.StmtContext(ctx, fixValidityStmt).ExecContext(
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	473 ctx,
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	474 code.String(),
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	475 &validity,
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	476 ); err != nil {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	477 key := pgxutils.ReadableError{Err: err}.Error()
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	478 databaseErrors[key] = append(databaseErrors[key], isrs)
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	479 unchanged++
5683 31973f6f5cca Get rid of deprecation warnings from staticcheck. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5670 diff changeset	480 return errContinue
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	481 }
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	482
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	483 // "Upsert" reference water levels
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	484 for _, wl := range []struct {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	485 level **erdms.RisreflevelcodeType
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	486 value **erdms.RisreflevelvalueType
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	487 }{
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	488 {&dr.Reflevel1code, &dr.Reflevel1value},
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	489 {&dr.Reflevel2code, &dr.Reflevel2value},
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	490 {&dr.Reflevel3code, &dr.Reflevel3value},
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	491 } {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	492 if wl.level == nil \|\| wl.value == nil {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	493 continue
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	494 }
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	495
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	496 var isNtSDepthRef bool
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	497 if err := tx.StmtContext(
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	498 ctx, isNtSDepthRefStmt).QueryRowContext(ctx,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	499 string(**wl.level),
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	500 ).Scan(
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	501 &isNtSDepthRef,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	502 ); err != nil {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	503 return err
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	504 }
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	505 if !isNtSDepthRef {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	506 feedback.Warn(
5670 b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	507 "Reference level code '%s' of %s is not in line "+
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	508 "with the NtS reference_code table",
5670 b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	509 string(**wl.level), isrs)
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	510 }
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	511
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	512 if _, err := tx.StmtContext(
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	513 ctx, insertWaterLevelStmt).ExecContext(ctx,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	514 code.CountryCode,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	515 code.LoCode,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	516 code.FairwaySection,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	517 code.Orc,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	518 code.Hectometre,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	519 &validity,
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	520 string(**wl.level),
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	521 int64(**wl.value),
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	522 ); err != nil {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	523 key := pgxutils.ReadableError{Err: err}.Error()
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	524 databaseErrors[key] = append(databaseErrors[key], isrs)
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	525 continue
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	526 }
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	527 }
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	528
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	529 return tx.Commit()
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	530 }()
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	531
5683 31973f6f5cca Get rid of deprecation warnings from staticcheck. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5670 diff changeset	532 if err != nil && err != errContinue {
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	533 return err, nil
3305 5932f9574493 Follow-up for 3350:e640f51b5a4e (Fix reporting of removed reference water levels). Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3302 diff changeset	534 }
3163 d9903cb34842 Handle failing INSERTs gracefully during gauges import Tom Gottfried <tom@intevation.de> parents: 2633 diff changeset	535 }
1835 f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	536 }
f7b926440449 Waterway gauge import: More stuggling with inserting gauges. Not working, yet. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1829 diff changeset	537
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	538 if len(invalidISRS) > 0 {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	539 feedback.Error("Invalid ISRS code: '%s'", strings.Join(invalidISRS, "', '"))
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	540 }
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	541
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	542 if len(startEndOrder) > 0 {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	543 feedback.Error("start date not before end date: %s",
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	544 strings.Join(startEndOrder, ", "))
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	545 }
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	546
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	547 if len(databaseErrors) > 0 {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	548 for err, iris := range databaseErrors {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	549 feedback.Error("%s: %s", err, strings.Join(iris, ", "))
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	550 }
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	551 }
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	552
5621 cf1e8ffe1ed5 Fix missing zero point logging in wg import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5620 diff changeset	553 if len(missingZeropoint) > 0 {
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	554 feedback.Error("Missing zeropoint: %s", strings.Join(missingZeropoint, ", "))
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	555 }
5621 cf1e8ffe1ed5 Fix missing zero point logging in wg import. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5620 diff changeset	556
5620 165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	557 if len(missingObjname) > 0 {
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	558 feedback.Error("Missing objname: %s", strings.Join(missingObjname, ", "))
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	559 }
165e77c5736a Simplified WG import logging. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5615 diff changeset	560
5670 b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	561 if len(updatedGauges) > 0 {
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	562 feedback.Info("Updated gauges: %s", strings.Join(updatedGauges, ", "))
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	563 }
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	564
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	565 if len(newGauges) > 0 {
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	566 feedback.Info("New gauges: %s", strings.Join(newGauges, ", "))
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	567 }
b75d0b303328 Various fixes and improvements of gauges import: Sascha Wilde <wilde@sha-bang.de> parents: 5621 diff changeset	568
3310 e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	569 if len(gauges) == 0 {
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	570 return nil, UnchangedError("No gauges returned from ERDMS")
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	571 }
e0dabe7b2fcf Simplify gauges import Tom Gottfried <tom@intevation.de> parents: 3305 diff changeset	572
3551 a5448426e4e2 Only erase gauges that have been requested from ERDMS Tom Gottfried <tom@intevation.de> parents: 3529 diff changeset	573 var pgCountries, pgGauges pgtype.VarcharArray
a5448426e4e2 Only erase gauges that have been requested from ERDMS Tom Gottfried <tom@intevation.de> parents: 3529 diff changeset	574 pgCountries.Set(countries)
3311 0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	575 pgGauges.Set(gauges)
3551 a5448426e4e2 Only erase gauges that have been requested from ERDMS Tom Gottfried <tom@intevation.de> parents: 3529 diff changeset	576 obsGauges, err := conn.QueryContext(ctx,
a5448426e4e2 Only erase gauges that have been requested from ERDMS Tom Gottfried <tom@intevation.de> parents: 3529 diff changeset	577 eraseObsoleteGaugesSQL,
a5448426e4e2 Only erase gauges that have been requested from ERDMS Tom Gottfried <tom@intevation.de> parents: 3529 diff changeset	578 &pgCountries,
a5448426e4e2 Only erase gauges that have been requested from ERDMS Tom Gottfried <tom@intevation.de> parents: 3529 diff changeset	579 &pgGauges)
3311 0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	580 if err != nil {
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	581 return nil, err
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	582 }
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	583 defer obsGauges.Close()
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	584 for obsGauges.Next() {
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	585 var isrs string
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	586 if err := obsGauges.Scan(&isrs); err != nil {
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	587 return nil, err
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	588 }
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	589 feedback.Info("Erased %s", isrs)
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	590 unchanged--
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	591 }
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	592 if err := obsGauges.Err(); err != nil {
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	593 return nil, err
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	594 }
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	595
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	596 if unchanged == len(gauges) {
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	597 return nil, UnchangedError("All gauges unchanged")
ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	598 }
5711 2dd155cc95ec Fix all revive issue (w/o machine generated stuff). Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5683 diff changeset	599 feedback.Info("Unchanged gauges: %d",
2dd155cc95ec Fix all revive issue (w/o machine generated stuff). Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5683 diff changeset	600 unchanged)
3302 ec6163c6687d 'Historicise' gauges on import Tom Gottfried <tom@intevation.de> parents: 3286 diff changeset	601
3311 0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	602 feedback.Info("Importing gauges took %s",
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	603 time.Since(start))
0f6b156cff55 Mark gauges as erased if they are missing in ERDMS response Tom Gottfried <tom@intevation.de> parents: 3310 diff changeset	604
1837 00d63eb9306a Waterway gauges import: Finalized with re-inserting the reference water levels. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1836 diff changeset	605 return nil, err
1808 77582da3adb0 Waterway gauges import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	606 }

Mercurial > gemma

annotate pkg/imports/wg.go @ 5711:2dd155cc95ec revive-cleanup