gemma: pkg/imports/sec.go annotate

annotate pkg/imports/sec.go @ 5670:b75d0b303328

Various fixes and improvements of gauges import: - Allow update of erased data (and thereby set erased to false) - Fix source_organization to work with ERDMS2 - Give ISRS of new and updated gauges in summary - Fixed reference of null pointers if revlevels are missing - Fixed reference of null pointer on update errors - Added ISRS to reference_code warning

author	Sascha Wilde <wilde@sha-bang.de>
date	Fri, 08 Dec 2023 17:29:56 +0100
parents	f2204f91d286
children	6270951dda28

rev	line source
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	1 // This is Free Software under GNU Affero General Public License v >= 3.0
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	2 // without warranty, see README.md and license for details.
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	3 //
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	4 // SPDX-License-Identifier: AGPL-3.0-or-later
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	5 // License-Filename: LICENSES/AGPL-3.0.txt
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	6 //
3496 f10d606b2931 Made sections visualization less bulky. Sascha Wilde <wilde@intevation.de> parents: 3307 diff changeset	7 // Copyright (C) 2018,2019 by via donau
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	8 // – Österreichische Wasserstraßen-Gesellschaft mbH
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	9 // Software engineering by Intevation GmbH
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	10 //
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	11 // Author(s):
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	13
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14 package imports
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16 import (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	17 "context"
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	18 "database/sql"
4798 ca6a5f722471 Added Description method to most imports. Sascha Wilde <wilde@intevation.de> parents: 4306 diff changeset	19 "strings"
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	20 "time"
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	21
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	22 "gemma.intevation.de/gemma/pkg/models"
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	23 )
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	24
4179 a376351d2774 Another round of doc strings to make 'golint' a little bit more happy with the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4147 diff changeset	25 // Section is a Job to create a section in the database.
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	26 type Section struct {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	27 Name string `json:"name"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	28 From models.Isrs `json:"from"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	29 To models.Isrs `json:"to"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	30 Tolerance float32 `json:"tolerance"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	31 ObjNam string `json:"objnam"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	32 NObjNam *string `json:"nobjnam"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	33 Source string `json:"source-organization"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	34 Date models.Date `json:"date-info"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	35 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	36
4852 046a07a33b19 Fixed the golint issues of the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4799 diff changeset	37 // Description gives a short info about relevant facts of this import.
5560 f2204f91d286 Join the log lines of imports to the log exports to recover data from them. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5034 diff changeset	38 func (sec *Section) Description([]string) (string, error) {
4799 f32d086b5dbf Removed the mechanical touch of the last commit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4798 diff changeset	39 return strings.Join([]string{
f32d086b5dbf Removed the mechanical touch of the last commit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4798 diff changeset	40 sec.Name,
f32d086b5dbf Removed the mechanical touch of the last commit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4798 diff changeset	41 sec.ObjNam,
f32d086b5dbf Removed the mechanical touch of the last commit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4798 diff changeset	42 sec.From.String(),
f32d086b5dbf Removed the mechanical touch of the last commit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4798 diff changeset	43 sec.To.String(),
f32d086b5dbf Removed the mechanical touch of the last commit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4798 diff changeset	44 }, "\|"), nil
4798 ca6a5f722471 Added Description method to most imports. Sascha Wilde <wilde@intevation.de> parents: 4306 diff changeset	45 }
ca6a5f722471 Added Description method to most imports. Sascha Wilde <wilde@intevation.de> parents: 4306 diff changeset	46
4179 a376351d2774 Another round of doc strings to make 'golint' a little bit more happy with the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4147 diff changeset	47 // SECJobKind is the import queue type identifier.
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	48 const SECJobKind JobKind = "sec"
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	49
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	50 type secJobCreator struct{}
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	51
4179 a376351d2774 Another round of doc strings to make 'golint' a little bit more happy with the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4147 diff changeset	52 func init() { RegisterJobCreator(SECJobKind, secJobCreator{}) }
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	53
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	54 func (secJobCreator) Description() string { return "section" }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	55
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	56 func (secJobCreator) AutoAccept() bool { return false }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	57
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	58 func (secJobCreator) Create() Job { return new(Section) }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	59
3219 4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3171 diff changeset	60 func (secJobCreator) Depends() [2][]string {
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3171 diff changeset	61 return [2][]string{
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3171 diff changeset	62 {"sections"},
3224 bfde4f8dd323 Add missing table dependencies of imports with area generation Tom Gottfried <tom@intevation.de> parents: 3219 diff changeset	63 {"distance_marks_virtual", "waterway_axis", "waterway_area"},
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	64 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	65 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	66
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	67 const (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	68 secDeleteSQL = `
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	69 DELETE FROM waterway.sections WHERE
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	70 staging_done AND name = (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	71 SELECT name
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	72 FROM waterway.sections WHERE
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	73 id = (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	74 SELECT key from import.track_imports
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	75 WHERE import_id = $1 AND
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	76 relation = 'waterway.sections'::regclass)
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	77 AND NOT staging_done
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	78 )`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	79
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	80 secStageDoneSQL = `
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	81 UPDATE waterway.sections SET staging_done = true
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	82 WHERE id IN (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	83 SELECT key from import.track_imports
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	84 WHERE import_id = $1 AND
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	85 relation = 'waterway.sections'::regclass)`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	86
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	87 secInsertSQL = `
3566 4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	88 WITH
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	89 bounds (b) AS (VALUES (
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	90 ($1::char(2),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	91 $2::char(3),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	92 $3::char(5),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	93 $4::char(5),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	94 $5::int)::isrs
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	95 ), (
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	96 ($6::char(2),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	97 $7::char(3),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	98 $8::char(5),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	99 $9::char(5),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	100 $10::int)::isrs)),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	101 r AS (SELECT isrsrange(
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	102 (SELECT b FROM bounds ORDER BY b USING <~ FETCH FIRST ROW ONLY),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	103 (SELECT b FROM bounds ORDER BY b USING >~ FETCH FIRST ROW ONLY)) AS r),
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	104 axs AS (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	105 SELECT ISRSrange_axis((SELECT r FROM r), $16::double precision) AS axs)
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	106 INSERT INTO waterway.sections (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	107 name,
3307 b90b17d0b5a9 added sections_geoserver view, renamed stretch column to section in schema Markus Kottlaender <markus@intevation.de> parents: 3224 diff changeset	108 section,
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	109 area,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	110 objnam,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	111 nobjnam,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	112 date_info,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	113 source_organization
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	114 ) VALUES (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	115 $11,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	116 (SELECT r FROM r),
4306 2336f75637d1 sections: Use blown up axis as geometry. (Nicked from current stretches) Sascha Wilde <wilde@intevation.de> parents: 4180 diff changeset	117 ST_Transform(ISRSrange_area(
2336f75637d1 sections: Use blown up axis as geometry. (Nicked from current stretches) Sascha Wilde <wilde@intevation.de> parents: 4180 diff changeset	118 (SELECT axs FROM axs),
2336f75637d1 sections: Use blown up axis as geometry. (Nicked from current stretches) Sascha Wilde <wilde@intevation.de> parents: 4180 diff changeset	119 (SELECT ST_Buffer(axs, 150) FROM axs)),
2336f75637d1 sections: Use blown up axis as geometry. (Nicked from current stretches) Sascha Wilde <wilde@intevation.de> parents: 4180 diff changeset	120 4326),
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	121 $12,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	122 $13,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	123 $14,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	124 $15)
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	125 RETURNING id`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	126 )
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	127
3307 b90b17d0b5a9 added sections_geoserver view, renamed stretch column to section in schema Markus Kottlaender <markus@intevation.de> parents: 3224 diff changeset	128 // StageDone moves the imported section out of the staging area.
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	129 func (secJobCreator) StageDone(
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	130 ctx context.Context,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	131 tx *sql.Tx,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	132 id int64,
5034 59a99655f34d Added feedback support for StageDone. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4852 diff changeset	133 _ Feedback,
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	134 ) error {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	135 if _, err := tx.ExecContext(ctx, secDeleteSQL, id); err != nil {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	136 return err
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	137 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	138 _, err := tx.ExecContext(ctx, secStageDoneSQL, id)
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	139 return err
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	140 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	141
3307 b90b17d0b5a9 added sections_geoserver view, renamed stretch column to section in schema Markus Kottlaender <markus@intevation.de> parents: 3224 diff changeset	142 // CleanUp of a section import is a NOP.
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	143 func (*Section) CleanUp() error { return nil }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	144
3307 b90b17d0b5a9 added sections_geoserver view, renamed stretch column to section in schema Markus Kottlaender <markus@intevation.de> parents: 3224 diff changeset	145 // Do executes the actual section import.
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	146 func (sec *Section) Do(
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	147 ctx context.Context,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	148 importID int64,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	149 conn *sql.Conn,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	150 feedback Feedback,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	151 ) (interface{}, error) {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	152
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	153 start := time.Now()
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	154
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	155 if sec.Date.Time.IsZero() {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	156 sec.Date = models.Date{Time: start}
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	157 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	158
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	159 feedback.Info("Storing section '%s'", sec.Name)
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	160
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	161 tx, err := conn.BeginTx(ctx, nil)
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	162 if err != nil {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	163 return nil, err
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	164 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	165 defer tx.Rollback()
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	166
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	167 var nobjnm sql.NullString
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	168 if sec.NObjNam != nil {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	169 nobjnm = sql.NullString{String: *sec.NObjNam, Valid: true}
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	170 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	171
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	172 feedback.Info("Section from %s to %s.", sec.From.String(), sec.To.String())
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	173 feedback.Info("Tolerance used to snap waterway axis: %g", sec.Tolerance)
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	174
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	175 var id int64
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	176 if err := tx.QueryRowContext(
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	177 ctx,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	178 secInsertSQL,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	179 sec.From.CountryCode,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	180 sec.From.LoCode,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	181 sec.From.FairwaySection,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	182 sec.From.Orc,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	183 sec.From.Hectometre,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	184 sec.To.CountryCode,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	185 sec.To.LoCode,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	186 sec.To.FairwaySection,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	187 sec.To.Orc,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	188 sec.To.Hectometre,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	189 sec.Name,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	190 sec.ObjNam,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	191 nobjnm,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	192 sec.Date.Time,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	193 sec.Source,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	194 sec.Tolerance,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	195 ).Scan(&id); err != nil {
4180 91cb4a7b1b13 Always try to translate to readable error if import failed Tom Gottfried <tom@intevation.de> parents: 4179 diff changeset	196 return nil, err
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	197 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	198
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	199 if err := track(ctx, tx, importID, "waterway.sections", id); err != nil {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	200 return nil, err
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	201 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	202
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	203 feedback.Info("Storing section '%s' took %s", sec.Name, time.Since(start))
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	204 if err := tx.Commit(); err != nil {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	205 return nil, err
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	206 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	207 feedback.Info("Import of section was successful")
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	208
3524 40bc7e3bc561 review details for sections Thomas Junk <thomas.junk@intevation.de> parents: 3496 diff changeset	209 summary := sec // to provide full data for review
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	210
3524 40bc7e3bc561 review details for sections Thomas Junk <thomas.junk@intevation.de> parents: 3496 diff changeset	211 return summary, nil
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	212 }

Mercurial > gemma

annotate pkg/imports/sec.go @ 5670:b75d0b303328