gemma: pkg/imports/sec.go annotate

annotate pkg/imports/sec.go @ 5560:f2204f91d286

Join the log lines of imports to the log exports to recover data from them. Used in SR export to extract information that where in the meta json but now are only found in the log.

author	Sascha L. Teichmann <sascha.teichmann@intevation.de>
date	Wed, 09 Feb 2022 18:34:40 +0100
parents	59a99655f34d
children	6270951dda28

rev	line source
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	1 // This is Free Software under GNU Affero General Public License v >= 3.0
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	2 // without warranty, see README.md and license for details.
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	3 //
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	4 // SPDX-License-Identifier: AGPL-3.0-or-later
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	5 // License-Filename: LICENSES/AGPL-3.0.txt
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	6 //
3496 f10d606b2931 Made sections visualization less bulky. Sascha Wilde <wilde@intevation.de> parents: 3307 diff changeset	7 // Copyright (C) 2018,2019 by via donau
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	8 // – Österreichische Wasserstraßen-Gesellschaft mbH
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	9 // Software engineering by Intevation GmbH
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	10 //
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	11 // Author(s):
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	13
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14 package imports
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16 import (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	17 "context"
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	18 "database/sql"
4798 ca6a5f722471 Added Description method to most imports. Sascha Wilde <wilde@intevation.de> parents: 4306 diff changeset	19 "strings"
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	20 "time"
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	21
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	22 "gemma.intevation.de/gemma/pkg/models"
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	23 )
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	24
4179 a376351d2774 Another round of doc strings to make 'golint' a little bit more happy with the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4147 diff changeset	25 // Section is a Job to create a section in the database.
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	26 type Section struct {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	27 Name string `json:"name"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	28 From models.Isrs `json:"from"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	29 To models.Isrs `json:"to"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	30 Tolerance float32 `json:"tolerance"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	31 ObjNam string `json:"objnam"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	32 NObjNam *string `json:"nobjnam"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	33 Source string `json:"source-organization"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	34 Date models.Date `json:"date-info"`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	35 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	36
4852 046a07a33b19 Fixed the golint issues of the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4799 diff changeset	37 // Description gives a short info about relevant facts of this import.
5560 f2204f91d286 Join the log lines of imports to the log exports to recover data from them. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5034 diff changeset	38 func (sec *Section) Description([]string) (string, error) {
4799 f32d086b5dbf Removed the mechanical touch of the last commit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4798 diff changeset	39 return strings.Join([]string{
f32d086b5dbf Removed the mechanical touch of the last commit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4798 diff changeset	40 sec.Name,
f32d086b5dbf Removed the mechanical touch of the last commit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4798 diff changeset	41 sec.ObjNam,
f32d086b5dbf Removed the mechanical touch of the last commit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4798 diff changeset	42 sec.From.String(),
f32d086b5dbf Removed the mechanical touch of the last commit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4798 diff changeset	43 sec.To.String(),
f32d086b5dbf Removed the mechanical touch of the last commit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4798 diff changeset	44 }, "\|"), nil
4798 ca6a5f722471 Added Description method to most imports. Sascha Wilde <wilde@intevation.de> parents: 4306 diff changeset	45 }
ca6a5f722471 Added Description method to most imports. Sascha Wilde <wilde@intevation.de> parents: 4306 diff changeset	46
4179 a376351d2774 Another round of doc strings to make 'golint' a little bit more happy with the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4147 diff changeset	47 // SECJobKind is the import queue type identifier.
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	48 const SECJobKind JobKind = "sec"
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	49
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	50 type secJobCreator struct{}
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	51
4179 a376351d2774 Another round of doc strings to make 'golint' a little bit more happy with the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4147 diff changeset	52 func init() { RegisterJobCreator(SECJobKind, secJobCreator{}) }
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	53
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	54 func (secJobCreator) Description() string { return "section" }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	55
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	56 func (secJobCreator) AutoAccept() bool { return false }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	57
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	58 func (secJobCreator) Create() Job { return new(Section) }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	59
3219 4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3171 diff changeset	60 func (secJobCreator) Depends() [2][]string {
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3171 diff changeset	61 return [2][]string{
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3171 diff changeset	62 {"sections"},
3224 bfde4f8dd323 Add missing table dependencies of imports with area generation Tom Gottfried <tom@intevation.de> parents: 3219 diff changeset	63 {"distance_marks_virtual", "waterway_axis", "waterway_area"},
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	64 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	65 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	66
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	67 const (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	68 secDeleteSQL = `
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	69 DELETE FROM waterway.sections WHERE
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	70 staging_done AND name = (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	71 SELECT name
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	72 FROM waterway.sections WHERE
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	73 id = (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	74 SELECT key from import.track_imports
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	75 WHERE import_id = $1 AND
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	76 relation = 'waterway.sections'::regclass)
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	77 AND NOT staging_done
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	78 )`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	79
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	80 secStageDoneSQL = `
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	81 UPDATE waterway.sections SET staging_done = true
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	82 WHERE id IN (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	83 SELECT key from import.track_imports
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	84 WHERE import_id = $1 AND
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	85 relation = 'waterway.sections'::regclass)`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	86
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	87 secInsertSQL = `
3566 4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	88 WITH
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	89 bounds (b) AS (VALUES (
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	90 ($1::char(2),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	91 $2::char(3),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	92 $3::char(5),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	93 $4::char(5),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	94 $5::int)::isrs
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	95 ), (
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	96 ($6::char(2),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	97 $7::char(3),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	98 $8::char(5),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	99 $9::char(5),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	100 $10::int)::isrs)),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	101 r AS (SELECT isrsrange(
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	102 (SELECT b FROM bounds ORDER BY b USING <~ FETCH FIRST ROW ONLY),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3560 diff changeset	103 (SELECT b FROM bounds ORDER BY b USING >~ FETCH FIRST ROW ONLY)) AS r),
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	104 axs AS (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	105 SELECT ISRSrange_axis((SELECT r FROM r), $16::double precision) AS axs)
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	106 INSERT INTO waterway.sections (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	107 name,
3307 b90b17d0b5a9 added sections_geoserver view, renamed stretch column to section in schema Markus Kottlaender <markus@intevation.de> parents: 3224 diff changeset	108 section,
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	109 area,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	110 objnam,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	111 nobjnam,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	112 date_info,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	113 source_organization
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	114 ) VALUES (
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	115 $11,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	116 (SELECT r FROM r),
4306 2336f75637d1 sections: Use blown up axis as geometry. (Nicked from current stretches) Sascha Wilde <wilde@intevation.de> parents: 4180 diff changeset	117 ST_Transform(ISRSrange_area(
2336f75637d1 sections: Use blown up axis as geometry. (Nicked from current stretches) Sascha Wilde <wilde@intevation.de> parents: 4180 diff changeset	118 (SELECT axs FROM axs),
2336f75637d1 sections: Use blown up axis as geometry. (Nicked from current stretches) Sascha Wilde <wilde@intevation.de> parents: 4180 diff changeset	119 (SELECT ST_Buffer(axs, 150) FROM axs)),
2336f75637d1 sections: Use blown up axis as geometry. (Nicked from current stretches) Sascha Wilde <wilde@intevation.de> parents: 4180 diff changeset	120 4326),
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	121 $12,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	122 $13,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	123 $14,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	124 $15)
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	125 RETURNING id`
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	126 )
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	127
3307 b90b17d0b5a9 added sections_geoserver view, renamed stretch column to section in schema Markus Kottlaender <markus@intevation.de> parents: 3224 diff changeset	128 // StageDone moves the imported section out of the staging area.
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	129 func (secJobCreator) StageDone(
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	130 ctx context.Context,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	131 tx *sql.Tx,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	132 id int64,
5034 59a99655f34d Added feedback support for StageDone. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4852 diff changeset	133 _ Feedback,
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	134 ) error {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	135 if _, err := tx.ExecContext(ctx, secDeleteSQL, id); err != nil {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	136 return err
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	137 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	138 _, err := tx.ExecContext(ctx, secStageDoneSQL, id)
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	139 return err
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	140 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	141
3307 b90b17d0b5a9 added sections_geoserver view, renamed stretch column to section in schema Markus Kottlaender <markus@intevation.de> parents: 3224 diff changeset	142 // CleanUp of a section import is a NOP.
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	143 func (*Section) CleanUp() error { return nil }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	144
3307 b90b17d0b5a9 added sections_geoserver view, renamed stretch column to section in schema Markus Kottlaender <markus@intevation.de> parents: 3224 diff changeset	145 // Do executes the actual section import.
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	146 func (sec *Section) Do(
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	147 ctx context.Context,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	148 importID int64,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	149 conn *sql.Conn,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	150 feedback Feedback,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	151 ) (interface{}, error) {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	152
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	153 start := time.Now()
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	154
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	155 if sec.Date.Time.IsZero() {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	156 sec.Date = models.Date{Time: start}
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	157 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	158
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	159 feedback.Info("Storing section '%s'", sec.Name)
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	160
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	161 tx, err := conn.BeginTx(ctx, nil)
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	162 if err != nil {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	163 return nil, err
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	164 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	165 defer tx.Rollback()
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	166
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	167 var nobjnm sql.NullString
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	168 if sec.NObjNam != nil {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	169 nobjnm = sql.NullString{String: *sec.NObjNam, Valid: true}
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	170 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	171
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	172 feedback.Info("Section from %s to %s.", sec.From.String(), sec.To.String())
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	173 feedback.Info("Tolerance used to snap waterway axis: %g", sec.Tolerance)
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	174
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	175 var id int64
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	176 if err := tx.QueryRowContext(
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	177 ctx,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	178 secInsertSQL,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	179 sec.From.CountryCode,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	180 sec.From.LoCode,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	181 sec.From.FairwaySection,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	182 sec.From.Orc,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	183 sec.From.Hectometre,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	184 sec.To.CountryCode,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	185 sec.To.LoCode,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	186 sec.To.FairwaySection,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	187 sec.To.Orc,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	188 sec.To.Hectometre,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	189 sec.Name,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	190 sec.ObjNam,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	191 nobjnm,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	192 sec.Date.Time,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	193 sec.Source,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	194 sec.Tolerance,
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	195 ).Scan(&id); err != nil {
4180 91cb4a7b1b13 Always try to translate to readable error if import failed Tom Gottfried <tom@intevation.de> parents: 4179 diff changeset	196 return nil, err
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	197 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	198
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	199 if err := track(ctx, tx, importID, "waterway.sections", id); err != nil {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	200 return nil, err
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	201 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	202
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	203 feedback.Info("Storing section '%s' took %s", sec.Name, time.Since(start))
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	204 if err := tx.Commit(); err != nil {
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	205 return nil, err
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	206 }
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	207 feedback.Info("Import of section was successful")
c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	208
3524 40bc7e3bc561 review details for sections Thomas Junk <thomas.junk@intevation.de> parents: 3496 diff changeset	209 summary := sec // to provide full data for review
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	210
3524 40bc7e3bc561 review details for sections Thomas Junk <thomas.junk@intevation.de> parents: 3496 diff changeset	211 return summary, nil
3171 c8ded555c2a8 Sections import: Added a sections import. Derived from the stretches import w/o the countries. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	212 }

Mercurial > gemma

annotate pkg/imports/sec.go @ 5560:f2204f91d286