gemma: pkg/imports/st.go annotate

annotate pkg/imports/st.go @ 4180:91cb4a7b1b13

Always try to translate to readable error if import failed

author	Tom Gottfried <tom@intevation.de>
date	Mon, 05 Aug 2019 18:21:19 +0200
parents	a376351d2774
children	f9bb06f2dbe3

rev	line source
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	1 // This is Free Software under GNU Affero General Public License v >= 3.0
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	2 // without warranty, see README.md and license for details.
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	3 //
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	4 // SPDX-License-Identifier: AGPL-3.0-or-later
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	5 // License-Filename: LICENSES/AGPL-3.0.txt
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	6 //
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	7 // Copyright (C) 2018 by via donau
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	8 // – Österreichische Wasserstraßen-Gesellschaft mbH
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	9 // Software engineering by Intevation GmbH
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	10 //
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	11 // Author(s):
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	13
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14 package imports
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16 import (
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	17 "context"
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	18 "database/sql"
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	19 "errors"
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	20 "time"
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	21
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	22 "gemma.intevation.de/gemma/pkg/models"
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	23 )
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	24
4179 a376351d2774 Another round of doc strings to make 'golint' a little bit more happy with the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4177 diff changeset	25 // Stretch is a Job to create a stretch in the database.
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	26 type Stretch struct {
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	27 Name string `json:"name"`
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	28 From models.Isrs `json:"from"`
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	29 To models.Isrs `json:"to"`
2563 dc4fae4bdb8f Expose axis snapping tolerance to users Tom Gottfried <tom@intevation.de> parents: 2548 diff changeset	30 Tolerance float32 `json:"tolerance"`
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	31 ObjNam string `json:"objnam"`
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	32 NObjNam *string `json:"nobjnam"`
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	33 Source string `json:"source-organization"`
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	34 Date models.Date `json:"date-info"`
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	35 Countries models.UniqueCountries `json:"countries"`
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	36 }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	37
4179 a376351d2774 Another round of doc strings to make 'golint' a little bit more happy with the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4177 diff changeset	38 // STJobKind is the import queue type identifier.
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	39 const STJobKind JobKind = "st"
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	40
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	41 type stJobCreator struct{}
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	42
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	43 func init() {
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	44 RegisterJobCreator(STJobKind, stJobCreator{})
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	45 }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	46
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	47 func (stJobCreator) Description() string { return "stretch" }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	48
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	49 func (stJobCreator) AutoAccept() bool { return false }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	50
2187 7c83b5277c1c Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2168 diff changeset	51 func (stJobCreator) Create() Job { return new(Stretch) }
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	52
3219 4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3170 diff changeset	53 func (stJobCreator) Depends() [2][]string {
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3170 diff changeset	54 return [2][]string{
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3170 diff changeset	55 {"stretches", "stretch_countries"},
3224 bfde4f8dd323 Add missing table dependencies of imports with area generation Tom Gottfried <tom@intevation.de> parents: 3219 diff changeset	56 {"distance_marks_virtual", "waterway_axis", "waterway_area"},
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	57 }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	58 }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	59
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	60 const (
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	61 stDeleteSQL = `
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	62 DELETE FROM waterway.stretches WHERE
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	63 staging_done AND name = (
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	64 SELECT name
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	65 FROM waterway.stretches WHERE
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	66 id = (
1995 59055c8301df Move import queue to its own database namespace Tom Gottfried <tom@intevation.de> parents: 1983 diff changeset	67 SELECT key from import.track_imports
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	68 WHERE import_id = $1 AND
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	69 relation = 'waterway.stretches'::regclass)
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	70 AND NOT staging_done
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	71 )`
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	72
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	73 stStageDoneSQL = `
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	74 UPDATE waterway.stretches SET staging_done = true
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	75 WHERE id IN (
1995 59055c8301df Move import queue to its own database namespace Tom Gottfried <tom@intevation.de> parents: 1983 diff changeset	76 SELECT key from import.track_imports
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	77 WHERE import_id = $1 AND
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	78 relation = 'waterway.stretches'::regclass)`
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	79
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	80 stInsertSQL = `
3566 4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	81 WITH
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	82 bounds (b) AS (VALUES (
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	83 ($1::char(2),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	84 $2::char(3),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	85 $3::char(5),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	86 $4::char(5),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	87 $5::int)::isrs
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	88 ), (
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	89 ($6::char(2),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	90 $7::char(3),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	91 $8::char(5),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	92 $9::char(5),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	93 $10::int)::isrs)),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	94 r AS (SELECT isrsrange(
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	95 (SELECT b FROM bounds ORDER BY b USING <~ FETCH FIRST ROW ONLY),
4c585b5d4fe8 Introduce hectometer based order for ranges of ISRS location codes Tom Gottfried <tom@intevation.de> parents: 3521 diff changeset	96 (SELECT b FROM bounds ORDER BY b USING >~ FETCH FIRST ROW ONLY)) AS r),
2911 a75c546ef498 Use buffer around axis for stretch area Tom Gottfried <tom@intevation.de> parents: 2755 diff changeset	97 axs AS (
a75c546ef498 Use buffer around axis for stretch area Tom Gottfried <tom@intevation.de> parents: 2755 diff changeset	98 SELECT ISRSrange_axis((SELECT r FROM r), $16::double precision) AS axs)
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	99 INSERT INTO waterway.stretches (
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	100 name,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	101 stretch,
1983 f9f1babe52ae Fix area generation from multipolygon input Tom Gottfried <tom@intevation.de> parents: 1924 diff changeset	102 area,
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	103 objnam,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	104 nobjnam,
1916 8750f50b6cee Stretch import: Fixed insert statement to store stretch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1912 diff changeset	105 date_info,
8750f50b6cee Stretch import: Fixed insert statement to store stretch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1912 diff changeset	106 source_organization
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	107 ) VALUES (
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	108 $11,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	109 (SELECT r FROM r),
2911 a75c546ef498 Use buffer around axis for stretch area Tom Gottfried <tom@intevation.de> parents: 2755 diff changeset	110 ST_Transform(ISRSrange_area(
a75c546ef498 Use buffer around axis for stretch area Tom Gottfried <tom@intevation.de> parents: 2755 diff changeset	111 (SELECT axs FROM axs),
a75c546ef498 Use buffer around axis for stretch area Tom Gottfried <tom@intevation.de> parents: 2755 diff changeset	112 (SELECT ST_Buffer(axs, 10000) FROM axs)),
a75c546ef498 Use buffer around axis for stretch area Tom Gottfried <tom@intevation.de> parents: 2755 diff changeset	113 4326),
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	114 $12,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	115 $13,
1916 8750f50b6cee Stretch import: Fixed insert statement to store stretch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1912 diff changeset	116 $14,
8750f50b6cee Stretch import: Fixed insert statement to store stretch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1912 diff changeset	117 $15)
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	118 RETURNING id`
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	119
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	120 stInsertCountrySQL = `
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	121 INSERT INTO waterway.stretch_countries (
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	122 stretches_id,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	123 country_code
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	124 ) VALUES (
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	125 $1,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	126 $2
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	127 )`
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	128 )
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	129
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	130 // StageDone moves the imported stretch out of the staging area.
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	131 func (stJobCreator) StageDone(
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	132 ctx context.Context,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	133 tx *sql.Tx,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	134 id int64,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	135 ) error {
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	136 if _, err := tx.ExecContext(ctx, stDeleteSQL, id); err != nil {
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	137 return err
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	138 }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	139 _, err := tx.ExecContext(ctx, stStageDoneSQL, id)
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	140 return err
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	141 }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	142
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	143 // CleanUp of a stretch import is a NOP.
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	144 func (*Stretch) CleanUp() error { return nil }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	145
1911 6ffc372cde1e Fixed JSON serialization of ISRS codes. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1906 diff changeset	146 // Do executes the actual stretch import.
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	147 func (st *Stretch) Do(
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	148 ctx context.Context,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	149 importID int64,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	150 conn *sql.Conn,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	151 feedback Feedback,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	152 ) (interface{}, error) {
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	153
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	154 start := time.Now()
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	155
1911 6ffc372cde1e Fixed JSON serialization of ISRS codes. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1906 diff changeset	156 if st.Date.Time.IsZero() {
1924 0ff9bd32c34c Fixed 'go vet' issue. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1920 diff changeset	157 st.Date = models.Date{Time: start}
1911 6ffc372cde1e Fixed JSON serialization of ISRS codes. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1906 diff changeset	158 }
6ffc372cde1e Fixed JSON serialization of ISRS codes. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1906 diff changeset	159
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	160 feedback.Info("Storing stretch '%s'", st.Name)
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	161
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	162 if len(st.Countries) == 0 {
4177 8b75ac5e243e Made 'staticcheck' happy with pgxutils package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4147 diff changeset	163 return nil, errors.New("list of countries is empty")
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	164 }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	165
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	166 tx, err := conn.BeginTx(ctx, nil)
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	167 if err != nil {
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	168 return nil, err
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	169 }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	170 defer tx.Rollback()
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	171
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	172 insertCountryStmt, err := tx.PrepareContext(ctx, stInsertCountrySQL)
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	173 if err != nil {
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	174 return nil, err
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	175 }
3170 5c8ecab9f2d4 Stretch import: Added missing prepared statement closing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2911 diff changeset	176 defer insertCountryStmt.Close()
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	177
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	178 var nobjnm sql.NullString
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	179 if st.NObjNam != nil {
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	180 nobjnm = sql.NullString{String: *st.NObjNam, Valid: true}
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	181 }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	182
2261 c5bbe2409a52 Stretch import: Call Stringer for ISRS codes directly when logging range. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2248 diff changeset	183 feedback.Info("Stretch from %s to %s.", st.From.String(), st.To.String())
2693 02d012272dd4 Log snapping tolerance Tom Gottfried <tom@intevation.de> parents: 2563 diff changeset	184 feedback.Info("Tolerance used to snap waterway axis: %g", st.Tolerance)
2231 97bba8b51b9c Stretch import: Be more verbose about range before inserting data. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	185
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	186 var id int64
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	187 if err := tx.QueryRowContext(
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	188 ctx,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	189 stInsertSQL,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	190 st.From.CountryCode,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	191 st.From.LoCode,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	192 st.From.FairwaySection,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	193 st.From.Orc,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	194 st.From.Hectometre,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	195 st.To.CountryCode,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	196 st.To.LoCode,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	197 st.To.FairwaySection,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	198 st.To.Orc,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	199 st.To.Hectometre,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	200 st.Name,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	201 st.ObjNam,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	202 nobjnm,
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	203 st.Date.Time,
1916 8750f50b6cee Stretch import: Fixed insert statement to store stretch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1912 diff changeset	204 st.Source,
2563 dc4fae4bdb8f Expose axis snapping tolerance to users Tom Gottfried <tom@intevation.de> parents: 2548 diff changeset	205 st.Tolerance,
1916 8750f50b6cee Stretch import: Fixed insert statement to store stretch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1912 diff changeset	206 ).Scan(&id); err != nil {
4180 91cb4a7b1b13 Always try to translate to readable error if import failed Tom Gottfried <tom@intevation.de> parents: 4179 diff changeset	207 return nil, err
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	208 }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	209
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	210 // store the associated countries.
2231 97bba8b51b9c Stretch import: Be more verbose about range before inserting data. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	211
97bba8b51b9c Stretch import: Be more verbose about range before inserting data. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2187 diff changeset	212 feedback.Info("Countries associated with stretch: %s.", st.Countries)
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	213 for _, c := range st.Countries {
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	214 if _, err := insertCountryStmt.ExecContext(ctx, id, c); err != nil {
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	215 return nil, err
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	216 }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	217 }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	218
1920 3dca0ee97da6 Stretches import: Track new ids. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1916 diff changeset	219 if err := track(ctx, tx, importID, "waterway.stretches", id); err != nil {
3dca0ee97da6 Stretches import: Track new ids. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1916 diff changeset	220 return nil, err
3dca0ee97da6 Stretches import: Track new ids. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1916 diff changeset	221 }
3dca0ee97da6 Stretches import: Track new ids. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1916 diff changeset	222
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	223 feedback.Info("Storing stretch '%s' took %s", st.Name, time.Since(start))
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	224 if err := tx.Commit(); err != nil {
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	225 return nil, err
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	226 }
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	227 feedback.Info("Import of stretch was successful")
32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	228
3521 9538a0aeacde stretch_import: provide full information for summary Thomas Junk <thomas.junk@intevation.de> parents: 3224 diff changeset	229 summary := st // provide full information for summary
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	230
3521 9538a0aeacde stretch_import: provide full information for summary Thomas Junk <thomas.junk@intevation.de> parents: 3224 diff changeset	231 return summary, nil
1906 32c56e6c089a Stretch import: Added forgotten source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	232 }

Mercurial > gemma

annotate pkg/imports/st.go @ 4180:91cb4a7b1b13