gemma: pkg/imports/isr.go annotate

annotate pkg/imports/isr.go @ 5560:f2204f91d286

Join the log lines of imports to the log exports to recover data from them. Used in SR export to extract information that where in the meta json but now are only found in the log.

author	Sascha L. Teichmann <sascha.teichmann@intevation.de>
date	Wed, 09 Feb 2022 18:34:40 +0100
parents	5f47eeea988d
children	2dd155cc95ec

rev	line source
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	1 // This is Free Software under GNU Affero General Public License v >= 3.0
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	2 // without warranty, see README.md and license for details.
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	3 //
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	4 // SPDX-License-Identifier: AGPL-3.0-or-later
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	5 // License-Filename: LICENSES/AGPL-3.0.txt
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	6 //
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	7 // Copyright (C) 2018 by via donau
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	8 // – Österreichische Wasserstraßen-Gesellschaft mbH
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	9 // Software engineering by Intevation GmbH
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	10 //
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	11 // Author(s):
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	13
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14 package imports
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16 import (
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	17 "context"
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	18 "database/sql"
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	19 "time"
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	20
5490 5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5436 diff changeset	21 "gemma.intevation.de/gemma/pkg/log"
4827 f4abfd0ee8ad Renamed octree package to mesh as there is no octree any more. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4768 diff changeset	22 "gemma.intevation.de/gemma/pkg/mesh"
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	23 "gemma.intevation.de/gemma/pkg/models"
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	24 )
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	25
4852 046a07a33b19 Fixed the golint issues of the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4827 diff changeset	26 // IsoRefresh is an import job to refresh the pre-calculated
046a07a33b19 Fixed the golint issues of the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4827 diff changeset	27 // iso morphology data when the class breaks were changed.
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	28 type IsoRefresh struct {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	29 ClassBreaks string `json:"class-breaks"`
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	30 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	31
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	32 // ISRJobKind is the unique name of this import job type.
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	33 const ISRJobKind JobKind = "isr"
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	34
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	35 func init() { RegisterJobCreator(ISRJobKind, isrJobCreator{}) }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	36
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	37 type isrJobCreator struct{}
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	38
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	39 func (isrJobCreator) Description() string { return "refresh iso lines" }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	40
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	41 func (isrJobCreator) AutoAccept() bool { return true }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	42
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	43 func (isrJobCreator) Create() Job { return new(IsoRefresh) }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	44
5035 56c589f7435d Enhance comments on implementations of StageDone() Tom Gottfried <tom@intevation.de> parents: 5034 diff changeset	45 // StageDone is a NOP for IsoRefresh imports.
5034 59a99655f34d Added feedback support for StageDone. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4852 diff changeset	46 func (isrJobCreator) StageDone(context.Context, *sql.Tx, int64, Feedback) error {
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	47 return nil
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	48 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	49
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	50 func (isrJobCreator) Depends() [2][]string {
4571 a413b4a89bcc Update areas if config has changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4257 diff changeset	51 return srJobCreator{}.Depends()
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	52 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	53
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	54 const (
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	55 fetchSoundingResultsIDsSQL = `
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	56 SELECT
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	57 bottleneck_id,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	58 id,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	59 surtyp
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	60 FROM waterway.sounding_results
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	61 ORDER BY bottleneck_id`
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	62
4571 a413b4a89bcc Update areas if config has changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4257 diff changeset	63 deleteIsoAreasSQL = `
a413b4a89bcc Update areas if config has changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4257 diff changeset	64 DELETE FROM waterway.sounding_results_iso_areas
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	65 WHERE sounding_result_id = $1`
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	66
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	67 fetchMarkingPointsSQL = `
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	68 SELECT ST_AsBinary(points::geometry(MULTIPOINTZ))
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	69 FROM waterway.sounding_results_marking_points
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	70 WHERE sounding_result_id = $1`
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	71
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	72 deleteMarkingPointsSQL = `
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	73 DELETE FROM waterway.sounding_results_marking_points
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	74 WHERE sounding_result_id = $1`
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	75 )
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	76
4852 046a07a33b19 Fixed the golint issues of the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4827 diff changeset	77 // CleanUp of a iso refresh import is a NOP.
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	78 func (isr *IsoRefresh) CleanUp() error { return nil }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	79
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	80 type (
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	81 scanResult struct {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	82 id int64
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	83 surveyType models.SurveyType
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	84 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	85 bottleneckSoundingResults struct {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	86 bn string
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	87 srs []scanResult
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	88 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	89 )
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	90
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	91 func fetchBottleneckResults(
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	92 ctx context.Context,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	93 conn *sql.Conn,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	94 ) ([]bottleneckSoundingResults, error) {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	95
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	96 rows, err := conn.QueryContext(ctx, fetchSoundingResultsIDsSQL)
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	97 if err != nil {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	98 return nil, err
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	99 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	100 defer rows.Close()
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	101
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	102 var bsrs []bottleneckSoundingResults
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	103
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	104 for rows.Next() {
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	105 var (
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	106 bn string
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	107 id int64
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	108 st string
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	109 )
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	110 if err := rows.Scan(&bn, &id, &st); err != nil {
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	111 return nil, err
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	112 }
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	113 sr := scanResult{id: id, surveyType: models.SurveyType(st)}
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	114
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	115 if l := len(bsrs); l == 0 \|\| bsrs[l-1].bn != bn {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	116 bsrs = append(bsrs, bottleneckSoundingResults{
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	117 bn: bn,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	118 srs: []scanResult{sr},
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	119 })
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	120 } else {
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	121 bsrs[l-1].srs = append(bsrs[l-1].srs, sr)
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	122 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	123 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	124 if err := rows.Err(); err != nil {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	125 return nil, err
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	126 }
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	127 return bsrs, nil
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	128 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	129
4767 f66e5b2fa894 Wording: Speak of iso areas instead of iso lines when refreshing the sounding results after a configuration change. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4669 diff changeset	130 // Do executes the actual refreshing of the iso areas.
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	131 func (isr *IsoRefresh) Do(
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	132 ctx context.Context,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	133 importID int64,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	134 conn *sql.Conn,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	135 feedback Feedback,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	136 ) (interface{}, error) {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	137
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	138 start := time.Now()
4767 f66e5b2fa894 Wording: Speak of iso areas instead of iso lines when refreshing the sounding results after a configuration change. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4669 diff changeset	139 feedback.Info("Regenerating iso areas for sounding results " +
4255 d69a3b19a1ae Indicate to the user what triggered the import Tom Gottfried <tom@intevation.de> parents: 4220 diff changeset	140 "after configuration change")
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	141 defer func() {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	142 feedback.Info(
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	143 "Processing all sounding results took %v.",
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	144 time.Since(start))
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	145 }()
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	146
4827 f4abfd0ee8ad Renamed octree package to mesh as there is no octree any more. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4768 diff changeset	147 heights, err := mesh.ParseClassBreaks(isr.ClassBreaks)
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	148 if err != nil {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	149 return nil, err
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	150 }
5428 b8d5f1cd15fb Simplified classification. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5421 diff changeset	151 heights = heights.Dedup()
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	152
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	153 bns, err := fetchBottleneckResults(ctx, conn)
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	154 if err != nil {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	155 return nil, err
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	156 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	157
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	158 isrs, err := newISRStmts(ctx, conn)
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	159 if err != nil {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	160 return nil, err
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	161 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	162 defer isrs.close()
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	163
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	164 for i := range bns {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	165 start := time.Now()
4216 cbc75527916f Refresh iso contour lines import: Improved logging a bit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4215 diff changeset	166 feedback.Info("Processing bottleneck '%s' ...", bns[i].bn)
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	167 err := isr.processBottleneck(
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	168 ctx, conn, feedback,
a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	169 isrs,
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	170 heights,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	171 &bns[i],
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	172 )
4216 cbc75527916f Refresh iso contour lines import: Improved logging a bit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4215 diff changeset	173 feedback.Info("Processing took %v.", time.Since(start))
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	174 if err != nil {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	175 return nil, err
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	176 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	177 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	178
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	179 return nil, nil
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	180 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	181
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	182 type isrStmts struct {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	183 insertAreas *sql.Stmt
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	184 deleteAreas *sql.Stmt
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	185 fetchMarkingPoints *sql.Stmt
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	186 deleteMarkingPoints *sql.Stmt
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	187 insertMarkingPoints *sql.Stmt
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	188 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	189
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	190 func newISRStmts(ctx context.Context, conn sql.Conn) (isrStmts, error) {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	191 var isrs isrStmts
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	192 for _, x := range []struct {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	193 stmt **sql.Stmt
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	194 query string
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	195 }{
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	196 {&isrs.insertAreas, insertIsoAreasSQL},
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	197 {&isrs.deleteAreas, deleteIsoAreasSQL},
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	198 {&isrs.fetchMarkingPoints, fetchMarkingPointsSQL},
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	199 {&isrs.deleteMarkingPoints, deleteMarkingPointsSQL},
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	200 {&isrs.insertMarkingPoints, insertMarkingPointsSQL},
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	201 } {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	202 var err error
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	203 if *x.stmt, err = conn.PrepareContext(ctx, x.query); err != nil {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	204 isrs.close()
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	205 return nil, err
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	206 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	207 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	208 return &isrs, nil
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	209 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	210
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	211 func (isrs *isrStmts) close() {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	212 for _, x := range []**sql.Stmt{
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	213 &isrs.insertAreas,
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	214 &isrs.deleteAreas,
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	215 &isrs.fetchMarkingPoints,
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	216 &isrs.deleteMarkingPoints,
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	217 &isrs.insertMarkingPoints,
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	218 } {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	219 if *x != nil {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	220 (*x).Close()
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	221 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	222 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	223 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	224
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	225 func (isr *IsoRefresh) processBottleneck(
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	226 ctx context.Context,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	227 conn *sql.Conn,
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	228 feedback Feedback,
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	229 isrs *isrStmts,
5412 34bc6041e61e Added a type for class breaks. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5035 diff changeset	230 heights mesh.ClassBreaks,
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	231 bn *bottleneckSoundingResults,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	232 ) error {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	233 // Do one transaction per bottleneck.
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	234 tx, err := conn.BeginTx(ctx, nil)
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	235 if err != nil {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	236 return nil
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	237 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	238 defer tx.Rollback()
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	239
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	240 var (
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	241 insertAreas = tx.StmtContext(ctx, isrs.insertAreas)
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	242 deleteAreas = tx.StmtContext(ctx, isrs.deleteAreas)
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	243 fetchMarkingPoints = tx.StmtContext(ctx, isrs.fetchMarkingPoints)
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	244 deleteMarkingPoints = tx.StmtContext(ctx, isrs.deleteMarkingPoints)
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	245 insertMarkingPoints = tx.StmtContext(ctx, isrs.insertMarkingPoints)
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	246 )
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	247
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	248 var markings, beams int
a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	249
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	250 // For all sounding results in bottleneck.
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	251 for _, sr := range bn.srs {
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	252 switch sr.surveyType {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	253 case models.SurveyTypeMarking:
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	254 markings++
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	255
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	256 // Read all points back in.
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	257
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	258 var points mesh.MultiPointZ
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	259
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	260 if err := func() error {
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	261 rows, err := fetchMarkingPoints.QueryContext(ctx, sr.id)
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	262 if err != nil {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	263 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	264 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	265 defer rows.Close()
4571 a413b4a89bcc Update areas if config has changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4257 diff changeset	266
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	267 for rows.Next() {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	268 var buf []byte
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	269 if err := rows.Scan(&buf); err != nil {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	270 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	271 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	272 var npoints mesh.MultiPointZ
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	273 if err := npoints.FromWKB(buf); err != nil {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	274 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	275 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	276 points = append(points, npoints...)
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	277 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	278 return rows.Err()
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	279 }(); err != nil {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	280 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	281 }
4768 a2f16bbcc846 Morph differences: Directly raster A and subtract B as a raster. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4767 diff changeset	282
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	283 // Delete old points
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	284 if _, err := deleteMarkingPoints.ExecContext(ctx, sr.id); err != nil {
4571 a413b4a89bcc Update areas if config has changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4257 diff changeset	285 return err
a413b4a89bcc Update areas if config has changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4257 diff changeset	286 }
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	287
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	288 // Re-classify points.
5428 b8d5f1cd15fb Simplified classification. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5421 diff changeset	289 classes := heights.Classify(points)
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	290
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	291 // Re-insert points
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	292 for i, class := range classes {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	293 // Ignore empty classes
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	294 if len(class) == 0 {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	295 continue
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	296 }
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	297 if _, err := insertMarkingPoints.ExecContext(
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	298 ctx, sr.id, heights[i], 4326, class.AsWKB(),
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	299 ); err != nil {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	300 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	301 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	302 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	303
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	304 case models.SurveyTypeMultiBeam, models.SurveyTypeSingleBeam:
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	305 beams++
a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	306
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	307 tree, err := mesh.FetchMeshDirectly(ctx, tx, sr.id)
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	308 if err != nil {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	309 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	310 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	311 hs := heights.ExtrapolateClassBreaks(tree.Min().Z, tree.Max().Z).Dedup()
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	312
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	313 // Delete the old iso areas.
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	314 if _, err := deleteAreas.ExecContext(ctx, sr.id); err != nil {
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	315 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	316 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	317
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	318 // Calculate and store the iso areas.
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	319 box := mesh.Box2D{
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	320 X1: tree.Min().X,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	321 Y1: tree.Min().Y,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	322 X2: tree.Max().X,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	323 Y2: tree.Max().Y,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	324 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	325
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	326 raster := mesh.NewRaster(box, isoCellSize)
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	327 raster.Rasterize(tree.Value)
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	328 areas := raster.Trace(hs)
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	329
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	330 for i, a := range areas {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	331 if len(a) == 0 {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	332 continue
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	333 }
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	334 if _, err := insertAreas.ExecContext(
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	335 ctx,
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	336 sr.id, hs[i], tree.EPSG(),
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	337 a.AsWKB(),
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	338 contourTolerance,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	339 ); err != nil {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	340 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	341 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	342 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	343 default:
5490 5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5436 diff changeset	344 log.Errorf("unknown survey type '%s'\n", sr.surveyType)
4571 a413b4a89bcc Update areas if config has changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4257 diff changeset	345 }
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	346 }
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	347 feedback.Info("Scan types: Single/Multi: %d Marking: %d", beams, markings)
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	348
4220 d8b736ba3f32 Refresh iso contour lines import: Commit the changes! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4216 diff changeset	349 return tx.Commit()
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	350 }

Mercurial > gemma

annotate pkg/imports/isr.go @ 5560:f2204f91d286