gemma: pkg/imports/isr.go annotate

annotate pkg/imports/isr.go @ 5670:b75d0b303328

Various fixes and improvements of gauges import: - Allow update of erased data (and thereby set erased to false) - Fix source_organization to work with ERDMS2 - Give ISRS of new and updated gauges in summary - Fixed reference of null pointers if revlevels are missing - Fixed reference of null pointer on update errors - Added ISRS to reference_code warning

author	Sascha Wilde <wilde@sha-bang.de>
date	Fri, 08 Dec 2023 17:29:56 +0100
parents	5f47eeea988d
children	2dd155cc95ec

rev	line source
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	1 // This is Free Software under GNU Affero General Public License v >= 3.0
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	2 // without warranty, see README.md and license for details.
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	3 //
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	4 // SPDX-License-Identifier: AGPL-3.0-or-later
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	5 // License-Filename: LICENSES/AGPL-3.0.txt
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	6 //
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	7 // Copyright (C) 2018 by via donau
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	8 // – Österreichische Wasserstraßen-Gesellschaft mbH
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	9 // Software engineering by Intevation GmbH
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	10 //
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	11 // Author(s):
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	13
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14 package imports
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16 import (
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	17 "context"
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	18 "database/sql"
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	19 "time"
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	20
5490 5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5436 diff changeset	21 "gemma.intevation.de/gemma/pkg/log"
4827 f4abfd0ee8ad Renamed octree package to mesh as there is no octree any more. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4768 diff changeset	22 "gemma.intevation.de/gemma/pkg/mesh"
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	23 "gemma.intevation.de/gemma/pkg/models"
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	24 )
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	25
4852 046a07a33b19 Fixed the golint issues of the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4827 diff changeset	26 // IsoRefresh is an import job to refresh the pre-calculated
046a07a33b19 Fixed the golint issues of the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4827 diff changeset	27 // iso morphology data when the class breaks were changed.
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	28 type IsoRefresh struct {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	29 ClassBreaks string `json:"class-breaks"`
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	30 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	31
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	32 // ISRJobKind is the unique name of this import job type.
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	33 const ISRJobKind JobKind = "isr"
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	34
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	35 func init() { RegisterJobCreator(ISRJobKind, isrJobCreator{}) }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	36
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	37 type isrJobCreator struct{}
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	38
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	39 func (isrJobCreator) Description() string { return "refresh iso lines" }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	40
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	41 func (isrJobCreator) AutoAccept() bool { return true }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	42
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	43 func (isrJobCreator) Create() Job { return new(IsoRefresh) }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	44
5035 56c589f7435d Enhance comments on implementations of StageDone() Tom Gottfried <tom@intevation.de> parents: 5034 diff changeset	45 // StageDone is a NOP for IsoRefresh imports.
5034 59a99655f34d Added feedback support for StageDone. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4852 diff changeset	46 func (isrJobCreator) StageDone(context.Context, *sql.Tx, int64, Feedback) error {
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	47 return nil
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	48 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	49
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	50 func (isrJobCreator) Depends() [2][]string {
4571 a413b4a89bcc Update areas if config has changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4257 diff changeset	51 return srJobCreator{}.Depends()
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	52 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	53
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	54 const (
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	55 fetchSoundingResultsIDsSQL = `
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	56 SELECT
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	57 bottleneck_id,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	58 id,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	59 surtyp
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	60 FROM waterway.sounding_results
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	61 ORDER BY bottleneck_id`
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	62
4571 a413b4a89bcc Update areas if config has changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4257 diff changeset	63 deleteIsoAreasSQL = `
a413b4a89bcc Update areas if config has changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4257 diff changeset	64 DELETE FROM waterway.sounding_results_iso_areas
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	65 WHERE sounding_result_id = $1`
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	66
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	67 fetchMarkingPointsSQL = `
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	68 SELECT ST_AsBinary(points::geometry(MULTIPOINTZ))
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	69 FROM waterway.sounding_results_marking_points
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	70 WHERE sounding_result_id = $1`
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	71
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	72 deleteMarkingPointsSQL = `
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	73 DELETE FROM waterway.sounding_results_marking_points
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	74 WHERE sounding_result_id = $1`
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	75 )
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	76
4852 046a07a33b19 Fixed the golint issues of the imports package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4827 diff changeset	77 // CleanUp of a iso refresh import is a NOP.
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	78 func (isr *IsoRefresh) CleanUp() error { return nil }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	79
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	80 type (
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	81 scanResult struct {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	82 id int64
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	83 surveyType models.SurveyType
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	84 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	85 bottleneckSoundingResults struct {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	86 bn string
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	87 srs []scanResult
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	88 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	89 )
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	90
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	91 func fetchBottleneckResults(
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	92 ctx context.Context,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	93 conn *sql.Conn,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	94 ) ([]bottleneckSoundingResults, error) {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	95
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	96 rows, err := conn.QueryContext(ctx, fetchSoundingResultsIDsSQL)
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	97 if err != nil {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	98 return nil, err
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	99 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	100 defer rows.Close()
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	101
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	102 var bsrs []bottleneckSoundingResults
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	103
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	104 for rows.Next() {
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	105 var (
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	106 bn string
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	107 id int64
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	108 st string
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	109 )
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	110 if err := rows.Scan(&bn, &id, &st); err != nil {
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	111 return nil, err
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	112 }
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	113 sr := scanResult{id: id, surveyType: models.SurveyType(st)}
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	114
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	115 if l := len(bsrs); l == 0 \|\| bsrs[l-1].bn != bn {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	116 bsrs = append(bsrs, bottleneckSoundingResults{
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	117 bn: bn,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	118 srs: []scanResult{sr},
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	119 })
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	120 } else {
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	121 bsrs[l-1].srs = append(bsrs[l-1].srs, sr)
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	122 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	123 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	124 if err := rows.Err(); err != nil {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	125 return nil, err
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	126 }
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	127 return bsrs, nil
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	128 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	129
4767 f66e5b2fa894 Wording: Speak of iso areas instead of iso lines when refreshing the sounding results after a configuration change. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4669 diff changeset	130 // Do executes the actual refreshing of the iso areas.
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	131 func (isr *IsoRefresh) Do(
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	132 ctx context.Context,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	133 importID int64,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	134 conn *sql.Conn,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	135 feedback Feedback,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	136 ) (interface{}, error) {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	137
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	138 start := time.Now()
4767 f66e5b2fa894 Wording: Speak of iso areas instead of iso lines when refreshing the sounding results after a configuration change. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4669 diff changeset	139 feedback.Info("Regenerating iso areas for sounding results " +
4255 d69a3b19a1ae Indicate to the user what triggered the import Tom Gottfried <tom@intevation.de> parents: 4220 diff changeset	140 "after configuration change")
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	141 defer func() {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	142 feedback.Info(
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	143 "Processing all sounding results took %v.",
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	144 time.Since(start))
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	145 }()
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	146
4827 f4abfd0ee8ad Renamed octree package to mesh as there is no octree any more. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4768 diff changeset	147 heights, err := mesh.ParseClassBreaks(isr.ClassBreaks)
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	148 if err != nil {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	149 return nil, err
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	150 }
5428 b8d5f1cd15fb Simplified classification. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5421 diff changeset	151 heights = heights.Dedup()
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	152
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	153 bns, err := fetchBottleneckResults(ctx, conn)
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	154 if err != nil {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	155 return nil, err
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	156 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	157
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	158 isrs, err := newISRStmts(ctx, conn)
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	159 if err != nil {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	160 return nil, err
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	161 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	162 defer isrs.close()
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	163
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	164 for i := range bns {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	165 start := time.Now()
4216 cbc75527916f Refresh iso contour lines import: Improved logging a bit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4215 diff changeset	166 feedback.Info("Processing bottleneck '%s' ...", bns[i].bn)
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	167 err := isr.processBottleneck(
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	168 ctx, conn, feedback,
a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	169 isrs,
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	170 heights,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	171 &bns[i],
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	172 )
4216 cbc75527916f Refresh iso contour lines import: Improved logging a bit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4215 diff changeset	173 feedback.Info("Processing took %v.", time.Since(start))
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	174 if err != nil {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	175 return nil, err
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	176 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	177 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	178
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	179 return nil, nil
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	180 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	181
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	182 type isrStmts struct {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	183 insertAreas *sql.Stmt
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	184 deleteAreas *sql.Stmt
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	185 fetchMarkingPoints *sql.Stmt
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	186 deleteMarkingPoints *sql.Stmt
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	187 insertMarkingPoints *sql.Stmt
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	188 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	189
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	190 func newISRStmts(ctx context.Context, conn sql.Conn) (isrStmts, error) {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	191 var isrs isrStmts
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	192 for _, x := range []struct {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	193 stmt **sql.Stmt
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	194 query string
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	195 }{
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	196 {&isrs.insertAreas, insertIsoAreasSQL},
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	197 {&isrs.deleteAreas, deleteIsoAreasSQL},
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	198 {&isrs.fetchMarkingPoints, fetchMarkingPointsSQL},
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	199 {&isrs.deleteMarkingPoints, deleteMarkingPointsSQL},
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	200 {&isrs.insertMarkingPoints, insertMarkingPointsSQL},
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	201 } {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	202 var err error
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	203 if *x.stmt, err = conn.PrepareContext(ctx, x.query); err != nil {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	204 isrs.close()
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	205 return nil, err
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	206 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	207 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	208 return &isrs, nil
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	209 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	210
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	211 func (isrs *isrStmts) close() {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	212 for _, x := range []**sql.Stmt{
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	213 &isrs.insertAreas,
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	214 &isrs.deleteAreas,
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	215 &isrs.fetchMarkingPoints,
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	216 &isrs.deleteMarkingPoints,
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	217 &isrs.insertMarkingPoints,
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	218 } {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	219 if *x != nil {
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	220 (*x).Close()
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	221 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	222 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	223 }
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	224
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	225 func (isr *IsoRefresh) processBottleneck(
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	226 ctx context.Context,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	227 conn *sql.Conn,
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	228 feedback Feedback,
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	229 isrs *isrStmts,
5412 34bc6041e61e Added a type for class breaks. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5035 diff changeset	230 heights mesh.ClassBreaks,
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	231 bn *bottleneckSoundingResults,
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	232 ) error {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	233 // Do one transaction per bottleneck.
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	234 tx, err := conn.BeginTx(ctx, nil)
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	235 if err != nil {
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	236 return nil
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	237 }
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	238 defer tx.Rollback()
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	239
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	240 var (
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	241 insertAreas = tx.StmtContext(ctx, isrs.insertAreas)
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	242 deleteAreas = tx.StmtContext(ctx, isrs.deleteAreas)
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	243 fetchMarkingPoints = tx.StmtContext(ctx, isrs.fetchMarkingPoints)
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	244 deleteMarkingPoints = tx.StmtContext(ctx, isrs.deleteMarkingPoints)
fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	245 insertMarkingPoints = tx.StmtContext(ctx, isrs.insertMarkingPoints)
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	246 )
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	247
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	248 var markings, beams int
a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	249
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	250 // For all sounding results in bottleneck.
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	251 for _, sr := range bn.srs {
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	252 switch sr.surveyType {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	253 case models.SurveyTypeMarking:
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	254 markings++
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	255
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	256 // Read all points back in.
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	257
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	258 var points mesh.MultiPointZ
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	259
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	260 if err := func() error {
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	261 rows, err := fetchMarkingPoints.QueryContext(ctx, sr.id)
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	262 if err != nil {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	263 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	264 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	265 defer rows.Close()
4571 a413b4a89bcc Update areas if config has changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4257 diff changeset	266
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	267 for rows.Next() {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	268 var buf []byte
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	269 if err := rows.Scan(&buf); err != nil {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	270 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	271 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	272 var npoints mesh.MultiPointZ
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	273 if err := npoints.FromWKB(buf); err != nil {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	274 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	275 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	276 points = append(points, npoints...)
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	277 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	278 return rows.Err()
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	279 }(); err != nil {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	280 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	281 }
4768 a2f16bbcc846 Morph differences: Directly raster A and subtract B as a raster. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4767 diff changeset	282
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	283 // Delete old points
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	284 if _, err := deleteMarkingPoints.ExecContext(ctx, sr.id); err != nil {
4571 a413b4a89bcc Update areas if config has changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4257 diff changeset	285 return err
a413b4a89bcc Update areas if config has changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4257 diff changeset	286 }
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	287
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	288 // Re-classify points.
5428 b8d5f1cd15fb Simplified classification. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5421 diff changeset	289 classes := heights.Classify(points)
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	290
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	291 // Re-insert points
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	292 for i, class := range classes {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	293 // Ignore empty classes
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	294 if len(class) == 0 {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	295 continue
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	296 }
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	297 if _, err := insertMarkingPoints.ExecContext(
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	298 ctx, sr.id, heights[i], 4326, class.AsWKB(),
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	299 ); err != nil {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	300 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	301 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	302 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	303
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	304 case models.SurveyTypeMultiBeam, models.SurveyTypeSingleBeam:
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	305 beams++
a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	306
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	307 tree, err := mesh.FetchMeshDirectly(ctx, tx, sr.id)
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	308 if err != nil {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	309 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	310 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	311 hs := heights.ExtrapolateClassBreaks(tree.Min().Z, tree.Max().Z).Dedup()
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	312
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	313 // Delete the old iso areas.
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	314 if _, err := deleteAreas.ExecContext(ctx, sr.id); err != nil {
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	315 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	316 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	317
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	318 // Calculate and store the iso areas.
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	319 box := mesh.Box2D{
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	320 X1: tree.Min().X,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	321 Y1: tree.Min().Y,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	322 X2: tree.Max().X,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	323 Y2: tree.Max().Y,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	324 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	325
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	326 raster := mesh.NewRaster(box, isoCellSize)
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	327 raster.Rasterize(tree.Value)
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	328 areas := raster.Trace(hs)
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	329
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	330 for i, a := range areas {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	331 if len(a) == 0 {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	332 continue
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	333 }
5431 fc79622783a7 ISR job: Moved statement preparations out out bottleneck processing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5428 diff changeset	334 if _, err := insertAreas.ExecContext(
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	335 ctx,
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	336 sr.id, hs[i], tree.EPSG(),
5421 c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	337 a.AsWKB(),
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	338 contourTolerance,
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	339 ); err != nil {
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	340 return err
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	341 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	342 }
c9da747d4109 Extended ISR import job to handle marking scans as well. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5412 diff changeset	343 default:
5490 5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5436 diff changeset	344 log.Errorf("unknown survey type '%s'\n", sr.surveyType)
4571 a413b4a89bcc Update areas if config has changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4257 diff changeset	345 }
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	346 }
5436 a65b0891e8fd Fixed SQL errors in ISR job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5431 diff changeset	347 feedback.Info("Scan types: Single/Multi: %d Marking: %d", beams, markings)
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	348
4220 d8b736ba3f32 Refresh iso contour lines import: Commit the changes! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4216 diff changeset	349 return tx.Commit()
4214 49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	350 }

Mercurial > gemma

annotate pkg/imports/isr.go @ 5670:b75d0b303328