gemma: pkg/imports/dmv.go annotate

annotate pkg/imports/dmv.go @ 2187:7c83b5277c1c

Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue.

author	Sascha L. Teichmann <sascha.teichmann@intevation.de>
date	Mon, 11 Feb 2019 18:35:01 +0100
parents	b868cb653c4d
children	5f300bd7316f

rev	line source
1855 bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	1 // This is Free Software under GNU Affero General Public License v >= 3.0
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	2 // without warranty, see README.md and license for details.
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	3 //
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	4 // SPDX-License-Identifier: AGPL-3.0-or-later
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	5 // License-Filename: LICENSES/AGPL-3.0.txt
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	6 //
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	7 // Copyright (C) 2018 by via donau
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	8 // – Österreichische Wasserstraßen-Gesellschaft mbH
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	9 // Software engineering by Intevation GmbH
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	10 //
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	11 // Author(s):
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	13
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14 package imports
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16 import (
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	17 "context"
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	18 "database/sql"
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	19 "errors"
1859 9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	20 "fmt"
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	21 "log"
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	22 "strings"
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	23 "time"
1855 bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	24
1859 9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	25 "gemma.intevation.de/gemma/pkg/models"
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	26 "gemma.intevation.de/gemma/pkg/soap"
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	27 "gemma.intevation.de/gemma/pkg/soap/erdms"
1855 bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	28 )
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	29
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	30 type DistanceMarksVirtual struct {
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	31 // URL is the URL of the SOAP service.
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	32 URL string `json:"url"`
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	33 // Username is the username used to authenticate.
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	34 Username string `json:"username"`
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	35 // Passwort is the password to authenticate.
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	36 Password string `json:"password"`
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	37 // Insecure indicates if HTTPS traffic
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	38 // should validate certificates or not.
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	39 Insecure bool `json:"insecure"`
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	40 }
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	41
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	42 const DMVJobKind JobKind = "dmv"
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	43
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	44 type dmvJobCreator struct{}
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	45
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	46 func init() {
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	47 RegisterJobCreator(DMVJobKind, dmvJobCreator{})
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	48 }
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	49
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	50 func (dmvJobCreator) Description() string { return "distance marks virtual" }
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	51
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	52 func (dmvJobCreator) AutoAccept() bool { return true }
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	53
2187 7c83b5277c1c Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2168 diff changeset	54 func (dmvJobCreator) Create() Job { return new(DistanceMarksVirtual) }
1855 bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	55
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	56 func (dmvJobCreator) Depends() []string {
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	57 return []string{
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	58 "distance_marks_virtual",
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	59 }
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	60 }
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	61
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	62 // StageDone does nothing as there is no staging for distance marks virtual.
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	63 func (dmvJobCreator) StageDone(context.Context, *sql.Tx, int64) error { return nil }
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	64
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	65 // CleanUp does nothing as there is nothing to cleanup with distance marks virtual.
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	66 func (*DistanceMarksVirtual) CleanUp() error { return nil }
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	67
1859 9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	68 const (
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	69 deleteDistanceMarksVirtualSQL = `
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	70 DELETE FROM waterway.distance_marks_virtual
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	71 WHERE (location_code).country_code = $1
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	72 `
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	73 insertDistanceMarksVirtualSQL = `
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	74 INSERT INTO waterway.distance_marks_virtual (
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	75 location_code,
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	76 geom,
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	77 related_enc
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	78 )
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	79 VALUES (
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	80 ($1::char(2), $2::char(3), $3::char(5), $4::char(5), $5::int),
1918 62b5f6e48e82 Distance marks virtual/Waterway gauge imports: Better fix for swapped Lat/Lon fields. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1917 diff changeset	81 ST_SetSRID(ST_MakePoint($6, $7), 4326)::geography,
1859 9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	82 $8
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	83 ) ON CONFLICT (location_code) DO UPDATE SET
1918 62b5f6e48e82 Distance marks virtual/Waterway gauge imports: Better fix for swapped Lat/Lon fields. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1917 diff changeset	84 geom = ST_SetSRID(ST_MakePoint($6, $7), 4326)::geography,
1859 9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	85 related_enc = $8
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	86 `
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	87 )
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	88
1855 bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	89 func (dmv *DistanceMarksVirtual) Do(
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	90 ctx context.Context,
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	91 importID int64,
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	92 conn *sql.Conn,
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	93 feedback Feedback,
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	94 ) (interface{}, error) {
bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	95
1859 9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	96 start := time.Now()
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	97
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	98 tx, err := conn.BeginTx(ctx, nil)
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	99 if err != nil {
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	100 return nil, err
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	101 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	102 defer tx.Rollback()
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	103
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	104 var country string
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	105 err = tx.QueryRowContext(ctx, selectCurrentUserCountrySQL).Scan(&country)
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	106 switch {
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	107 case err == sql.ErrNoRows:
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	108 return nil, errors.New("Cannot figure out user country")
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	109 case err != nil:
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	110 return nil, err
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	111 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	112
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	113 country = strings.ToUpper(country)
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	114 feedback.Info("Using country '%s'.", country)
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	115
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	116 var auth *soap.BasicAuth
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	117 if dmv.Username != "" {
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	118 auth = &soap.BasicAuth{
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	119 Login: dmv.Username,
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	120 Password: dmv.Password,
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	121 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	122 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	123
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	124 client := erdms.NewRefService(dmv.URL, dmv.Insecure, auth)
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	125
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	126 request := &erdms.GetRisDataXML{
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	127 GetRisDataXMLType: &erdms.GetRisDataXMLType{
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	128 Subcode: erdms.NoNS{Text: country + "%"},
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	129 Funcode: erdms.NoNS{Text: "DISMAR"},
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	130 },
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	131 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	132
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	133 data, err := client.GetRisDataXML(request)
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	134
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	135 if err != nil {
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	136 log.Printf("error: %v\n", err)
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	137 return nil, fmt.Errorf("Error requesting ERDMS service: %v", err)
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	138 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	139
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	140 if _, err := tx.ExecContext(ctx, deleteDistanceMarksVirtualSQL, country); err != nil {
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	141 return nil, err
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	142 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	143
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	144 insertStmt, err := tx.PrepareContext(ctx, insertDistanceMarksVirtualSQL)
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	145 if err != nil {
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	146 return nil, err
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	147 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	148 defer insertStmt.Close()
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	149
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	150 var ignored, features int
1855 bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	151
1859 9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	152 for _, dr := range data.RisdataReturn {
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	153 if dr.RisidxCode == nil {
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	154 ignored++
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	155 continue
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	156 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	157
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	158 code, err := models.IsrsFromString(string(*dr.RisidxCode))
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	159 if err != nil {
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	160 feedback.Warn("invalid ISRS code %v", err)
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	161 ignored++
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	162 continue
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	163 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	164
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	165 if dr.Lat == nil \|\| dr.Lon == nil {
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	166 feedback.Warn("missing lat/lon: %s", code)
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	167 ignored++
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	168 continue
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	169 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	170
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	171 if dr.Relenc == nil {
1994 a7c4005b723f Improve some import log messages Tom Gottfried <tom@intevation.de> parents: 1918 diff changeset	172 feedback.Warn("missing relenc: %s", code)
1859 9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	173 ignored++
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	174 continue
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	175 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	176
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	177 if _, err := insertStmt.ExecContext(
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	178 ctx,
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	179 code.CountryCode,
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	180 code.LoCode,
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	181 code.FairwaySection,
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	182 code.Orc,
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	183 code.Hectometre,
1918 62b5f6e48e82 Distance marks virtual/Waterway gauge imports: Better fix for swapped Lat/Lon fields. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1917 diff changeset	184 float64(dr.Lon), float64(dr.Lat),
1859 9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	185 string(*dr.Relenc),
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	186 ); err != nil {
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	187 return nil, err
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	188 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	189 features++
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	190 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	191 feedback.Info("ignored: %d", ignored)
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	192 feedback.Info("features: %d", features)
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	193
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	194 if features == 0 {
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	195 return nil, errors.New("No features found")
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	196 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	197
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	198 if err = tx.Commit(); err == nil {
1994 a7c4005b723f Improve some import log messages Tom Gottfried <tom@intevation.de> parents: 1918 diff changeset	199 feedback.Info("Refreshing distance marks (virtual) successfully took %s.",
1859 9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	200 time.Since(start))
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	201 } else {
1994 a7c4005b723f Improve some import log messages Tom Gottfried <tom@intevation.de> parents: 1918 diff changeset	202 feedback.Error("Refreshing distance marks (virtual) failed after %s.",
1859 9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	203 time.Since(start))
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	204 }
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	205
9780cb4ef6a6 Distance marks virtual import: Implemented. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1855 diff changeset	206 return nil, nil
1855 bbd653a43a6a Distance marks virtual import: Added stub. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	207 }

Mercurial > gemma

annotate pkg/imports/dmv.go @ 2187:7c83b5277c1c