Mercurial > gemma

annotate pkg/imports/wa.go @ 2549:9bf6b767a56a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
client: refactored and improved splitscreen for diagrams To make different diagrams possible, the splitscreen view needed to be decoupled from the cross profiles. Also the style has changed to make it more consistent with the rest of the app. The standard box header is now used and there are collapse and expand animations.
author Markus Kottlaender <markus@intevation.de>
date Fri, 08 Mar 2019 08:50:47 +0100
parents 7c83b5277c1c
children 02505fcff63c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1785
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
3 //
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
6 //
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
10 //
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
11 // Author(s):
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
13
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
14 package imports
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
15
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
16 import (
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
17 "context"
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
18 "database/sql"
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
19 "encoding/json"
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
20 "errors"
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
21 "fmt"
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
22 "io"
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
23 "strconv"
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
24 "time"
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
25
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
26 "gemma.intevation.de/gemma/pkg/wfs"
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
27 )
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
28
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
29 // WaterwayArea is an import job to import
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
30 // the waterway area in form of polygon geometries
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
31 // and attribute data from a WFS service.
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
32 type WaterwayArea struct {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
33 // URL the GetCapabilities URL of the WFS service.
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
34 URL string `json:"url"`
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
35 // FeatureType selects the feature type of the WFS service.
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
36 FeatureType string `json:"feature-type"`
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
37 // SortBy works around misconfigured services to
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
38 // establish a sort order to get the features.
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
39 SortBy string `json:"sort-by"`
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
40 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
41
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
42 // WAJobKind is the import queue type identifier.
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
43 const WAJobKind JobKind = "wa"
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
44
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
45 type waJobCreator struct{}
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
46
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
47 func init() {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
48 RegisterJobCreator(WAJobKind, waJobCreator{})
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
49 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
50
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
51 func (waJobCreator) Description() string { return "waterway area" }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
52
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
53 func (waJobCreator) AutoAccept() bool { return true }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
54
2187
7c83b5277c1c Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
55 func (waJobCreator) Create() Job { return new(WaterwayArea) }
1785
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
56
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
57 func (waJobCreator) Depends() []string {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
58 return []string{
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
59 "waterway_area",
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
60 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
61 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
62
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
63 // StageDone is a NOP for waterway area imports.
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
64 func (waJobCreator) StageDone(context.Context, *sql.Tx, int64) error {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
65 return nil
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
66 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
67
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
68 // CleanUp for waterway area imports is a NOP.
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
69 func (*WaterwayArea) CleanUp() error { return nil }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
70
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
71 type waterwayAreaProperties struct {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
72 Catccl *string `json:"ienc_catccl"`
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
73 Dirimp *string `json:"ienc_dirimp"`
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
74 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
75
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
76 const (
1796
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
77 deleteWaterwayAreaSQL = `
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
78 WITH resp AS (
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
79 SELECT best_utm(area::geometry) AS t,
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
80 ST_Transform(area::geometry, best_utm(area::geometry)) AS a
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
81 FROM users.responsibility_areas
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
82 WHERE country = users.current_user_country()
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
83 )
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
84 DELETE FROM waterway.waterway_area
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
85 WHERE ST_Covers(
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
86 (SELECT a FROM resp),
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
87 ST_Transform(area::geometry, (SELECT t FROM resp)))
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
88 `
1785
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
89 insertWaterwayAreaSQL = `
1796
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
90 WITH resp AS (
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
91 SELECT best_utm(area::geometry) AS t,
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
92 ST_Transform(area::geometry, best_utm(area::geometry)) AS a
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
93 FROM users.responsibility_areas
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
94 WHERE country = users.current_user_country()
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
95 )
1785
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
96 INSERT INTO waterway.waterway_area (area, catccl, dirimp)
1796
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
97 SELECT ST_Transform(clipped.geom, 4326)::geography, $3, $4 FROM (
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
98 SELECT (ST_Dump(
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
99 ST_Intersection(
1899
272133cd65da Workarround for failing waterway_area imports.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1828
diff changeset
100 (SELECT ST_Buffer(a, 0.0001) FROM resp),
272133cd65da Workarround for failing waterway_area imports.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1828
diff changeset
101 ST_CollectionExtract(ST_MakeValid(ST_Transform(
1796
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
102 ST_GeomFromWKB($1, $2::integer),
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
103 (SELECT t FROM resp)
1899
272133cd65da Workarround for failing waterway_area imports.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 1828
diff changeset
104 )),3)
1796
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
105 )
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
106 )).geom AS geom
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
107 ) AS clipped
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
108 WHERE clipped.geom IS NOT NULL
904c7e853adc Waterway area import: Clip features against responsibility area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
109 `
1785
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
110 )
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
111
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
112 // Do executes the actual waterway axis import.
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
113 func (wx *WaterwayArea) Do(
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
114 ctx context.Context,
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
115 importID int64,
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
116 conn *sql.Conn,
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
117 feedback Feedback,
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
118 ) (interface{}, error) {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
119
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
120 start := time.Now()
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
121
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
122 feedback.Info("Import waterway area")
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
123
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
124 feedback.Info("Loading capabilities from %s", wx.URL)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
125 caps, err := wfs.GetCapabilities(wx.URL)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
126 if err != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
127 feedback.Error("Loading capabilities failed: %v", err)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
128 return nil, err
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
129 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
130
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
131 ft := caps.FindFeatureType(wx.FeatureType)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
132 if ft == nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
133 return nil, fmt.Errorf("Unknown feature type '%s'", wx.FeatureType)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
134 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
135
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
136 feedback.Info("Found feature type '%s", wx.FeatureType)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
137
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
138 epsg, err := wfs.CRSToEPSG(ft.DefaultCRS)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
139 if err != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
140 feedback.Error("Unsupported CRS name '%s'", ft.DefaultCRS)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
141 return nil, err
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
142 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
143
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
144 if wx.SortBy != "" {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
145 feedback.Info("Features will be sorted by '%s'", wx.SortBy)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
146 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
147
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
148 urls, err := wfs.GetFeaturesGET(
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
149 caps, wx.FeatureType, "application/json", wx.SortBy)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
150 if err != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
151 feedback.Error("Cannot create GetFeature URLs. %v", err)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
152 return nil, err
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
153 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
154
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
155 tx, err := conn.BeginTx(ctx, nil)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
156 if err != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
157 return nil, err
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
158 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
159 defer tx.Rollback()
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
160
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
161 insertStmt, err := tx.PrepareContext(ctx, insertWaterwayAreaSQL)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
162 if err != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
163 return nil, err
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
164 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
165 defer insertStmt.Close()
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
166
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
167 // Delete the old features.
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
168 if _, err := tx.ExecContext(ctx, deleteWaterwayAreaSQL); err != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
169 return nil, err
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
170 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
171
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
172 var (
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
173 unsupported = stringCounter{}
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
174 missingProperties int
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
175 badProperties int
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
176 features int
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
177 )
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
178
2106
2b72f5e005aa WFS imports: Write get GetFeature URLs into import log.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1899
diff changeset
179 if err := wfs.DownloadURLs(urls, func(url string, r io.Reader) error {
2b72f5e005aa WFS imports: Write get GetFeature URLs into import log.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1899
diff changeset
180 feedback.Info("Get features from: '%s'", url)
1785
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
181 rfc, err := wfs.ParseRawFeatureCollection(r)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
182 if err != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
183 return fmt.Errorf("parsing GetFeature document failed: %v", err)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
184 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
185 if rfc.CRS != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
186 crsName := rfc.CRS.Properties.Name
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
187 if epsg, err = wfs.CRSToEPSG(crsName); err != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
188 feedback.Error("Unsupported CRS: %d", crsName)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
189 return err
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
190 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
191 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
192
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
193 // No features -> ignore.
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
194 if rfc.Features == nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
195 return nil
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
196 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
197
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
198 feedback.Info("Using EPSG: %d", epsg)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
199
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
200 for _, feature := range rfc.Features {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
201 if feature.Properties == nil || feature.Geometry.Coordinates == nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
202 missingProperties++
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
203 continue
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
204 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
205
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
206 var props waterwayAreaProperties
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
207
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
208 if err := json.Unmarshal(*feature.Properties, &props); err != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
209 badProperties++
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
210 continue
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
211 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
212
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
213 var catccl sql.NullInt64
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
214 if props.Catccl != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
215 if value, err := strconv.ParseInt(*props.Catccl, 10, 64); err == nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
216 catccl = sql.NullInt64{Int64: value, Valid: true}
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
217 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
218 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
219 var dirimp sql.NullInt64
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
220 if props.Dirimp != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
221 if value, err := strconv.ParseInt(*props.Dirimp, 10, 64); err == nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
222 dirimp = sql.NullInt64{Int64: value, Valid: true}
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
223 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
224 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
225
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
226 switch feature.Geometry.Type {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
227 case "Polygon":
1786
09349ca27dd7 Imports: Removed duplicated code path to store WKB polygons into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
228 var p polygonSlice
1785
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
229 if err := json.Unmarshal(*feature.Geometry.Coordinates, &p); err != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
230 return err
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
231 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
232 if _, err := insertStmt.ExecContext(
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
233 ctx,
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
234 p.asWKB(),
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
235 epsg,
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
236 catccl,
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
237 dirimp,
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
238 ); err != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
239 return err
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
240 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
241 features++
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
242 default:
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
243 unsupported[feature.Geometry.Type]++
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
244 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
245 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
246 return nil
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
247 }); err != nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
248 return nil, err
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
249 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
250
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
251 if badProperties > 0 {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
252 feedback.Warn("Bad properties: %d", badProperties)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
253 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
254
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
255 if missingProperties > 0 {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
256 feedback.Warn("Missing properties: %d", missingProperties)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
257 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
258
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
259 if len(unsupported) != 0 {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
260 feedback.Warn("Unsupported types found: %s", unsupported)
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
261 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
262
1828
1ecfcf46e4da Waterway area/axis: Be more informative if features are bad.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1796
diff changeset
263 if features == 0 {
1ecfcf46e4da Waterway area/axis: Be more informative if features are bad.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1796
diff changeset
264 err := errors.New("No features found")
1ecfcf46e4da Waterway area/axis: Be more informative if features are bad.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1796
diff changeset
265 feedback.Error("%v", err)
1ecfcf46e4da Waterway area/axis: Be more informative if features are bad.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1796
diff changeset
266 return nil, err
1ecfcf46e4da Waterway area/axis: Be more informative if features are bad.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1796
diff changeset
267 }
1ecfcf46e4da Waterway area/axis: Be more informative if features are bad.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1796
diff changeset
268
1785
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
269 if err = tx.Commit(); err == nil {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
270 feedback.Info("Storing %d features took %s",
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
271 features, time.Since(start))
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
272 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
273
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
274 return nil, err
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff changeset
275 }