Mercurial > gemma
annotate pkg/wfs/download.go @ 3334:8c96b7379aea
client: fairway availability: made store action work with both types of feature objects
author | Markus Kottlaender <markus@intevation.de> |
---|---|
date | Mon, 20 May 2019 17:14:47 +0200 |
parents | 2b6f73c30016 |
children | 04876d865528 |
rev | line source |
---|---|
1689
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
1 // This is Free Software under GNU Affero General Public License v >= 3.0.Reader. |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
2 // without warranty, see README.md and license for details. |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
3 // |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
4 // SPDX-License-Identifier: AGPL-3.0-or-later |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
5 // License-Filename: LICENSES/AGPL-3.0.txt |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
6 // |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
7 // Copyright (C) 2018 by via donau |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
8 // – Österreichische Wasserstraßen-Gesellschaft mbH |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
9 // Software engineering by Intevation GmbH |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
10 // |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
11 // Author(s): |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de> |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
13 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
14 package wfs |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
15 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
16 import ( |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
17 "bufio" |
2110
fc4ebe7eaeff
WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2109
diff
changeset
|
18 "bytes" |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
19 "encoding/xml" |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
20 "errors" |
1614
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
21 "fmt" |
1624
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
22 "io" |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
23 "log" |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
24 "net/http" |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
25 "net/url" |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
26 "strconv" |
2118
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
27 "strings" |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
28 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
29 "golang.org/x/net/html/charset" |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
30 ) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
31 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
32 var ( |
1689
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
33 // ErrNoSuchFeatureType is returned when a feature is not supported. |
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
34 ErrNoSuchFeatureType = errors.New("No such feature type") |
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
35 // ErrGetFeatureNotSupported is returned when GetFeature is not supported. |
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
36 ErrGetFeatureNotSupported = errors.New("GetFeature not supported") |
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
37 // ErrMethodGetNotSupported is returned when the GET is not supported. |
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
38 ErrMethodGetNotSupported = errors.New("GET not supported") |
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
39 // ErrNoNumberMatchedFound is returned if feature count cannot be extracted. |
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
40 ErrNoNumberMatchedFound = errors.New("No numberMatched attribute found") |
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
41 // ErrOutputFormatNotSupported is returned if a output format is |
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
42 // not supported. |
1616
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
43 ErrOutputFormatNotSupported = errors.New("Output format not supported") |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
44 ) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
45 |
1689
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
46 // GetCapabilities downloads a capabilities document for a given URL. |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
47 func GetCapabilities(capURL string) (*Capabilities, error) { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
48 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
49 base, err := url.Parse(capURL) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
50 if err != nil { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
51 return nil, err |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
52 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
53 v := url.Values{} |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
54 v.Set("SERVICE", "WFS") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
55 v.Set("REQUEST", "GetCapabilities") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
56 v.Set("ACCEPTVERSIONS", "2.0.0,1.1.0,1.0.0") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
57 base.RawQuery = v.Encode() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
58 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
59 baseURL := base.String() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
60 resp, err := http.Get(baseURL) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
61 if err != nil { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
62 return nil, err |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
63 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
64 defer resp.Body.Close() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
65 caps, err := ParseCapabilities(bufio.NewReader(resp.Body)) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
66 if err == nil { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
67 caps.BaseURL = baseURL |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
68 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
69 return caps, err |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
70 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
71 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
72 func numberFeaturesGET(u *url.URL, featureType, version string) (int, error) { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
73 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
74 v := url.Values{} |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
75 v.Set("SERVICE", "WFS") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
76 v.Set("REQUEST", "GetFeature") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
77 v.Set("resultType", "hits") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
78 v.Set("VERSION", version) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
79 v.Set("TYPENAMES", featureType) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
80 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
81 q := *u |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
82 q.RawQuery = v.Encode() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
83 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
84 resp, err := http.Get(q.String()) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
85 if err != nil { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
86 return 0, err |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
87 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
88 defer resp.Body.Close() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
89 dec := xml.NewDecoder(resp.Body) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
90 dec.CharsetReader = charset.NewReaderLabel |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
91 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
92 var result struct { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
93 NumberMatched *int `xml:"numberMatched,attr"` |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
94 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
95 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
96 if err := dec.Decode(&result); err != nil { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
97 return 0, err |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
98 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
99 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
100 if result.NumberMatched == nil { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
101 return 0, ErrNoNumberMatchedFound |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
102 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
103 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
104 return *result.NumberMatched, nil |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
105 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
106 |
1689
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
107 // GetFeaturesGET constructs a list of URLs to get features |
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
108 // for a given feature type from a WFS servers. |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
109 func GetFeaturesGET( |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
110 caps *Capabilities, |
2622
02505fcff63c
WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2620
diff
changeset
|
111 featureTypeName string, |
02505fcff63c
WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2620
diff
changeset
|
112 outputFormats []string, |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
113 sortBy string, |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
114 ) ([]string, error) { |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
115 |
1614
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
116 feature := caps.FindFeatureType(featureTypeName) |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
117 if feature == nil { |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
118 return nil, ErrNoSuchFeatureType |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
119 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
120 op := caps.FindOperation("GetFeature") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
121 if op == nil { |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
122 return nil, ErrGetFeatureNotSupported |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
123 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
124 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
125 if op.DCP.HTTP.Get == nil { |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
126 return nil, ErrMethodGetNotSupported |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
127 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
128 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
129 getRaw := op.DCP.HTTP.Get.HRef |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
130 getU, err := url.Parse(getRaw) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
131 if err != nil { |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
132 return nil, err |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
133 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
134 // The URL could be relative so resolve against Capabilities URL. |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
135 if !getU.IsAbs() { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
136 base, err := url.Parse(caps.BaseURL) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
137 if err != nil { |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
138 return nil, err |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
139 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
140 getU = getU.ResolveReference(base) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
141 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
142 |
2622
02505fcff63c
WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2620
diff
changeset
|
143 var outputFormat string |
02505fcff63c
WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2620
diff
changeset
|
144 |
02505fcff63c
WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2620
diff
changeset
|
145 if len(outputFormats) > 0 { |
02505fcff63c
WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2620
diff
changeset
|
146 if outputFormat = op.SupportsOutputFormat(outputFormats...); outputFormat == "" { |
02505fcff63c
WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2620
diff
changeset
|
147 return nil, ErrOutputFormatNotSupported |
02505fcff63c
WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2620
diff
changeset
|
148 } |
1616
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
149 } |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
150 |
1689
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
151 wfsVersion := caps.HighestWFSVersion(WFS200) |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
152 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
153 featuresPerPage, supportsPaging := op.FeaturesPerPage() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
154 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
155 var numFeatures int |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
156 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
157 if supportsPaging { |
2096
8b7dee291488
WFS downloader: Harmonized logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1935
diff
changeset
|
158 log.Printf("info: Paging supported with %d feature per page.\n", |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
159 featuresPerPage) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
160 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
161 if !op.SupportsHits() { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
162 supportsPaging = false |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
163 } else { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
164 numFeatures, err = numberFeaturesGET(getU, featureTypeName, wfsVersion) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
165 if err != nil { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
166 log.Printf("error: %v\n", err) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
167 supportsPaging = false |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
168 } else { |
2096
8b7dee291488
WFS downloader: Harmonized logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1935
diff
changeset
|
169 log.Printf("info: Number of features: %d\n", numFeatures) |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
170 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
171 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
172 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
173 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
174 var downloadURLs []string |
1689
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
175 wfs2 := !versionIsLess(wfsVersion, WFS200) |
1614
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
176 |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
177 addNS := func(v url.Values) { |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
178 if len(feature.Namespaces) == 0 { |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
179 return |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
180 } |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
181 // Only use first namespace |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
182 ns := feature.Namespaces[0] |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
183 if wfs2 { |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
184 v.Set("NAMESPACES", fmt.Sprintf("(%s,%s)", ns.Space, ns.Local)) |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
185 } else { |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
186 v.Set("NAMESPACE", fmt.Sprintf("(%s:%s)", ns.Space, ns.Local)) |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
187 } |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
188 } |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
189 |
1616
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
190 addOutputFormat := func(v url.Values) { |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
191 if outputFormat != "" { |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
192 v.Set("outputFormat", outputFormat) |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
193 } |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
194 } |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
195 |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
196 addSortBy := func(v url.Values) { |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
197 if sortBy != "" { |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
198 v.Set("sortBy", sortBy) |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
199 } |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
200 } |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
201 |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
202 if supportsPaging { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
203 pagedURL := func(ofs, count int) string { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
204 v := url.Values{} |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
205 v.Set("SERVICE", "WFS") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
206 v.Set("REQUEST", "GetFeature") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
207 v.Set("VERSION", wfsVersion) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
208 v.Set("startIndex", strconv.Itoa(ofs)) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
209 if wfs2 { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
210 v.Set("count", strconv.Itoa(count)) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
211 } else { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
212 v.Set("maxFeatures", strconv.Itoa(count)) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
213 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
214 v.Set("TYPENAMES", featureTypeName) |
1614
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
215 addNS(v) |
1616
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
216 addOutputFormat(v) |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
217 addSortBy(v) |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
218 q := *getU |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
219 q.RawQuery = v.Encode() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
220 return q.String() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
221 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
222 if numFeatures <= featuresPerPage { |
2096
8b7dee291488
WFS downloader: Harmonized logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1935
diff
changeset
|
223 log.Println("info: All features can be fetched in one page.") |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
224 downloadURLs = []string{pagedURL(0, numFeatures)} |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
225 } else { |
2096
8b7dee291488
WFS downloader: Harmonized logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1935
diff
changeset
|
226 log.Println("info: Features need to be downloaded in pages.") |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
227 for pos := 0; pos < numFeatures; { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
228 var count int |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
229 if rest := numFeatures - pos; rest >= numFeatures { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
230 count = numFeatures |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
231 } else { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
232 count = rest |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
233 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
234 downloadURLs = append(downloadURLs, pagedURL(pos, count)) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
235 pos += count |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
236 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
237 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
238 } else { // No paging support. |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
239 v := url.Values{} |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
240 v.Set("SERVICE", "WFS") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
241 v.Set("REQUEST", "GetFeature") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
242 v.Set("VERSION", wfsVersion) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
243 v.Set("TYPENAMES", featureTypeName) |
1614
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
244 addNS(v) |
1616
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
245 addOutputFormat(v) |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
246 addSortBy(v) |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
247 q := *getU |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
248 q.RawQuery = v.Encode() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
249 downloadURLs = []string{q.String()} |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
250 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
251 |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
252 return downloadURLs, nil |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
253 } |
1624
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
254 |
2719
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
255 func downloadURL(user, password, url string, handler func(string, io.Reader) error) error { |
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
256 req, err := http.NewRequest(http.MethodGet, url, nil) |
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
257 if err != nil { |
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
258 return err |
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
259 } |
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
260 |
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
261 if user != "" || password != "" { |
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
262 req.SetBasicAuth(user, password) |
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
263 } |
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
264 |
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
265 resp, err := http.DefaultClient.Do(req) |
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
266 |
1624
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
267 if err != nil { |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
268 return err |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
269 } |
1626
92da44ba610c
WFS downloader: Parse into raw feature collections.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1624
diff
changeset
|
270 if resp.StatusCode < 200 || resp.StatusCode > 299 { |
92da44ba610c
WFS downloader: Parse into raw feature collections.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1624
diff
changeset
|
271 return fmt.Errorf("Invalid HTTP status code: %d (%s)", |
92da44ba610c
WFS downloader: Parse into raw feature collections.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1624
diff
changeset
|
272 resp.StatusCode, resp.Status) |
92da44ba610c
WFS downloader: Parse into raw feature collections.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1624
diff
changeset
|
273 } |
1624
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
274 defer resp.Body.Close() |
2110
fc4ebe7eaeff
WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2109
diff
changeset
|
275 |
fc4ebe7eaeff
WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2109
diff
changeset
|
276 var already bytes.Buffer |
2109
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
277 |
2110
fc4ebe7eaeff
WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2109
diff
changeset
|
278 // Prevent the XML reader from consuming everything. |
fc4ebe7eaeff
WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2109
diff
changeset
|
279 limit := io.LimitReader(resp.Body, 16*1024) |
fc4ebe7eaeff
WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2109
diff
changeset
|
280 in := io.TeeReader(limit, &already) |
2109
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
281 |
2110
fc4ebe7eaeff
WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2109
diff
changeset
|
282 if err := scanExceptionReport(in); err != nil { |
2109
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
283 return err |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
284 } |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
285 |
2110
fc4ebe7eaeff
WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2109
diff
changeset
|
286 multi := io.MultiReader(bytes.NewReader(already.Bytes()), resp.Body) |
fc4ebe7eaeff
WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2109
diff
changeset
|
287 return handler(url, multi) |
2109
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
288 } |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
289 |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
290 type ExceptionReport struct { |
2118
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
291 Code string |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
292 Text string |
2109
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
293 } |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
294 |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
295 func (er *ExceptionReport) Error() string { |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
296 return fmt.Sprintf( |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
297 "WFS GetFeature error: ExceptionCode: '%s' / ExceptionText: %s", |
2118
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
298 er.Code, er.Text) |
2109
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
299 } |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
300 |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
301 func scanExceptionReport(r io.Reader) *ExceptionReport { |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
302 |
2118
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
303 const ows = "http://www.opengis.net/ows/1.1" |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
304 |
2109
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
305 decoder := xml.NewDecoder(r) |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
306 decoder.CharsetReader = charset.NewReaderLabel |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
307 |
2118
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
308 var ( |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
309 isError bool |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
310 code string |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
311 depth int |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
312 text strings.Builder |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
313 ) |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
314 |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
315 type tokenFunc func(xml.Token) tokenFunc |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
316 |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
317 var exceptionReportFn, exceptionFn, exceptionTextFn, collectTextFn tokenFunc |
2109
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
318 |
2118
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
319 exceptionReportFn = func(t xml.Token) tokenFunc { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
320 e, ok := t.(xml.StartElement) |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
321 if !ok { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
322 return exceptionReportFn |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
323 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
324 if e.Name.Local != "ExceptionReport" && e.Name.Space != ows { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
325 return nil |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
326 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
327 isError = true |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
328 return exceptionFn |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
329 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
330 |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
331 exceptionFn = func(t xml.Token) tokenFunc { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
332 e, ok := t.(xml.StartElement) |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
333 if !ok { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
334 return exceptionFn |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
335 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
336 if e.Name.Local == "Exception" && e.Name.Space == ows { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
337 for i := range e.Attr { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
338 at := &e.Attr[i] |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
339 if at.Name.Local == "code" || at.Name.Local == "exceptionCode" { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
340 code = at.Value |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
341 break |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
342 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
343 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
344 return exceptionTextFn |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
345 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
346 return exceptionFn |
2109
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
347 } |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
348 |
2118
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
349 exceptionTextFn = func(t xml.Token) tokenFunc { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
350 e, ok := t.(xml.StartElement) |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
351 if ok && e.Name.Local == "ExceptionText" && e.Name.Space == ows { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
352 return collectTextFn |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
353 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
354 return exceptionTextFn |
2109
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
355 } |
86c88fc0ff5e
WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2106
diff
changeset
|
356 |
2118
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
357 collectTextFn = func(t xml.Token) tokenFunc { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
358 switch e := t.(type) { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
359 case xml.StartElement: |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
360 depth++ |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
361 case xml.CharData: |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
362 if depth == 0 { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
363 text.Write(e) |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
364 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
365 case xml.EndElement: |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
366 if depth == 0 { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
367 return nil |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
368 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
369 depth-- |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
370 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
371 return collectTextFn |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
372 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
373 |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
374 tokens: |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
375 for fn := exceptionReportFn; fn != nil; { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
376 tok, err := decoder.Token() |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
377 switch { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
378 case tok == nil && err == io.EOF: |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
379 break tokens |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
380 case err != nil: |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
381 return nil |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
382 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
383 fn = fn(tok) |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
384 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
385 |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
386 if isError { |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
387 return &ExceptionReport{ |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
388 Code: code, |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
389 Text: text.String(), |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
390 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
391 } |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
392 |
8a6ed058af22
WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2110
diff
changeset
|
393 return nil |
1624
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
394 } |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
395 |
1689
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
396 // DownloadURLs does the actual GetFeature requests downloads |
6caf5cd6249e
WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1626
diff
changeset
|
397 // and hands the resulting io.Readers over to the given handler. |
2719
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
398 func DownloadURLs(user, password string, urls []string, handler func(string, io.Reader) error) error { |
1624
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
399 for _, url := range urls { |
2719
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
400 if err := downloadURL(user, password, url, handler); err != nil { |
1935
73f9e15d0384
WFS downloader: die correctly if an error was detected when downloading features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1689
diff
changeset
|
401 return err |
1624
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
402 } |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
403 } |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
404 return nil |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
405 } |