Mercurial > gemma

annotate pkg/wfs/download.go @ 4880:ed0de319d09a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Fix parameter format for WFS 2.0 See table 7 in the WFS 2.0 Interface Standard
author Tom Gottfried <tom@intevation.de>
date Wed, 29 Jan 2020 14:06:36 +0100
parents 04876d865528
children 16259efa828f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0.Reader.
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
3 //
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
6 //
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 //
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 // Author(s):
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
13
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14 package wfs
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import (
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 "bufio"
2110
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
18 "bytes"
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 "encoding/xml"
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
20 "errors"
1614
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
21 "fmt"
1624
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
22 "io"
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
23 "log"
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
24 "net/http"
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
25 "net/url"
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26 "strconv"
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
27 "strings"
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
28
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
29 "golang.org/x/net/html/charset"
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
30 )
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
31
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
32 var (
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
33 // ErrNoSuchFeatureType is returned when a feature is not supported.
4166
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
34 ErrNoSuchFeatureType = errors.New("no such feature type")
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
35 // ErrGetFeatureNotSupported is returned when GetFeature is not supported.
4166
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
36 ErrGetFeatureNotSupported = errors.New("method GetFeature not supported")
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
37 // ErrMethodGetNotSupported is returned when the GET is not supported.
4166
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
38 ErrMethodGetNotSupported = errors.New("method GET not supported")
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
39 // ErrNoNumberMatchedFound is returned if feature count cannot be extracted.
4166
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
40 ErrNoNumberMatchedFound = errors.New("no numberMatched attribute found")
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
41 // ErrOutputFormatNotSupported is returned if a output format is
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
42 // not supported.
4166
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
43 ErrOutputFormatNotSupported = errors.New("output format not supported")
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
44 )
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
45
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
46 // GetCapabilities downloads a capabilities document for a given URL.
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
47 func GetCapabilities(capURL string) (*Capabilities, error) {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
48
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
49 base, err := url.Parse(capURL)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
50 if err != nil {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
51 return nil, err
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
52 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
53 v := url.Values{}
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
54 v.Set("SERVICE", "WFS")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
55 v.Set("REQUEST", "GetCapabilities")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
56 v.Set("ACCEPTVERSIONS", "2.0.0,1.1.0,1.0.0")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
57 base.RawQuery = v.Encode()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
58
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
59 baseURL := base.String()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
60 resp, err := http.Get(baseURL)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
61 if err != nil {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
62 return nil, err
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
64 defer resp.Body.Close()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
65 caps, err := ParseCapabilities(bufio.NewReader(resp.Body))
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
66 if err == nil {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
67 caps.BaseURL = baseURL
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
68 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
69 return caps, err
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
70 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
71
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
72 func numberFeaturesGET(u *url.URL, featureType, version string) (int, error) {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
73
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
74 v := url.Values{}
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
75 v.Set("SERVICE", "WFS")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
76 v.Set("REQUEST", "GetFeature")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
77 v.Set("resultType", "hits")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
78 v.Set("VERSION", version)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
79 v.Set("TYPENAMES", featureType)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
80
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
81 q := *u
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
82 q.RawQuery = v.Encode()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
83
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
84 resp, err := http.Get(q.String())
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
85 if err != nil {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
86 return 0, err
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
87 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
88 defer resp.Body.Close()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
89 dec := xml.NewDecoder(resp.Body)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
90 dec.CharsetReader = charset.NewReaderLabel
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
91
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
92 var result struct {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
93 NumberMatched *int `xml:"numberMatched,attr"`
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
94 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
95
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
96 if err := dec.Decode(&result); err != nil {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
97 return 0, err
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
98 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
99
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
100 if result.NumberMatched == nil {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
101 return 0, ErrNoNumberMatchedFound
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
102 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
103
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
104 return *result.NumberMatched, nil
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
105 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
106
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
107 // GetFeaturesGET constructs a list of URLs to get features
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
108 // for a given feature type from a WFS servers.
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
109 func GetFeaturesGET(
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
110 caps *Capabilities,
2622
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
111 featureTypeName string,
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
112 outputFormats []string,
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
113 sortBy string,
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
114 ) ([]string, error) {
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
115
1614
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
116 feature := caps.FindFeatureType(featureTypeName)
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
117 if feature == nil {
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
118 return nil, ErrNoSuchFeatureType
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
119 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
120 op := caps.FindOperation("GetFeature")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
121 if op == nil {
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
122 return nil, ErrGetFeatureNotSupported
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
123 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
124
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
125 if op.DCP.HTTP.Get == nil {
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
126 return nil, ErrMethodGetNotSupported
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
127 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
128
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
129 getRaw := op.DCP.HTTP.Get.HRef
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
130 getU, err := url.Parse(getRaw)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
131 if err != nil {
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
132 return nil, err
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
133 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
134 // The URL could be relative so resolve against Capabilities URL.
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
135 if !getU.IsAbs() {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
136 base, err := url.Parse(caps.BaseURL)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
137 if err != nil {
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
138 return nil, err
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
139 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
140 getU = getU.ResolveReference(base)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
141 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
142
2622
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
143 var outputFormat string
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
144
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
145 if len(outputFormats) > 0 {
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
146 if outputFormat = op.SupportsOutputFormat(outputFormats...); outputFormat == "" {
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
147 return nil, ErrOutputFormatNotSupported
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
148 }
1616
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
149 }
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
150
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
151 wfsVersion := caps.HighestWFSVersion(WFS200)
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
152
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
153 featuresPerPage, supportsPaging := op.FeaturesPerPage()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
154
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
155 var numFeatures int
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
156
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
157 if supportsPaging {
2096
8b7dee291488 WFS downloader: Harmonized logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1935
diff changeset
158 log.Printf("info: Paging supported with %d feature per page.\n",
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
159 featuresPerPage)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
160
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
161 if !op.SupportsHits() {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
162 supportsPaging = false
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
163 } else {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
164 numFeatures, err = numberFeaturesGET(getU, featureTypeName, wfsVersion)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
165 if err != nil {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
166 log.Printf("error: %v\n", err)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
167 supportsPaging = false
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
168 } else {
2096
8b7dee291488 WFS downloader: Harmonized logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1935
diff changeset
169 log.Printf("info: Number of features: %d\n", numFeatures)
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
170 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
171 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
172 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
173
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
174 var downloadURLs []string
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
175 wfs2 := !versionIsLess(wfsVersion, WFS200)
1614
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
176
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
177 addNS := func(v url.Values) {
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
178 if len(feature.Namespaces) == 0 {
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
179 return
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
180 }
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
181 // Only use first namespace
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
182 ns := feature.Namespaces[0]
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
183 if wfs2 {
4880
ed0de319d09a Fix parameter format for WFS 2.0
Tom Gottfried <tom@intevation.de>
parents: 4166
diff changeset
184 v.Set(
ed0de319d09a Fix parameter format for WFS 2.0
Tom Gottfried <tom@intevation.de>
parents: 4166
diff changeset
185 "NAMESPACES", fmt.Sprintf("xmlns(%s,%s)", ns.Space, ns.Local))
1614
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
186 } else {
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
187 v.Set("NAMESPACE", fmt.Sprintf("(%s:%s)", ns.Space, ns.Local))
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
188 }
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
189 }
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
190
1616
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
191 addOutputFormat := func(v url.Values) {
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
192 if outputFormat != "" {
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
193 v.Set("outputFormat", outputFormat)
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
194 }
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
195 }
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
196
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
197 addSortBy := func(v url.Values) {
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
198 if sortBy != "" {
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
199 v.Set("sortBy", sortBy)
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
200 }
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
201 }
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
202
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
203 if supportsPaging {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
204 pagedURL := func(ofs, count int) string {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
205 v := url.Values{}
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
206 v.Set("SERVICE", "WFS")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
207 v.Set("REQUEST", "GetFeature")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
208 v.Set("VERSION", wfsVersion)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
209 v.Set("startIndex", strconv.Itoa(ofs))
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
210 if wfs2 {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
211 v.Set("count", strconv.Itoa(count))
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
212 } else {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
213 v.Set("maxFeatures", strconv.Itoa(count))
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
214 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
215 v.Set("TYPENAMES", featureTypeName)
1614
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
216 addNS(v)
1616
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
217 addOutputFormat(v)
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
218 addSortBy(v)
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
219 q := *getU
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
220 q.RawQuery = v.Encode()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
221 return q.String()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
222 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
223 if numFeatures <= featuresPerPage {
2096
8b7dee291488 WFS downloader: Harmonized logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1935
diff changeset
224 log.Println("info: All features can be fetched in one page.")
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
225 downloadURLs = []string{pagedURL(0, numFeatures)}
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
226 } else {
2096
8b7dee291488 WFS downloader: Harmonized logging.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1935
diff changeset
227 log.Println("info: Features need to be downloaded in pages.")
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
228 for pos := 0; pos < numFeatures; {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
229 var count int
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
230 if rest := numFeatures - pos; rest >= numFeatures {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
231 count = numFeatures
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
232 } else {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
233 count = rest
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
234 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
235 downloadURLs = append(downloadURLs, pagedURL(pos, count))
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
236 pos += count
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
237 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
238 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
239 } else { // No paging support.
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
240 v := url.Values{}
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
241 v.Set("SERVICE", "WFS")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
242 v.Set("REQUEST", "GetFeature")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
243 v.Set("VERSION", wfsVersion)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
244 v.Set("TYPENAMES", featureTypeName)
1614
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
245 addNS(v)
1616
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
246 addOutputFormat(v)
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
247 addSortBy(v)
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
248 q := *getU
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
249 q.RawQuery = v.Encode()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
250 downloadURLs = []string{q.String()}
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
251 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
252
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
253 return downloadURLs, nil
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
254 }
1624
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
255
2719
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
256 func downloadURL(user, password, url string, handler func(string, io.Reader) error) error {
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
257 req, err := http.NewRequest(http.MethodGet, url, nil)
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
258 if err != nil {
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
259 return err
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
260 }
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
261
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
262 if user != "" || password != "" {
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
263 req.SetBasicAuth(user, password)
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
264 }
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
265
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
266 resp, err := http.DefaultClient.Do(req)
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
267
1624
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
268 if err != nil {
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
269 return err
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
270 }
1626
92da44ba610c WFS downloader: Parse into raw feature collections.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1624
diff changeset
271 if resp.StatusCode < 200 || resp.StatusCode > 299 {
4166
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
272 return fmt.Errorf("invalid HTTP status code: %d (%s)",
1626
92da44ba610c WFS downloader: Parse into raw feature collections.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1624
diff changeset
273 resp.StatusCode, resp.Status)
92da44ba610c WFS downloader: Parse into raw feature collections.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1624
diff changeset
274 }
1624
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
275 defer resp.Body.Close()
2110
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
276
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
277 var already bytes.Buffer
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
278
2110
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
279 // Prevent the XML reader from consuming everything.
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
280 limit := io.LimitReader(resp.Body, 16*1024)
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
281 in := io.TeeReader(limit, &already)
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
282
2110
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
283 if err := scanExceptionReport(in); err != nil {
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
284 return err
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
285 }
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
286
2110
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
287 multi := io.MultiReader(bytes.NewReader(already.Bytes()), resp.Body)
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
288 return handler(url, multi)
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
289 }
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
290
4166
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
291 // ExceptionReport is an error with the extract code and
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
292 // text from an OWS exception document.
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
293 type ExceptionReport struct {
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
294 Code string
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
295 Text string
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
296 }
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
297
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
298 func (er *ExceptionReport) Error() string {
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
299 return fmt.Sprintf(
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
300 "WFS GetFeature error: ExceptionCode: '%s' / ExceptionText: %s",
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
301 er.Code, er.Text)
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
302 }
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
303
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
304 func scanExceptionReport(r io.Reader) *ExceptionReport {
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
305
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
306 const ows = "http://www.opengis.net/ows/1.1"
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
307
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
308 decoder := xml.NewDecoder(r)
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
309 decoder.CharsetReader = charset.NewReaderLabel
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
310
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
311 var (
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
312 isError bool
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
313 code string
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
314 depth int
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
315 text strings.Builder
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
316 )
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
317
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
318 type tokenFunc func(xml.Token) tokenFunc
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
319
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
320 var exceptionReportFn, exceptionFn, exceptionTextFn, collectTextFn tokenFunc
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
321
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
322 exceptionReportFn = func(t xml.Token) tokenFunc {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
323 e, ok := t.(xml.StartElement)
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
324 if !ok {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
325 return exceptionReportFn
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
326 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
327 if e.Name.Local != "ExceptionReport" && e.Name.Space != ows {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
328 return nil
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
329 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
330 isError = true
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
331 return exceptionFn
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
332 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
333
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
334 exceptionFn = func(t xml.Token) tokenFunc {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
335 e, ok := t.(xml.StartElement)
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
336 if !ok {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
337 return exceptionFn
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
338 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
339 if e.Name.Local == "Exception" && e.Name.Space == ows {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
340 for i := range e.Attr {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
341 at := &e.Attr[i]
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
342 if at.Name.Local == "code" || at.Name.Local == "exceptionCode" {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
343 code = at.Value
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
344 break
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
345 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
346 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
347 return exceptionTextFn
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
348 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
349 return exceptionFn
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
350 }
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
351
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
352 exceptionTextFn = func(t xml.Token) tokenFunc {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
353 e, ok := t.(xml.StartElement)
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
354 if ok && e.Name.Local == "ExceptionText" && e.Name.Space == ows {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
355 return collectTextFn
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
356 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
357 return exceptionTextFn
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
358 }
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
359
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
360 collectTextFn = func(t xml.Token) tokenFunc {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
361 switch e := t.(type) {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
362 case xml.StartElement:
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
363 depth++
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
364 case xml.CharData:
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
365 if depth == 0 {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
366 text.Write(e)
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
367 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
368 case xml.EndElement:
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
369 if depth == 0 {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
370 return nil
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
371 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
372 depth--
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
373 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
374 return collectTextFn
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
375 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
376
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
377 tokens:
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
378 for fn := exceptionReportFn; fn != nil; {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
379 tok, err := decoder.Token()
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
380 switch {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
381 case tok == nil && err == io.EOF:
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
382 break tokens
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
383 case err != nil:
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
384 return nil
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
385 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
386 fn = fn(tok)
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
387 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
388
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
389 if isError {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
390 return &ExceptionReport{
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
391 Code: code,
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
392 Text: text.String(),
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
393 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
394 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
395
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
396 return nil
1624
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
397 }
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
398
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
399 // DownloadURLs does the actual GetFeature requests downloads
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
400 // and hands the resulting io.Readers over to the given handler.
2719
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
401 func DownloadURLs(user, password string, urls []string, handler func(string, io.Reader) error) error {
1624
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
402 for _, url := range urls {
2719
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
403 if err := downloadURL(user, password, url, handler); err != nil {
1935
73f9e15d0384 WFS downloader: die correctly if an error was detected when downloading features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1689
diff changeset
404 return err
1624
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
405 }
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
406 }
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
407 return nil
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
408 }