Mercurial > gemma

annotate pkg/wfs/download.go @ 5490:5f47eeea988d logging

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Use own logging package.
author Sascha L. Teichmann <sascha.teichmann@intevation.de>
date Mon, 20 Sep 2021 17:45:39 +0200
parents 16259efa828f
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0.Reader.
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
3 //
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
6 //
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 //
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 // Author(s):
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
13
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14 package wfs
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import (
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 "bufio"
2110
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
18 "bytes"
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 "encoding/xml"
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
20 "errors"
1614
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
21 "fmt"
1624
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
22 "io"
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
23 "net/http"
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
24 "net/url"
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
25 "strconv"
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
26 "strings"
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
27
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
28 "golang.org/x/net/html/charset"
5490
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4990
diff changeset
29
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4990
diff changeset
30 "gemma.intevation.de/gemma/pkg/log"
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
31 )
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
32
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
33 var (
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
34 // ErrNoSuchFeatureType is returned when a feature is not supported.
4166
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
35 ErrNoSuchFeatureType = errors.New("no such feature type")
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
36 // ErrGetFeatureNotSupported is returned when GetFeature is not supported.
4166
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
37 ErrGetFeatureNotSupported = errors.New("method GetFeature not supported")
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
38 // ErrMethodGetNotSupported is returned when the GET is not supported.
4166
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
39 ErrMethodGetNotSupported = errors.New("method GET not supported")
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
40 // ErrNoNumberMatchedFound is returned if feature count cannot be extracted.
4166
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
41 ErrNoNumberMatchedFound = errors.New("no numberMatched attribute found")
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
42 // ErrOutputFormatNotSupported is returned if a output format is
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
43 // not supported.
4166
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
44 ErrOutputFormatNotSupported = errors.New("output format not supported")
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
45 )
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
46
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
47 // GetCapabilities downloads a capabilities document for a given URL.
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
48 func GetCapabilities(capURL string) (*Capabilities, error) {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
49
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
50 base, err := url.Parse(capURL)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
51 if err != nil {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
52 return nil, err
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
53 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
54 v := url.Values{}
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
55 v.Set("SERVICE", "WFS")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
56 v.Set("REQUEST", "GetCapabilities")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
57 v.Set("ACCEPTVERSIONS", "2.0.0,1.1.0,1.0.0")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
58 base.RawQuery = v.Encode()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
59
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
60 baseURL := base.String()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
61 resp, err := http.Get(baseURL)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
62 if err != nil {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63 return nil, err
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
64 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
65 defer resp.Body.Close()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
66 caps, err := ParseCapabilities(bufio.NewReader(resp.Body))
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
67 if err == nil {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
68 caps.BaseURL = baseURL
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
69 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
70 return caps, err
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
71 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
72
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
73 func numberFeaturesGET(u *url.URL, featureType, version string) (int, error) {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
74
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
75 v := url.Values{}
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
76 v.Set("SERVICE", "WFS")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
77 v.Set("REQUEST", "GetFeature")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
78 v.Set("resultType", "hits")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
79 v.Set("VERSION", version)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
80 v.Set("TYPENAMES", featureType)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
81
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
82 q := *u
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
83 q.RawQuery = v.Encode()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
84
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
85 resp, err := http.Get(q.String())
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
86 if err != nil {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
87 return 0, err
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
88 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
89 defer resp.Body.Close()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
90 dec := xml.NewDecoder(resp.Body)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
91 dec.CharsetReader = charset.NewReaderLabel
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
92
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
93 var result struct {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
94 NumberMatched *int `xml:"numberMatched,attr"`
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
95 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
96
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
97 if err := dec.Decode(&result); err != nil {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
98 return 0, err
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
99 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
100
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
101 if result.NumberMatched == nil {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
102 return 0, ErrNoNumberMatchedFound
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
103 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
104
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
105 return *result.NumberMatched, nil
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
106 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
107
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
108 // GetFeaturesGET constructs a list of URLs to get features
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
109 // for a given feature type from a WFS servers.
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
110 func GetFeaturesGET(
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
111 caps *Capabilities,
2622
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
112 featureTypeName string,
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
113 outputFormats []string,
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
114 sortBy string,
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
115 ) ([]string, error) {
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
116
1614
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
117 feature := caps.FindFeatureType(featureTypeName)
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
118 if feature == nil {
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
119 return nil, ErrNoSuchFeatureType
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
120 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
121 op := caps.FindOperation("GetFeature")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
122 if op == nil {
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
123 return nil, ErrGetFeatureNotSupported
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
124 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
125
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
126 if op.DCP.HTTP.Get == nil {
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
127 return nil, ErrMethodGetNotSupported
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
128 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
129
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
130 getRaw := op.DCP.HTTP.Get.HRef
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
131 getU, err := url.Parse(getRaw)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
132 if err != nil {
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
133 return nil, err
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
134 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
135 // The URL could be relative so resolve against Capabilities URL.
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
136 if !getU.IsAbs() {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
137 base, err := url.Parse(caps.BaseURL)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
138 if err != nil {
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
139 return nil, err
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
140 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
141 getU = getU.ResolveReference(base)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
142 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
143
2622
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
144 var outputFormat string
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
145
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
146 if len(outputFormats) > 0 {
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
147 if outputFormat = op.SupportsOutputFormat(outputFormats...); outputFormat == "" {
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
148 return nil, ErrOutputFormatNotSupported
02505fcff63c WFS downloads: Accept 'geojson' as mime type, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2620
diff changeset
149 }
1616
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
150 }
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
151
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
152 wfsVersion := caps.HighestWFSVersion(WFS200)
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
153
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
154 featuresPerPage, supportsPaging := op.FeaturesPerPage()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
155
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
156 var numFeatures int
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
157
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
158 if supportsPaging {
5490
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4990
diff changeset
159 log.Infof("Paging supported with %d feature per page.\n",
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
160 featuresPerPage)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
161
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
162 if !op.SupportsHits() {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
163 supportsPaging = false
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
164 } else {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
165 numFeatures, err = numberFeaturesGET(getU, featureTypeName, wfsVersion)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
166 if err != nil {
5490
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4990
diff changeset
167 log.Errorf("%v\n", err)
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
168 supportsPaging = false
4990
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
169 } else if numFeatures == 0 {
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
170 return nil, nil
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
171 } else {
5490
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4990
diff changeset
172 log.Infof("Number of features: %d\n", numFeatures)
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
173 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
174 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
175 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
176
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
177 var downloadURLs []string
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
178 wfs2 := !versionIsLess(wfsVersion, WFS200)
1614
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
179
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
180 addNS := func(v url.Values) {
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
181 if len(feature.Namespaces) == 0 {
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
182 return
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
183 }
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
184 // Only use first namespace
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
185 ns := feature.Namespaces[0]
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
186 if wfs2 {
4880
ed0de319d09a Fix parameter format for WFS 2.0
Tom Gottfried <tom@intevation.de>
parents: 4166
diff changeset
187 v.Set(
ed0de319d09a Fix parameter format for WFS 2.0
Tom Gottfried <tom@intevation.de>
parents: 4166
diff changeset
188 "NAMESPACES", fmt.Sprintf("xmlns(%s,%s)", ns.Space, ns.Local))
1614
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
189 } else {
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
190 v.Set("NAMESPACE", fmt.Sprintf("(%s:%s)", ns.Space, ns.Local))
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
191 }
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
192 }
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
193
1616
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
194 addOutputFormat := func(v url.Values) {
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
195 if outputFormat != "" {
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
196 v.Set("outputFormat", outputFormat)
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
197 }
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
198 }
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
199
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
200 addSortBy := func(v url.Values) {
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
201 if sortBy != "" {
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
202 v.Set("sortBy", sortBy)
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
203 }
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
204 }
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
205
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
206 if supportsPaging {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
207 pagedURL := func(ofs, count int) string {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
208 v := url.Values{}
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
209 v.Set("SERVICE", "WFS")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
210 v.Set("REQUEST", "GetFeature")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
211 v.Set("VERSION", wfsVersion)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
212 v.Set("startIndex", strconv.Itoa(ofs))
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
213 if wfs2 {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
214 v.Set("count", strconv.Itoa(count))
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
215 } else {
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
216 v.Set("maxFeatures", strconv.Itoa(count))
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
217 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
218 v.Set("TYPENAMES", featureTypeName)
1614
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
219 addNS(v)
1616
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
220 addOutputFormat(v)
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
221 addSortBy(v)
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
222 q := *getU
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
223 q.RawQuery = v.Encode()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
224 return q.String()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
225 }
4990
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
226 if numFeatures > 0 {
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
227 if numFeatures <= featuresPerPage {
5490
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4990
diff changeset
228 log.Infof("all features can be fetched in one page.")
4990
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
229 downloadURLs = []string{pagedURL(0, numFeatures)}
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
230 } else {
5490
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4990
diff changeset
231 log.Infof("features need to be downloaded in pages.")
4990
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
232 for pos := 0; pos < numFeatures; {
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
233 var count int
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
234 if rest := numFeatures - pos; rest >= numFeatures {
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
235 count = numFeatures
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
236 } else {
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
237 count = rest
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
238 }
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
239 downloadURLs = append(downloadURLs, pagedURL(pos, count))
16259efa828f Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4880
diff changeset
240 pos += count
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
241 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
242 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
243 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
244 } else { // No paging support.
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
245 v := url.Values{}
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
246 v.Set("SERVICE", "WFS")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
247 v.Set("REQUEST", "GetFeature")
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
248 v.Set("VERSION", wfsVersion)
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
249 v.Set("TYPENAMES", featureTypeName)
1614
efc409e330a6 WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1608
diff changeset
250 addNS(v)
1616
f59550310143 WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1614
diff changeset
251 addOutputFormat(v)
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
252 addSortBy(v)
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
253 q := *getU
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
254 q.RawQuery = v.Encode()
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
255 downloadURLs = []string{q.String()}
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
256 }
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
257
1619
3093bab05c81 WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1616
diff changeset
258 return downloadURLs, nil
1608
427f9010b4a9 WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
259 }
1624
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
260
2719
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
261 func downloadURL(user, password, url string, handler func(string, io.Reader) error) error {
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
262 req, err := http.NewRequest(http.MethodGet, url, nil)
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
263 if err != nil {
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
264 return err
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
265 }
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
266
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
267 if user != "" || password != "" {
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
268 req.SetBasicAuth(user, password)
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
269 }
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
270
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
271 resp, err := http.DefaultClient.Do(req)
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
272
1624
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
273 if err != nil {
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
274 return err
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
275 }
1626
92da44ba610c WFS downloader: Parse into raw feature collections.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1624
diff changeset
276 if resp.StatusCode < 200 || resp.StatusCode > 299 {
4166
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
277 return fmt.Errorf("invalid HTTP status code: %d (%s)",
1626
92da44ba610c WFS downloader: Parse into raw feature collections.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1624
diff changeset
278 resp.StatusCode, resp.Status)
92da44ba610c WFS downloader: Parse into raw feature collections.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1624
diff changeset
279 }
1624
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
280 defer resp.Body.Close()
2110
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
281
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
282 var already bytes.Buffer
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
283
2110
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
284 // Prevent the XML reader from consuming everything.
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
285 limit := io.LimitReader(resp.Body, 16*1024)
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
286 in := io.TeeReader(limit, &already)
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
287
2110
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
288 if err := scanExceptionReport(in); err != nil {
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
289 return err
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
290 }
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
291
2110
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
292 multi := io.MultiReader(bytes.NewReader(already.Bytes()), resp.Body)
fc4ebe7eaeff WFS downloader: Don't use temp files. TODO: Use a streaming scanner for the detection of the exception documents.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2109
diff changeset
293 return handler(url, multi)
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
294 }
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
295
4166
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
296 // ExceptionReport is an error with the extract code and
04876d865528 Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2719
diff changeset
297 // text from an OWS exception document.
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
298 type ExceptionReport struct {
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
299 Code string
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
300 Text string
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
301 }
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
302
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
303 func (er *ExceptionReport) Error() string {
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
304 return fmt.Sprintf(
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
305 "WFS GetFeature error: ExceptionCode: '%s' / ExceptionText: %s",
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
306 er.Code, er.Text)
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
307 }
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
308
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
309 func scanExceptionReport(r io.Reader) *ExceptionReport {
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
310
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
311 const ows = "http://www.opengis.net/ows/1.1"
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
312
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
313 decoder := xml.NewDecoder(r)
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
314 decoder.CharsetReader = charset.NewReaderLabel
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
315
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
316 var (
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
317 isError bool
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
318 code string
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
319 depth int
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
320 text strings.Builder
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
321 )
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
322
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
323 type tokenFunc func(xml.Token) tokenFunc
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
324
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
325 var exceptionReportFn, exceptionFn, exceptionTextFn, collectTextFn tokenFunc
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
326
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
327 exceptionReportFn = func(t xml.Token) tokenFunc {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
328 e, ok := t.(xml.StartElement)
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
329 if !ok {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
330 return exceptionReportFn
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
331 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
332 if e.Name.Local != "ExceptionReport" && e.Name.Space != ows {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
333 return nil
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
334 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
335 isError = true
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
336 return exceptionFn
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
337 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
338
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
339 exceptionFn = func(t xml.Token) tokenFunc {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
340 e, ok := t.(xml.StartElement)
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
341 if !ok {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
342 return exceptionFn
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
343 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
344 if e.Name.Local == "Exception" && e.Name.Space == ows {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
345 for i := range e.Attr {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
346 at := &e.Attr[i]
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
347 if at.Name.Local == "code" || at.Name.Local == "exceptionCode" {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
348 code = at.Value
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
349 break
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
350 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
351 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
352 return exceptionTextFn
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
353 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
354 return exceptionFn
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
355 }
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
356
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
357 exceptionTextFn = func(t xml.Token) tokenFunc {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
358 e, ok := t.(xml.StartElement)
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
359 if ok && e.Name.Local == "ExceptionText" && e.Name.Space == ows {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
360 return collectTextFn
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
361 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
362 return exceptionTextFn
2109
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
363 }
86c88fc0ff5e WFS downloader: Parse for exception reports when calling GetFeature. Not very efficient (uses temp files).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2106
diff changeset
364
2118
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
365 collectTextFn = func(t xml.Token) tokenFunc {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
366 switch e := t.(type) {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
367 case xml.StartElement:
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
368 depth++
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
369 case xml.CharData:
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
370 if depth == 0 {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
371 text.Write(e)
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
372 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
373 case xml.EndElement:
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
374 if depth == 0 {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
375 return nil
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
376 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
377 depth--
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
378 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
379 return collectTextFn
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
380 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
381
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
382 tokens:
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
383 for fn := exceptionReportFn; fn != nil; {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
384 tok, err := decoder.Token()
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
385 switch {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
386 case tok == nil && err == io.EOF:
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
387 break tokens
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
388 case err != nil:
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
389 return nil
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
390 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
391 fn = fn(tok)
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
392 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
393
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
394 if isError {
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
395 return &ExceptionReport{
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
396 Code: code,
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
397 Text: text.String(),
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
398 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
399 }
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
400
8a6ed058af22 WFS downloader: Use a custom XML streaming parser to detect error reports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2110
diff changeset
401 return nil
1624
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
402 }
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
403
1689
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
404 // DownloadURLs does the actual GetFeature requests downloads
6caf5cd6249e WFS: Made golint happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1626
diff changeset
405 // and hands the resulting io.Readers over to the given handler.
2719
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
406 func DownloadURLs(user, password string, urls []string, handler func(string, io.Reader) error) error {
1624
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
407 for _, url := range urls {
2719
2b6f73c30016 WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2702
diff changeset
408 if err := downloadURL(user, password, url, handler); err != nil {
1935
73f9e15d0384 WFS downloader: die correctly if an error was detected when downloading features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1689
diff changeset
409 return err
1624
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
410 }
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
411 }
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
412 return nil
943823d03d50 WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1619
diff changeset
413 }