Mercurial > gemma
annotate pkg/wfs/download.go @ 1665:da0d1a19ebe6
Fairway availability import: Made schedulable, too.
author | Sascha L. Teichmann <sascha.teichmann@intevation.de> |
---|---|
date | Sun, 23 Dec 2018 13:30:33 +0100 |
parents | 92da44ba610c |
children | 6caf5cd6249e |
rev | line source |
---|---|
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
1 // This is Free Software under GNU Affero General Public License v >= 3.0 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
2 // without warranty, see README.md and license for details. |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
3 // |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
4 // SPDX-License-Identifier: AGPL-3.0-or-later |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
5 // License-Filename: LICENSES/AGPL-3.0.txt |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
6 // |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
7 // Copyright (C) 2018 by via donau |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
8 // – Österreichische Wasserstraßen-Gesellschaft mbH |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
9 // Software engineering by Intevation GmbH |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
10 // |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
11 // Author(s): |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de> |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
13 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
14 package wfs |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
15 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
16 import ( |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
17 "bufio" |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
18 "encoding/xml" |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
19 "errors" |
1614
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
20 "fmt" |
1624
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
21 "io" |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
22 "log" |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
23 "net/http" |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
24 "net/url" |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
25 "strconv" |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
26 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
27 "golang.org/x/net/html/charset" |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
28 ) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
29 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
30 var ( |
1616
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
31 ErrNoSuchFeatureType = errors.New("No such feature type") |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
32 ErrGetFeatureNotSupported = errors.New("GetFeature not supported") |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
33 ErrMethodGetNotSupported = errors.New("GET not supported") |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
34 ErrNoNumberMatchedFound = errors.New("No numberMatched attribute found") |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
35 ErrOutputFormatNotSupported = errors.New("Output format not supported") |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
36 ) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
37 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
38 func GetCapabilities(capURL string) (*Capabilities, error) { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
39 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
40 base, err := url.Parse(capURL) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
41 if err != nil { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
42 return nil, err |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
43 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
44 v := url.Values{} |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
45 v.Set("SERVICE", "WFS") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
46 v.Set("REQUEST", "GetCapabilities") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
47 v.Set("ACCEPTVERSIONS", "2.0.0,1.1.0,1.0.0") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
48 base.RawQuery = v.Encode() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
49 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
50 baseURL := base.String() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
51 resp, err := http.Get(baseURL) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
52 if err != nil { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
53 return nil, err |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
54 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
55 defer resp.Body.Close() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
56 caps, err := ParseCapabilities(bufio.NewReader(resp.Body)) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
57 if err == nil { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
58 caps.BaseURL = baseURL |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
59 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
60 return caps, err |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
61 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
62 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
63 func numberFeaturesGET(u *url.URL, featureType, version string) (int, error) { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
64 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
65 v := url.Values{} |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
66 v.Set("SERVICE", "WFS") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
67 v.Set("REQUEST", "GetFeature") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
68 v.Set("resultType", "hits") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
69 v.Set("VERSION", version) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
70 v.Set("TYPENAMES", featureType) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
71 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
72 q := *u |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
73 q.RawQuery = v.Encode() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
74 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
75 resp, err := http.Get(q.String()) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
76 if err != nil { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
77 return 0, err |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
78 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
79 defer resp.Body.Close() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
80 dec := xml.NewDecoder(resp.Body) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
81 dec.CharsetReader = charset.NewReaderLabel |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
82 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
83 var result struct { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
84 NumberMatched *int `xml:"numberMatched,attr"` |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
85 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
86 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
87 if err := dec.Decode(&result); err != nil { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
88 return 0, err |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
89 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
90 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
91 if result.NumberMatched == nil { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
92 return 0, ErrNoNumberMatchedFound |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
93 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
94 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
95 return *result.NumberMatched, nil |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
96 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
97 |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
98 func GetFeaturesGET( |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
99 caps *Capabilities, |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
100 featureTypeName, |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
101 outputFormat string, |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
102 sortBy string, |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
103 ) ([]string, error) { |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
104 |
1614
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
105 feature := caps.FindFeatureType(featureTypeName) |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
106 if feature == nil { |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
107 return nil, ErrNoSuchFeatureType |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
108 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
109 op := caps.FindOperation("GetFeature") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
110 if op == nil { |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
111 return nil, ErrGetFeatureNotSupported |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
112 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
113 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
114 if op.DCP.HTTP.Get == nil { |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
115 return nil, ErrMethodGetNotSupported |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
116 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
117 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
118 getRaw := op.DCP.HTTP.Get.HRef |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
119 getU, err := url.Parse(getRaw) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
120 if err != nil { |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
121 return nil, err |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
122 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
123 // The URL could be relative so resolve against Capabilities URL. |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
124 if !getU.IsAbs() { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
125 base, err := url.Parse(caps.BaseURL) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
126 if err != nil { |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
127 return nil, err |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
128 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
129 getU = getU.ResolveReference(base) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
130 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
131 |
1616
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
132 if !op.SupportsOutputFormat(outputFormat) { |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
133 return nil, ErrOutputFormatNotSupported |
1616
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
134 } |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
135 |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
136 wfsVersion := caps.HighestWFSVersion(WFS2_0_0) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
137 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
138 featuresPerPage, supportsPaging := op.FeaturesPerPage() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
139 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
140 var numFeatures int |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
141 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
142 if supportsPaging { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
143 log.Printf("Paging supported with %d feature per page.\n", |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
144 featuresPerPage) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
145 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
146 if !op.SupportsHits() { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
147 supportsPaging = false |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
148 } else { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
149 numFeatures, err = numberFeaturesGET(getU, featureTypeName, wfsVersion) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
150 if err != nil { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
151 log.Printf("error: %v\n", err) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
152 supportsPaging = false |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
153 } else { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
154 log.Printf("Number of features: %d\n", numFeatures) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
155 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
156 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
157 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
158 |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
159 var downloadURLs []string |
1614
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
160 wfs2 := !versionIsLess(wfsVersion, WFS2_0_0) |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
161 |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
162 addNS := func(v url.Values) { |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
163 if len(feature.Namespaces) == 0 { |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
164 return |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
165 } |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
166 // Only use first namespace |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
167 ns := feature.Namespaces[0] |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
168 if wfs2 { |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
169 v.Set("NAMESPACES", fmt.Sprintf("(%s,%s)", ns.Space, ns.Local)) |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
170 } else { |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
171 v.Set("NAMESPACE", fmt.Sprintf("(%s:%s)", ns.Space, ns.Local)) |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
172 } |
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
173 } |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
174 |
1616
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
175 addOutputFormat := func(v url.Values) { |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
176 if outputFormat != "" { |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
177 v.Set("outputFormat", outputFormat) |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
178 } |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
179 } |
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
180 |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
181 addSortBy := func(v url.Values) { |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
182 if sortBy != "" { |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
183 v.Set("sortBy", sortBy) |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
184 } |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
185 } |
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
186 |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
187 if supportsPaging { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
188 pagedURL := func(ofs, count int) string { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
189 v := url.Values{} |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
190 v.Set("SERVICE", "WFS") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
191 v.Set("REQUEST", "GetFeature") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
192 v.Set("VERSION", wfsVersion) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
193 v.Set("startIndex", strconv.Itoa(ofs)) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
194 if wfs2 { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
195 v.Set("count", strconv.Itoa(count)) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
196 } else { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
197 v.Set("maxFeatures", strconv.Itoa(count)) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
198 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
199 v.Set("TYPENAMES", featureTypeName) |
1614
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
200 addNS(v) |
1616
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
201 addOutputFormat(v) |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
202 addSortBy(v) |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
203 q := *getU |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
204 q.RawQuery = v.Encode() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
205 return q.String() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
206 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
207 if numFeatures <= featuresPerPage { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
208 log.Println("All features can be fetched in one page") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
209 downloadURLs = []string{pagedURL(0, numFeatures)} |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
210 } else { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
211 log.Println("Features need to be downloaded in pages.") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
212 for pos := 0; pos < numFeatures; { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
213 var count int |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
214 if rest := numFeatures - pos; rest >= numFeatures { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
215 count = numFeatures |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
216 } else { |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
217 count = rest |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
218 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
219 downloadURLs = append(downloadURLs, pagedURL(pos, count)) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
220 pos += count |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
221 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
222 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
223 } else { // No paging support. |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
224 v := url.Values{} |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
225 v.Set("SERVICE", "WFS") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
226 v.Set("REQUEST", "GetFeature") |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
227 v.Set("VERSION", wfsVersion) |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
228 v.Set("TYPENAMES", featureTypeName) |
1614
efc409e330a6
WFS downloader: Propagate namespaces of feature type into GetFeature request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1608
diff
changeset
|
229 addNS(v) |
1616
f59550310143
WFS downloader: Support different feature output formats.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1614
diff
changeset
|
230 addOutputFormat(v) |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
231 addSortBy(v) |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
232 q := *getU |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
233 q.RawQuery = v.Encode() |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
234 downloadURLs = []string{q.String()} |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
235 } |
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
236 |
1619
3093bab05c81
WFS downloader: Dump features to stdout for testing purposes.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1616
diff
changeset
|
237 return downloadURLs, nil |
1608
427f9010b4a9
WFS download: Started with GET downloader (paged and unpaged).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
238 } |
1624
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
239 |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
240 func downloadURL(url string, handler func(io.Reader) error) error { |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
241 resp, err := http.Get(url) |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
242 if err != nil { |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
243 return err |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
244 } |
1626
92da44ba610c
WFS downloader: Parse into raw feature collections.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1624
diff
changeset
|
245 if resp.StatusCode < 200 || resp.StatusCode > 299 { |
92da44ba610c
WFS downloader: Parse into raw feature collections.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1624
diff
changeset
|
246 return fmt.Errorf("Invalid HTTP status code: %d (%s)", |
92da44ba610c
WFS downloader: Parse into raw feature collections.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1624
diff
changeset
|
247 resp.StatusCode, resp.Status) |
92da44ba610c
WFS downloader: Parse into raw feature collections.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1624
diff
changeset
|
248 } |
1624
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
249 defer resp.Body.Close() |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
250 return handler(resp.Body) |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
251 } |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
252 |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
253 func DownloadURLs(urls []string, handler func(io.Reader) error) error { |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
254 for _, url := range urls { |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
255 if err := downloadURL(url, handler); err != nil { |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
256 return nil |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
257 } |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
258 } |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
259 return nil |
943823d03d50
WFS downloader: Started with mapping return features to Go structs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1619
diff
changeset
|
260 } |