Mercurial > gemma
annotate pkg/wfs/global.go @ 5710:37c8feeecb4d
Merged branch sr-v2 into default.
author | Sascha L. Teichmann <sascha.teichmann@intevation.de> |
---|---|
date | Tue, 20 Feb 2024 21:28:56 +0100 |
parents | 31973f6f5cca |
children |
rev | line source |
---|---|
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
1 // This is Free Software under GNU Affero General Public License v >= 3.0 |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
2 // without warranty, see README.md and license for details. |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
3 // |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
4 // SPDX-License-Identifier: AGPL-3.0-or-later |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
5 // License-Filename: LICENSES/AGPL-3.0.txt |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
6 // |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
7 // Copyright (C) 2019 by via donau |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
8 // – Österreichische Wasserstraßen-Gesellschaft mbH |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
9 // Software engineering by Intevation GmbH |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
10 // |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
11 // Author(s): |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de> |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
13 |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
14 package wfs |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
15 |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
16 import ( |
2711
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
17 "bufio" |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
18 "fmt" |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
19 "io" |
2711
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
20 "os" |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
21 "os/exec" |
2711
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
22 "path/filepath" |
5491
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
23 "sync" |
2711
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
24 |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
25 "gemma.intevation.de/gemma/pkg/config" |
5490
5f47eeea988d
Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4990
diff
changeset
|
26 "gemma.intevation.de/gemma/pkg/log" |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
27 ) |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
28 |
2702
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
29 var ( |
4166
04876d865528
Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2719
diff
changeset
|
30 // FormatGeoJSON is a list of MIME types of |
04876d865528
Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2719
diff
changeset
|
31 // GeoJSON documents. |
2702
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
32 FormatGeoJSON = []string{"geojson", "application/json"} |
4166
04876d865528
Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2719
diff
changeset
|
33 // FormatGML is a list of MIME types |
04876d865528
Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2719
diff
changeset
|
34 // GML documents. |
04876d865528
Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2719
diff
changeset
|
35 FormatGML = []string{ |
2711
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
36 "application/gml+xml; version=3.2", |
2702
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
37 "gml2", "gml3", "gml32", |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
38 "text/xml; subtype=gml/2.1.2", |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
39 "text/xml; subtype=gml/3.1.1", |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
40 "text/xml; subtype=gml/3.2", |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
41 } |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
42 ) |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
43 |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
44 type ( |
4166
04876d865528
Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2719
diff
changeset
|
45 // Downloader abstracts the idea of a download function for WFS. |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
46 Downloader interface { |
2719
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2711
diff
changeset
|
47 Download(user, password string, handler func(string, io.Reader) error) error |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
48 } |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
49 |
4166
04876d865528
Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2719
diff
changeset
|
50 // GeoJSONDownloader is a Downloader for GeoJSON. |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
51 GeoJSONDownloader []string |
4166
04876d865528
Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2719
diff
changeset
|
52 // GMLDownloader is a Downloader for GML. |
04876d865528
Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2719
diff
changeset
|
53 GMLDownloader []string |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
54 ) |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
55 |
5491
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
56 var ( |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
57 getFeaturesOnce sync.Once |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
58 getFeatures func(*Capabilities, string, string) (Downloader, error) |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
59 ) |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
60 |
5491
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
61 func getFeaturesOnceFunc() { |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
62 path, err := exec.LookPath("ogr2ogr") |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
63 if err != nil { |
5490
5f47eeea988d
Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4990
diff
changeset
|
64 log.Infoln("ogr2ogr not installed. Using direct GeoJSON WFS download.") |
5491
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
65 getFeatures = getFeaturesGeoJSON |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
66 } else { |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
67 log.Infof("ogr2ogr found at %s. Using GML WFS download.\n", path) |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
68 getFeatures = getFeaturesGML |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
69 } |
5491
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
70 } |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
71 |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
72 // GetFeatures is the default Downloader in this Gemma server. |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
73 func GetFeatures( |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
74 caps *Capabilities, |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
75 featureTypeName string, |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
76 sortBy string, |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
77 ) (Downloader, error) { |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
78 getFeaturesOnce.Do(getFeaturesOnceFunc) |
09346efa7f69
Do ogr2ogr check only once when it's needed the first time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5490
diff
changeset
|
79 return getFeatures(caps, featureTypeName, sortBy) |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
80 } |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
81 |
4166
04876d865528
Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2719
diff
changeset
|
82 func getFeaturesGeoJSON( |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
83 caps *Capabilities, |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
84 featureTypeName string, |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
85 sortBy string, |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
86 ) (Downloader, error) { |
2702
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
87 urls, err := GetFeaturesGET( |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
88 caps, |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
89 featureTypeName, |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
90 FormatGeoJSON, |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
91 sortBy) |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
92 return GeoJSONDownloader(urls), err |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
93 } |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
94 |
4166
04876d865528
Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2719
diff
changeset
|
95 func getFeaturesGML( |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
96 caps *Capabilities, |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
97 featureTypeName string, |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
98 sortBy string, |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
99 ) (Downloader, error) { |
2702
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
100 urls, err := GetFeaturesGET( |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
101 caps, |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
102 featureTypeName, |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
103 FormatGML, |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
104 sortBy) |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
105 return GMLDownloader(urls), err |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
106 } |
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
107 |
4166
04876d865528
Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2719
diff
changeset
|
108 // Download is the GeoJSON implementation of the Downloader. |
2719
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2711
diff
changeset
|
109 func (gjd GeoJSONDownloader) Download(user, password string, handler func(string, io.Reader) error) error { |
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2711
diff
changeset
|
110 return DownloadURLs(user, password, []string(gjd), handler) |
2702
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
111 } |
d7ef169fd0d3
WFS downloader: More on downloading GML. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2634
diff
changeset
|
112 |
2711
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
113 func places(n int) int { |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
114 places := 1 |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
115 if n < 0 { |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
116 places++ |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
117 n = -n |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
118 } |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
119 for value := 10; n >= value; value *= 10 { |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
120 places++ |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
121 } |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
122 return places |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
123 } |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
124 |
4166
04876d865528
Made 'golint' and 'staticcheck' happy with wfs package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2719
diff
changeset
|
125 // Download is GML implementaion of the Downloader. |
2719
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2711
diff
changeset
|
126 func (gmd GMLDownloader) Download(user, password string, handler func(string, io.Reader) error) error { |
2711
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
127 |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
128 if len(gmd) == 0 { |
4990
16259efa828f
Dont try to download from WFS if they dont have features.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4166
diff
changeset
|
129 return nil |
2711
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
130 } |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
131 |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
132 tmpDir := config.TmpDir() |
5683
31973f6f5cca
Get rid of deprecation warnings from staticcheck.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5491
diff
changeset
|
133 dlDir, err := os.MkdirTemp(tmpDir, "wfs-downloads") |
2711
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
134 if err != nil { |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
135 return err |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
136 } |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
137 defer os.RemoveAll(dlDir) |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
138 digits := places(len(gmd)) |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
139 |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
140 var files []string |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
141 |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
142 for i, url := range gmd { |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
143 fname := filepath.Join(dlDir, fmt.Sprintf("%0*d.gml", digits, i)) |
2719
2b6f73c30016
WFS downloader: Add BasicAuth support for downloading. TODO: Use it.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2711
diff
changeset
|
144 if err := downloadURL(user, password, url, func(_ string, r io.Reader) error { |
2711
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
145 f, err := os.Create(fname) |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
146 if err != nil { |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
147 return err |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
148 } |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
149 if _, err = io.Copy(f, r); err != nil { |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
150 f.Close() |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
151 return err |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
152 } |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
153 return f.Close() |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
154 }); err != nil { |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
155 return err |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
156 } |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
157 files = append(files, fname) |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
158 } |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
159 |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
160 out := filepath.Join(dlDir, "out.geojson") |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
161 |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
162 cmd := exec.Command("ogr2ogr", append([]string{"-f", "GeoJSON", out}, files...)...) |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
163 |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
164 msg, err := cmd.CombinedOutput() |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
165 if err != nil { |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
166 return fmt.Errorf("ogr2ogr failed: %s", msg) |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
167 } |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
168 |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
169 in, err := os.Open(out) |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
170 if err != nil { |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
171 return err |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
172 } |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
173 defer in.Close() |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
174 |
3956de9b6b32
WFS downloader: Implemented GML download to file, ogr2ogr to GeoJSON.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2702
diff
changeset
|
175 return handler(gmd[0], bufio.NewReader(in)) |
2634
49ce3c11ca72
WFS downloads: Started with GML based downloads based on installation of ogr2ogr.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
176 } |