Mercurial > gemma

annotate pkg/controllers/wpimports.go @ 2317:8a8680e70d2e

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Cleanup schema for reference water levels Remove duplicate lookup table and leave an out-commented foreign key constraint to make clear what should be referenced. The removed comment in surveys.go mostly became obsolete and was otherwise trying to explain SQL basics.
author Tom Gottfried <tom@intevation.de>
date Mon, 18 Feb 2019 18:45:12 +0100
parents 4d6979dedb11
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2073
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
3 //
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
6 //
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 //
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 // Author(s):
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
13
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14 package controllers
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import (
2094
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
17 "fmt"
2073
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 "log"
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 "net/http"
2094
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
20 "strconv"
2073
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
21 "time"
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
22
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
23 "gemma.intevation.de/gemma/pkg/auth"
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
24 "gemma.intevation.de/gemma/pkg/common"
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
25 "gemma.intevation.de/gemma/pkg/imports"
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2094
diff changeset
26 "gemma.intevation.de/gemma/pkg/misc"
2073
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
27 )
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
28
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
29 const (
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
30 maxWaterwayProfilesSize = 25 * 1024 * 1024
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
31 waterwayProfilesName = "wp"
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
32 )
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
33
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
34 func importWaterwayProfiles(rw http.ResponseWriter, req *http.Request) {
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
35
2086
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
36 url := req.FormValue("url")
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
37 if url == "" {
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
38 http.Error(rw, "missing 'url' parameter", http.StatusBadRequest)
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
39 return
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
40 }
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
41
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
42 featureType := req.FormValue("feature-type")
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
43 if featureType == "" {
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
44 http.Error(rw, "missing 'feature-type' parameter", http.StatusBadRequest)
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
45 return
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
46 }
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
47
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2094
diff changeset
48 dir, err := misc.StoreUploadedFile(
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2094
diff changeset
49 req,
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2094
diff changeset
50 waterwayProfilesName,
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2094
diff changeset
51 "wp.csv",
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2094
diff changeset
52 maxWaterwayProfilesSize)
2073
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
53 if err != nil {
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
54 log.Printf("error: %v\n", err)
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
55 http.Error(rw, "error: "+err.Error(), http.StatusInternalServerError)
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
56 return
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
57 }
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
58
2086
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
59 sortBy := req.FormValue("sort-by")
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
60
2094
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
61 var precision *float64
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
62 if p := req.FormValue("precision"); p != "" {
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
63 if v, err := strconv.ParseFloat(p, 64); err != nil {
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
64 http.Error(rw,
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
65 fmt.Sprintf("Invalid 'precision' parameter: %v", err),
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
66 http.StatusBadRequest)
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
67 return
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
68 } else {
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
69 precision = &v
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
70 }
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
71 }
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
72
2086
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
73 wp := &imports.WaterwayProfiles{
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
74 Dir: dir,
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
75 URL: url,
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
76 FeatureType: featureType,
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
77 SortBy: sortBy,
2094
7a3378c14397 Waterway profiles import: Added a precision parameter in meters to find lines for points (default 10m).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2086
diff changeset
78 Precision: precision,
2086
6096ec4951f8 Waterway profiles imports: Download the geometries from WPS and join them with the CSV data.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2078
diff changeset
79 }
2073
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
80
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
81 serialized, err := common.ToJSONString(wp)
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
82 if err != nil {
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
83 log.Printf("error: %v\n", err)
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
84 http.Error(rw, "error: "+err.Error(), http.StatusInternalServerError)
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
85 return
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
86 }
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
87
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
88 session, _ := auth.GetSession(req)
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
89
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
90 sendEmail := req.FormValue("email") != ""
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
91
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
92 jobID, err := imports.AddJob(
2078
dacf79a0658e Waterway profile import: Parse header line. TODO: Parse the body and store the data into the database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2073
diff changeset
93 imports.WPJobKind,
2073
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
94 time.Time{}, // due
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
95 nil, // trys
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
96 nil, // wait retry
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
97 session.User,
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
98 sendEmail,
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
99 serialized)
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
100
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
101 if err != nil {
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
102 log.Printf("error: %v\n", err)
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
103 http.Error(rw, "error: "+err.Error(), http.StatusInternalServerError)
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
104 return
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
105 }
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
106
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
107 log.Printf("info: added import #%d to queue\n", jobID)
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
108
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
109 result := struct {
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
110 ID int64 `json:"id"`
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
111 }{
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
112 ID: jobID,
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
113 }
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
114 SendJSON(rw, http.StatusCreated, &result)
e6dccc7a3ea1 Waterway profiles import: Added upload route.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
115 }