Mercurial > gemma

annotate pkg/controllers/srimports.go @ 5490:5f47eeea988d logging

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Use own logging package.
author Sascha L. Teichmann <sascha.teichmann@intevation.de>
date Mon, 20 Sep 2021 17:45:39 +0200
parents 850f5847d18a
children 6270951dda28
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1017
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1010
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1010
diff changeset
2 // without warranty, see README.md and license for details.
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1010
diff changeset
3 //
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1010
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1010
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1010
diff changeset
6 //
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1010
diff changeset
7 // Copyright (C) 2018 by via donau
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1010
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1010
diff changeset
9 // Software engineering by Intevation GmbH
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1010
diff changeset
10 //
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1010
diff changeset
11 // Author(s):
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1010
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1010
diff changeset
13
978
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14 package controllers
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import (
1225
4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1224
diff changeset
17 "archive/zip"
1221
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
18 "encoding/hex"
1226
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
19 "fmt"
978
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
20 "net/http"
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
21 "os"
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
22 "path/filepath"
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
23 "strconv"
3585
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2995
diff changeset
24 "strings"
1226
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
25 "sync"
1221
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
26 "time"
978
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
27
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
28 "github.com/gorilla/mux"
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
29
979
7934b5c1a910 Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 978
diff changeset
30 "gemma.intevation.de/gemma/pkg/auth"
1226
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
31 "gemma.intevation.de/gemma/pkg/common"
978
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
32 "gemma.intevation.de/gemma/pkg/config"
979
7934b5c1a910 Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 978
diff changeset
33 "gemma.intevation.de/gemma/pkg/imports"
5490
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5407
diff changeset
34 "gemma.intevation.de/gemma/pkg/log"
1221
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
35 "gemma.intevation.de/gemma/pkg/misc"
1226
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
36 "gemma.intevation.de/gemma/pkg/models"
4244
4394daeea96a Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4243
diff changeset
37
4394daeea96a Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4243
diff changeset
38 mw "gemma.intevation.de/gemma/pkg/middleware"
978
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
39 )
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
40
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
41 const (
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2075
diff changeset
42 soundingResultName = "soundingresult"
4875
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4244
diff changeset
43 maxSoundingResultSize = 50 * 1024 * 1024
978
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
44 )
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
45
1224
bc4b642c8d04 Started with an upload sounding result to temp upload area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1221
diff changeset
46 func fetchSoundingResult(req *http.Request) (string, error) {
978
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
47
1221
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
48 // Check first if we have a token.
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
49 if token := req.FormValue("token"); token != "" {
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
50 if _, err := hex.DecodeString(token); err != nil {
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
51 return "", err
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
52 }
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
53 dir := config.TmpDir()
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
54 if dir == "" {
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
55 dir = os.TempDir()
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
56 }
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
57 // XXX: This should hopefully be race-free enough.
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
58 now := time.Now().Format("2006-15-04-05")
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
59 dst := filepath.Join(dir, soundingResultName+"-"+token+"-"+now)
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
60 if err := misc.UnmakeTempFile(token, dst); err != nil {
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
61 return "", err
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
62 }
1241
a45fa8943254 Sounding result import: Forgot to return the token temp file to the importer. Doh!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1239
diff changeset
63 return dst, nil
1221
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
64 }
c193649d4f11 Add an area for temp uploads on the server to be addressed
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1203
diff changeset
65
4875
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4244
diff changeset
66 return misc.StoreUploadedFileCheck(
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2075
diff changeset
67 req,
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2075
diff changeset
68 soundingResultName,
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2075
diff changeset
69 "sr.zip",
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2075
diff changeset
70 maxSoundingResultSize,
4875
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4244
diff changeset
71 true,
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2075
diff changeset
72 )
978
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
73 }
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
74
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
75 func fetchSoundingResultMetaOverrides(sr *imports.SoundingResult, req *http.Request) error {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
76
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
77 if v := req.FormValue("epsg"); v != "" {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
78 epsg, err := strconv.ParseUint(v, 10, 32)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
79 if err != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
80 return err
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
81 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
82 srid := uint(epsg)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
83 sr.EPSG = &srid
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
84 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
85
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
86 if v := req.FormValue("date"); v != "" {
2059
ae0021feaac8 Imports: Made stretch import marshable (even if it is not needed).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1985
diff changeset
87 date, err := time.Parse(common.DateFormat, v)
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
88 if err != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
89 return err
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
90 }
1900
6a67cd819e93 To prepare stretch import made some model data types re-usable.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1787
diff changeset
91 sr.Date = &models.Date{Time: date}
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
92 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
93
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
94 if v := req.FormValue("depth-reference"); v != "" {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
95 sr.DepthReference = &v
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
96 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
97
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
98 if v := req.FormValue("bottleneck"); v != "" {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
99 sr.Bottleneck = &v
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
100 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
101
3945
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
102 if v := req.FormValue("negate-z"); v != "" {
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
103 var negateZ bool
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
104 switch strings.ToLower(v) {
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
105 case "true", "1":
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
106 negateZ = true
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
107 case "false", "0":
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
108 negateZ = false
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
109 default:
3976
c412dff6e1da Fixed cases for error strings.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3946
diff changeset
110 return fmt.Errorf("unknown negate-z '%s'", v)
3945
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
111 }
3946
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
112 sr.NegateZ = &negateZ
3945
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
113 }
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
114
5407
850f5847d18a Re-establish compat with old single-beam interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5403
diff changeset
115 // Kept this in for compat.
850f5847d18a Re-establish compat with old single-beam interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5403
diff changeset
116 if v := req.FormValue("single-beam"); v != "" {
850f5847d18a Re-establish compat with old single-beam interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5403
diff changeset
117 var surveyType models.SurveyType
850f5847d18a Re-establish compat with old single-beam interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5403
diff changeset
118 switch strings.ToLower(v) {
850f5847d18a Re-establish compat with old single-beam interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5403
diff changeset
119 case "true", "1", "singlebeam", "single-beam":
850f5847d18a Re-establish compat with old single-beam interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5403
diff changeset
120 surveyType = models.SurveyTypeSingleBeam
850f5847d18a Re-establish compat with old single-beam interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5403
diff changeset
121 case "false", "0", "multibeam", "multi-beam":
850f5847d18a Re-establish compat with old single-beam interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5403
diff changeset
122 surveyType = models.SurveyTypeMultiBeam
850f5847d18a Re-establish compat with old single-beam interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5403
diff changeset
123 default:
850f5847d18a Re-establish compat with old single-beam interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5403
diff changeset
124 return fmt.Errorf("unknown single-beam '%s'", v)
850f5847d18a Re-establish compat with old single-beam interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5403
diff changeset
125 }
850f5847d18a Re-establish compat with old single-beam interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5403
diff changeset
126 sr.SurveyType = &surveyType
850f5847d18a Re-establish compat with old single-beam interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5403
diff changeset
127 }
850f5847d18a Re-establish compat with old single-beam interface.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5403
diff changeset
128
5403
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
129 if v := req.FormValue("survey-type"); v != "" {
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
130 var surveyType models.SurveyType
3585
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2995
diff changeset
131 switch strings.ToLower(v) {
5403
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
132 case "2", "marking":
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
133 surveyType = models.SurveyTypeMarking
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
134 case "true", "1", "singlebeam", "single-beam", "single":
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
135 surveyType = models.SurveyTypeSingleBeam
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
136 case "false", "0", "multibeam", "multi-beam", "multi":
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
137 surveyType = models.SurveyTypeMultiBeam
3585
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2995
diff changeset
138 default:
5403
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
139 return fmt.Errorf("unknown survey-type '%s'", v)
3585
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2995
diff changeset
140 }
5403
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
141 sr.SurveyType = &surveyType
3585
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2995
diff changeset
142 }
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2995
diff changeset
143
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
144 return nil
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
145 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
146
978
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
147 func importSoundingResult(rw http.ResponseWriter, req *http.Request) {
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
148
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
149 sr := new(imports.SoundingResult)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
150
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
151 if err := fetchSoundingResultMetaOverrides(sr, req); err != nil {
5490
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5407
diff changeset
152 log.Errorf("%v\n", err)
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
153 http.Error(rw, "error: "+err.Error(), http.StatusBadRequest)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
154 return
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
155 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
156
1224
bc4b642c8d04 Started with an upload sounding result to temp upload area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1221
diff changeset
157 dir, err := fetchSoundingResult(req)
978
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
158 if err != nil {
5490
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5407
diff changeset
159 log.Errorf("%v\n", err)
978
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
160 http.Error(rw, "error: "+err.Error(), http.StatusInternalServerError)
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
161 return
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
162 }
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
163 sr.Dir = dir
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
164
1531
24445a618513 Added stub for bottleneck importer.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1528
diff changeset
165 serialized, err := common.ToJSONString(sr)
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
166 if err != nil {
5490
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5407
diff changeset
167 log.Errorf("%v\n", err)
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
168 http.Error(rw, "error: "+err.Error(), http.StatusInternalServerError)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
169 return
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1228
diff changeset
170 }
978
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
171
979
7934b5c1a910 Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 978
diff changeset
172 session, _ := auth.GetSession(req)
7934b5c1a910 Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 978
diff changeset
173
2995
5222bfe5b4af Upload imports: spell form field 'send-email' instead of 'email'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2194
diff changeset
174 sendEmail := req.FormValue("send-email") != ""
1646
a0982c38eac0 Import queue: Implemented email notifications.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1642
diff changeset
175
1642
49c04bb64e0a Import queue: Implemented auto-accept and email sending.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1531
diff changeset
176 jobID, err := imports.AddJob(
49c04bb64e0a Import queue: Implemented auto-accept and email sending.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1531
diff changeset
177 imports.SRJobKind,
2075
e7f3199384ec Sounding results import: Removed senseless parameters.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2059
diff changeset
178 time.Time{}, // due
e7f3199384ec Sounding results import: Removed senseless parameters.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2059
diff changeset
179 nil, // trys
e7f3199384ec Sounding results import: Removed senseless parameters.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2059
diff changeset
180 nil, // retry wait
1642
49c04bb64e0a Import queue: Implemented auto-accept and email sending.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1531
diff changeset
181 session.User,
1754
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
182 sendEmail,
1642
49c04bb64e0a Import queue: Implemented auto-accept and email sending.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1531
diff changeset
183 serialized)
49c04bb64e0a Import queue: Implemented auto-accept and email sending.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1531
diff changeset
184
998
75e65599ea52 Persist job queue in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
185 if err != nil {
5490
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5407
diff changeset
186 log.Errorf("%v\n", err)
998
75e65599ea52 Persist job queue in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
187 http.Error(rw, "error: "+err.Error(), http.StatusInternalServerError)
75e65599ea52 Persist job queue in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
188 return
75e65599ea52 Persist job queue in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
189 }
75e65599ea52 Persist job queue in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
190
5490
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5407
diff changeset
191 log.Infof("added import #%d to queue\n", jobID)
979
7934b5c1a910 Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 978
diff changeset
192
7934b5c1a910 Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 978
diff changeset
193 result := struct {
7934b5c1a910 Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 978
diff changeset
194 ID int64 `json:"id"`
7934b5c1a910 Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 978
diff changeset
195 }{
7934b5c1a910 Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 978
diff changeset
196 ID: jobID,
7934b5c1a910 Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 978
diff changeset
197 }
4244
4394daeea96a Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4243
diff changeset
198 mw.SendJSON(rw, http.StatusCreated, &result)
978
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
199 }
1224
bc4b642c8d04 Started with an upload sounding result to temp upload area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1221
diff changeset
200
1226
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
201 func loadMeta(f *zip.File) (*models.SoundingResultMeta, error) {
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
202 r, err := f.Open()
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
203 if err != nil {
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
204 return nil, err
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
205 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
206 defer r.Close()
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
207 var m models.SoundingResultMeta
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
208 return &m, m.Decode(r)
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
209 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
210
4244
4394daeea96a Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4243
diff changeset
211 func uploadSoundingResult(req *http.Request) (jr mw.JSONResult, err error) {
1224
bc4b642c8d04 Started with an upload sounding result to temp upload area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1221
diff changeset
212
bc4b642c8d04 Started with an upload sounding result to temp upload area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1221
diff changeset
213 var dir string
4875
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4244
diff changeset
214 if dir, err = misc.StoreUploadedFileCheck(
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2075
diff changeset
215 req,
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2075
diff changeset
216 soundingResultName,
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2075
diff changeset
217 "sr.zip",
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2075
diff changeset
218 maxSoundingResultSize,
4875
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4244
diff changeset
219 true,
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2075
diff changeset
220 ); err != nil {
1224
bc4b642c8d04 Started with an upload sounding result to temp upload area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1221
diff changeset
221 return
bc4b642c8d04 Started with an upload sounding result to temp upload area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1221
diff changeset
222 }
bc4b642c8d04 Started with an upload sounding result to temp upload area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1221
diff changeset
223
3751
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3713
diff changeset
224 var messages []string
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3713
diff changeset
225
1225
4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1224
diff changeset
226 srFile := filepath.Join(dir, "sr.zip")
4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1224
diff changeset
227
4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1224
diff changeset
228 var zr *zip.ReadCloser
1226
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
229 var once sync.Once
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
230 closeOnce := func() { zr.Close() }
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
231
4183
34623265eac1 Made 'golint' happy (again) with the controllers package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3976
diff changeset
232 var isZIP bool
3751
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3713
diff changeset
233 if zr, err = zip.OpenReader(srFile); err != nil {
3762
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
234 messages = append(messages, fmt.Sprintf("%v - "+
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
235 "Trying TXT file mode.", err))
3751
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3713
diff changeset
236 } else {
3762
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
237 isZIP = true
3751
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3713
diff changeset
238 defer once.Do(closeOnce)
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3713
diff changeset
239 }
1226
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
240
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
241 var result struct {
1227
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1226
diff changeset
242 Token string `json:"token,omitempty"`
1226
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
243 Meta interface{} `json:"meta,omitempty"`
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
244 Messages []string `json:"messages,omitempty"`
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
245 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
246
1528
5874cedd7f91 Sounding result import: Accept *.txt files for XYZ data, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
247 find := func(ext string) *zip.File { return common.FindInZIP(zr, ext) }
5874cedd7f91 Sounding result import: Accept *.txt files for XYZ data, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1328
diff changeset
248
3751
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3713
diff changeset
249 var noXYZ bool
3762
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
250 if isZIP {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
251 if zr != nil {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
252 noXYZ = find(".xyz") == nil && find(".txt") == nil
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
253 }
3751
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3713
diff changeset
254
3762
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
255 if noXYZ {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
256 messages = append(messages, "no .xyz or .txt file found.")
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
257 }
1225
4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1224
diff changeset
258
3762
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
259 if mj := find("meta.json"); mj == nil {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
260 messages = append(messages, "no 'meta.json' file found.")
1226
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
261 } else {
3762
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
262 if meta, err := loadMeta(mj); err != nil {
1226
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
263 messages = append(messages,
3762
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
264 fmt.Sprintf("'meta.json' found but invalid: %v", err))
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
265 } else {
4244
4394daeea96a Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4243
diff changeset
266 errs := meta.Validate(req.Context(), mw.JSONConn(req))
3762
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
267 for _, err := range errs {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
268 messages = append(messages,
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
269 fmt.Sprintf("invalid 'meta.json': %v", err))
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
270 }
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3752
diff changeset
271 result.Meta = meta
1226
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
272 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
273 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
274 }
3751
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3713
diff changeset
275
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3713
diff changeset
276 if zr != nil {
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3713
diff changeset
277 once.Do(closeOnce)
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3713
diff changeset
278 }
1224
bc4b642c8d04 Started with an upload sounding result to temp upload area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1221
diff changeset
279
1227
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1226
diff changeset
280 code := http.StatusCreated
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1226
diff changeset
281
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1226
diff changeset
282 // If there are no XYZ data we cant help the user anyway.
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1226
diff changeset
283 if noXYZ {
1228
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
284 code = http.StatusBadRequest
1227
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1226
diff changeset
285 if err2 := os.RemoveAll(dir); err2 != nil {
5490
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5407
diff changeset
286 log.Errorf("%v\n", err2)
1227
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1226
diff changeset
287 }
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1226
diff changeset
288 } else if result.Token, err = misc.MakeTempFile(dir); err != nil {
1226
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
289 if err2 := os.RemoveAll(dir); err2 != nil {
5490
5f47eeea988d Use own logging package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5407
diff changeset
290 log.Errorf("%v\n", err2)
1226
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
291 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
292 return
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
293 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
294
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
295 result.Messages = messages
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
296
4244
4394daeea96a Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4243
diff changeset
297 jr = mw.JSONResult{
1227
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1226
diff changeset
298 Code: code,
1226
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
299 Result: &result,
2e65e8ddacab Finished the sounding result upload controller to temp uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
300 }
1224
bc4b642c8d04 Started with an upload sounding result to temp upload area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1221
diff changeset
301 return
bc4b642c8d04 Started with an upload sounding result to temp upload area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1221
diff changeset
302 }
1228
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
303
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
304 func deleteSoundingUpload(rw http.ResponseWriter, req *http.Request) {
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
305 token := mux.Vars(req)["token"]
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
306 if _, err := hex.DecodeString(token); err != nil {
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
307 http.Error(rw, "Invalid token", http.StatusBadRequest)
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
308 return
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
309 }
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
310 if err := misc.DeleteTempFile(token); err != nil {
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
311 http.Error(rw, fmt.Sprintf("error: %v", err), http.StatusInternalServerError)
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
312 return
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
313 }
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
314 result := struct {
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
315 Message string `json:"message"`
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
316 }{
1297
1c0c9190fcf2 Import sounding result: If uploading a zip with a meta.json which has
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1241
diff changeset
317 Message: fmt.Sprintf("Token %s deleted.", token),
1228
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
318 }
4244
4394daeea96a Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4243
diff changeset
319 mw.SendJSON(rw, http.StatusOK, &result)
1228
17131f0f9fcb Added endpoint to explicitly delete a temp upload.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1227
diff changeset
320 }