gemma: pkg/controllers/srimports.go annotate

annotate pkg/controllers/srimports.go @ 5591:0011f50cf216 surveysperbottleneckid

Removed no longer used alternative api for surveys/ endpoint. As bottlenecks in the summary for SR imports are now identified by their id and no longer by the (not guarantied to be unique!) name, there is no longer the need to request survey data by the name+date tuple (which isn't reliable anyway). So the workaround was now reversed.

author	Sascha Wilde <wilde@sha-bang.de>
date	Wed, 06 Apr 2022 13:30:29 +0200
parents	5f47eeea988d
children	6270951dda28

rev	line source
1017 a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	1 // This is Free Software under GNU Affero General Public License v >= 3.0
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	2 // without warranty, see README.md and license for details.
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	3 //
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	4 // SPDX-License-Identifier: AGPL-3.0-or-later
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	5 // License-Filename: LICENSES/AGPL-3.0.txt
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	6 //
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	7 // Copyright (C) 2018 by via donau
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	8 // – Österreichische Wasserstraßen-Gesellschaft mbH
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	9 // Software engineering by Intevation GmbH
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	10 //
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	11 // Author(s):
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	13
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14 package controllers
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16 import (
1225 4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1224 diff changeset	17 "archive/zip"
1221 c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	18 "encoding/hex"
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	19 "fmt"
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	20 "net/http"
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	21 "os"
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	22 "path/filepath"
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	23 "strconv"
3585 479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	24 "strings"
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	25 "sync"
1221 c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	26 "time"
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	27
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	28 "github.com/gorilla/mux"
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	29
979 7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	30 "gemma.intevation.de/gemma/pkg/auth"
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	31 "gemma.intevation.de/gemma/pkg/common"
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	32 "gemma.intevation.de/gemma/pkg/config"
979 7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	33 "gemma.intevation.de/gemma/pkg/imports"
5490 5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5407 diff changeset	34 "gemma.intevation.de/gemma/pkg/log"
1221 c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	35 "gemma.intevation.de/gemma/pkg/misc"
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	36 "gemma.intevation.de/gemma/pkg/models"
4244 4394daeea96a Moved JSONHandler into middleware package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4243 diff changeset	37
4394daeea96a Moved JSONHandler into middleware package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4243 diff changeset	38 mw "gemma.intevation.de/gemma/pkg/middleware"
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	39 )
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	40
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	41 const (
2194 4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	42 soundingResultName = "soundingresult"
4875 6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4244 diff changeset	43 maxSoundingResultSize = 50 * 1024 * 1024
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	44 )
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	45
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	46 func fetchSoundingResult(req *http.Request) (string, error) {
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	47
1221 c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	48 // Check first if we have a token.
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	49 if token := req.FormValue("token"); token != "" {
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	50 if _, err := hex.DecodeString(token); err != nil {
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	51 return "", err
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	52 }
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	53 dir := config.TmpDir()
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	54 if dir == "" {
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	55 dir = os.TempDir()
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	56 }
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	57 // XXX: This should hopefully be race-free enough.
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	58 now := time.Now().Format("2006-15-04-05")
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	59 dst := filepath.Join(dir, soundingResultName+"-"+token+"-"+now)
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	60 if err := misc.UnmakeTempFile(token, dst); err != nil {
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	61 return "", err
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	62 }
1241 a45fa8943254 Sounding result import: Forgot to return the token temp file to the importer. Doh! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1239 diff changeset	63 return dst, nil
1221 c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	64 }
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	65
4875 6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4244 diff changeset	66 return misc.StoreUploadedFileCheck(
2194 4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	67 req,
4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	68 soundingResultName,
4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	69 "sr.zip",
4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	70 maxSoundingResultSize,
4875 6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4244 diff changeset	71 true,
2194 4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	72 )
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	73 }
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	74
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	75 func fetchSoundingResultMetaOverrides(sr imports.SoundingResult, req http.Request) error {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	76
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	77 if v := req.FormValue("epsg"); v != "" {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	78 epsg, err := strconv.ParseUint(v, 10, 32)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	79 if err != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	80 return err
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	81 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	82 srid := uint(epsg)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	83 sr.EPSG = &srid
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	84 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	85
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	86 if v := req.FormValue("date"); v != "" {
2059 ae0021feaac8 Imports: Made stretch import marshable (even if it is not needed). Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1985 diff changeset	87 date, err := time.Parse(common.DateFormat, v)
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	88 if err != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	89 return err
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	90 }
1900 6a67cd819e93 To prepare stretch import made some model data types re-usable. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1787 diff changeset	91 sr.Date = &models.Date{Time: date}
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	92 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	93
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	94 if v := req.FormValue("depth-reference"); v != "" {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	95 sr.DepthReference = &v
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	96 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	97
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	98 if v := req.FormValue("bottleneck"); v != "" {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	99 sr.Bottleneck = &v
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	100 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	101
3945 3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	102 if v := req.FormValue("negate-z"); v != "" {
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	103 var negateZ bool
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	104 switch strings.ToLower(v) {
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	105 case "true", "1":
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	106 negateZ = true
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	107 case "false", "0":
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	108 negateZ = false
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	109 default:
3976 c412dff6e1da Fixed cases for error strings. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3946 diff changeset	110 return fmt.Errorf("unknown negate-z '%s'", v)
3945 3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	111 }
3946 d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3945 diff changeset	112 sr.NegateZ = &negateZ
3945 3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	113 }
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	114
5407 850f5847d18a Re-establish compat with old single-beam interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5403 diff changeset	115 // Kept this in for compat.
850f5847d18a Re-establish compat with old single-beam interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5403 diff changeset	116 if v := req.FormValue("single-beam"); v != "" {
850f5847d18a Re-establish compat with old single-beam interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5403 diff changeset	117 var surveyType models.SurveyType
850f5847d18a Re-establish compat with old single-beam interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5403 diff changeset	118 switch strings.ToLower(v) {
850f5847d18a Re-establish compat with old single-beam interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5403 diff changeset	119 case "true", "1", "singlebeam", "single-beam":
850f5847d18a Re-establish compat with old single-beam interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5403 diff changeset	120 surveyType = models.SurveyTypeSingleBeam
850f5847d18a Re-establish compat with old single-beam interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5403 diff changeset	121 case "false", "0", "multibeam", "multi-beam":
850f5847d18a Re-establish compat with old single-beam interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5403 diff changeset	122 surveyType = models.SurveyTypeMultiBeam
850f5847d18a Re-establish compat with old single-beam interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5403 diff changeset	123 default:
850f5847d18a Re-establish compat with old single-beam interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5403 diff changeset	124 return fmt.Errorf("unknown single-beam '%s'", v)
850f5847d18a Re-establish compat with old single-beam interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5403 diff changeset	125 }
850f5847d18a Re-establish compat with old single-beam interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5403 diff changeset	126 sr.SurveyType = &surveyType
850f5847d18a Re-establish compat with old single-beam interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5403 diff changeset	127 }
850f5847d18a Re-establish compat with old single-beam interface. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5403 diff changeset	128
5403 85f19e924a43 Adjusted the import model to be able to handle marking single beam scans. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4875 diff changeset	129 if v := req.FormValue("survey-type"); v != "" {
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4875 diff changeset	130 var surveyType models.SurveyType
3585 479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	131 switch strings.ToLower(v) {
5403 85f19e924a43 Adjusted the import model to be able to handle marking single beam scans. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4875 diff changeset	132 case "2", "marking":
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4875 diff changeset	133 surveyType = models.SurveyTypeMarking
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4875 diff changeset	134 case "true", "1", "singlebeam", "single-beam", "single":
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4875 diff changeset	135 surveyType = models.SurveyTypeSingleBeam
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4875 diff changeset	136 case "false", "0", "multibeam", "multi-beam", "multi":
85f19e924a43 Adjusted the import model to be able to handle marking single beam scans. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4875 diff changeset	137 surveyType = models.SurveyTypeMultiBeam
3585 479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	138 default:
5403 85f19e924a43 Adjusted the import model to be able to handle marking single beam scans. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4875 diff changeset	139 return fmt.Errorf("unknown survey-type '%s'", v)
3585 479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	140 }
5403 85f19e924a43 Adjusted the import model to be able to handle marking single beam scans. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4875 diff changeset	141 sr.SurveyType = &surveyType
3585 479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	142 }
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	143
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	144 return nil
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	145 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	146
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	147 func importSoundingResult(rw http.ResponseWriter, req *http.Request) {
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	148
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	149 sr := new(imports.SoundingResult)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	150
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	151 if err := fetchSoundingResultMetaOverrides(sr, req); err != nil {
5490 5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5407 diff changeset	152 log.Errorf("%v\n", err)
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	153 http.Error(rw, "error: "+err.Error(), http.StatusBadRequest)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	154 return
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	155 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	156
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	157 dir, err := fetchSoundingResult(req)
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	158 if err != nil {
5490 5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5407 diff changeset	159 log.Errorf("%v\n", err)
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	160 http.Error(rw, "error: "+err.Error(), http.StatusInternalServerError)
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	161 return
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	162 }
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	163 sr.Dir = dir
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	164
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1528 diff changeset	165 serialized, err := common.ToJSONString(sr)
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	166 if err != nil {
5490 5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5407 diff changeset	167 log.Errorf("%v\n", err)
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	168 http.Error(rw, "error: "+err.Error(), http.StatusInternalServerError)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	169 return
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	170 }
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	171
979 7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	172 session, _ := auth.GetSession(req)
7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	173
2995 5222bfe5b4af Upload imports: spell form field 'send-email' instead of 'email'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2194 diff changeset	174 sendEmail := req.FormValue("send-email") != ""
1646 a0982c38eac0 Import queue: Implemented email notifications. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1642 diff changeset	175
1642 49c04bb64e0a Import queue: Implemented auto-accept and email sending. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1531 diff changeset	176 jobID, err := imports.AddJob(
49c04bb64e0a Import queue: Implemented auto-accept and email sending. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1531 diff changeset	177 imports.SRJobKind,
2075 e7f3199384ec Sounding results import: Removed senseless parameters. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2059 diff changeset	178 time.Time{}, // due
e7f3199384ec Sounding results import: Removed senseless parameters. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2059 diff changeset	179 nil, // trys
e7f3199384ec Sounding results import: Removed senseless parameters. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2059 diff changeset	180 nil, // retry wait
1642 49c04bb64e0a Import queue: Implemented auto-accept and email sending. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1531 diff changeset	181 session.User,
1754 807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1708 diff changeset	182 sendEmail,
1642 49c04bb64e0a Import queue: Implemented auto-accept and email sending. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1531 diff changeset	183 serialized)
49c04bb64e0a Import queue: Implemented auto-accept and email sending. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1531 diff changeset	184
998 75e65599ea52 Persist job queue in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	185 if err != nil {
5490 5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5407 diff changeset	186 log.Errorf("%v\n", err)
998 75e65599ea52 Persist job queue in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	187 http.Error(rw, "error: "+err.Error(), http.StatusInternalServerError)
75e65599ea52 Persist job queue in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	188 return
75e65599ea52 Persist job queue in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	189 }
75e65599ea52 Persist job queue in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	190
5490 5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5407 diff changeset	191 log.Infof("added import #%d to queue\n", jobID)
979 7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	192
7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	193 result := struct {
7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	194 ID int64 `json:"id"`
7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	195 }{
7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	196 ID: jobID,
7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	197 }
4244 4394daeea96a Moved JSONHandler into middleware package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4243 diff changeset	198 mw.SendJSON(rw, http.StatusCreated, &result)
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	199 }
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	200
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	201 func loadMeta(f zip.File) (models.SoundingResultMeta, error) {
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	202 r, err := f.Open()
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	203 if err != nil {
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	204 return nil, err
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	205 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	206 defer r.Close()
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	207 var m models.SoundingResultMeta
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	208 return &m, m.Decode(r)
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	209 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	210
4244 4394daeea96a Moved JSONHandler into middleware package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4243 diff changeset	211 func uploadSoundingResult(req *http.Request) (jr mw.JSONResult, err error) {
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	212
bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	213 var dir string
4875 6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4244 diff changeset	214 if dir, err = misc.StoreUploadedFileCheck(
2194 4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	215 req,
4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	216 soundingResultName,
4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	217 "sr.zip",
4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	218 maxSoundingResultSize,
4875 6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4244 diff changeset	219 true,
2194 4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	220 ); err != nil {
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	221 return
bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	222 }
bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	223
3751 96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	224 var messages []string
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	225
1225 4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1224 diff changeset	226 srFile := filepath.Join(dir, "sr.zip")
4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1224 diff changeset	227
4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1224 diff changeset	228 var zr *zip.ReadCloser
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	229 var once sync.Once
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	230 closeOnce := func() { zr.Close() }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	231
4183 34623265eac1 Made 'golint' happy (again) with the controllers package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3976 diff changeset	232 var isZIP bool
3751 96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	233 if zr, err = zip.OpenReader(srFile); err != nil {
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	234 messages = append(messages, fmt.Sprintf("%v - "+
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	235 "Trying TXT file mode.", err))
3751 96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	236 } else {
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	237 isZIP = true
3751 96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	238 defer once.Do(closeOnce)
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	239 }
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	240
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	241 var result struct {
1227 737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	242 Token string `json:"token,omitempty"`
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	243 Meta interface{} `json:"meta,omitempty"`
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	244 Messages []string `json:"messages,omitempty"`
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	245 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	246
1528 5874cedd7f91 Sounding result import: Accept .txt files for XYZ data, too. Sascha L. Teichmann <sascha.teichmann@intevation.de>* parents: 1328 diff changeset	247 find := func(ext string) *zip.File { return common.FindInZIP(zr, ext) }
5874cedd7f91 Sounding result import: Accept .txt files for XYZ data, too. Sascha L. Teichmann <sascha.teichmann@intevation.de>* parents: 1328 diff changeset	248
3751 96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	249 var noXYZ bool
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	250 if isZIP {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	251 if zr != nil {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	252 noXYZ = find(".xyz") == nil && find(".txt") == nil
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	253 }
3751 96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	254
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	255 if noXYZ {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	256 messages = append(messages, "no .xyz or .txt file found.")
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	257 }
1225 4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1224 diff changeset	258
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	259 if mj := find("meta.json"); mj == nil {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	260 messages = append(messages, "no 'meta.json' file found.")
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	261 } else {
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	262 if meta, err := loadMeta(mj); err != nil {
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	263 messages = append(messages,
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	264 fmt.Sprintf("'meta.json' found but invalid: %v", err))
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	265 } else {
4244 4394daeea96a Moved JSONHandler into middleware package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4243 diff changeset	266 errs := meta.Validate(req.Context(), mw.JSONConn(req))
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	267 for _, err := range errs {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	268 messages = append(messages,
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	269 fmt.Sprintf("invalid 'meta.json': %v", err))
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	270 }
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	271 result.Meta = meta
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	272 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	273 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	274 }
3751 96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	275
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	276 if zr != nil {
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	277 once.Do(closeOnce)
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	278 }
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	279
1227 737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	280 code := http.StatusCreated
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	281
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	282 // If there are no XYZ data we cant help the user anyway.
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	283 if noXYZ {
1228 17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	284 code = http.StatusBadRequest
1227 737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	285 if err2 := os.RemoveAll(dir); err2 != nil {
5490 5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5407 diff changeset	286 log.Errorf("%v\n", err2)
1227 737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	287 }
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	288 } else if result.Token, err = misc.MakeTempFile(dir); err != nil {
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	289 if err2 := os.RemoveAll(dir); err2 != nil {
5490 5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5407 diff changeset	290 log.Errorf("%v\n", err2)
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	291 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	292 return
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	293 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	294
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	295 result.Messages = messages
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	296
4244 4394daeea96a Moved JSONHandler into middleware package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4243 diff changeset	297 jr = mw.JSONResult{
1227 737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	298 Code: code,
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	299 Result: &result,
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	300 }
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	301 return
bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	302 }
1228 17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	303
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	304 func deleteSoundingUpload(rw http.ResponseWriter, req *http.Request) {
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	305 token := mux.Vars(req)["token"]
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	306 if _, err := hex.DecodeString(token); err != nil {
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	307 http.Error(rw, "Invalid token", http.StatusBadRequest)
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	308 return
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	309 }
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	310 if err := misc.DeleteTempFile(token); err != nil {
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	311 http.Error(rw, fmt.Sprintf("error: %v", err), http.StatusInternalServerError)
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	312 return
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	313 }
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	314 result := struct {
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	315 Message string `json:"message"`
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	316 }{
1297 1c0c9190fcf2 Import sounding result: If uploading a zip with a meta.json which has Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1241 diff changeset	317 Message: fmt.Sprintf("Token %s deleted.", token),
1228 17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	318 }
4244 4394daeea96a Moved JSONHandler into middleware package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4243 diff changeset	319 mw.SendJSON(rw, http.StatusOK, &result)
1228 17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	320 }

Mercurial > gemma

annotate pkg/controllers/srimports.go @ 5591:0011f50cf216 surveysperbottleneckid