gemma: pkg/controllers/srimports.go annotate

annotate pkg/controllers/srimports.go @ 4875:6237e6165041

* Raise the upload limit for sounding results from 25GB up to 50GB. * Issue an error if sounding results uploads are over the limit instead of silently truncating them. * Fix a file handle leak when storing all uploads.

author	Sascha L. Teichmann <sascha.teichmann@intevation.de>
date	Fri, 31 Jan 2020 23:17:37 +0100
parents	4394daeea96a
children	85f19e924a43

rev	line source
1017 a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	1 // This is Free Software under GNU Affero General Public License v >= 3.0
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	2 // without warranty, see README.md and license for details.
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	3 //
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	4 // SPDX-License-Identifier: AGPL-3.0-or-later
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	5 // License-Filename: LICENSES/AGPL-3.0.txt
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	6 //
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	7 // Copyright (C) 2018 by via donau
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	8 // – Österreichische Wasserstraßen-Gesellschaft mbH
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	9 // Software engineering by Intevation GmbH
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	10 //
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	11 // Author(s):
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1010 diff changeset	13
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14 package controllers
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16 import (
1225 4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1224 diff changeset	17 "archive/zip"
1221 c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	18 "encoding/hex"
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	19 "fmt"
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	20 "log"
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	21 "net/http"
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	22 "os"
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	23 "path/filepath"
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	24 "strconv"
3585 479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	25 "strings"
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	26 "sync"
1221 c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	27 "time"
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	28
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	29 "github.com/gorilla/mux"
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	30
979 7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	31 "gemma.intevation.de/gemma/pkg/auth"
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	32 "gemma.intevation.de/gemma/pkg/common"
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	33 "gemma.intevation.de/gemma/pkg/config"
979 7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	34 "gemma.intevation.de/gemma/pkg/imports"
1221 c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	35 "gemma.intevation.de/gemma/pkg/misc"
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	36 "gemma.intevation.de/gemma/pkg/models"
4244 4394daeea96a Moved JSONHandler into middleware package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4243 diff changeset	37
4394daeea96a Moved JSONHandler into middleware package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4243 diff changeset	38 mw "gemma.intevation.de/gemma/pkg/middleware"
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	39 )
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	40
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	41 const (
2194 4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	42 soundingResultName = "soundingresult"
4875 6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4244 diff changeset	43 maxSoundingResultSize = 50 * 1024 * 1024
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	44 )
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	45
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	46 func fetchSoundingResult(req *http.Request) (string, error) {
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	47
1221 c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	48 // Check first if we have a token.
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	49 if token := req.FormValue("token"); token != "" {
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	50 if _, err := hex.DecodeString(token); err != nil {
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	51 return "", err
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	52 }
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	53 dir := config.TmpDir()
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	54 if dir == "" {
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	55 dir = os.TempDir()
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	56 }
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	57 // XXX: This should hopefully be race-free enough.
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	58 now := time.Now().Format("2006-15-04-05")
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	59 dst := filepath.Join(dir, soundingResultName+"-"+token+"-"+now)
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	60 if err := misc.UnmakeTempFile(token, dst); err != nil {
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	61 return "", err
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	62 }
1241 a45fa8943254 Sounding result import: Forgot to return the token temp file to the importer. Doh! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1239 diff changeset	63 return dst, nil
1221 c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	64 }
c193649d4f11 Add an area for temp uploads on the server to be addressed Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1203 diff changeset	65
4875 6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4244 diff changeset	66 return misc.StoreUploadedFileCheck(
2194 4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	67 req,
4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	68 soundingResultName,
4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	69 "sr.zip",
4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	70 maxSoundingResultSize,
4875 6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4244 diff changeset	71 true,
2194 4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	72 )
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	73 }
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	74
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	75 func fetchSoundingResultMetaOverrides(sr imports.SoundingResult, req http.Request) error {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	76
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	77 if v := req.FormValue("epsg"); v != "" {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	78 epsg, err := strconv.ParseUint(v, 10, 32)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	79 if err != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	80 return err
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	81 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	82 srid := uint(epsg)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	83 sr.EPSG = &srid
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	84 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	85
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	86 if v := req.FormValue("date"); v != "" {
2059 ae0021feaac8 Imports: Made stretch import marshable (even if it is not needed). Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1985 diff changeset	87 date, err := time.Parse(common.DateFormat, v)
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	88 if err != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	89 return err
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	90 }
1900 6a67cd819e93 To prepare stretch import made some model data types re-usable. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1787 diff changeset	91 sr.Date = &models.Date{Time: date}
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	92 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	93
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	94 if v := req.FormValue("depth-reference"); v != "" {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	95 sr.DepthReference = &v
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	96 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	97
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	98 if v := req.FormValue("bottleneck"); v != "" {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	99 sr.Bottleneck = &v
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	100 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	101
3945 3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	102 if v := req.FormValue("negate-z"); v != "" {
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	103 var negateZ bool
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	104 switch strings.ToLower(v) {
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	105 case "true", "1":
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	106 negateZ = true
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	107 case "false", "0":
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	108 negateZ = false
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	109 default:
3976 c412dff6e1da Fixed cases for error strings. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3946 diff changeset	110 return fmt.Errorf("unknown negate-z '%s'", v)
3945 3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	111 }
3946 d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3945 diff changeset	112 sr.NegateZ = &negateZ
3945 3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	113 }
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3762 diff changeset	114
3713 ac168fcf210e Eat single-beam parameter in form of 'single-beam' mit accepted values of 'true, false, 0, 1, singlebeam, single-beam, multibeam, multi-beam'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3711 diff changeset	115 if v := req.FormValue("single-beam"); v != "" {
3585 479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	116 var singleBeam bool
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	117 switch strings.ToLower(v) {
3713 ac168fcf210e Eat single-beam parameter in form of 'single-beam' mit accepted values of 'true, false, 0, 1, singlebeam, single-beam, multibeam, multi-beam'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3711 diff changeset	118 case "true", "1", "singlebeam", "single-beam":
ac168fcf210e Eat single-beam parameter in form of 'single-beam' mit accepted values of 'true, false, 0, 1, singlebeam, single-beam, multibeam, multi-beam'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3711 diff changeset	119 singleBeam = true
ac168fcf210e Eat single-beam parameter in form of 'single-beam' mit accepted values of 'true, false, 0, 1, singlebeam, single-beam, multibeam, multi-beam'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3711 diff changeset	120 case "false", "0", "multibeam", "multi-beam":
3585 479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	121 singleBeam = false
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	122 default:
3976 c412dff6e1da Fixed cases for error strings. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3946 diff changeset	123 return fmt.Errorf("unknown single-beam '%s'", v)
3585 479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	124 }
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	125 sr.SingleBeam = &singleBeam
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	126 }
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2995 diff changeset	127
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	128 return nil
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	129 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	130
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	131 func importSoundingResult(rw http.ResponseWriter, req *http.Request) {
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	132
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	133 sr := new(imports.SoundingResult)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	134
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	135 if err := fetchSoundingResultMetaOverrides(sr, req); err != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	136 log.Printf("error: %v\n", err)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	137 http.Error(rw, "error: "+err.Error(), http.StatusBadRequest)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	138 return
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	139 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	140
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	141 dir, err := fetchSoundingResult(req)
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	142 if err != nil {
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	143 log.Printf("error: %v\n", err)
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	144 http.Error(rw, "error: "+err.Error(), http.StatusInternalServerError)
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	145 return
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	146 }
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	147 sr.Dir = dir
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	148
1531 24445a618513 Added stub for bottleneck importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1528 diff changeset	149 serialized, err := common.ToJSONString(sr)
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	150 if err != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	151 log.Printf("error: %v\n", err)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	152 http.Error(rw, "error: "+err.Error(), http.StatusInternalServerError)
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	153 return
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1228 diff changeset	154 }
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	155
979 7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	156 session, _ := auth.GetSession(req)
7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	157
2995 5222bfe5b4af Upload imports: spell form field 'send-email' instead of 'email'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2194 diff changeset	158 sendEmail := req.FormValue("send-email") != ""
1646 a0982c38eac0 Import queue: Implemented email notifications. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1642 diff changeset	159
1642 49c04bb64e0a Import queue: Implemented auto-accept and email sending. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1531 diff changeset	160 jobID, err := imports.AddJob(
49c04bb64e0a Import queue: Implemented auto-accept and email sending. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1531 diff changeset	161 imports.SRJobKind,
2075 e7f3199384ec Sounding results import: Removed senseless parameters. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2059 diff changeset	162 time.Time{}, // due
e7f3199384ec Sounding results import: Removed senseless parameters. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2059 diff changeset	163 nil, // trys
e7f3199384ec Sounding results import: Removed senseless parameters. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2059 diff changeset	164 nil, // retry wait
1642 49c04bb64e0a Import queue: Implemented auto-accept and email sending. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1531 diff changeset	165 session.User,
1754 807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1708 diff changeset	166 sendEmail,
1642 49c04bb64e0a Import queue: Implemented auto-accept and email sending. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1531 diff changeset	167 serialized)
49c04bb64e0a Import queue: Implemented auto-accept and email sending. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1531 diff changeset	168
998 75e65599ea52 Persist job queue in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	169 if err != nil {
75e65599ea52 Persist job queue in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	170 log.Printf("error: %v\n", err)
75e65599ea52 Persist job queue in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	171 http.Error(rw, "error: "+err.Error(), http.StatusInternalServerError)
75e65599ea52 Persist job queue in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	172 return
75e65599ea52 Persist job queue in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	173 }
75e65599ea52 Persist job queue in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	174
1010 8f23ec811afb Fixed and harmonized wording in importer queue a bit. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 998 diff changeset	175 log.Printf("info: added import #%d to queue\n", jobID)
979 7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	176
7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	177 result := struct {
7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	178 ID int64 `json:"id"`
7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	179 }{
7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	180 ID: jobID,
7934b5c1a910 Finally enqueue sounding result import job to import jobs. Sascha L. Teichmann <teichmann@intevation.de> parents: 978 diff changeset	181 }
4244 4394daeea96a Moved JSONHandler into middleware package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4243 diff changeset	182 mw.SendJSON(rw, http.StatusCreated, &result)
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	183 }
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	184
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	185 func loadMeta(f zip.File) (models.SoundingResultMeta, error) {
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	186 r, err := f.Open()
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	187 if err != nil {
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	188 return nil, err
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	189 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	190 defer r.Close()
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	191 var m models.SoundingResultMeta
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	192 return &m, m.Decode(r)
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	193 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	194
4244 4394daeea96a Moved JSONHandler into middleware package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4243 diff changeset	195 func uploadSoundingResult(req *http.Request) (jr mw.JSONResult, err error) {
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	196
bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	197 var dir string
4875 6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4244 diff changeset	198 if dir, err = misc.StoreUploadedFileCheck(
2194 4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	199 req,
4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	200 soundingResultName,
4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	201 "sr.zip",
4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	202 maxSoundingResultSize,
4875 6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4244 diff changeset	203 true,
2194 4d6979dedb11 Imports: Deduplicted file upload code. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2075 diff changeset	204 ); err != nil {
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	205 return
bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	206 }
bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	207
3751 96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	208 var messages []string
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	209
1225 4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1224 diff changeset	210 srFile := filepath.Join(dir, "sr.zip")
4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1224 diff changeset	211
4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1224 diff changeset	212 var zr *zip.ReadCloser
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	213 var once sync.Once
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	214 closeOnce := func() { zr.Close() }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	215
4183 34623265eac1 Made 'golint' happy (again) with the controllers package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3976 diff changeset	216 var isZIP bool
3751 96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	217 if zr, err = zip.OpenReader(srFile); err != nil {
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	218 messages = append(messages, fmt.Sprintf("%v - "+
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	219 "Trying TXT file mode.", err))
3751 96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	220 } else {
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	221 isZIP = true
3751 96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	222 defer once.Do(closeOnce)
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	223 }
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	224
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	225 var result struct {
1227 737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	226 Token string `json:"token,omitempty"`
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	227 Meta interface{} `json:"meta,omitempty"`
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	228 Messages []string `json:"messages,omitempty"`
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	229 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	230
1528 5874cedd7f91 Sounding result import: Accept .txt files for XYZ data, too. Sascha L. Teichmann <sascha.teichmann@intevation.de>* parents: 1328 diff changeset	231 find := func(ext string) *zip.File { return common.FindInZIP(zr, ext) }
5874cedd7f91 Sounding result import: Accept .txt files for XYZ data, too. Sascha L. Teichmann <sascha.teichmann@intevation.de>* parents: 1328 diff changeset	232
3751 96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	233 var noXYZ bool
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	234 if isZIP {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	235 if zr != nil {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	236 noXYZ = find(".xyz") == nil && find(".txt") == nil
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	237 }
3751 96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	238
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	239 if noXYZ {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	240 messages = append(messages, "no .xyz or .txt file found.")
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	241 }
1225 4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1224 diff changeset	242
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	243 if mj := find("meta.json"); mj == nil {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	244 messages = append(messages, "no 'meta.json' file found.")
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	245 } else {
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	246 if meta, err := loadMeta(mj); err != nil {
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	247 messages = append(messages,
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	248 fmt.Sprintf("'meta.json' found but invalid: %v", err))
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	249 } else {
4244 4394daeea96a Moved JSONHandler into middleware package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4243 diff changeset	250 errs := meta.Validate(req.Context(), mw.JSONConn(req))
3762 98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	251 for _, err := range errs {
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	252 messages = append(messages,
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	253 fmt.Sprintf("invalid 'meta.json': %v", err))
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	254 }
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR. Sascha Wilde <wilde@intevation.de> parents: 3752 diff changeset	255 result.Meta = meta
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	256 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	257 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	258 }
3751 96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	259
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	260 if zr != nil {
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	261 once.Do(closeOnce)
96d6e6417819 SR import: Allow upload of none-ZIP files and assume they are plain XYZ files. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 3713 diff changeset	262 }
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	263
1227 737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	264 code := http.StatusCreated
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	265
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	266 // If there are no XYZ data we cant help the user anyway.
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	267 if noXYZ {
1228 17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	268 code = http.StatusBadRequest
1227 737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	269 if err2 := os.RemoveAll(dir); err2 != nil {
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	270 log.Printf("error: %v\n", err2)
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	271 }
737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	272 } else if result.Token, err = misc.MakeTempFile(dir); err != nil {
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	273 if err2 := os.RemoveAll(dir); err2 != nil {
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	274 log.Printf("error: %v\n", err2)
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	275 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	276 return
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	277 }
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	278
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	279 result.Messages = messages
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	280
4244 4394daeea96a Moved JSONHandler into middleware package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4243 diff changeset	281 jr = mw.JSONResult{
1227 737e1acea1f1 If there are no XYZ files in the ZIP we cant help the user. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1226 diff changeset	282 Code: code,
1226 2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	283 Result: &result,
2e65e8ddacab Finished the sounding result upload controller to temp uploads. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	284 }
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	285 return
bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1221 diff changeset	286 }
1228 17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	287
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	288 func deleteSoundingUpload(rw http.ResponseWriter, req *http.Request) {
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	289 token := mux.Vars(req)["token"]
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	290 if _, err := hex.DecodeString(token); err != nil {
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	291 http.Error(rw, "Invalid token", http.StatusBadRequest)
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	292 return
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	293 }
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	294 if err := misc.DeleteTempFile(token); err != nil {
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	295 http.Error(rw, fmt.Sprintf("error: %v", err), http.StatusInternalServerError)
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	296 return
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	297 }
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	298 result := struct {
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	299 Message string `json:"message"`
17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	300 }{
1297 1c0c9190fcf2 Import sounding result: If uploading a zip with a meta.json which has Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1241 diff changeset	301 Message: fmt.Sprintf("Token %s deleted.", token),
1228 17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	302 }
4244 4394daeea96a Moved JSONHandler into middleware package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4243 diff changeset	303 mw.SendJSON(rw, http.StatusOK, &result)
1228 17131f0f9fcb Added endpoint to explicitly delete a temp upload. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1227 diff changeset	304 }

Mercurial > gemma

annotate pkg/controllers/srimports.go @ 4875:6237e6165041