Mercurial > gemma

annotate pkg/misc/http.go @ 5710:37c8feeecb4d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Merged branch sr-v2 into default.
author Sascha L. Teichmann <sascha.teichmann@intevation.de>
date Tue, 20 Feb 2024 21:28:56 +0100
parents 31973f6f5cca
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
3 //
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
6 //
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 //
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 // Author(s):
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
13
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14 package misc
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import (
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 "bufio"
4875
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
18 "fmt"
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 "io"
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
20 "net/http"
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
21 "os"
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
22 "path/filepath"
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
23
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
24 "gemma.intevation.de/gemma/pkg/config"
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
25 )
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26
4172
9cbed444b8a4 Made 'golint' and 'staticcheck' happy with misc package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2194
diff changeset
27 // StoreUploadedFile stores a file upload file from
9cbed444b8a4 Made 'golint' and 'staticcheck' happy with misc package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2194
diff changeset
28 // a given HTTP request identified by a given field name
9cbed444b8a4 Made 'golint' and 'staticcheck' happy with misc package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2194
diff changeset
29 // in a file with a path build by the config.TmpDir and
9cbed444b8a4 Made 'golint' and 'staticcheck' happy with misc package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2194
diff changeset
30 // the given file name.
9cbed444b8a4 Made 'golint' and 'staticcheck' happy with misc package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2194
diff changeset
31 // If the file is long than the given limit maxSize
9cbed444b8a4 Made 'golint' and 'staticcheck' happy with misc package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2194
diff changeset
32 // this function returns an error.
4875
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
33 func StoreUploadedFile(
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
34 req *http.Request,
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
35 field, fname string,
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
36 maxSize int64) (string, error) {
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
37 return StoreUploadedFileCheck(req, field, fname, maxSize, false)
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
38 }
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
39
5601
1222b777f51f Made golint finally happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
40 // StoreUploadedFileCheck does the same as StoreUploadedFile
1222b777f51f Made golint finally happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
41 // with the optional check if the upload is too large or not.
1222b777f51f Made golint finally happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
42 func StoreUploadedFileCheck(
1222b777f51f Made golint finally happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
43 req *http.Request,
1222b777f51f Made golint finally happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
44 field, fname string,
1222b777f51f Made golint finally happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
45 maxSize int64,
1222b777f51f Made golint finally happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
46 errorOverMax bool,
1222b777f51f Made golint finally happy.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4875
diff changeset
47 ) (string, error) {
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
48
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
49 // Check for direct upload.
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
50 f, _, err := req.FormFile(field)
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
51 if err != nil {
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
52 return "", err
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
53 }
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
54 defer f.Close()
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
55
5683
31973f6f5cca Get rid of deprecation warnings from staticcheck.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 5601
diff changeset
56 dir, err := os.MkdirTemp(config.TmpDir(), field)
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
57 if err != nil {
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
58 return "", err
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
59 }
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
60
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
61 o, err := os.Create(filepath.Join(dir, fname))
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
62 if err != nil {
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63 os.RemoveAll(dir)
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
64 return "", err
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
65 }
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
66
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
67 out := bufio.NewWriter(o)
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
68
4875
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
69 // Little trick to check if we are over the limit.
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
70 size := maxSize
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
71 if errorOverMax {
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
72 size++
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
73 }
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
74
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
75 cleanup := func() {
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
76 o.Close()
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
77 os.RemoveAll(dir)
4875
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
78 }
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
79
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
80 r, err := io.Copy(out, io.LimitReader(f, size))
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
81 if err != nil {
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
82 cleanup()
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
83 return "", err
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
84 }
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
85
4875
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
86 if errorOverMax && r > maxSize {
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
87 cleanup()
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
88 return "", fmt.Errorf("upload exceeded limit of %d bytes", maxSize)
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
89 }
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
90
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
91 if err = out.Flush(); err != nil {
4875
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
92 cleanup()
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
93 return "", err
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
94 }
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
95
6237e6165041 * Raise the upload limit for sounding results from 25GB up to 50GB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4172
diff changeset
96 if err := o.Close(); err != nil {
2194
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
97 os.RemoveAll(dir)
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
98 return "", err
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
99 }
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
100
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
101 return dir, nil
4d6979dedb11 Imports: Deduplicted file upload code.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
102 }