Mercurial > gemma
annotate pkg/controllers/imports.go @ 1199:181b8a947ecd
fixed property of null bug
author | Markus Kottlaender <markus@intevation.de> |
---|---|
date | Mon, 19 Nov 2018 08:51:44 +0100 |
parents | a244b18cb916 |
children |
rev | line source |
---|---|
1017
a244b18cb916
Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1010
diff
changeset
|
1 // This is Free Software under GNU Affero General Public License v >= 3.0 |
a244b18cb916
Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1010
diff
changeset
|
2 // without warranty, see README.md and license for details. |
a244b18cb916
Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1010
diff
changeset
|
3 // |
a244b18cb916
Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1010
diff
changeset
|
4 // SPDX-License-Identifier: AGPL-3.0-or-later |
a244b18cb916
Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1010
diff
changeset
|
5 // License-Filename: LICENSES/AGPL-3.0.txt |
a244b18cb916
Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1010
diff
changeset
|
6 // |
a244b18cb916
Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1010
diff
changeset
|
7 // Copyright (C) 2018 by via donau |
a244b18cb916
Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1010
diff
changeset
|
8 // – Österreichische Wasserstraßen-Gesellschaft mbH |
a244b18cb916
Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1010
diff
changeset
|
9 // Software engineering by Intevation GmbH |
a244b18cb916
Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1010
diff
changeset
|
10 // |
a244b18cb916
Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1010
diff
changeset
|
11 // Author(s): |
a244b18cb916
Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1010
diff
changeset
|
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de> |
a244b18cb916
Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1010
diff
changeset
|
13 |
978
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
14 package controllers |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
15 |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
16 import ( |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
17 "bufio" |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
18 "io" |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
19 "io/ioutil" |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
20 "log" |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
21 "net/http" |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
22 "os" |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
23 "path/filepath" |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
24 |
979
7934b5c1a910
Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
978
diff
changeset
|
25 "gemma.intevation.de/gemma/pkg/auth" |
978
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
26 "gemma.intevation.de/gemma/pkg/config" |
979
7934b5c1a910
Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
978
diff
changeset
|
27 "gemma.intevation.de/gemma/pkg/imports" |
978
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
28 ) |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
29 |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
30 const ( |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
31 maxSoundingResultSize = 25 * 1024 * 1024 |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
32 soundingResultName = "soundingresult" |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
33 ) |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
34 |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
35 func downloadSoundingResult(req *http.Request) (string, error) { |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
36 |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
37 f, _, err := req.FormFile(soundingResultName) |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
38 if err != nil { |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
39 return "", err |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
40 } |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
41 defer f.Close() |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
42 |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
43 dir, err := ioutil.TempDir(config.TmpDir(), soundingResultName) |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
44 if err != nil { |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
45 return "", err |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
46 } |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
47 |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
48 o, err := os.Create(filepath.Join(dir, "sr.zip")) |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
49 if err != nil { |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
50 os.RemoveAll(dir) |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
51 return "", err |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
52 } |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
53 |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
54 out := bufio.NewWriter(o) |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
55 |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
56 if _, err = io.Copy(out, io.LimitReader(f, maxSoundingResultSize)); err != nil { |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
57 o.Close() |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
58 os.RemoveAll(dir) |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
59 return "", err |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
60 } |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
61 |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
62 if err = out.Flush(); err != nil { |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
63 o.Close() |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
64 os.RemoveAll(dir) |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
65 return "", err |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
66 } |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
67 |
984
3c9ea8ab856a
Fixed small glitches in sounding result importer.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
979
diff
changeset
|
68 return dir, nil |
978
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
69 } |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
70 |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
71 func importSoundingResult(rw http.ResponseWriter, req *http.Request) { |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
72 |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
73 dir, err := downloadSoundingResult(req) |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
74 if err != nil { |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
75 log.Printf("error: %v\n", err) |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
76 http.Error(rw, "error: "+err.Error(), http.StatusInternalServerError) |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
77 return |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
78 } |
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
79 |
979
7934b5c1a910
Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
978
diff
changeset
|
80 session, _ := auth.GetSession(req) |
7934b5c1a910
Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
978
diff
changeset
|
81 |
998
75e65599ea52
Persist job queue in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
991
diff
changeset
|
82 jobID, err := imports.AddJob(imports.SRJobKind, session.User, dir) |
75e65599ea52
Persist job queue in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
991
diff
changeset
|
83 if err != nil { |
75e65599ea52
Persist job queue in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
991
diff
changeset
|
84 log.Printf("error: %v\n", err) |
75e65599ea52
Persist job queue in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
991
diff
changeset
|
85 http.Error(rw, "error: "+err.Error(), http.StatusInternalServerError) |
75e65599ea52
Persist job queue in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
991
diff
changeset
|
86 return |
75e65599ea52
Persist job queue in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
991
diff
changeset
|
87 } |
75e65599ea52
Persist job queue in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
991
diff
changeset
|
88 |
1010
8f23ec811afb
Fixed and harmonized wording in importer queue a bit.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
998
diff
changeset
|
89 log.Printf("info: added import #%d to queue\n", jobID) |
979
7934b5c1a910
Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
978
diff
changeset
|
90 |
7934b5c1a910
Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
978
diff
changeset
|
91 result := struct { |
7934b5c1a910
Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
978
diff
changeset
|
92 ID int64 `json:"id"` |
7934b5c1a910
Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
978
diff
changeset
|
93 }{ |
7934b5c1a910
Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
978
diff
changeset
|
94 ID: jobID, |
7934b5c1a910
Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
978
diff
changeset
|
95 } |
7934b5c1a910
Finally enqueue sounding result import job to import jobs.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
978
diff
changeset
|
96 SendJSON(rw, http.StatusCreated, &result) |
978
544a5cfe07cd
Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
97 } |