Mercurial > gemma

annotate pkg/imports/ugm.go @ 4606:dfe9cde6a20c geoserver_sql_views

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Reflect database model changes for SQL views in backend In principle, we could use many datasources with different database schemas, but this would imply changing GeoServer initialization, service filtering, endpoints and eventually more. Since we do not need it, just hard-code the schema name as a constant.
author Tom Gottfried <tom@intevation.de>
date Thu, 05 Sep 2019 12:23:31 +0200
parents 49012340336c
children 59a99655f34d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2245
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
3 //
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
6 //
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 //
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 // Author(s):
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
13
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14 package imports
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import (
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 "context"
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 "database/sql"
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 "errors"
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
20 "os"
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
21 "path/filepath"
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
22
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
23 "gemma.intevation.de/gemma/pkg/soap"
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
24 "gemma.intevation.de/gemma/pkg/soap/nts"
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
25 )
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26
4182
49012340336c Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4177
diff changeset
27 // UploadedGaugeMeasurement is an Jon to extract gauge measurement data
49012340336c Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4177
diff changeset
28 // from an uploaded XML file and stores it into the database.
2249
35db2e11c966 Uploaded gauge measurement: Implemented in terms of normal gauge measurement import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2246
diff changeset
29 type UploadedGaugeMeasurement struct {
2245
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
30 Dir string `json:"dir"`
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
31 }
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
32
4182
49012340336c Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4177
diff changeset
33 // UGMJobKind is the unique name of this import job type.
2245
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
34 const UGMJobKind JobKind = "ugm"
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
35
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
36 type ugmJobCreator struct{}
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
37
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
38 func init() { RegisterJobCreator(UGMJobKind, ugmJobCreator{}) }
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
39
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
40 func (ugmJobCreator) Description() string { return "uploaded gauge measurements" }
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
41
2249
35db2e11c966 Uploaded gauge measurement: Implemented in terms of normal gauge measurement import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2246
diff changeset
42 func (ugmJobCreator) Create() Job { return new(UploadedGaugeMeasurement) }
2245
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
43
3219
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2278
diff changeset
44 func (ugmJobCreator) Depends() [2][]string { return gmJobCreator{}.Depends() }
2245
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
45
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
46 func (ugmJobCreator) AutoAccept() bool { return true }
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
47
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
48 func (ugmJobCreator) StageDone(context.Context, *sql.Tx, int64) error { return nil }
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
49
4182
49012340336c Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4177
diff changeset
50 // CleanUp removes the temporary files from the filesystem.
2249
35db2e11c966 Uploaded gauge measurement: Implemented in terms of normal gauge measurement import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2246
diff changeset
51 func (ugm *UploadedGaugeMeasurement) CleanUp() error { return os.RemoveAll(ugm.Dir) }
2245
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
52
2249
35db2e11c966 Uploaded gauge measurement: Implemented in terms of normal gauge measurement import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2246
diff changeset
53 // Do executes the actual uploaded gauge measurement import.
35db2e11c966 Uploaded gauge measurement: Implemented in terms of normal gauge measurement import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2246
diff changeset
54 func (ugm *UploadedGaugeMeasurement) Do(
2245
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
55 ctx context.Context,
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
56 importID int64,
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
57 conn *sql.Conn,
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
58 feedback Feedback,
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
59 ) (interface{}, error) {
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
60
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
61 fetch := func() ([]*nts.RIS_Message_Type, error) {
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
62
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63 var dst nts.Get_messages_result
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
64
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
65 if err := soap.ValidateFile(
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
66 filepath.Join(ugm.Dir, "data.xml"),
2278
a3e7e727209a UGM import: Use special schema.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2249
diff changeset
67 "NtS.xsd",
2245
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
68 &dst,
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
69 ); err != nil {
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
70 return nil, err
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
71 }
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
72
2249
35db2e11c966 Uploaded gauge measurement: Implemented in terms of normal gauge measurement import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2246
diff changeset
73 if len(dst.Result_message) == 0 {
4177
8b75ac5e243e Made 'staticcheck' happy with pgxutils package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
74 return nil, errors.New("no gauge measurements found")
2245
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
75 }
2249
35db2e11c966 Uploaded gauge measurement: Implemented in terms of normal gauge measurement import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2246
diff changeset
76 return dst.Result_message, nil
2245
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
77 }
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
78
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
79 return storeGaugeMeasurements(
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
80 ctx,
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
81 importID,
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
82 fetch,
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
83 conn,
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
84 feedback,
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
85 )
7d784840a9a7 Uploaded gauge measurements: Initial commit. TODO: Add endpoint.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
86 }