Mercurial > gemma
annotate pkg/imports/ubn.go @ 5670:b75d0b303328
Various fixes and improvements of gauges import:
- Allow update of erased data (and thereby set erased to false)
- Fix source_organization to work with ERDMS2
- Give ISRS of new and updated gauges in summary
- Fixed reference of null pointers if revlevels are missing
- Fixed reference of null pointer on update errors
- Added ISRS to reference_code warning
author | Sascha Wilde <wilde@sha-bang.de> |
---|---|
date | Fri, 08 Dec 2023 17:29:56 +0100 |
parents | 59a99655f34d |
children | 6270951dda28 |
rev | line source |
---|---|
2191
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
1 // This is Free Software under GNU Affero General Public License v >= 3.0 |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
2 // without warranty, see README.md and license for details. |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
3 // |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
4 // SPDX-License-Identifier: AGPL-3.0-or-later |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
5 // License-Filename: LICENSES/AGPL-3.0.txt |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
6 // |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
7 // Copyright (C) 2018 by via donau |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
8 // – Österreichische Wasserstraßen-Gesellschaft mbH |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
9 // Software engineering by Intevation GmbH |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
10 // |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
11 // Author(s): |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de> |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
13 |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
14 package imports |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
15 |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
16 import ( |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
17 "context" |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
18 "database/sql" |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
19 "errors" |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
20 "os" |
2195
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
21 "path/filepath" |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
22 |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
23 "gemma.intevation.de/gemma/pkg/soap" |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
24 "gemma.intevation.de/gemma/pkg/soap/ifbn" |
2191
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
25 ) |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
26 |
4179
a376351d2774
Another round of doc strings to make 'golint' a little bit more happy with the imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4177
diff
changeset
|
27 // UploadedBottleneck is a Job to import bottlenecks from |
a376351d2774
Another round of doc strings to make 'golint' a little bit more happy with the imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4177
diff
changeset
|
28 // an uploaded XML file into the database. |
2191
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
29 type UploadedBottleneck struct { |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
30 Dir string `json:"dir"` |
2563
dc4fae4bdb8f
Expose axis snapping tolerance to users
Tom Gottfried <tom@intevation.de>
parents:
2253
diff
changeset
|
31 // Tolerance used for axis snapping |
dc4fae4bdb8f
Expose axis snapping tolerance to users
Tom Gottfried <tom@intevation.de>
parents:
2253
diff
changeset
|
32 Tolerance float64 `json:"tolerance"` |
2191
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
33 } |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
34 |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
35 // UBNJobKind is the import queue type identifier. |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
36 const UBNJobKind JobKind = "ubn" |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
37 |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
38 type ubnJobCreator struct{} |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
39 |
2253
cfc523c70e90
Uploaded imports: Small code clean-ups.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2195
diff
changeset
|
40 func init() { RegisterJobCreator(UBNJobKind, ubnJobCreator{}) } |
2191
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
41 |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
42 func (ubnJobCreator) Description() string { return "uploaded bottlenecks" } |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
43 |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
44 func (ubnJobCreator) AutoAccept() bool { return false } |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
45 |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
46 func (ubnJobCreator) Create() Job { return new(UploadedBottleneck) } |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
47 |
3219
4acbee65275d
Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2563
diff
changeset
|
48 func (ubnJobCreator) Depends() [2][]string { |
2195
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
49 // Same as normal bottleneck import. |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
50 return bnJobCreator{}.Depends() |
2191
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
51 } |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
52 |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
53 // StageDone moves the imported bottleneck out of the staging area. |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
54 func (ubnJobCreator) StageDone( |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
55 ctx context.Context, |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
56 tx *sql.Tx, |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
57 id int64, |
5034
59a99655f34d
Added feedback support for StageDone.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4179
diff
changeset
|
58 feedback Feedback, |
2191
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
59 ) error { |
2195
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
60 // Same as normal bottleneck import. |
5034
59a99655f34d
Added feedback support for StageDone.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4179
diff
changeset
|
61 return bnJobCreator{}.StageDone(ctx, tx, id, feedback) |
2191
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
62 } |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
63 |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
64 // CleanUp of a uploaded bottleneck import removes the temp dir. |
2253
cfc523c70e90
Uploaded imports: Small code clean-ups.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2195
diff
changeset
|
65 func (ubn *UploadedBottleneck) CleanUp() error { return os.RemoveAll(ubn.Dir) } |
2191
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
66 |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
67 // Do executes the actual uploaded bottleneck import. |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
68 func (ubn *UploadedBottleneck) Do( |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
69 ctx context.Context, |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
70 importID int64, |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
71 conn *sql.Conn, |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
72 feedback Feedback, |
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
73 ) (interface{}, error) { |
2195
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
74 |
2253
cfc523c70e90
Uploaded imports: Small code clean-ups.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2195
diff
changeset
|
75 fetch := func() ([]*ifbn.BottleNeckType, error) { |
2195
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
76 var dst ifbn.Export_bn_by_isrsResponse |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
77 if err := soap.ValidateFile( |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
78 filepath.Join(ubn.Dir, "data.xml"), |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
79 "IFBN.xsd", |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
80 &dst, |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
81 ); err != nil { |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
82 return nil, err |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
83 } |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
84 |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
85 if dst.Export_bn_by_isrsResult == nil { |
4177
8b75ac5e243e
Made 'staticcheck' happy with pgxutils package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3219
diff
changeset
|
86 return nil, errors.New("no bottlenecks found") |
2195
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
87 } |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
88 |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
89 return dst.Export_bn_by_isrsResult.BottleNeckType, nil |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
90 } |
3437ceee1529
Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2191
diff
changeset
|
91 |
2563
dc4fae4bdb8f
Expose axis snapping tolerance to users
Tom Gottfried <tom@intevation.de>
parents:
2253
diff
changeset
|
92 return storeBottlenecks( |
dc4fae4bdb8f
Expose axis snapping tolerance to users
Tom Gottfried <tom@intevation.de>
parents:
2253
diff
changeset
|
93 ctx, fetch, importID, conn, feedback, ubn.Tolerance) |
2191
4b9496752d89
Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
94 } |