Mercurial > gemma

annotate pkg/imports/ubn.go @ 5591:0011f50cf216 surveysperbottleneckid

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Removed no longer used alternative api for surveys/ endpoint. As bottlenecks in the summary for SR imports are now identified by their id and no longer by the (not guarantied to be unique!) name, there is no longer the need to request survey data by the name+date tuple (which isn't reliable anyway). So the workaround was now reversed.
author Sascha Wilde <wilde@sha-bang.de>
date Wed, 06 Apr 2022 13:30:29 +0200
parents 59a99655f34d
children 6270951dda28
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2191
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
3 //
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
6 //
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 //
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 // Author(s):
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
13
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14 package imports
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import (
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 "context"
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 "database/sql"
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 "errors"
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
20 "os"
2195
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
21 "path/filepath"
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
22
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
23 "gemma.intevation.de/gemma/pkg/soap"
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
24 "gemma.intevation.de/gemma/pkg/soap/ifbn"
2191
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
25 )
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26
4179
a376351d2774 Another round of doc strings to make 'golint' a little bit more happy with the imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4177
diff changeset
27 // UploadedBottleneck is a Job to import bottlenecks from
a376351d2774 Another round of doc strings to make 'golint' a little bit more happy with the imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4177
diff changeset
28 // an uploaded XML file into the database.
2191
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
29 type UploadedBottleneck struct {
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
30 Dir string `json:"dir"`
2563
dc4fae4bdb8f Expose axis snapping tolerance to users
Tom Gottfried <tom@intevation.de>
parents: 2253
diff changeset
31 // Tolerance used for axis snapping
dc4fae4bdb8f Expose axis snapping tolerance to users
Tom Gottfried <tom@intevation.de>
parents: 2253
diff changeset
32 Tolerance float64 `json:"tolerance"`
2191
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
33 }
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
34
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
35 // UBNJobKind is the import queue type identifier.
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
36 const UBNJobKind JobKind = "ubn"
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
37
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
38 type ubnJobCreator struct{}
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
39
2253
cfc523c70e90 Uploaded imports: Small code clean-ups.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2195
diff changeset
40 func init() { RegisterJobCreator(UBNJobKind, ubnJobCreator{}) }
2191
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
41
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
42 func (ubnJobCreator) Description() string { return "uploaded bottlenecks" }
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
43
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
44 func (ubnJobCreator) AutoAccept() bool { return false }
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
45
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
46 func (ubnJobCreator) Create() Job { return new(UploadedBottleneck) }
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
47
3219
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2563
diff changeset
48 func (ubnJobCreator) Depends() [2][]string {
2195
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
49 // Same as normal bottleneck import.
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
50 return bnJobCreator{}.Depends()
2191
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
51 }
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
52
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
53 // StageDone moves the imported bottleneck out of the staging area.
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
54 func (ubnJobCreator) StageDone(
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
55 ctx context.Context,
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
56 tx *sql.Tx,
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
57 id int64,
5034
59a99655f34d Added feedback support for StageDone.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4179
diff changeset
58 feedback Feedback,
2191
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
59 ) error {
2195
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
60 // Same as normal bottleneck import.
5034
59a99655f34d Added feedback support for StageDone.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4179
diff changeset
61 return bnJobCreator{}.StageDone(ctx, tx, id, feedback)
2191
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
62 }
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
64 // CleanUp of a uploaded bottleneck import removes the temp dir.
2253
cfc523c70e90 Uploaded imports: Small code clean-ups.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2195
diff changeset
65 func (ubn *UploadedBottleneck) CleanUp() error { return os.RemoveAll(ubn.Dir) }
2191
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
66
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
67 // Do executes the actual uploaded bottleneck import.
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
68 func (ubn *UploadedBottleneck) Do(
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
69 ctx context.Context,
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
70 importID int64,
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
71 conn *sql.Conn,
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
72 feedback Feedback,
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
73 ) (interface{}, error) {
2195
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
74
2253
cfc523c70e90 Uploaded imports: Small code clean-ups.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2195
diff changeset
75 fetch := func() ([]*ifbn.BottleNeckType, error) {
2195
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
76 var dst ifbn.Export_bn_by_isrsResponse
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
77 if err := soap.ValidateFile(
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
78 filepath.Join(ubn.Dir, "data.xml"),
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
79 "IFBN.xsd",
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
80 &dst,
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
81 ); err != nil {
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
82 return nil, err
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
83 }
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
84
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
85 if dst.Export_bn_by_isrsResult == nil {
4177
8b75ac5e243e Made 'staticcheck' happy with pgxutils package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
86 return nil, errors.New("no bottlenecks found")
2195
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
87 }
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
88
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
89 return dst.Export_bn_by_isrsResult.BottleNeckType, nil
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
90 }
3437ceee1529 Uploaded bottleneck import: Implemented in terms of normal bottleneck import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2191
diff changeset
91
2563
dc4fae4bdb8f Expose axis snapping tolerance to users
Tom Gottfried <tom@intevation.de>
parents: 2253
diff changeset
92 return storeBottlenecks(
dc4fae4bdb8f Expose axis snapping tolerance to users
Tom Gottfried <tom@intevation.de>
parents: 2253
diff changeset
93 ctx, fetch, importID, conn, feedback, ubn.Tolerance)
2191
4b9496752d89 Upload bottleneck import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
94 }