Mercurial > gemma

annotate pkg/imports/ufa.go @ 5560:f2204f91d286

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Join the log lines of imports to the log exports to recover data from them. Used in SR export to extract information that where in the meta json but now are only found in the log.
author Sascha L. Teichmann <sascha.teichmann@intevation.de>
date Wed, 09 Feb 2022 18:34:40 +0100
parents 56c589f7435d
children 2dd155cc95ec
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2203
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
3 //
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
6 //
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 //
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 // Author(s):
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
13
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14 package imports
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import (
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 "context"
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 "database/sql"
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 "errors"
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
20 "os"
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
21 "path/filepath"
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
22
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
23 "gemma.intevation.de/gemma/pkg/soap"
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
24 "gemma.intevation.de/gemma/pkg/soap/ifaf"
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
25 )
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26
4182
49012340336c Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4179
diff changeset
27 // UploadedFairwayAvailability is Job to extract
49012340336c Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4179
diff changeset
28 // fairway availability data from an XML file and stores
49012340336c Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4179
diff changeset
29 // it into the database.
2203
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
30 type UploadedFairwayAvailability struct {
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
31 Dir string
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
32 }
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
33
4182
49012340336c Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4179
diff changeset
34 // UFAJobKind is the unique name of this import job type.
2203
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
35 const UFAJobKind JobKind = "ufa"
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
36
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
37 type ufaJobCreator struct{}
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
38
4179
a376351d2774 Another round of doc strings to make 'golint' a little bit more happy with the imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4177
diff changeset
39 func init() { RegisterJobCreator(UFAJobKind, ufaJobCreator{}) }
2203
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
40
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
41 func (ufaJobCreator) Description() string {
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
42 return "uploaded fairway availability"
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
43 }
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
44
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
45 func (ufaJobCreator) Create() Job { return new(UploadedFairwayAvailability) }
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
46
3219
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2204
diff changeset
47 func (ufaJobCreator) Depends() [2][]string {
2203
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
48 // Same as faJobCreator
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
49 return faJobCreator{}.Depends()
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
50 }
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
51
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
52 func (ufaJobCreator) AutoAccept() bool { return true }
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
53
5035
56c589f7435d Enhance comments on implementations of StageDone()
Tom Gottfried <tom@intevation.de>
parents: 5034
diff changeset
54 // StageDone is a NOP for fairway availability imports.
5034
59a99655f34d Added feedback support for StageDone.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4182
diff changeset
55 func (ufaJobCreator) StageDone(context.Context, *sql.Tx, int64, Feedback) error {
2203
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
56 return nil
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
57 }
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
58
4182
49012340336c Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4179
diff changeset
59 // CleanUp removes the temporary files from the filesystem.
2203
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
60 func (ufa *UploadedFairwayAvailability) CleanUp() error {
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
61 return os.RemoveAll(ufa.Dir)
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
62 }
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
64 // Do executes the actual uploaded fairway availability import.
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
65 func (ufa *UploadedFairwayAvailability) Do(
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
66 ctx context.Context,
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
67 importID int64,
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
68 conn *sql.Conn,
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
69 feedback Feedback,
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
70 ) (interface{}, error) {
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
71
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
72 fetch := func(
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
73 ctx context.Context,
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
74 tx *sql.Tx,
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
75 bns bottlenecks,
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
76 ) ([]*ifaf.FairwayAvailability, error) {
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
77
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
78 var response ifaf.Get_bottleneck_faResponse
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
79
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
80 if err := soap.ValidateFile(
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
81 filepath.Join(ufa.Dir, "data.xml"),
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
82 "IFAF.xsd",
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
83 &response,
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
84 ); err != nil {
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
85 return nil, err
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
86 }
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
87
2204
25f73251a6ac Uploaded fairway availability import: cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2203
diff changeset
88 result := response.Get_bottleneck_faResult
25f73251a6ac Uploaded fairway availability import: cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2203
diff changeset
89 if result == nil {
4177
8b75ac5e243e Made 'staticcheck' happy with pgxutils package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
90 return nil, errors.New("no bottlenecks found")
2203
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
91 }
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
92
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
93 return result.FairwayAvailability, nil
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
94 }
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
95
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
96 return storeFairwayAvailability(ctx, conn, feedback, fetch)
8d1a945d0c3b Uploaded fairway availabilty import: Implemented in terms of normal fairway availabilty import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
97 }