Mercurial > gemma

annotate pkg/imports/scheduled.go @ 1828:1ecfcf46e4da

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Waterway area/axis: Be more informative if features are bad.
author Sascha L. Teichmann <sascha.teichmann@intevation.de>
date Wed, 16 Jan 2019 15:35:23 +0100
parents 77582da3adb0
children bbd653a43a6a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1551
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
3 //
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
6 //
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 //
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 // Author(s):
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
13
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14 package imports
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import (
1748
c11807dc6940 Comfigured imports: Run enqueing of configured imports if triggered by REST with the database connection of the logged in user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1746
diff changeset
17 "context"
c11807dc6940 Comfigured imports: Run enqueing of configured imports if triggered by REST with the database connection of the logged in user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1746
diff changeset
18 "database/sql"
1705
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
19 "errors"
1729
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
20 "fmt"
1551
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
21 "log"
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
22
1661
51a0ba4ede41 Scheduled imports: Made gauge measurement import able to be scheduled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1654
diff changeset
23 "gemma.intevation.de/gemma/pkg/common"
1551
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
24 "gemma.intevation.de/gemma/pkg/scheduler"
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
25 )
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26
1729
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
27 // JobKindSetups maps JobKinds to special setup functions.
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
28 var JobKindSetups = map[JobKind]func(*IDConfig) (interface{}, error){
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
29
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
30 GMJobKind: func(cfg *IDConfig) (interface{}, error) {
1665
da0d1a19ebe6 Fairway availability import: Made schedulable, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1661
diff changeset
31 log.Println("info: schedule 'gm' import")
1705
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
32 insecure := cfg.Attributes.Bool("insecure")
1665
da0d1a19ebe6 Fairway availability import: Made schedulable, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1661
diff changeset
33 return &GaugeMeasurement{
da0d1a19ebe6 Fairway availability import: Made schedulable, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1661
diff changeset
34 URL: *cfg.URL,
1705
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
35 Insecure: insecure,
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
36 }, nil
1729
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
37 },
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
38
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
39 FAJobKind: func(cfg *IDConfig) (interface{}, error) {
1665
da0d1a19ebe6 Fairway availability import: Made schedulable, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1661
diff changeset
40 log.Println("info: schedule 'fa' import")
1705
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
41 insecure := cfg.Attributes.Bool("insecure")
1665
da0d1a19ebe6 Fairway availability import: Made schedulable, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1661
diff changeset
42 return &FairwayAvailability{
da0d1a19ebe6 Fairway availability import: Made schedulable, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1661
diff changeset
43 URL: *cfg.URL,
1705
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
44 Insecure: insecure,
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
45 }, nil
1729
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
46 },
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
47
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
48 BNJobKind: func(cfg *IDConfig) (interface{}, error) {
1666
56b29406a163 Bottleneck import: Made schedulable, too. Even if it's not required.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1665
diff changeset
49 log.Println("info: schedule 'bn' import")
1705
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
50 insecure := cfg.Attributes.Bool("insecure")
1666
56b29406a163 Bottleneck import: Made schedulable, too. Even if it's not required.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1665
diff changeset
51 return &Bottleneck{
56b29406a163 Bottleneck import: Made schedulable, too. Even if it's not required.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1665
diff changeset
52 URL: *cfg.URL,
1705
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
53 Insecure: insecure,
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
54 }, nil
1729
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
55 },
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
56
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
57 WXJobKind: func(cfg *IDConfig) (interface{}, error) {
1678
8fdb57173e3f Waterway axis import: Made schedulable, too. TODO: Remove hard-coded parameters.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1666
diff changeset
58 log.Println("info: schedule 'wx' import")
1705
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
59 ft, found := cfg.Attributes.Get("feature-type")
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
60 if !found {
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
61 return nil, errors.New("cannot find 'feature-type' attribute")
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
62 }
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
63 sb, found := cfg.Attributes.Get("sort-by")
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
64 if !found {
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
65 return nil, errors.New("cannot find 'sort-by' attribute")
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
66 }
1678
8fdb57173e3f Waterway axis import: Made schedulable, too. TODO: Remove hard-coded parameters.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1666
diff changeset
67 return &WaterwayAxis{
8fdb57173e3f Waterway axis import: Made schedulable, too. TODO: Remove hard-coded parameters.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1666
diff changeset
68 URL: *cfg.URL,
1705
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
69 FeatureType: ft,
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
70 SortBy: sb,
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
71 }, nil
1729
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
72 },
1785
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
73
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
74 WAJobKind: func(cfg *IDConfig) (interface{}, error) {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
75 log.Println("info: schedule 'wa' import")
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
76 ft, found := cfg.Attributes.Get("feature-type")
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
77 if !found {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
78 return nil, errors.New("cannot find 'feature-type' attribute")
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
79 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
80 sb, found := cfg.Attributes.Get("sort-by")
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
81 if !found {
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
82 return nil, errors.New("cannot find 'sort-by' attribute")
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
83 }
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
84 return &WaterwayArea{
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
85 URL: *cfg.URL,
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
86 FeatureType: ft,
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
87 SortBy: sb,
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
88 }, nil
614c6c766691 Waterway area import: Implemented.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 1754
diff changeset
89 },
1808
77582da3adb0 Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
90
77582da3adb0 Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
91 WGJobKind: func(cfg *IDConfig) (interface{}, error) {
77582da3adb0 Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
92 log.Println("info: schedule 'wg' import")
77582da3adb0 Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
93 username, _ := cfg.Attributes.Get("username")
77582da3adb0 Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
94 password, _ := cfg.Attributes.Get("password")
77582da3adb0 Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
95 insecure := cfg.Attributes.Bool("insecure")
77582da3adb0 Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
96 return &WaterwayGauge{
77582da3adb0 Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
97 URL: *cfg.URL,
77582da3adb0 Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
98 Username: username,
77582da3adb0 Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
99 Password: password,
77582da3adb0 Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
100 Insecure: insecure,
77582da3adb0 Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
101 }, nil
77582da3adb0 Waterway gauges import: Added stub.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1785
diff changeset
102 },
1551
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
103 }
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
104
1729
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
105 func init() {
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
106 run := func(cfgID int64) {
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
107 jobID, err := RunConfiguredImport(cfgID)
1665
da0d1a19ebe6 Fairway availability import: Made schedulable, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1661
diff changeset
108 if err != nil {
1729
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
109 log.Printf("error: running scheduled import failed: %v\n", err)
1705
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
110 return
dcbe2a7dc532 Scheduled imports: Take extra configuration parameters from attributes table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1678
diff changeset
111 }
1665
da0d1a19ebe6 Fairway availability import: Made schedulable, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1661
diff changeset
112 log.Printf("info: added import #%d to queue\n", jobID)
1661
51a0ba4ede41 Scheduled imports: Made gauge measurement import able to be scheduled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1654
diff changeset
113 }
51a0ba4ede41 Scheduled imports: Made gauge measurement import able to be scheduled.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1654
diff changeset
114
1729
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
115 for kind := range JobKindSetups {
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
116 scheduler.RegisterAction(string(kind), run)
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
117 }
1551
d9eba69f6515 Registered a dummy callback function for scheduled fairway imports in the scheduler.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
118 }
1729
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
119
1748
c11807dc6940 Comfigured imports: Run enqueing of configured imports if triggered by REST with the database connection of the logged in user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1746
diff changeset
120 // RunConfiguredImportContext runs an import configured from the database.
c11807dc6940 Comfigured imports: Run enqueing of configured imports if triggered by REST with the database connection of the logged in user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1746
diff changeset
121 func RunConfiguredImportContext(ctx context.Context, conn *sql.Conn, id int64) (int64, error) {
c11807dc6940 Comfigured imports: Run enqueing of configured imports if triggered by REST with the database connection of the logged in user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1746
diff changeset
122 cfg, err := LoadIDConfigContext(ctx, conn, id)
c11807dc6940 Comfigured imports: Run enqueing of configured imports if triggered by REST with the database connection of the logged in user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1746
diff changeset
123 return runConfiguredImport(id, cfg, err)
c11807dc6940 Comfigured imports: Run enqueing of configured imports if triggered by REST with the database connection of the logged in user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1746
diff changeset
124 }
c11807dc6940 Comfigured imports: Run enqueing of configured imports if triggered by REST with the database connection of the logged in user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1746
diff changeset
125
1729
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
126 // RunConfiguredImport runs an import configured from the database.
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
127 func RunConfiguredImport(id int64) (int64, error) {
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
128 cfg, err := loadIDConfig(id)
1748
c11807dc6940 Comfigured imports: Run enqueing of configured imports if triggered by REST with the database connection of the logged in user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1746
diff changeset
129 return runConfiguredImport(id, cfg, err)
c11807dc6940 Comfigured imports: Run enqueing of configured imports if triggered by REST with the database connection of the logged in user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1746
diff changeset
130 }
c11807dc6940 Comfigured imports: Run enqueing of configured imports if triggered by REST with the database connection of the logged in user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1746
diff changeset
131
c11807dc6940 Comfigured imports: Run enqueing of configured imports if triggered by REST with the database connection of the logged in user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1746
diff changeset
132 func runConfiguredImport(id int64, cfg *IDConfig, err error) (int64, error) {
c11807dc6940 Comfigured imports: Run enqueing of configured imports if triggered by REST with the database connection of the logged in user.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1746
diff changeset
133
1729
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
134 if err != nil {
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
135 return 0, err
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
136 }
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
137 if cfg == nil {
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
138 return 0, fmt.Errorf("no config found for id %d.\n", id)
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
139 }
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
140 if cfg.URL == nil {
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
141 return 0, errors.New("error: No URL specified")
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
142 }
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
143
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
144 kind := JobKind(cfg.Kind)
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
145
1746
09f51f2b3b0d Cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1729
diff changeset
146 setup := JobKindSetups[kind]
09f51f2b3b0d Cosmetics.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1729
diff changeset
147 if setup == nil {
1729
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
148 return 0, fmt.Errorf("unknown job kind: %s", cfg.Kind)
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
149 }
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
150
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
151 what, err := setup(cfg)
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
152 if err != nil {
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
153 return 0, err
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
154 }
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
155
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
156 var serialized string
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
157 if serialized, err = common.ToJSONString(what); err != nil {
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
158 return 0, err
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
159 }
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
160
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
161 due, _ := cfg.Attributes.Time("due")
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
162
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
163 retries, found := cfg.Attributes.Int("retries")
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
164 if !found {
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
165 retries = -1
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
166 }
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
167
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
168 var jobID int64
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
169 if jobID, err = AddJob(
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
170 kind,
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
171 due, retries,
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
172 cfg.User,
1754
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1748
diff changeset
173 cfg.SendEMail,
1729
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
174 serialized,
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
175 ); err != nil {
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
176 return 0, err
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
177 }
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
178
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
179 return jobID, nil
74f7d4c531bc Scheduled imports: Refactored calling of database configured import to be reusable elsewhere.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1708
diff changeset
180 }