Mercurial > gemma
annotate pkg/controllers/importqueue.go @ 5349:a352a695b69c extented-report
XLSX templater: Added column to coord conversions.
author | Sascha L. Teichmann <sascha.teichmann@intevation.de> |
---|---|
date | Mon, 21 Jun 2021 18:10:49 +0200 |
parents | adf7b9f1273b |
children | 5f47eeea988d |
rev | line source |
---|---|
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
1 // This is Free Software under GNU Affero General Public License v >= 3.0 |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
2 // without warranty, see README.md and license for details. |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
3 // |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
4 // SPDX-License-Identifier: AGPL-3.0-or-later |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
5 // License-Filename: LICENSES/AGPL-3.0.txt |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
6 // |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
7 // Copyright (C) 2018 by via donau |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
8 // – Österreichische Wasserstraßen-Gesellschaft mbH |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
9 // Software engineering by Intevation GmbH |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
10 // |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
11 // Author(s): |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de> |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
13 |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
14 package controllers |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
15 |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
16 import ( |
2663
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
17 "context" |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
18 "database/sql" |
4791
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
19 "encoding/csv" |
1392
0e1d89241cda
Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1359
diff
changeset
|
20 "encoding/json" |
1031
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
21 "fmt" |
1189
3d50f558870c
REST GET call to /imports now has the ability to be filtered by kinds or states.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1037
diff
changeset
|
22 "log" |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
23 "net/http" |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
24 "strconv" |
1189
3d50f558870c
REST GET call to /imports now has the ability to be filtered by kinds or states.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1037
diff
changeset
|
25 "strings" |
5123
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
26 "sync" |
2663
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
27 "time" |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
28 |
1392
0e1d89241cda
Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1359
diff
changeset
|
29 "github.com/gorilla/mux" |
0e1d89241cda
Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1359
diff
changeset
|
30 |
1194
7db850de0952
Added a signer who makes the final decison on an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1193
diff
changeset
|
31 "gemma.intevation.de/gemma/pkg/auth" |
4078
80bdcd137a1d
Parse timezones from time inputs and send timezones in results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3776
diff
changeset
|
32 "gemma.intevation.de/gemma/pkg/common" |
1189
3d50f558870c
REST GET call to /imports now has the ability to be filtered by kinds or states.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1037
diff
changeset
|
33 "gemma.intevation.de/gemma/pkg/imports" |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
34 "gemma.intevation.de/gemma/pkg/models" |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
35 |
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
36 mw "gemma.intevation.de/gemma/pkg/middleware" |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
37 ) |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
38 |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
39 const ( |
4187
65a5501dc13d
Enable faster access to import logs
Tom Gottfried <tom@intevation.de>
parents:
4148
diff
changeset
|
40 selectImportsCountSQL = ` |
2682
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
41 SELECT count(*) |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
42 FROM import.imports |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
43 WHERE |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
44 ` |
4187
65a5501dc13d
Enable faster access to import logs
Tom Gottfried <tom@intevation.de>
parents:
4148
diff
changeset
|
45 selectImportsSQL = ` |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
46 SELECT |
2627
3a242e6aa56d
Import log: Add filter for log entries with warnings only: GET /api/imports?warnings=true
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2179
diff
changeset
|
47 imports.id AS id, |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
48 state::varchar, |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
49 enqueued, |
4748
47922c1a088d
Added a 'changed' column to the import.imports table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4244
diff
changeset
|
50 changed, |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
51 kind, |
1194
7db850de0952
Added a signer who makes the final decison on an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1193
diff
changeset
|
52 username, |
1392
0e1d89241cda
Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1359
diff
changeset
|
53 signer, |
2663
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
54 summary IS NOT NULL AS has_summary, |
4187
65a5501dc13d
Enable faster access to import logs
Tom Gottfried <tom@intevation.de>
parents:
4148
diff
changeset
|
55 EXISTS(SELECT 1 FROM import.import_logs |
4196
948e312e87bc
Add flag and filter for imports having errors
Tom Gottfried <tom@intevation.de>
parents:
4195
diff
changeset
|
56 WHERE kind = 'error'::log_type and import_id = imports.id) AS has_errors, |
948e312e87bc
Add flag and filter for imports having errors
Tom Gottfried <tom@intevation.de>
parents:
4195
diff
changeset
|
57 EXISTS(SELECT 1 FROM import.import_logs |
4187
65a5501dc13d
Enable faster access to import logs
Tom Gottfried <tom@intevation.de>
parents:
4148
diff
changeset
|
58 WHERE kind = 'warn'::log_type and import_id = imports.id) AS has_warnings |
1995
59055c8301df
Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents:
1693
diff
changeset
|
59 FROM import.imports |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
60 WHERE |
1189
3d50f558870c
REST GET call to /imports now has the ability to be filtered by kinds or states.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1037
diff
changeset
|
61 ` |
4791
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
62 selectExportSQL = ` |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
63 SELECT |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
64 imports.id AS id, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
65 state::varchar, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
66 enqueued, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
67 changed, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
68 kind, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
69 username, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
70 (SELECT country FROM users.list_users lu |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
71 WHERE lu.username = import.imports.username) AS country, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
72 signer, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
73 EXISTS(SELECT 1 FROM import.import_logs |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
74 WHERE kind = 'warn'::log_type and import_id = imports.id) AS has_warnings, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
75 data |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
76 FROM import.imports |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
77 WHERE |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
78 ` |
4187
65a5501dc13d
Enable faster access to import logs
Tom Gottfried <tom@intevation.de>
parents:
4148
diff
changeset
|
79 selectEnqueuedSQL = ` |
2672
b997e1fd1d3d
Fixed warning SQL prefix for selection.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2671
diff
changeset
|
80 SELECT enqueued FROM import.imports |
b997e1fd1d3d
Fixed warning SQL prefix for selection.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2671
diff
changeset
|
81 WHERE |
b997e1fd1d3d
Fixed warning SQL prefix for selection.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2671
diff
changeset
|
82 ` |
2795
241e7f05a538
/api/imports/{id} result now also contains the 'enqueued' time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2772
diff
changeset
|
83 selectImportSummarySQL = ` |
241e7f05a538
/api/imports/{id} result now also contains the 'enqueued' time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2772
diff
changeset
|
84 SELECT summary, enqueued FROM import.imports WHERE id = $1` |
1031
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
85 |
1037
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
86 selectHasNoRunningImportSQL = ` |
1995
59055c8301df
Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents:
1693
diff
changeset
|
87 SELECT true FROM import.imports |
59055c8301df
Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents:
1693
diff
changeset
|
88 WHERE id = $1 AND state <> 'running'::import_state` |
1037
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
89 |
1026
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
90 selectImportLogsSQL = ` |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
91 SELECT |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
92 time, |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
93 kind::varchar, |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
94 msg |
1995
59055c8301df
Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents:
1693
diff
changeset
|
95 FROM import.import_logs |
1026
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
96 WHERE import_id = $1 |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
97 ORDER BY time` |
1037
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
98 |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
99 deleteLogsSQL = ` |
1995
59055c8301df
Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents:
1693
diff
changeset
|
100 DELETE FROM import.import_logs WHERE import_id = $1` |
1037
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
101 |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
102 deleteImportSQL = ` |
1995
59055c8301df
Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents:
1693
diff
changeset
|
103 DELETE FROM import.imports WHERE id = $1` |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
104 ) |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
105 |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
106 type filledStmt struct { |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
107 stmt strings.Builder |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
108 args []interface{} |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
109 } |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
110 |
4791
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
111 func buildFilters(projection string, req *http.Request) (*filledStmt, *filledStmt, *filledStmt, error) { |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
112 |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
113 var l, a, b filterAnd |
1189
3d50f558870c
REST GET call to /imports now has the ability to be filtered by kinds or states.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1037
diff
changeset
|
114 |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
115 var noBefore, noAfter bool |
1189
3d50f558870c
REST GET call to /imports now has the ability to be filtered by kinds or states.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1037
diff
changeset
|
116 |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
117 cond := func(format string, args ...interface{}) { |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
118 term := &filterTerm{format: format, args: args} |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
119 l = append(l, term) |
4195
ad13c581de7c
Fixed building filters for listing import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4187
diff
changeset
|
120 a = append(a, term) |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
121 b = append(b, term) |
2677
fe93c48f76c9
Add /api/imports?query=TXT to search TXT in kind, username, signer or logs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2673
diff
changeset
|
122 } |
fe93c48f76c9
Add /api/imports?query=TXT to search TXT in kind, username, signer or logs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2673
diff
changeset
|
123 |
5108
89639e02cff0
Do not show internal review jobs to users
Tom Gottfried <tom@intevation.de>
parents:
5093
diff
changeset
|
124 // Always filter review jobs. They are only for internal use. |
5109
c0ceec7e6f85
Define constants for review jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5108
diff
changeset
|
125 cond(` NOT kind LIKE '%%` + imports.ReviewJobSuffix + `'`) |
5108
89639e02cff0
Do not show internal review jobs to users
Tom Gottfried <tom@intevation.de>
parents:
5093
diff
changeset
|
126 |
2677
fe93c48f76c9
Add /api/imports?query=TXT to search TXT in kind, username, signer or logs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2673
diff
changeset
|
127 if query := req.FormValue("query"); query != "" { |
fe93c48f76c9
Add /api/imports?query=TXT to search TXT in kind, username, signer or logs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2673
diff
changeset
|
128 query = "%" + query + "%" |
fe93c48f76c9
Add /api/imports?query=TXT to search TXT in kind, username, signer or logs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2673
diff
changeset
|
129 cond(` (kind ILIKE $%d OR username ILIKE $%d OR signer ILIKE $%d OR `+ |
fe93c48f76c9
Add /api/imports?query=TXT to search TXT in kind, username, signer or logs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2673
diff
changeset
|
130 `id IN (SELECT import_id FROM import.import_logs WHERE msg ILIKE $%d)) `, |
fe93c48f76c9
Add /api/imports?query=TXT to search TXT in kind, username, signer or logs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2673
diff
changeset
|
131 query, query, query, query) |
fe93c48f76c9
Add /api/imports?query=TXT to search TXT in kind, username, signer or logs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2673
diff
changeset
|
132 } |
fe93c48f76c9
Add /api/imports?query=TXT to search TXT in kind, username, signer or logs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2673
diff
changeset
|
133 |
3776
6521c962a7b6
Add 'cc' parameter to search imports for a comma separated list of country codes for the the importing person.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3217
diff
changeset
|
134 if cc := req.FormValue("cc"); cc != "" { |
6521c962a7b6
Add 'cc' parameter to search imports for a comma separated list of country codes for the the importing person.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3217
diff
changeset
|
135 codes := sliceToTextArray(splitUpper(cc)) |
6521c962a7b6
Add 'cc' parameter to search imports for a comma separated list of country codes for the the importing person.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3217
diff
changeset
|
136 cond(" username IN "+ |
6521c962a7b6
Add 'cc' parameter to search imports for a comma separated list of country codes for the the importing person.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3217
diff
changeset
|
137 "(SELECT username FROM internal.user_profiles "+ |
6521c962a7b6
Add 'cc' parameter to search imports for a comma separated list of country codes for the the importing person.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3217
diff
changeset
|
138 "WHERE country = ANY($%d)) ", |
6521c962a7b6
Add 'cc' parameter to search imports for a comma separated list of country codes for the the importing person.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3217
diff
changeset
|
139 codes) |
6521c962a7b6
Add 'cc' parameter to search imports for a comma separated list of country codes for the the importing person.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3217
diff
changeset
|
140 } |
6521c962a7b6
Add 'cc' parameter to search imports for a comma separated list of country codes for the the importing person.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3217
diff
changeset
|
141 |
2667
5ece2c51d1f0
More filter simplification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2665
diff
changeset
|
142 if st := req.FormValue("states"); st != "" { |
5ece2c51d1f0
More filter simplification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2665
diff
changeset
|
143 states := toTextArray(st, imports.ImportStateNames) |
2677
fe93c48f76c9
Add /api/imports?query=TXT to search TXT in kind, username, signer or logs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2673
diff
changeset
|
144 cond(" state = ANY($%d) ", states) |
1189
3d50f558870c
REST GET call to /imports now has the ability to be filtered by kinds or states.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1037
diff
changeset
|
145 } |
3d50f558870c
REST GET call to /imports now has the ability to be filtered by kinds or states.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1037
diff
changeset
|
146 |
2667
5ece2c51d1f0
More filter simplification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2665
diff
changeset
|
147 if ks := req.FormValue("kinds"); ks != "" { |
5ece2c51d1f0
More filter simplification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2665
diff
changeset
|
148 kinds := toTextArray(ks, imports.ImportKindNames()) |
2677
fe93c48f76c9
Add /api/imports?query=TXT to search TXT in kind, username, signer or logs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2673
diff
changeset
|
149 cond(" kind = ANY($%d) ", kinds) |
1655
34315277f2d6
Added ids filter to importqueue listing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1495
diff
changeset
|
150 } |
34315277f2d6
Added ids filter to importqueue listing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1495
diff
changeset
|
151 |
2667
5ece2c51d1f0
More filter simplification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2665
diff
changeset
|
152 if idss := req.FormValue("ids"); idss != "" { |
5ece2c51d1f0
More filter simplification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2665
diff
changeset
|
153 ids := toInt8Array(idss) |
2677
fe93c48f76c9
Add /api/imports?query=TXT to search TXT in kind, username, signer or logs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2673
diff
changeset
|
154 cond(" id = ANY($%d) ", ids) |
2663
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
155 } |
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
156 |
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
157 if from := req.FormValue("from"); from != "" { |
4078
80bdcd137a1d
Parse timezones from time inputs and send timezones in results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3776
diff
changeset
|
158 fromTime, err := common.ParseTime(from) |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
159 if err != nil { |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
160 return nil, nil, nil, err |
2663
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
161 } |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
162 l = append(l, buildFilterTerm("enqueued >= $%d", fromTime)) |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
163 b = append(b, buildFilterTerm("enqueued < $%d", fromTime)) |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
164 } else { |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
165 noBefore = true |
2663
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
166 } |
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
167 |
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
168 if to := req.FormValue("to"); to != "" { |
4078
80bdcd137a1d
Parse timezones from time inputs and send timezones in results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3776
diff
changeset
|
169 toTime, err := common.ParseTime(to) |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
170 if err != nil { |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
171 return nil, nil, nil, err |
2663
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
172 } |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
173 l = append(l, buildFilterTerm("enqueued <= $%d", toTime)) |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
174 a = append(a, buildFilterTerm("enqueued > $%d", toTime)) |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
175 } else { |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
176 noAfter = true |
2663
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
177 } |
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
178 |
2665
fea12fc850d2
Simplified warnings filtering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2663
diff
changeset
|
179 switch warn := strings.ToLower(req.FormValue("warnings")); warn { |
fea12fc850d2
Simplified warnings filtering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2663
diff
changeset
|
180 case "1", "t", "true": |
4187
65a5501dc13d
Enable faster access to import logs
Tom Gottfried <tom@intevation.de>
parents:
4148
diff
changeset
|
181 cond(` EXISTS(SELECT 1 FROM import.import_logs |
65a5501dc13d
Enable faster access to import logs
Tom Gottfried <tom@intevation.de>
parents:
4148
diff
changeset
|
182 WHERE kind = 'warn'::log_type and import_id = imports.id)`) |
2665
fea12fc850d2
Simplified warnings filtering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2663
diff
changeset
|
183 } |
fea12fc850d2
Simplified warnings filtering.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2663
diff
changeset
|
184 |
4196
948e312e87bc
Add flag and filter for imports having errors
Tom Gottfried <tom@intevation.de>
parents:
4195
diff
changeset
|
185 switch errors := strings.ToLower(req.FormValue("errors")); errors { |
948e312e87bc
Add flag and filter for imports having errors
Tom Gottfried <tom@intevation.de>
parents:
4195
diff
changeset
|
186 case "1", "t", "true": |
948e312e87bc
Add flag and filter for imports having errors
Tom Gottfried <tom@intevation.de>
parents:
4195
diff
changeset
|
187 cond(` EXISTS(SELECT 1 FROM import.import_logs |
948e312e87bc
Add flag and filter for imports having errors
Tom Gottfried <tom@intevation.de>
parents:
4195
diff
changeset
|
188 WHERE kind = 'error'::log_type and import_id = imports.id)`) |
948e312e87bc
Add flag and filter for imports having errors
Tom Gottfried <tom@intevation.de>
parents:
4195
diff
changeset
|
189 } |
948e312e87bc
Add flag and filter for imports having errors
Tom Gottfried <tom@intevation.de>
parents:
4195
diff
changeset
|
190 |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
191 fl := &filledStmt{} |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
192 fa := &filledStmt{} |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
193 fb := &filledStmt{} |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
194 |
4187
65a5501dc13d
Enable faster access to import logs
Tom Gottfried <tom@intevation.de>
parents:
4148
diff
changeset
|
195 fa.stmt.WriteString(selectEnqueuedSQL) |
65a5501dc13d
Enable faster access to import logs
Tom Gottfried <tom@intevation.de>
parents:
4148
diff
changeset
|
196 fb.stmt.WriteString(selectEnqueuedSQL) |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
197 |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
198 var counting bool |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
199 |
4791
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
200 if projection != "" { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
201 fl.stmt.WriteString(projection) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
202 } else { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
203 switch count := strings.ToLower(req.FormValue("count")); count { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
204 case "1", "t", "true": |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
205 counting = true |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
206 fl.stmt.WriteString(selectImportsCountSQL) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
207 default: |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
208 fl.stmt.WriteString(selectImportsSQL) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
209 } |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
210 } |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
211 |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
212 if len(l) == 0 { |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
213 fl.stmt.WriteString(" TRUE ") |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
214 } else { |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
215 l.serialize(&fl.stmt, &fl.args) |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
216 } |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
217 |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
218 if len(b) == 0 { |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
219 fb.stmt.WriteString(" TRUE ") |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
220 } else { |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
221 b.serialize(&fb.stmt, &fb.args) |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
222 } |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
223 |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
224 if len(a) == 0 { |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
225 fa.stmt.WriteString(" TRUE ") |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
226 } else { |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
227 a.serialize(&fa.stmt, &fa.args) |
1655
34315277f2d6
Added ids filter to importqueue listing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1495
diff
changeset
|
228 } |
34315277f2d6
Added ids filter to importqueue listing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1495
diff
changeset
|
229 |
2682
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
230 if !counting { |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
231 fl.stmt.WriteString(" ORDER BY enqueued DESC ") |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
232 fa.stmt.WriteString(" ORDER BY enqueued LIMIT 1") |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
233 fb.stmt.WriteString(" ORDER BY enqueued DESC LIMIT 1") |
2682
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
234 } |
1189
3d50f558870c
REST GET call to /imports now has the ability to be filtered by kinds or states.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1037
diff
changeset
|
235 |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
236 if noBefore { |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
237 fb = nil |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
238 } |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
239 if noAfter { |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
240 fa = nil |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
241 } |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
242 |
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
243 return fl, fb, fa, nil |
2663
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
244 } |
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
245 |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
246 func neighbored(ctx context.Context, conn *sql.Conn, fb *filledStmt) *models.ImportTime { |
1189
3d50f558870c
REST GET call to /imports now has the ability to be filtered by kinds or states.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1037
diff
changeset
|
247 |
2663
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
248 var when time.Time |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
249 err := conn.QueryRowContext(ctx, fb.stmt.String(), fb.args...).Scan(&when) |
2663
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
250 switch { |
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
251 case err == sql.ErrNoRows: |
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
252 return nil |
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
253 case err != nil: |
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
254 log.Printf("warn: %v\n", err) |
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
255 return nil |
1189
3d50f558870c
REST GET call to /imports now has the ability to be filtered by kinds or states.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1037
diff
changeset
|
256 } |
4084
350a24c92848
Deliver times from import queue in UTC.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4078
diff
changeset
|
257 return &models.ImportTime{Time: when.UTC()} |
2663
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
258 } |
1189
3d50f558870c
REST GET call to /imports now has the ability to be filtered by kinds or states.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1037
diff
changeset
|
259 |
4791
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
260 func exportImports(rw http.ResponseWriter, req *http.Request) { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
261 |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
262 list, _, _, err := buildFilters(selectExportSQL, req) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
263 if err != nil { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
264 http.Error(rw, "error: "+err.Error(), http.StatusBadRequest) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
265 return |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
266 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
267 |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
268 rw.Header().Add("Content-Type", "text/csv") |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
269 out := csv.NewWriter(rw) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
270 |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
271 record := []string{ |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
272 "#id", |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
273 "#kind", |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
274 "#enqueued", |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
275 "#changed", |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
276 "#user", |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
277 "#country", |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
278 "#signer", |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
279 "#state", |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
280 "#warnings", |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
281 "#source", |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
282 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
283 |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
284 if err := out.Write(record); err != nil { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
285 // Too late for HTTP status message. |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
286 log.Printf("error: %v\n", err) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
287 return |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
288 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
289 |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
290 conn := mw.GetDBConn(req) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
291 ctx := req.Context() |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
292 var rows *sql.Rows |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
293 if rows, err = conn.QueryContext(ctx, list.stmt.String(), list.args...); err != nil { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
294 log.Printf("error: %v\n", err) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
295 return |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
296 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
297 defer rows.Close() |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
298 |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
299 stringString := func(s sql.NullString) string { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
300 if s.Valid { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
301 return s.String |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
302 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
303 return "" |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
304 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
305 |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
306 // Extract some meta infos from the import. |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
307 type Description interface { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
308 Description() (string, error) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
309 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
310 |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
311 for rows.Next() { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
312 var ( |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
313 id int64 |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
314 state string |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
315 enqueued time.Time |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
316 changed time.Time |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
317 kind string |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
318 user string |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
319 country string |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
320 signer sql.NullString |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
321 warnings bool |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
322 data string |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
323 description string |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
324 ) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
325 if err = rows.Scan( |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
326 &id, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
327 &state, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
328 &enqueued, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
329 &changed, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
330 &kind, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
331 &user, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
332 &country, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
333 &signer, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
334 &warnings, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
335 &data, |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
336 ); err != nil { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
337 return |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
338 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
339 |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
340 // Do some introspection on the job to be more verbose. |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
341 if jc := imports.FindJobCreator(imports.JobKind(kind)); jc != nil { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
342 job := jc.Create() |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
343 if err := common.FromJSONString(data, job); err != nil { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
344 log.Printf("error: %v\n", err) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
345 } else if desc, ok := job.(Description); ok { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
346 if description, err = desc.Description(); err != nil { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
347 log.Printf("error: %v\n", err) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
348 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
349 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
350 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
351 |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
352 record[0] = strconv.FormatInt(id, 10) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
353 record[1] = kind |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
354 record[2] = enqueued.UTC().Format(common.TimeFormat) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
355 record[3] = changed.UTC().Format(common.TimeFormat) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
356 record[4] = user |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
357 record[5] = country |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
358 record[6] = stringString(signer) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
359 record[7] = state |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
360 record[8] = strconv.FormatBool(warnings) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
361 record[9] = strings.Replace(description, ",", "|", -1) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
362 |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
363 if err := out.Write(record); err != nil { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
364 log.Printf("error: %v\n", err) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
365 return |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
366 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
367 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
368 |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
369 out.Flush() |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
370 if err := out.Error(); err != nil { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
371 log.Printf("error: %v\n", err) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
372 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
373 |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
374 if err = rows.Err(); err != nil { |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
375 log.Printf("error: %v\n", err) |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
376 return |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
377 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
378 } |
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
379 |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
380 func listImports(req *http.Request) (jr mw.JSONResult, err error) { |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
381 |
3194
eeff2cc4ff9d
controllers: re-factored the SQL filter to a tree like structure to be of more general use.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
3190
diff
changeset
|
382 var list, before, after *filledStmt |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
383 |
4791
1fef4679b07a
Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4748
diff
changeset
|
384 if list, before, after, err = buildFilters("", req); err != nil { |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
385 return |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
386 } |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
387 |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
388 ctx := req.Context() |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
389 |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
390 conn := mw.JSONConn(req) |
4242
1458c9b0fdaa
Made the sql.Conn in function accessible via the context of the request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4196
diff
changeset
|
391 |
2682
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
392 // Fast path for counting |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
393 |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
394 switch count := strings.ToLower(req.FormValue("count")); count { |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
395 case "1", "t", "true": |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
396 var count int64 |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
397 err = conn.QueryRowContext(ctx, list.stmt.String(), list.args...).Scan(&count) |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
398 switch { |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
399 case err == sql.ErrNoRows: |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
400 count, err = 0, nil |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
401 case err != nil: |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
402 return |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
403 } |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
404 jr = mw.JSONResult{Result: count} |
2682
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
405 return |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
406 } |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
407 |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
408 // Generate the list |
9a493d27bf3f
Added /api/imports?count=true to only count the results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2681
diff
changeset
|
409 |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
410 var rows *sql.Rows |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
411 if rows, err = conn.QueryContext(ctx, list.stmt.String(), list.args...); err != nil { |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
412 return |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
413 } |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
414 defer rows.Close() |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
415 |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
416 imports := make([]*models.Import, 0, 20) |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
417 |
2662
f90557a8c960
Moved summary from /api/imports overview to /api/imports/{id} details.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2627
diff
changeset
|
418 var signer sql.NullString |
1195
486d66a9565c
Be aware that the signer is null till the final decision is made about an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1194
diff
changeset
|
419 |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
420 for rows.Next() { |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
421 var it models.Import |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
422 var enqueued time.Time |
4748
47922c1a088d
Added a 'changed' column to the import.imports table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4244
diff
changeset
|
423 var changed time.Time |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
424 if err = rows.Scan( |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
425 &it.ID, |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
426 &it.State, |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
427 &enqueued, |
4748
47922c1a088d
Added a 'changed' column to the import.imports table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4244
diff
changeset
|
428 &changed, |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
429 &it.Kind, |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
430 &it.User, |
1195
486d66a9565c
Be aware that the signer is null till the final decision is made about an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1194
diff
changeset
|
431 &signer, |
2662
f90557a8c960
Moved summary from /api/imports overview to /api/imports/{id} details.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2627
diff
changeset
|
432 &it.Summary, |
4196
948e312e87bc
Add flag and filter for imports having errors
Tom Gottfried <tom@intevation.de>
parents:
4195
diff
changeset
|
433 &it.Errors, |
2179
20d9b71f4125
Import queue listing: Add another column 'warnings' (bool) to listing output which indicates if an import has warnings or not.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1995
diff
changeset
|
434 &it.Warnings, |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
435 ); err != nil { |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
436 return |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
437 } |
1195
486d66a9565c
Be aware that the signer is null till the final decision is made about an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1194
diff
changeset
|
438 if signer.Valid { |
486d66a9565c
Be aware that the signer is null till the final decision is made about an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1194
diff
changeset
|
439 it.Signer = signer.String |
486d66a9565c
Be aware that the signer is null till the final decision is made about an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1194
diff
changeset
|
440 } |
4084
350a24c92848
Deliver times from import queue in UTC.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4078
diff
changeset
|
441 it.Enqueued = models.ImportTime{Time: enqueued.UTC()} |
4748
47922c1a088d
Added a 'changed' column to the import.imports table.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4244
diff
changeset
|
442 it.Changed = models.ImportTime{Time: changed.UTC()} |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
443 imports = append(imports, &it) |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
444 } |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
445 |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
446 if err = rows.Err(); err != nil { |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
447 return |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
448 } |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
449 |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
450 var prev, next *models.ImportTime |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
451 |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
452 if before != nil { |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
453 prev = neighbored(ctx, conn, before) |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
454 } |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
455 |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
456 if after != nil { |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
457 next = neighbored(ctx, conn, after) |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
458 } |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
459 |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
460 jr = mw.JSONResult{ |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
461 Result: struct { |
2663
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
462 Prev *models.ImportTime `json:"prev,omitempty"` |
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
463 Next *models.ImportTime `json:"next,omitempty"` |
386ff766dfcd
Replaced limit/offset filters with from/to as timestamps (format '2006-01-02T15:04:05.000').The output document also contains 'prev' and 'next' fields (if there is any matching data) from the times before and after 'from' and 'to'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2662
diff
changeset
|
464 Imports []*models.Import `json:"imports"` |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
465 }{ |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
466 Imports: imports, |
2671
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
467 Prev: prev, |
8f3facf902dd
Filter prev/next, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2670
diff
changeset
|
468 Next: next, |
1023
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
469 }, |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
470 } |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
471 return |
337a7f4c8a16
Add endpoint to list all or some import jobs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
472 } |
1026
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
473 |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
474 func importLogs(req *http.Request) (jr mw.JSONResult, err error) { |
1031
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
475 |
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
476 ctx := req.Context() |
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
477 |
1026
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
478 id, _ := strconv.ParseInt(mux.Vars(req)["id"], 10, 64) |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
479 |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
480 conn := mw.JSONConn(req) |
4242
1458c9b0fdaa
Made the sql.Conn in function accessible via the context of the request.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4196
diff
changeset
|
481 |
1031
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
482 // Check if he have such a import job first. |
2662
f90557a8c960
Moved summary from /api/imports overview to /api/imports/{id} details.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2627
diff
changeset
|
483 var summary sql.NullString |
2795
241e7f05a538
/api/imports/{id} result now also contains the 'enqueued' time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2772
diff
changeset
|
484 var enqueued time.Time |
241e7f05a538
/api/imports/{id} result now also contains the 'enqueued' time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2772
diff
changeset
|
485 err = conn.QueryRowContext(ctx, selectImportSummarySQL, id).Scan( |
241e7f05a538
/api/imports/{id} result now also contains the 'enqueued' time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2772
diff
changeset
|
486 &summary, |
241e7f05a538
/api/imports/{id} result now also contains the 'enqueued' time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2772
diff
changeset
|
487 &enqueued, |
241e7f05a538
/api/imports/{id} result now also contains the 'enqueued' time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2772
diff
changeset
|
488 ) |
1031
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
489 switch { |
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
490 case err == sql.ErrNoRows: |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
491 err = mw.JSONError{ |
1031
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
492 Code: http.StatusNotFound, |
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
493 Message: fmt.Sprintf("Cannot find import #%d.", id), |
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
494 } |
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
495 return |
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
496 case err != nil: |
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
497 return |
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
498 } |
4084
350a24c92848
Deliver times from import queue in UTC.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4078
diff
changeset
|
499 enqueued = enqueued.UTC() |
1026
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
500 |
2662
f90557a8c960
Moved summary from /api/imports overview to /api/imports/{id} details.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2627
diff
changeset
|
501 var sum interface{} |
f90557a8c960
Moved summary from /api/imports overview to /api/imports/{id} details.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2627
diff
changeset
|
502 if summary.Valid { |
f90557a8c960
Moved summary from /api/imports overview to /api/imports/{id} details.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2627
diff
changeset
|
503 if err = json.NewDecoder( |
f90557a8c960
Moved summary from /api/imports overview to /api/imports/{id} details.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2627
diff
changeset
|
504 strings.NewReader(summary.String)).Decode(&sum); err != nil { |
f90557a8c960
Moved summary from /api/imports overview to /api/imports/{id} details.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2627
diff
changeset
|
505 return |
f90557a8c960
Moved summary from /api/imports overview to /api/imports/{id} details.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2627
diff
changeset
|
506 } |
f90557a8c960
Moved summary from /api/imports overview to /api/imports/{id} details.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2627
diff
changeset
|
507 } |
f90557a8c960
Moved summary from /api/imports overview to /api/imports/{id} details.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2627
diff
changeset
|
508 |
1031
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
509 // We have it -> generate log entries. |
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
510 var rows *sql.Rows |
c0f5dedf5753
Check first if we have a import before generating log import entries list.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1026
diff
changeset
|
511 rows, err = conn.QueryContext(ctx, selectImportLogsSQL, id) |
1026
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
512 if err != nil { |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
513 return |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
514 } |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
515 defer rows.Close() |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
516 |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
517 entries := make([]*models.ImportLogEntry, 0, 10) |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
518 |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
519 for rows.Next() { |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
520 var entry models.ImportLogEntry |
4084
350a24c92848
Deliver times from import queue in UTC.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4078
diff
changeset
|
521 var t time.Time |
350a24c92848
Deliver times from import queue in UTC.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4078
diff
changeset
|
522 if err = rows.Scan(&t, &entry.Kind, &entry.Message); err != nil { |
1026
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
523 return |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
524 } |
4148
1b1218bbd414
Made 'go vet' happy with controllers again.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4084
diff
changeset
|
525 entry.Time = models.ImportTime{Time: t.UTC()} |
1026
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
526 entries = append(entries, &entry) |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
527 } |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
528 |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
529 if err = rows.Err(); err != nil { |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
530 return |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
531 } |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
532 |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
533 jr = mw.JSONResult{ |
1026
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
534 Result: struct { |
2795
241e7f05a538
/api/imports/{id} result now also contains the 'enqueued' time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2772
diff
changeset
|
535 Enqueued models.ImportTime `json:"enqueued"` |
241e7f05a538
/api/imports/{id} result now also contains the 'enqueued' time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2772
diff
changeset
|
536 Summary interface{} `json:"summary,omitempty"` |
241e7f05a538
/api/imports/{id} result now also contains the 'enqueued' time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2772
diff
changeset
|
537 Entries []*models.ImportLogEntry `json:"entries"` |
1026
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
538 }{ |
3089
813309225e35
Made 'go vet' happy again.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2954
diff
changeset
|
539 Enqueued: models.ImportTime{Time: enqueued}, |
2795
241e7f05a538
/api/imports/{id} result now also contains the 'enqueued' time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2772
diff
changeset
|
540 Summary: sum, |
241e7f05a538
/api/imports/{id} result now also contains the 'enqueued' time.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
2772
diff
changeset
|
541 Entries: entries, |
1026
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
542 }, |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
543 } |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
544 return |
3de54d7b7d30
Added endpoint to show the detail logs of an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1023
diff
changeset
|
545 } |
1037
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
546 |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
547 func deleteImport(req *http.Request) (jr mw.JSONResult, err error) { |
1037
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
548 |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
549 ctx := req.Context() |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
550 id, _ := strconv.ParseInt(mux.Vars(req)["id"], 10, 64) |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
551 |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
552 var tx *sql.Tx |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
553 tx, err = mw.JSONConn(req).BeginTx(ctx, nil) |
1037
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
554 if err != nil { |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
555 return |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
556 } |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
557 defer tx.Rollback() |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
558 |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
559 // Check if he have such a import job first. |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
560 var dummy bool |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
561 err = tx.QueryRowContext(ctx, selectHasNoRunningImportSQL, id).Scan(&dummy) |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
562 switch { |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
563 case err == sql.ErrNoRows: |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
564 err = mw.JSONError{ |
1037
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
565 Code: http.StatusNotFound, |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
566 Message: fmt.Sprintf("Cannot find import #%d.", id), |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
567 } |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
568 return |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
569 case err != nil: |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
570 return |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
571 } |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
572 |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
573 if _, err = tx.ExecContext(ctx, deleteLogsSQL, id); err != nil { |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
574 return |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
575 } |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
576 |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
577 if _, err = tx.ExecContext(ctx, deleteImportSQL, id); err != nil { |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
578 return |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
579 } |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
580 |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
581 if err = tx.Commit(); err != nil { |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
582 return |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
583 } |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
584 |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
585 jr = mw.JSONResult{Code: http.StatusNoContent} |
1037
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
586 |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
587 return |
a04126989d91
Added endpoint to delete job from import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1031
diff
changeset
|
588 } |
1192
3afa71405b87
Added REST endpoint to accept or decline an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1189
diff
changeset
|
589 |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
590 func reviewImports(req *http.Request) (mw.JSONResult, error) { |
1469
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
591 |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
592 rs := *mw.JSONInput(req).(*[]models.Review) |
1469
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
593 |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
594 type reviewResult struct { |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
595 ID int64 `json:"id"` |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
596 Message string `json:"message,omitempty"` |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
597 Error string `json:"error,omitempty"` |
1468 | 598 } |
1469
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
599 |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
600 results := make([]reviewResult, len(rs)) |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
601 |
5123
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
602 for i := range results { |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
603 results[i].ID = rs[i].ID |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
604 results[i].Message = fmt.Sprintf("Finalizing import #%d in progress.", rs[i].ID) |
1469
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
605 } |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
606 |
5123
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
607 var wg sync.WaitGroup |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
608 var mu sync.Mutex |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
609 |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
610 for i := range rs { |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
611 wg.Add(1) |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
612 go func(idx int) { |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
613 defer wg.Done() |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
614 rev := &rs[idx] |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
615 msg, err := decideImport(req, rev.ID, string(rev.State)) |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
616 mu.Lock() |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
617 if err != nil { |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
618 results[idx].Error = err.Error() |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
619 } |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
620 results[idx].Message = msg |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
621 mu.Unlock() |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
622 }(i) |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
623 } |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
624 |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
625 done := make(chan struct{}) |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
626 go func() { |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
627 defer close(done) |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
628 wg.Wait() |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
629 }() |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
630 |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
631 select { |
5131
52e3980e3462
review decisions controller: Lowered immediate feedback timeout to 5 secs.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5123
diff
changeset
|
632 case <-time.After(5 * time.Second): |
5123
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
633 case <-done: |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
634 } |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
635 |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
636 out := make([]reviewResult, len(rs)) |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
637 mu.Lock() |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
638 copy(out, results) |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
639 mu.Unlock() |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
640 |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
641 return mw.JSONResult{Result: out}, nil |
1468 | 642 } |
643 | |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
644 func reviewImport(req *http.Request) (jr mw.JSONResult, err error) { |
1192
3afa71405b87
Added REST endpoint to accept or decline an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1189
diff
changeset
|
645 |
3afa71405b87
Added REST endpoint to accept or decline an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1189
diff
changeset
|
646 vars := mux.Vars(req) |
3afa71405b87
Added REST endpoint to accept or decline an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1189
diff
changeset
|
647 id, _ := strconv.ParseInt(vars["id"], 10, 64) |
3afa71405b87
Added REST endpoint to accept or decline an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1189
diff
changeset
|
648 state := vars["state"] |
1469
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
649 |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
650 var msg string |
5028
d727641911a5
Moved import desision logic to import queue (where it belongs).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4791
diff
changeset
|
651 if msg, err = decideImport(req, id, state); err != nil { |
1469
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
652 return |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
653 } |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
654 |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
655 result := struct { |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
656 Message string `json:"message"` |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
657 }{ |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
658 Message: msg, |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
659 } |
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
660 |
4244
4394daeea96a
Moved JSONHandler into middleware package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4243
diff
changeset
|
661 jr = mw.JSONResult{Result: &result} |
1469
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
662 return |
1468 | 663 } |
1192
3afa71405b87
Added REST endpoint to accept or decline an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1189
diff
changeset
|
664 |
1469
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
665 func decideImport( |
1468 | 666 req *http.Request, |
667 id int64, | |
668 state string, | |
1469
286a3306f6bf
Import bulk review: Cleaned up input JSON document handling. Added proper error handling.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1468
diff
changeset
|
669 ) (message string, err error) { |
1192
3afa71405b87
Added REST endpoint to accept or decline an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1189
diff
changeset
|
670 |
5028
d727641911a5
Moved import desision logic to import queue (where it belongs).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4791
diff
changeset
|
671 session, _ := auth.GetSession(req) |
d727641911a5
Moved import desision logic to import queue (where it belongs).
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
4791
diff
changeset
|
672 reviewer := session.User |
1192
3afa71405b87
Added REST endpoint to accept or decline an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1189
diff
changeset
|
673 |
5123
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
674 ctx := req.Context() |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
675 accepted := state == "accepted" |
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
676 |
5153
adf7b9f1273b
Send the info if an import job was re-scheduled to sync callers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5131
diff
changeset
|
677 switch err = imports.DecideImport(ctx, id, accepted, reviewer); { |
adf7b9f1273b
Send the info if an import job was re-scheduled to sync callers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5131
diff
changeset
|
678 case err == imports.ErrRetrying: |
adf7b9f1273b
Send the info if an import job was re-scheduled to sync callers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5131
diff
changeset
|
679 return "", fmt.Errorf("Finalizing import #%d is re-scheduled.", id) |
adf7b9f1273b
Send the info if an import job was re-scheduled to sync callers.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5131
diff
changeset
|
680 case err != nil: |
5123
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
681 return "", err |
1355
b0ebb77f4ab0
Return nice JSON result if import was accepted or declined.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1351
diff
changeset
|
682 } |
b0ebb77f4ab0
Return nice JSON result if import was accepted or declined.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1351
diff
changeset
|
683 |
5123
eeb45e3e0a5a
Added mechanism to have sync import jobs on import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
5109
diff
changeset
|
684 return fmt.Sprintf("Import #%d is %s.", id, state), nil |
1192
3afa71405b87
Added REST endpoint to accept or decline an import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1189
diff
changeset
|
685 } |