gemma: pkg/imports/dsec.go annotate

annotate pkg/imports/dsec.go @ 5560:f2204f91d286

Join the log lines of imports to the log exports to recover data from them. Used in SR export to extract information that where in the meta json but now are only found in the log.

author	Sascha L. Teichmann <sascha.teichmann@intevation.de>
date	Wed, 09 Feb 2022 18:34:40 +0100
parents	5f47eeea988d
children	6270951dda28

rev	line source
4188 f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	1 // This is Free Software under GNU Affero General Public License v >= 3.0
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	2 // without warranty, see README.md and license for details.
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	3 //
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	4 // SPDX-License-Identifier: AGPL-3.0-or-later
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	5 // License-Filename: LICENSES/AGPL-3.0.txt
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	6 //
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	7 // Copyright (C) 2019 by via donau
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	8 // – Österreichische Wasserstraßen-Gesellschaft mbH
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	9 // Software engineering by Intevation GmbH
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	10 //
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	11 // Author(s):
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	13
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14 package imports
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16 import (
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	17 "context"
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	18 "database/sql"
4678 e1cf039ea25a Section deletion: return error if nothing deleted Tom Gottfried <tom@intevation.de> parents: 4276 diff changeset	19 "errors"
4188 f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	20 "fmt"
5490 5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5034 diff changeset	21
5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5034 diff changeset	22 "gemma.intevation.de/gemma/pkg/log"
4188 f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	23 )
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	24
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	25 // DeleteSection is a Job to delete a section from the database.
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	26 type DeleteSection struct {
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	27 ID int64 `json:"id"`
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	28 }
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	29
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	30 // DSECJobKind is the import queue type identifier.
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	31 const DSECJobKind JobKind = "dsec"
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	32
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	33 type dsecJobCreator struct{}
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	34
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	35 func init() { RegisterJobCreator(DSECJobKind, dsecJobCreator{}) }
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	36
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	37 func (dsecJobCreator) Description() string { return "delete section" }
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	38
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	39 func (dsecJobCreator) AutoAccept() bool { return false }
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	40
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	41 func (dsecJobCreator) Create() Job { return new(DeleteSection) }
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	42
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	43 func (dsecJobCreator) Depends() [2][]string {
5020 e4ab338e7ba9 Removed unnecessary creatings of empty slices in expressing read dependencies in imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4847 diff changeset	44 return [2][]string{{"sections"}}
4188 f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	45 }
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	46
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	47 const (
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	48 dsecExistsSQL = `
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	49 SELECT EXISTS (
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	50 SELECT 1 FROM waterway.sections
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	51 WHERE id = $1 AND staging_done)
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	52 `
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	53 dsecStageDoneSQL = `
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	54 DELETE FROM waterway.sections
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	55 WHERE id IN (
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	56 SELECT key from import.track_imports
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	57 WHERE import_id = $1 AND
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	58 deletion AND
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	59 relation = 'waterway.sections'::regclass)`
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	60 )
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	61
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	62 // StageDone finally removes the section from the database.
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	63 func (dsecJobCreator) StageDone(
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	64 ctx context.Context,
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	65 tx *sql.Tx,
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	66 id int64,
5034 59a99655f34d Added feedback support for StageDone. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5020 diff changeset	67 _ Feedback,
4188 f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	68 ) error {
4678 e1cf039ea25a Section deletion: return error if nothing deleted Tom Gottfried <tom@intevation.de> parents: 4276 diff changeset	69 res, err := tx.ExecContext(ctx, dsecStageDoneSQL, id)
e1cf039ea25a Section deletion: return error if nothing deleted Tom Gottfried <tom@intevation.de> parents: 4276 diff changeset	70 if err != nil {
e1cf039ea25a Section deletion: return error if nothing deleted Tom Gottfried <tom@intevation.de> parents: 4276 diff changeset	71 return err
e1cf039ea25a Section deletion: return error if nothing deleted Tom Gottfried <tom@intevation.de> parents: 4276 diff changeset	72 }
e1cf039ea25a Section deletion: return error if nothing deleted Tom Gottfried <tom@intevation.de> parents: 4276 diff changeset	73 n, err := res.RowsAffected()
e1cf039ea25a Section deletion: return error if nothing deleted Tom Gottfried <tom@intevation.de> parents: 4276 diff changeset	74 if err != nil {
5490 5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5034 diff changeset	75 log.Errorf(
5f47eeea988d Use own logging package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 5034 diff changeset	76 "RowsAffected returned an error. Missing driver support?")
4678 e1cf039ea25a Section deletion: return error if nothing deleted Tom Gottfried <tom@intevation.de> parents: 4276 diff changeset	77 return nil
e1cf039ea25a Section deletion: return error if nothing deleted Tom Gottfried <tom@intevation.de> parents: 4276 diff changeset	78 }
e1cf039ea25a Section deletion: return error if nothing deleted Tom Gottfried <tom@intevation.de> parents: 4276 diff changeset	79 if n == 0 {
e1cf039ea25a Section deletion: return error if nothing deleted Tom Gottfried <tom@intevation.de> parents: 4276 diff changeset	80 return errors.New(
4847 4847ac70103a Made staticcheck happy. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4678 diff changeset	81 "nothing deleted. Probably due to missing access rights")
4678 e1cf039ea25a Section deletion: return error if nothing deleted Tom Gottfried <tom@intevation.de> parents: 4276 diff changeset	82 }
4188 f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	83 return err
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	84 }
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	85
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	86 // CleanUp of a section delete import is a NOP.
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	87 func (*DeleteSection) CleanUp() error { return nil }
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	88
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	89 // Do prepares the deletion of the section.
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	90 func (dsec *DeleteSection) Do(
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	91 ctx context.Context,
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	92 importID int64,
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	93 conn *sql.Conn,
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	94 feedback Feedback,
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	95 ) (interface{}, error) {
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	96
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	97 tx, err := conn.BeginTx(ctx, nil)
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	98 if err != nil {
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	99 return nil, err
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	100 }
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	101 defer tx.Rollback()
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	102
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	103 var found bool
4191 a1cc14c9ce69 Delete section import: Fixed SQL usage. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4188 diff changeset	104 if err := tx.QueryRowContext(ctx, dsecExistsSQL, dsec.ID).Scan(&found); err != nil {
4188 f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	105 return nil, err
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	106 }
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	107
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	108 if !found {
4276 3dbe05de54ea Fixed typo in import logging message. Sascha L. Teichmann <teichmann@intevation.de> parents: 4192 diff changeset	109 return nil, fmt.Errorf("no section with id %d found", dsec.ID)
4188 f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	110 }
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	111
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	112 feedback.Info("Prepare deletion of section with id %d", dsec.ID)
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	113
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	114 if _, err := tx.ExecContext(
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	115 ctx,
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	116 trackImportDeletionSQL,
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	117 importID,
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	118 "waterway.sections",
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	119 dsec.ID,
4192 095731c880b4 Delete section import: Fixed swapped arguments for tracking. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 4191 diff changeset	120 true,
4188 f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	121 ); err != nil {
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	122 return nil, err
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	123 }
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	124
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	125 if err := tx.Commit(); err != nil {
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	126 return nil, err
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	127 }
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	128
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	129 return dsec, nil
f8b7db7e392a Added an "import" job to delete a section from the database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	130 }

Mercurial > gemma

annotate pkg/imports/dsec.go @ 5560:f2204f91d286