gemma: pkg/imports/sr.go annotate

annotate pkg/imports/sr.go @ 2873:b1707f60f241

SR import: Log name of bottleneck and survey date into import log.

author	Sascha L. Teichmann <sascha.teichmann@intevation.de>
date	Mon, 01 Apr 2019 10:46:18 +0200
parents	1b6840093eac
children	27ffd94afcb5

rev	line source
1017 a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	1 // This is Free Software under GNU Affero General Public License v >= 3.0
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	2 // without warranty, see README.md and license for details.
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	3 //
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	4 // SPDX-License-Identifier: AGPL-3.0-or-later
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	5 // License-Filename: LICENSES/AGPL-3.0.txt
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	6 //
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	7 // Copyright (C) 2018 by via donau
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	8 // – Österreichische Wasserstraßen-Gesellschaft mbH
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	9 // Software engineering by Intevation GmbH
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	10 //
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	11 // Author(s):
a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
1337 53057ab27ff2 add headers for licensing to some middleware files Fadi Abbud <fadi.abbud@intevation.de> parents: 1292 diff changeset	13 // * Tom Gottfried <tom@intevation.de>
1017 a244b18cb916 Added GNU Affero General Public License. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 991 diff changeset	14
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15 package imports
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	17 import (
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	18 "archive/zip"
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	19 "bufio"
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	20 "context"
973 b6fec8f85599 Generate TINs and octrees in sounding result importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 972 diff changeset	21 "crypto/sha1"
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	22 "database/sql"
973 b6fec8f85599 Generate TINs and octrees in sounding result importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 972 diff changeset	23 "encoding/hex"
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	24 "errors"
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	25 "fmt"
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	26 "io"
1184 064d44ccc6f2 Adjust contour lines heights to multiples of step width Tom Gottfried <tom@intevation.de> parents: 1168 diff changeset	27 "math"
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	28 "os"
960 e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	29 "path"
978 544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 977 diff changeset	30 "path/filepath"
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	31 "strconv"
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	32 "strings"
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	33 "time"
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	34
960 e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	35 shp "github.com/jonas-p/go-shp"
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	36
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1193 diff changeset	37 "gemma.intevation.de/gemma/pkg/common"
bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1193 diff changeset	38 "gemma.intevation.de/gemma/pkg/models"
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	39 "gemma.intevation.de/gemma/pkg/octree"
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	40 )
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	41
1498 5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	42 // SoundingResult is a Job to import sounding reults
5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	43 // from a ZIP file into the database.
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	44 type SoundingResult struct {
1498 5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	45 // Dir is the folder in the file system where the
5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	46 // 'sr.zip' is expect to be.
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	47 Dir string `json:"dir"`
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	48
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	49 // Override data
1498 5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	50 // Date if given overrides the date value from the meta.json.
1900 6a67cd819e93 To prepare stretch import made some model data types re-usable. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1786 diff changeset	51 Date *models.Date `json:"date,omitempty"`
1498 5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	52 // Date if given overrides the name of the bottleneck from the meta.json.
5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	53 Bottleneck *string `json:"bottleneck,omitempty"`
5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	54 // EPSG if given overrides the EPSG code from the meta.json.
5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	55 // Defaults to WGS84.
5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	56 EPSG *uint `json:"epsg,omitempty"`
5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	57 // DepthReference if given overides the DepthReference value
5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	58 // from the meta.json.
5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	59 DepthReference *string `json:"depth-reference,omitempty"`
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	60 }
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	61
964 1e2dce348cfb Serialize boundary polygon of sounding result as WKB. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 961 diff changeset	62 const (
1130 42617bba8709 Go from 50cm to 10cm stepwidth in calculating the contorlines. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1017 diff changeset	63 contourStepWidth = 0.1
977 4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	64 contourTolerance = 0.1
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	65 )
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	66
1498 5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	67 // SRJobKind is the unique name of a SoundingResult import job.
991 a301d240905f Decoupled import job creation and job execution with a factory function. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 988 diff changeset	68 const SRJobKind JobKind = "sr"
a301d240905f Decoupled import job creation and job execution with a factory function. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 988 diff changeset	69
1136 a5069da2f0b7 Independent imports in terms of affected tables/dependencies are now run concurrently. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1131 diff changeset	70 type srJobCreator struct{}
a5069da2f0b7 Independent imports in terms of affected tables/dependencies are now run concurrently. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1131 diff changeset	71
1193 58acc343b1b6 Implemented the db stuff of the review process. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1184 diff changeset	72 func init() {
58acc343b1b6 Implemented the db stuff of the review process. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1184 diff changeset	73 RegisterJobCreator(SRJobKind, srJobCreator{})
58acc343b1b6 Implemented the db stuff of the review process. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1184 diff changeset	74 }
58acc343b1b6 Implemented the db stuff of the review process. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1184 diff changeset	75
1754 807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1664 diff changeset	76 func (srJobCreator) Description() string { return "sounding results" }
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1664 diff changeset	77
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1664 diff changeset	78 func (srJobCreator) AutoAccept() bool { return false }
1646 a0982c38eac0 Import queue: Implemented email notifications. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1572 diff changeset	79
2187 7c83b5277c1c Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2168 diff changeset	80 func (srJobCreator) Create() Job { return new(SoundingResult) }
1136 a5069da2f0b7 Independent imports in terms of affected tables/dependencies are now run concurrently. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1131 diff changeset	81
a5069da2f0b7 Independent imports in terms of affected tables/dependencies are now run concurrently. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1131 diff changeset	82 func (srJobCreator) Depends() []string {
a5069da2f0b7 Independent imports in terms of affected tables/dependencies are now run concurrently. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1131 diff changeset	83 return []string{
1664 819f67c31dfb Imports: Stripped schema prefixes from table dependencies. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1646 diff changeset	84 "sounding_results",
819f67c31dfb Imports: Stripped schema prefixes from table dependencies. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1646 diff changeset	85 "sounding_results_contour_lines",
819f67c31dfb Imports: Stripped schema prefixes from table dependencies. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1646 diff changeset	86 "bottlenecks",
1136 a5069da2f0b7 Independent imports in terms of affected tables/dependencies are now run concurrently. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1131 diff changeset	87 }
a5069da2f0b7 Independent imports in terms of affected tables/dependencies are now run concurrently. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1131 diff changeset	88 }
a5069da2f0b7 Independent imports in terms of affected tables/dependencies are now run concurrently. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1131 diff changeset	89
1193 58acc343b1b6 Implemented the db stuff of the review process. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1184 diff changeset	90 func (srJobCreator) StageDone(
1328 d753ce6cf588 To make golint happier made context.Context to be the first argument in all calls. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1317 diff changeset	91 ctx context.Context,
1193 58acc343b1b6 Implemented the db stuff of the review process. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1184 diff changeset	92 tx *sql.Tx,
58acc343b1b6 Implemented the db stuff of the review process. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1184 diff changeset	93 id int64,
58acc343b1b6 Implemented the db stuff of the review process. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1184 diff changeset	94 ) error {
58acc343b1b6 Implemented the db stuff of the review process. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1184 diff changeset	95 _, err := tx.ExecContext(ctx, srStageDoneSQL, id)
58acc343b1b6 Implemented the db stuff of the review process. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1184 diff changeset	96 return err
991 a301d240905f Decoupled import job creation and job execution with a factory function. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 988 diff changeset	97 }
a301d240905f Decoupled import job creation and job execution with a factory function. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 988 diff changeset	98
977 4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	99 const (
1193 58acc343b1b6 Implemented the db stuff of the review process. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1184 diff changeset	100 srStageDoneSQL = `
58acc343b1b6 Implemented the db stuff of the review process. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1184 diff changeset	101 UPDATE waterway.sounding_results SET staging_done = true
1502 0abcc82ef976 Sounding results: Be a bit more precise when setting stageing_done. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1501 diff changeset	102 WHERE id = (
1995 59055c8301df Move import queue to its own database namespace Tom Gottfried <tom@intevation.de> parents: 1900 diff changeset	103 SELECT key from import.track_imports
1502 0abcc82ef976 Sounding results: Be a bit more precise when setting stageing_done. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1501 diff changeset	104 WHERE import_id = $1 AND
0abcc82ef976 Sounding results: Be a bit more precise when setting stageing_done. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1501 diff changeset	105 relation = 'waterway.sounding_results'::regclass)`
1193 58acc343b1b6 Implemented the db stuff of the review process. Needs testing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1184 diff changeset	106
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	107 insertHullSQL = `
961 a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 960 diff changeset	108 INSERT INTO waterway.sounding_results (
a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 960 diff changeset	109 bottleneck_id,
a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 960 diff changeset	110 date_info,
a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 960 diff changeset	111 depth_reference,
a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 960 diff changeset	112 area
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	113 ) SELECT
1572 056a86b24be2 Made bottleneck primary key an int. Attention: This may break something! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1531 diff changeset	114 (SELECT id from waterway.bottlenecks where objnam = $1),
961 a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 960 diff changeset	115 $2::date,
a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 960 diff changeset	116 $3,
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	117 (SELECT
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	118 CASE WHEN $5::bytea IS NULL THEN
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	119 ST_Transform(ST_ConcaveHull(ST_Force2D(ST_GeomFromWKB($4, $6::integer)), 0.7), 4326)::geography
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	120 ELSE
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	121 ST_Transform(ST_GeomFromWKB($5, $6::integer), 4326)::geography
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	122 END)
971 f9fb6c399f3f Moved generating of tins to octree package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 964 diff changeset	123 RETURNING
f9fb6c399f3f Moved generating of tins to octree package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 964 diff changeset	124 id,
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	125 ST_X(ST_Centroid(area::geometry)),
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	126 ST_Y(ST_Centroid(area::geometry)),
2833 1b6840093eac Prevent calculation of wrong UTM zones Tom Gottfried <tom@intevation.de> parents: 2612 diff changeset	127 best_utm(area),
1b6840093eac Prevent calculation of wrong UTM zones Tom Gottfried <tom@intevation.de> parents: 2612 diff changeset	128 ST_AsBinary(ST_Transform(area::geometry, best_utm(area)))
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	129 `
973 b6fec8f85599 Generate TINs and octrees in sounding result importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 972 diff changeset	130
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	131 reprojectPointsSQL = `
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	132 SELECT ST_AsBinary(ST_Transform(ST_GeomFromWKB($1, $2::integer), $3::integer))
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	133 `
973 b6fec8f85599 Generate TINs and octrees in sounding result importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 972 diff changeset	134 insertOctreeSQL = `
1131 2e6b47cdb2ca Store octrees along with sounding results Tom Gottfried <tom@intevation.de> parents: 1130 diff changeset	135 UPDATE waterway.sounding_results SET
2e6b47cdb2ca Store octrees along with sounding results Tom Gottfried <tom@intevation.de> parents: 1130 diff changeset	136 octree_checksum = $2, octree_index = $3
2e6b47cdb2ca Store octrees along with sounding results Tom Gottfried <tom@intevation.de> parents: 1130 diff changeset	137 WHERE id = $1`
977 4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	138
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	139 insertContourSQL = `
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	140 INSERT INTO waterway.sounding_results_contour_lines (
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	141 sounding_result_id,
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	142 height,
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	143 lines
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	144 )
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	145 SELECT
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	146 $1,
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	147 $2,
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	148 ST_Transform(
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	149 ST_Multi(
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	150 ST_CollectionExtract(
1137 2fb6c0a6ec8a Avoid intersection of each isoline with area on insert Tom Gottfried <tom@intevation.de> parents: 1136 diff changeset	151 ST_SimplifyPreserveTopology(
1410 eb82076fdffe Sounding result import: Ensure that the generated iso-lines are valid. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1392 diff changeset	152 ST_Multi(ST_Collectionextract(
eb82076fdffe Sounding result import: Ensure that the generated iso-lines are valid. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1392 diff changeset	153 ST_MakeValid(ST_GeomFromWKB($4, $3::integer)), 2)),
1137 2fb6c0a6ec8a Avoid intersection of each isoline with area on insert Tom Gottfried <tom@intevation.de> parents: 1136 diff changeset	154 $5
977 4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	155 ),
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	156 2
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	157 )
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	158 ),
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	159 4326
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	160 )
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	161 FROM waterway.sounding_results sr
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	162 WHERE id = $1
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	163 `
961 a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 960 diff changeset	164 )
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	165
1498 5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	166 // Do executes the actual sounding result import.
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	167 func (sr *SoundingResult) Do(
1328 d753ce6cf588 To make golint happier made context.Context to be the first argument in all calls. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1317 diff changeset	168 ctx context.Context,
1168 930fdd8b474f Track successfull imports in a separate table to be able to remove them later. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1138 diff changeset	169 importID int64,
930fdd8b474f Track successfull imports in a separate table to be able to remove them later. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1138 diff changeset	170 conn *sql.Conn,
930fdd8b474f Track successfull imports in a separate table to be able to remove them later. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1138 diff changeset	171 feedback Feedback,
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	172 ) (interface{}, error) {
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	173
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	174 begin := time.Now()
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	175
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	176 z, err := zip.OpenReader(filepath.Join(sr.Dir, "sr.zip"))
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	177 if err != nil {
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	178 return nil, err
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	179 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	180 defer z.Close()
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	181
1138 443fc80a315f Don't issue new lines at end of log messages when importing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1137 diff changeset	182 feedback.Info("Looking for 'meta.json'")
1225 4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1224 diff changeset	183 mf := common.FindInZIP(z, "meta.json")
1292 5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1239 diff changeset	184 if mf == nil && !sr.completeOverride() {
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	185 return nil, errors.New("Cannot find 'meta.json'")
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	186 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	187
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	188 m, err := sr.loadMeta(mf)
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	189 if err != nil {
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	190 return nil, err
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	191 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	192
2873 b1707f60f241 SR import: Log name of bottleneck and survey date into import log. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2833 diff changeset	193 feedback.Info("Bottleneck: %s", m.Bottleneck)
b1707f60f241 SR import: Log name of bottleneck and survey date into import log. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2833 diff changeset	194 feedback.Info("Survey date: %s", m.Date.Format(common.DateFormat))
b1707f60f241 SR import: Log name of bottleneck and survey date into import log. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2833 diff changeset	195
1328 d753ce6cf588 To make golint happier made context.Context to be the first argument in all calls. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1317 diff changeset	196 if err := m.Validate(ctx, conn); err != nil {
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	197 return nil, common.ToError(err)
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	198 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	199
1528 5874cedd7f91 Sounding result import: Accept .txt files for XYZ data, too. Sascha L. Teichmann <sascha.teichmann@intevation.de>* parents: 1502 diff changeset	200 var xyzf *zip.File
5874cedd7f91 Sounding result import: Accept .txt files for XYZ data, too. Sascha L. Teichmann <sascha.teichmann@intevation.de>* parents: 1502 diff changeset	201 for _, ext := range []string{".xyz", ".txt"} {
5874cedd7f91 Sounding result import: Accept .txt files for XYZ data, too. Sascha L. Teichmann <sascha.teichmann@intevation.de>* parents: 1502 diff changeset	202 feedback.Info("Looking for '*%s'", ext)
5874cedd7f91 Sounding result import: Accept .txt files for XYZ data, too. Sascha L. Teichmann <sascha.teichmann@intevation.de>* parents: 1502 diff changeset	203 if xyzf = common.FindInZIP(z, ext); xyzf != nil {
5874cedd7f91 Sounding result import: Accept .txt files for XYZ data, too. Sascha L. Teichmann <sascha.teichmann@intevation.de>* parents: 1502 diff changeset	204 break
5874cedd7f91 Sounding result import: Accept .txt files for XYZ data, too. Sascha L. Teichmann <sascha.teichmann@intevation.de>* parents: 1502 diff changeset	205 }
5874cedd7f91 Sounding result import: Accept .txt files for XYZ data, too. Sascha L. Teichmann <sascha.teichmann@intevation.de>* parents: 1502 diff changeset	206 }
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	207 if xyzf == nil {
1528 5874cedd7f91 Sounding result import: Accept .txt files for XYZ data, too. Sascha L. Teichmann <sascha.teichmann@intevation.de>* parents: 1502 diff changeset	208 return nil, errors.New("Cannot find any .xyz or .txt file")
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	209 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	210
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	211 xyz, err := loadXYZ(xyzf, feedback)
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	212 if err != nil {
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	213 return nil, err
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	214 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	215
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	216 if len(xyz) == 0 {
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	217 return nil, errors.New("XYZ does not contain any vertices")
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	218 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	219
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	220 // Is there a boundary shapefile in the ZIP archive?
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	221 polygon, err := loadBoundary(z)
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	222 if err != nil {
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	223 return nil, err
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	224 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	225
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	226 tx, err := conn.BeginTx(ctx, nil)
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	227 if err != nil {
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	228 return nil, err
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	229 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	230 defer tx.Rollback()
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	231
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	232 var (
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	233 id int64
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	234 epsg uint32
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	235 lat, lon float64
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	236 )
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	237 start := time.Now()
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	238
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	239 var hull []byte
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	240
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	241 xyzWKB := xyz.AsWKB()
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	242
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	243 err = tx.QueryRow(insertHullSQL,
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	244 m.Bottleneck,
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	245 m.Date.Time,
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	246 m.DepthReference,
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	247 xyzWKB,
1496 3bf1f0de0763 Un-export point, line string and polygon types in import package as these are only used locally. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1410 diff changeset	248 polygon.asWKB(),
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	249 m.EPSG,
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	250 ).Scan(
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	251 &id,
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	252 &lat,
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	253 &lon,
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	254 &epsg,
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	255 &hull,
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	256 )
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	257 xyz, polygon = nil, nil // not need from now on.
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	258 feedback.Info("Calculating hull took %s.", time.Since(start))
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	259 if err != nil {
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	260 return nil, err
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	261 }
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	262 feedback.Info("Best suited UTM EPSG: %d", epsg)
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	263
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	264 start = time.Now()
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	265
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	266 var clippingPolygon octree.Polygon
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	267 if err := clippingPolygon.FromWKB(hull); err != nil {
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	268 return nil, err
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	269 }
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	270 clippingPolygon.Indexify()
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	271 feedback.Info("Building clipping polygon took %v.", time.Since(start))
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	272
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	273 start = time.Now()
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	274
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	275 var reproj []byte
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	276
2530 3f61b84ae7a6 SR import: Set missing EPSG code to TIN. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2529 diff changeset	277 if err = tx.QueryRow(reprojectPointsSQL,
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	278 xyzWKB,
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	279 m.EPSG,
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	280 epsg,
2530 3f61b84ae7a6 SR import: Set missing EPSG code to TIN. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2529 diff changeset	281 ).Scan(&reproj); err != nil {
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	282 return nil, err
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	283 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	284
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	285 if err := xyz.FromWKB(reproj); err != nil {
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	286 return nil, err
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	287 }
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	288
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	289 feedback.Info("Reprojecting points took %v.", time.Since(start))
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	290 feedback.Info("Number of reprojected points: %d", len(xyz))
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	291
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	292 start = time.Now()
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	293
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	294 tri, err := octree.Triangulate(xyz)
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	295 if err != nil {
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	296 return nil, err
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	297 }
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	298 feedback.Info("Triangulation took %v.", time.Since(start))
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	299
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	300 start = time.Now()
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	301
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	302 tin := tri.Tin()
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	303
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	304 var str octree.STRTree
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	305 str.Build(tin)
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	306 feedback.Info("Building STR tree took %v", time.Since(start))
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	307
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	308 start = time.Now()
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	309
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	310 removed := str.Clip(&clippingPolygon)
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	311 feedback.Info("Clipping STR tree took %v.", time.Since(start))
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	312 feedback.Info("Number of triangles to clip %d.", len(removed))
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	313
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	314 start = time.Now()
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	315
2530 3f61b84ae7a6 SR import: Set missing EPSG code to TIN. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2529 diff changeset	316 tin.EPSG = epsg
3f61b84ae7a6 SR import: Set missing EPSG code to TIN. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2529 diff changeset	317
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	318 builder := octree.NewBuilder(tin)
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	319 builder.Build(removed)
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	320 octreeIndex, err := builder.Bytes()
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	321 if err != nil {
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	322 return nil, err
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	323 }
2530 3f61b84ae7a6 SR import: Set missing EPSG code to TIN. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2529 diff changeset	324 feedback.Info("Building octree took %v.", time.Since(start))
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	325
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	326 start = time.Now()
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	327 h := sha1.New()
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	328 h.Write(octreeIndex)
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	329 checksum := hex.EncodeToString(h.Sum(nil))
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	330 _, err = tx.Exec(insertOctreeSQL, id, checksum, octreeIndex)
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	331 if err != nil {
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	332 return nil, err
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	333 }
2530 3f61b84ae7a6 SR import: Set missing EPSG code to TIN. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2529 diff changeset	334 feedback.Info("Storing octree index took %s.", time.Since(start))
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	335
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	336 tree := builder.Tree()
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	337
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	338 start = time.Now()
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	339 err = generateContours(tree, tx, id)
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	340 if err != nil {
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	341 return nil, err
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	342 }
2530 3f61b84ae7a6 SR import: Set missing EPSG code to TIN. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2529 diff changeset	343 feedback.Info("Generating and storing contour lines took %s.",
3f61b84ae7a6 SR import: Set missing EPSG code to TIN. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2529 diff changeset	344 time.Since(start))
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	345
1168 930fdd8b474f Track successfull imports in a separate table to be able to remove them later. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1138 diff changeset	346 // Store for potential later removal.
1328 d753ce6cf588 To make golint happier made context.Context to be the first argument in all calls. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1317 diff changeset	347 if err = track(ctx, tx, importID, "waterway.sounding_results", id); err != nil {
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	348 return nil, err
1168 930fdd8b474f Track successfull imports in a separate table to be able to remove them later. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1138 diff changeset	349 }
930fdd8b474f Track successfull imports in a separate table to be able to remove them later. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1138 diff changeset	350
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	351 if err = tx.Commit(); err != nil {
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	352 feedback.Error(
2530 3f61b84ae7a6 SR import: Set missing EPSG code to TIN. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2529 diff changeset	353 "Storing sounding result failed after %v.", time.Since(begin))
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	354 return nil, err
1138 443fc80a315f Don't issue new lines at end of log messages when importing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1137 diff changeset	355 }
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	356
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	357 feedback.Info(
2530 3f61b84ae7a6 SR import: Set missing EPSG code to TIN. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2529 diff changeset	358 "Storing sounding result was successful after %v.", time.Since(begin))
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	359
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	360 summary := struct {
1900 6a67cd819e93 To prepare stretch import made some model data types re-usable. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1786 diff changeset	361 Bottleneck string `json:"bottleneck"`
6a67cd819e93 To prepare stretch import made some model data types re-usable. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1786 diff changeset	362 Date models.Date `json:"date"`
6a67cd819e93 To prepare stretch import made some model data types re-usable. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1786 diff changeset	363 Lat float64 `json:"lat"`
6a67cd819e93 To prepare stretch import made some model data types re-usable. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1786 diff changeset	364 Lon float64 `json:"lon"`
1392 0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	365 }{
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	366 Bottleneck: m.Bottleneck,
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	367 Date: m.Date,
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	368 Lat: lat,
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	369 Lon: lon,
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1384 diff changeset	370 }
2530 3f61b84ae7a6 SR import: Set missing EPSG code to TIN. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2529 diff changeset	371
3f61b84ae7a6 SR import: Set missing EPSG code to TIN. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2529 diff changeset	372 return &summary, nil
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	373 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	374
1498 5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	375 // CleanUp removes the folder containing the ZIP file with the
5df748916fcf Finished API docs for sounding result imports. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1496 diff changeset	376 // the sounding result import.
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	377 func (sr *SoundingResult) CleanUp() error {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	378 return os.RemoveAll(sr.Dir)
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	379 }
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	380
1292 5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1239 diff changeset	381 func (sr *SoundingResult) completeOverride() bool {
1384 8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1338 diff changeset	382 // sr.EPSG == nil -> WGS84
8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1338 diff changeset	383 return sr.Bottleneck != nil && sr.Date != nil && sr.DepthReference != nil
1292 5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1239 diff changeset	384 }
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1239 diff changeset	385
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	386 func (sr SoundingResult) loadMeta(f zip.File) (*models.SoundingResultMeta, error) {
1292 5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1239 diff changeset	387 if f == nil {
1384 8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1338 diff changeset	388 var epsg uint
8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1338 diff changeset	389 if sr.EPSG != nil {
8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1338 diff changeset	390 epsg = *sr.EPSG
8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1338 diff changeset	391 } else {
8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1338 diff changeset	392 epsg = models.WGS84
8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1338 diff changeset	393 }
1292 5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1239 diff changeset	394 return &models.SoundingResultMeta{
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1239 diff changeset	395 Date: *sr.Date,
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1239 diff changeset	396 Bottleneck: *sr.Bottleneck,
1384 8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1338 diff changeset	397 EPSG: epsg,
1292 5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1239 diff changeset	398 DepthReference: *sr.DepthReference,
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1239 diff changeset	399 }, nil
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1239 diff changeset	400 }
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	401 r, err := f.Open()
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	402 if err != nil {
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	403 return nil, err
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	404 }
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	405 defer r.Close()
1224 bc4b642c8d04 Started with an upload sounding result to temp upload area. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1193 diff changeset	406 var m models.SoundingResultMeta
1239 d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	407 if err := m.Decode(r); err != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	408 return nil, err
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	409 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	410
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	411 // Apply overrides
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	412 if sr.Date != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	413 m.Date = *sr.Date
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	414 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	415 if sr.Bottleneck != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	416 m.Bottleneck = *sr.Bottleneck
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	417 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	418 if sr.EPSG != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	419 m.EPSG = *sr.EPSG
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	420 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	421 if sr.DepthReference != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	422 m.DepthReference = *sr.DepthReference
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	423 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	424
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1225 diff changeset	425 return &m, nil
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	426 }
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	427
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	428 func loadXYZReader(r io.Reader, feedback Feedback) (octree.MultiPointZ, error) {
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	429 mpz := make(octree.MultiPointZ, 0, 250000)
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	430 s := bufio.NewScanner(r)
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	431
2612 c7ce8b011bcb SR import: Negate negative Z values and issue a warning if a negative Z value is found. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2530 diff changeset	432 var hasNegZ bool
c7ce8b011bcb SR import: Negate negative Z values and issue a warning if a negative Z value is found. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2530 diff changeset	433
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	434 for line := 1; s.Scan(); line++ {
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	435 text := s.Text()
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	436 var p octree.Vertex
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	437 // fmt.Sscanf(text, "%f,%f,%f") is 4 times slower.
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	438 idx := strings.IndexByte(text, ',')
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	439 if idx == -1 {
1138 443fc80a315f Don't issue new lines at end of log messages when importing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1137 diff changeset	440 feedback.Warn("format error in line %d", line)
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	441 continue
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	442 }
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	443 var err error
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	444 if p.X, err = strconv.ParseFloat(text[:idx], 64); err != nil {
1138 443fc80a315f Don't issue new lines at end of log messages when importing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1137 diff changeset	445 feedback.Warn("format error in line %d: %v", line, err)
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	446 continue
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	447 }
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	448 text = text[idx+1:]
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	449 if idx = strings.IndexByte(text, ','); idx == -1 {
1138 443fc80a315f Don't issue new lines at end of log messages when importing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1137 diff changeset	450 feedback.Warn("format error in line %d", line)
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	451 continue
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	452 }
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	453 if p.Y, err = strconv.ParseFloat(text[:idx], 64); err != nil {
1138 443fc80a315f Don't issue new lines at end of log messages when importing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1137 diff changeset	454 feedback.Warn("format error in line %d: %v", line, err)
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	455 continue
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	456 }
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	457 text = text[idx+1:]
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	458 if p.Z, err = strconv.ParseFloat(text, 64); err != nil {
1138 443fc80a315f Don't issue new lines at end of log messages when importing. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1137 diff changeset	459 feedback.Warn("format error in line %d: %v", line, err)
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	460 continue
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	461 }
2612 c7ce8b011bcb SR import: Negate negative Z values and issue a warning if a negative Z value is found. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2530 diff changeset	462 if p.Z < 0 {
c7ce8b011bcb SR import: Negate negative Z values and issue a warning if a negative Z value is found. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2530 diff changeset	463 p.Z = -p.Z
c7ce8b011bcb SR import: Negate negative Z values and issue a warning if a negative Z value is found. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2530 diff changeset	464 if !hasNegZ {
c7ce8b011bcb SR import: Negate negative Z values and issue a warning if a negative Z value is found. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2530 diff changeset	465 hasNegZ = true
c7ce8b011bcb SR import: Negate negative Z values and issue a warning if a negative Z value is found. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2530 diff changeset	466 feedback.Warn("Negative Z value found: Using -Z")
c7ce8b011bcb SR import: Negate negative Z values and issue a warning if a negative Z value is found. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2530 diff changeset	467 }
c7ce8b011bcb SR import: Negate negative Z values and issue a warning if a negative Z value is found. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2530 diff changeset	468 }
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	469 mpz = append(mpz, p)
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	470 }
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	471
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	472 if err := s.Err(); err != nil {
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	473 return nil, err
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	474 }
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	475
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	476 return mpz, nil
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	477 }
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	478
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	479 func loadXYZ(f *zip.File, feedback Feedback) (octree.MultiPointZ, error) {
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	480 r, err := f.Open()
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	481 if err != nil {
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	482 return nil, err
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	483 }
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	484 defer r.Close()
988 7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface. Sascha L. Teichmann <teichmann@intevation.de> parents: 984 diff changeset	485 return loadXYZReader(r, feedback)
959 6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	486 }
6ab012d0f0c2 Started with writing an importer job for sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	487
1786 09349ca27dd7 Imports: Removed duplicated code path to store WKB polygons into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1754 diff changeset	488 func loadBoundary(z *zip.ReadCloser) (polygonSlice, error) {
1225 4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1224 diff changeset	489 shpF := common.FindInZIP(z, ".shp")
960 e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	490 if shpF == nil {
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	491 return nil, nil
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	492 }
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	493 prefix := strings.TrimSuffix(shpF.Name, path.Ext(shpF.Name))
1225 4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 1224 diff changeset	494 dbfF := common.FindInZIP(z, prefix+".dbf")
960 e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	495 if dbfF == nil {
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	496 return nil, fmt.Errorf("No DBF file found for %s", shpF.Name)
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	497 }
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	498
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	499 shpR, err := shpF.Open()
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	500 if err != nil {
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	501 return nil, err
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	502 }
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	503
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	504 dbfR, err := dbfF.Open()
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	505 if err != nil {
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	506 shpR.Close()
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	507 return nil, err
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	508 }
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	509 sr := shp.SequentialReaderFromExt(shpR, dbfR)
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	510 defer sr.Close()
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	511
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	512 if !sr.Next() {
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	513 return nil, sr.Err()
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	514 }
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	515
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	516 _, s := sr.Shape()
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	517 if s == nil {
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	518 return nil, sr.Err()
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	519 }
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	520
972 17a03a84b0e8 Split out polygon code out of sounding result importer source file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 971 diff changeset	521 return shapeToPolygon(s)
960 e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	522 }
e23ae2c83427 Load boundary polygon of sounding result from ZIP file. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 959 diff changeset	523
977 4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	524 func generateContours(tree octree.Tree, tx sql.Tx, id int64) error {
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	525 stmt, err := tx.Prepare(insertContourSQL)
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	526 if err != nil {
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	527 return err
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	528 }
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	529 defer stmt.Close()
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	530
1184 064d44ccc6f2 Adjust contour lines heights to multiples of step width Tom Gottfried <tom@intevation.de> parents: 1168 diff changeset	531 // Adjust contour lines heights to multiples of contourStepWidth
2530 3f61b84ae7a6 SR import: Set missing EPSG code to TIN. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2529 diff changeset	532 var heights []float64
1184 064d44ccc6f2 Adjust contour lines heights to multiples of step width Tom Gottfried <tom@intevation.de> parents: 1168 diff changeset	533 h := contourStepWidth * math.Ceil(tree.Min.Z/contourStepWidth)
064d44ccc6f2 Adjust contour lines heights to multiples of step width Tom Gottfried <tom@intevation.de> parents: 1168 diff changeset	534 for ; h <= tree.Max.Z; h += contourStepWidth {
064d44ccc6f2 Adjust contour lines heights to multiples of step width Tom Gottfried <tom@intevation.de> parents: 1168 diff changeset	535 heights = append(heights, h)
064d44ccc6f2 Adjust contour lines heights to multiples of step width Tom Gottfried <tom@intevation.de> parents: 1168 diff changeset	536 }
064d44ccc6f2 Adjust contour lines heights to multiples of step width Tom Gottfried <tom@intevation.de> parents: 1168 diff changeset	537
064d44ccc6f2 Adjust contour lines heights to multiples of step width Tom Gottfried <tom@intevation.de> parents: 1168 diff changeset	538 octree.DoContours(tree, heights, func(res *octree.ContourResult) {
2529 45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	539 if err == nil && len(res.Lines) > 0 {
977 4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	540 _, err = stmt.Exec(
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	541 id, res.Height, tree.EPSG,
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	542 res.Lines.AsWKB2D(),
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	543 contourTolerance)
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	544 }
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	545 })
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	546
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	547 return err
4a2ca0e20006 Fixed build error. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 976 diff changeset	548 }

Mercurial > gemma

annotate pkg/imports/sr.go @ 2873:b1707f60f241