Mercurial > gemma

annotate pkg/imports/sr.go @ 4791:1fef4679b07a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Added an endpoint GET api/imports/export to export imports.
author Sascha L. Teichmann <sascha.teichmann@intevation.de>
date Fri, 25 Oct 2019 13:03:50 +0200
parents 0feddad5bb6d
children f4abfd0ee8ad
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1017
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
2 // without warranty, see README.md and license for details.
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
3 //
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
6 //
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
7 // Copyright (C) 2018 by via donau
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
9 // Software engineering by Intevation GmbH
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
10 //
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
11 // Author(s):
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
1337
53057ab27ff2 add headers for licensing to some middleware files
Fadi Abbud <fadi.abbud@intevation.de>
parents: 1292
diff changeset
13 // * Tom Gottfried <tom@intevation.de>
1017
a244b18cb916 Added GNU Affero General Public License.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 991
diff changeset
14
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15 package imports
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 import (
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 "archive/zip"
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 "bufio"
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
20 "context"
973
b6fec8f85599 Generate TINs and octrees in sounding result importer.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 972
diff changeset
21 "crypto/sha1"
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
22 "database/sql"
973
b6fec8f85599 Generate TINs and octrees in sounding result importer.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 972
diff changeset
23 "encoding/hex"
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
24 "errors"
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
25 "fmt"
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26 "io"
1184
064d44ccc6f2 Adjust contour lines heights to multiples of step width
Tom Gottfried <tom@intevation.de>
parents: 1168
diff changeset
27 "math"
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
28 "os"
960
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
29 "path"
978
544a5cfe07cd Started with endpoint for uploading sounding result and trigger respective import job.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 977
diff changeset
30 "path/filepath"
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
31 "strconv"
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
32 "strings"
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
33 "time"
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
34
960
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
35 shp "github.com/jonas-p/go-shp"
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
36
1224
bc4b642c8d04 Started with an upload sounding result to temp upload area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1193
diff changeset
37 "gemma.intevation.de/gemma/pkg/common"
bc4b642c8d04 Started with an upload sounding result to temp upload area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1193
diff changeset
38 "gemma.intevation.de/gemma/pkg/models"
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
39 "gemma.intevation.de/gemma/pkg/octree"
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
40 "gemma.intevation.de/gemma/pkg/wkb"
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
41 )
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
42
1498
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
43 // SoundingResult is a Job to import sounding reults
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
44 // from a ZIP file into the database.
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
45 type SoundingResult struct {
1498
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
46 // Dir is the folder in the file system where the
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
47 // 'sr.zip' is expect to be.
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
48 Dir string `json:"dir"`
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
49
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
50 // Override data
1498
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
51 // Date if given overrides the date value from the meta.json.
1900
6a67cd819e93 To prepare stretch import made some model data types re-usable.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1786
diff changeset
52 Date *models.Date `json:"date,omitempty"`
1498
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
53 // Date if given overrides the name of the bottleneck from the meta.json.
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
54 Bottleneck *string `json:"bottleneck,omitempty"`
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
55 // EPSG if given overrides the EPSG code from the meta.json.
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
56 // Defaults to WGS84.
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
57 EPSG *uint `json:"epsg,omitempty"`
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
58 // DepthReference if given overides the DepthReference value
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
59 // from the meta.json.
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
60 DepthReference *string `json:"depth-reference,omitempty"`
3585
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3518
diff changeset
61 // SingleBeam indicates that the sounding is a single beam scan.
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3518
diff changeset
62 SingleBeam *bool `json:"single-beam,omitempty"`
3945
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
63 // NegateZ indicated that the Z values of thy XYZ input should be
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
64 // multiplied by -1.
3946
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
65 NegateZ *bool `json:"negate-z,omitempty"`
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
66 }
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
67
964
1e2dce348cfb Serialize boundary polygon of sounding result as WKB.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 961
diff changeset
68 const (
1130
42617bba8709 Go from 50cm to 10cm stepwidth in calculating the contorlines.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1017
diff changeset
69 contourStepWidth = 0.1
977
4a2ca0e20006 Fixed build error.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 976
diff changeset
70 contourTolerance = 0.1
4a2ca0e20006 Fixed build error.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 976
diff changeset
71 )
4a2ca0e20006 Fixed build error.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 976
diff changeset
72
3646
810b28f59b8b Generate random points for second mesh.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3609
diff changeset
73 const (
4751
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
74 // multiBeamThreshold is the number of points m² which
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
75 // is assumed that greater values are indicators for
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
76 // an already interpolated point cloud.
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
77 multiBeamThreshold = 1.0 / 5.0
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
78 )
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
79
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
80 const (
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
81 // pointsPerSquareMeter is the average number of points
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
82 // when generating a artifical height model for single beam scans.
4754
64979fec89a7 SR import: Reduce the point density in interpolation 1 point per meter².
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4751
diff changeset
83 pointsPerSquareMeter = 1.0
3646
810b28f59b8b Generate random points for second mesh.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3609
diff changeset
84 )
810b28f59b8b Generate random points for second mesh.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3609
diff changeset
85
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
86 const (
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
87 // isoCellSize is the side length of a raster cell when tracing
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
88 // iso areas.
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
89 isoCellSize = 0.5
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
90 )
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
91
4182
49012340336c Made 'golint' finally happy with imports package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4177
diff changeset
92 // SRJobKind is the unique name of this import job type.
991
a301d240905f Decoupled import job creation and job execution with a factory function.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 988
diff changeset
93 const SRJobKind JobKind = "sr"
a301d240905f Decoupled import job creation and job execution with a factory function.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 988
diff changeset
94
1136
a5069da2f0b7 Independent imports in terms of affected tables/dependencies are now run concurrently.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1131
diff changeset
95 type srJobCreator struct{}
a5069da2f0b7 Independent imports in terms of affected tables/dependencies are now run concurrently.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1131
diff changeset
96
4214
49564382ffff Added a import queue job to recalculate the contour lines of the sounding results if the heights have changed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4182
diff changeset
97 func init() { RegisterJobCreator(SRJobKind, srJobCreator{}) }
1193
58acc343b1b6 Implemented the db stuff of the review process. Needs testing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1184
diff changeset
98
1754
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1664
diff changeset
99 func (srJobCreator) Description() string { return "sounding results" }
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1664
diff changeset
100
807569b08513 Import queue: Auto acceptance is now a property of the import kind itself and is not configurable any more.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1664
diff changeset
101 func (srJobCreator) AutoAccept() bool { return false }
1646
a0982c38eac0 Import queue: Implemented email notifications.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1572
diff changeset
102
2187
7c83b5277c1c Import queue: Removed boilerplate code to deserialize jobs from JSON by making it part of the import queue.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2168
diff changeset
103 func (srJobCreator) Create() Job { return new(SoundingResult) }
1136
a5069da2f0b7 Independent imports in terms of affected tables/dependencies are now run concurrently.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1131
diff changeset
104
3219
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2963
diff changeset
105 func (srJobCreator) Depends() [2][]string {
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2963
diff changeset
106 return [2][]string{
4578
c657dec6b0fa Removed old contours from backend and database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4570
diff changeset
107 {"sounding_results", "sounding_results_iso_areas"},
3219
4acbee65275d Import queue: Split locked dependencies in exclusively and multiple uses.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2963
diff changeset
108 {"bottlenecks"},
1136
a5069da2f0b7 Independent imports in terms of affected tables/dependencies are now run concurrently.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1131
diff changeset
109 }
a5069da2f0b7 Independent imports in terms of affected tables/dependencies are now run concurrently.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1131
diff changeset
110 }
a5069da2f0b7 Independent imports in terms of affected tables/dependencies are now run concurrently.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1131
diff changeset
111
1193
58acc343b1b6 Implemented the db stuff of the review process. Needs testing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1184
diff changeset
112 func (srJobCreator) StageDone(
1328
d753ce6cf588 To make golint happier made context.Context to be the first argument in all calls.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1317
diff changeset
113 ctx context.Context,
1193
58acc343b1b6 Implemented the db stuff of the review process. Needs testing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1184
diff changeset
114 tx *sql.Tx,
58acc343b1b6 Implemented the db stuff of the review process. Needs testing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1184
diff changeset
115 id int64,
58acc343b1b6 Implemented the db stuff of the review process. Needs testing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1184
diff changeset
116 ) error {
58acc343b1b6 Implemented the db stuff of the review process. Needs testing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1184
diff changeset
117 _, err := tx.ExecContext(ctx, srStageDoneSQL, id)
58acc343b1b6 Implemented the db stuff of the review process. Needs testing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1184
diff changeset
118 return err
991
a301d240905f Decoupled import job creation and job execution with a factory function.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 988
diff changeset
119 }
a301d240905f Decoupled import job creation and job execution with a factory function.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 988
diff changeset
120
977
4a2ca0e20006 Fixed build error.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 976
diff changeset
121 const (
1193
58acc343b1b6 Implemented the db stuff of the review process. Needs testing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1184
diff changeset
122 srStageDoneSQL = `
58acc343b1b6 Implemented the db stuff of the review process. Needs testing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1184
diff changeset
123 UPDATE waterway.sounding_results SET staging_done = true
1502
0abcc82ef976 Sounding results: Be a bit more precise when setting stageing_done.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1501
diff changeset
124 WHERE id = (
1995
59055c8301df Move import queue to its own database namespace
Tom Gottfried <tom@intevation.de>
parents: 1900
diff changeset
125 SELECT key from import.track_imports
1502
0abcc82ef976 Sounding results: Be a bit more precise when setting stageing_done.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1501
diff changeset
126 WHERE import_id = $1 AND
0abcc82ef976 Sounding results: Be a bit more precise when setting stageing_done.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1501
diff changeset
127 relation = 'waterway.sounding_results'::regclass)`
1193
58acc343b1b6 Implemented the db stuff of the review process. Needs testing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1184
diff changeset
128
2529
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2516
diff changeset
129 insertHullSQL = `
961
a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 960
diff changeset
130 INSERT INTO waterway.sounding_results (
a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 960
diff changeset
131 bottleneck_id,
a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 960
diff changeset
132 date_info,
a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 960
diff changeset
133 depth_reference,
4629
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
134 area,
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
135 surtyp
2529
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2516
diff changeset
136 ) SELECT
3656
2a079d0a71c1 Ensure sounding results are associated to matching bottleneck version
Tom Gottfried <tom@intevation.de>
parents: 3645
diff changeset
137 bottleneck_id,
961
a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 960
diff changeset
138 $2::date,
a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 960
diff changeset
139 $3,
2529
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2516
diff changeset
140 (SELECT
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2516
diff changeset
141 CASE WHEN $5::bytea IS NULL THEN
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2516
diff changeset
142 ST_Transform(ST_ConcaveHull(ST_Force2D(ST_GeomFromWKB($4, $6::integer)), 0.7), 4326)::geography
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2516
diff changeset
143 ELSE
3652
f415c5ed60f6 Another attempt to fix the ST_IsValid check contstraint problem ... and also failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3651
diff changeset
144 ST_MakeValid(ST_Transform(ST_GeomFromWKB($5, $6::integer), 4326))::geography
4629
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
145 END),
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
146 $7
3656
2a079d0a71c1 Ensure sounding results are associated to matching bottleneck version
Tom Gottfried <tom@intevation.de>
parents: 3645
diff changeset
147 FROM waterway.bottlenecks
2a079d0a71c1 Ensure sounding results are associated to matching bottleneck version
Tom Gottfried <tom@intevation.de>
parents: 3645
diff changeset
148 WHERE objnam = $1 AND validity @> CAST($2 AS timestamptz)
971
f9fb6c399f3f Moved generating of tins to octree package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 964
diff changeset
149 RETURNING
f9fb6c399f3f Moved generating of tins to octree package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 964
diff changeset
150 id,
2529
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2516
diff changeset
151 ST_X(ST_Centroid(area::geometry)),
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2516
diff changeset
152 ST_Y(ST_Centroid(area::geometry)),
2833
1b6840093eac Prevent calculation of wrong UTM zones
Tom Gottfried <tom@intevation.de>
parents: 2612
diff changeset
153 best_utm(area),
1b6840093eac Prevent calculation of wrong UTM zones
Tom Gottfried <tom@intevation.de>
parents: 2612
diff changeset
154 ST_AsBinary(ST_Transform(area::geometry, best_utm(area)))
2529
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2516
diff changeset
155 `
973
b6fec8f85599 Generate TINs and octrees in sounding result importer.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 972
diff changeset
156
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
157 reprojectPointsBufferedSQL = `
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
158 SELECT
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
159 ST_AsBinary(ST_Transform(ST_GeomFromWKB($1, $2::integer), $3::integer)),
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
160 ST_AsBinary(ST_Buffer(ST_Transform(ST_GeomFromWKB($1, $2::integer), $3::integer), 0.1)),
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
161 ST_Area(ST_Transform(ST_GeomFromWKB($1, $2::integer), $3::integer))`
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
162
4660
6eab3ac0e849 Speak of meshes instead od octrees as they are not longer used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4654
diff changeset
163 insertMeshSQL = `
1131
2e6b47cdb2ca Store octrees along with sounding results
Tom Gottfried <tom@intevation.de>
parents: 1130
diff changeset
164 UPDATE waterway.sounding_results SET
4654
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
165 mesh_checksum = $2, mesh_index = $3
1131
2e6b47cdb2ca Store octrees along with sounding results
Tom Gottfried <tom@intevation.de>
parents: 1130
diff changeset
166 WHERE id = $1`
977
4a2ca0e20006 Fixed build error.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 976
diff changeset
167
3651
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
168 repairBoundarySQL = `
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
169 SELECT
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
170 ST_AsBinary(ST_Buffer(ST_MakeValid(ST_GeomFromWKB($1, $2::integer)), 0.0)),
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
171 ST_AsBinary(ST_Buffer(ST_MakeValid(ST_GeomFromWKB($1, $2::integer)), 0.1))`
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
172
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
173 insertIsoAreasSQL = `
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
174 INSERT INTO waterway.sounding_results_iso_areas (
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
175 sounding_result_id,
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
176 height,
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
177 areas
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
178 )
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
179 SELECT
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
180 $1,
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
181 $2,
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
182 ST_Transform(
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
183 ST_Multi(
4734
56bd9ba0354c Removed ST_MakeValid() call from storing iso areas in SR import, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4686
diff changeset
184 ST_Collectionextract(
56bd9ba0354c Removed ST_MakeValid() call from storing iso areas in SR import, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4686
diff changeset
185 ST_SimplifyPreserveTopology(ST_GeomFromWKB($4, $3::integer), $5),
4563
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
186 3
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
187 )
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
188 ),
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
189 4326
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
190 )
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
191 FROM waterway.sounding_results sr
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
192 WHERE id = $1
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
193 `
3479
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
194
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
195 selectGaugeLDCSQL = `
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
196 SELECT
3518
4cd4375efd5d ZP for sounding result imports: Use precise name of the depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3504
diff changeset
197 grwl.value,
4cd4375efd5d ZP for sounding result imports: Use precise name of the depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3504
diff changeset
198 grwl.depth_reference
3479
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
199 FROM waterway.gauges_reference_water_levels grwl
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
200 JOIN waterway.bottlenecks bns
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
201 ON grwl.location = bns.gauge_location
4055
98e2041c7ebf Adapted sr-import to new historization model.
Sascha Wilde <wilde@intevation.de>
parents: 3977
diff changeset
202 AND grwl.validity @> CAST($2 AS timestamptz)
3645
02951a62e8c6 'Historicise' bottlenecks on import
Tom Gottfried <tom@intevation.de>
parents: 3609
diff changeset
203 WHERE bns.objnam = $1
02951a62e8c6 'Historicise' bottlenecks on import
Tom Gottfried <tom@intevation.de>
parents: 3609
diff changeset
204 AND bns.validity @> CAST($2 AS timestamptz)
02951a62e8c6 'Historicise' bottlenecks on import
Tom Gottfried <tom@intevation.de>
parents: 3609
diff changeset
205 AND grwl.depth_reference like 'LDC%'
3479
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
206 `
3591
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
207
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
208 reprojectPointsSingleBeamSQL = `
3600
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
209 SELECT
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
210 ST_AsBinary(
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
211 ST_Transform(
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
212 ST_GeomFromWKB($1, $2::integer),
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
213 best_utm(CAST(ST_Transform(ST_GeomFromWKB($1, $2::integer), 4326) AS geography)))),
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
214 best_utm(CAST(ST_Transform(ST_GeomFromWKB($1, $2::integer), 4326) AS geography))
3591
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
215 `
961
a4c92e0ef2e1 Stored point cloud and bounding area of sounding result in database. WIP.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 960
diff changeset
216 )
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
217
4791
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
218 func (sr *SoundingResult) Description() (string, error) {
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
219
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
220 var descs []string
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
221
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
222 if sr.Bottleneck != nil {
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
223 descs = append(descs, *sr.Bottleneck)
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
224 }
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
225 if sr.Date != nil {
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
226 descs = append(descs, (*sr).Date.Format(common.DateFormat))
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
227 }
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
228 if sr.NegateZ != nil && *sr.NegateZ {
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
229 descs = append(descs, "negateZ")
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
230 }
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
231 return strings.Join(descs, "|"), nil
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
232 }
1fef4679b07a Added an endpoint GET api/imports/export to export imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4776
diff changeset
233
3946
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
234 func (sr *SoundingResult) singleBeam() bool {
3585
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3518
diff changeset
235 return sr.SingleBeam != nil && *sr.SingleBeam
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3518
diff changeset
236 }
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3518
diff changeset
237
4629
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
238 func (sr *SoundingResult) surtype() string {
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
239 if sr.singleBeam() {
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
240 return "single"
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
241 }
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
242 return "multi"
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
243 }
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
244
3946
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
245 func (sr *SoundingResult) negateZ() bool {
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
246 return sr.NegateZ != nil && *sr.NegateZ
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
247 }
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
248
1498
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
249 // Do executes the actual sounding result import.
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
250 func (sr *SoundingResult) Do(
1328
d753ce6cf588 To make golint happier made context.Context to be the first argument in all calls.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1317
diff changeset
251 ctx context.Context,
1168
930fdd8b474f Track successfull imports in a separate table to be able to remove them later.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1138
diff changeset
252 importID int64,
930fdd8b474f Track successfull imports in a separate table to be able to remove them later.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1138
diff changeset
253 conn *sql.Conn,
930fdd8b474f Track successfull imports in a separate table to be able to remove them later.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1138
diff changeset
254 feedback Feedback,
1392
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1384
diff changeset
255 ) (interface{}, error) {
988
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
256
3587
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
257 start := time.Now()
2529
45d51a49f191 SR import: Use own triangulation and clipping when importing sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2516
diff changeset
258
3748
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
259 zpath := filepath.Join(sr.Dir, "sr.zip")
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
260
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
261 z, err := zip.OpenReader(zpath)
988
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
262 if err != nil {
3762
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3748
diff changeset
263 feedback.Info("%v", err)
98d5dd2f0ca1 Don't show unnecessary warnings when uploading TXT file for SR.
Sascha Wilde <wilde@intevation.de>
parents: 3748
diff changeset
264 feedback.Info("Falling back to TXT file mode.")
3748
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
265 z = nil
988
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
266 }
3748
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
267 if z != nil {
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
268 defer z.Close()
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
269 }
988
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
270
1138
443fc80a315f Don't issue new lines at end of log messages when importing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1137
diff changeset
271 feedback.Info("Looking for 'meta.json'")
3600
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
272
1225
4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1224
diff changeset
273 mf := common.FindInZIP(z, "meta.json")
1292
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1239
diff changeset
274 if mf == nil && !sr.completeOverride() {
4177
8b75ac5e243e Made 'staticcheck' happy with pgxutils package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4171
diff changeset
275 return nil, errors.New("cannot find 'meta.json'")
988
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
276 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
277
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
278 m, err := sr.loadMeta(mf)
988
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
279 if err != nil {
1392
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1384
diff changeset
280 return nil, err
988
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
281 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
282
2873
b1707f60f241 SR import: Log name of bottleneck and survey date into import log.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2833
diff changeset
283 feedback.Info("Bottleneck: %s", m.Bottleneck)
b1707f60f241 SR import: Log name of bottleneck and survey date into import log.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2833
diff changeset
284 feedback.Info("Survey date: %s", m.Date.Format(common.DateFormat))
b1707f60f241 SR import: Log name of bottleneck and survey date into import log.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2833
diff changeset
285
3487
fdb0439850d5 Generalized transformation of vertices during sounding result import a bit.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3479
diff changeset
286 var xform vertexTransform
fdb0439850d5 Generalized transformation of vertices during sounding result import a bit.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3479
diff changeset
287
3946
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
288 if sr.negateZ() {
3945
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
289 xform = negateZTransform
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
290 } else {
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
291 xform = identityTransform
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
292 }
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
293
3479
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
294 if m.DepthReference == "ZPG" {
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
295 feedback.Info("Found ZPG as reference system -> translating Z values to LDC")
3487
fdb0439850d5 Generalized transformation of vertices during sounding result import a bit.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3479
diff changeset
296 var ldc float64
3518
4cd4375efd5d ZP for sounding result imports: Use precise name of the depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3504
diff changeset
297 var depthReference string
3645
02951a62e8c6 'Historicise' bottlenecks on import
Tom Gottfried <tom@intevation.de>
parents: 3609
diff changeset
298 err := conn.QueryRowContext(ctx,
02951a62e8c6 'Historicise' bottlenecks on import
Tom Gottfried <tom@intevation.de>
parents: 3609
diff changeset
299 selectGaugeLDCSQL,
02951a62e8c6 'Historicise' bottlenecks on import
Tom Gottfried <tom@intevation.de>
parents: 3609
diff changeset
300 m.Bottleneck,
02951a62e8c6 'Historicise' bottlenecks on import
Tom Gottfried <tom@intevation.de>
parents: 3609
diff changeset
301 m.Date.Time,
02951a62e8c6 'Historicise' bottlenecks on import
Tom Gottfried <tom@intevation.de>
parents: 3609
diff changeset
302 ).Scan(
3518
4cd4375efd5d ZP for sounding result imports: Use precise name of the depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3504
diff changeset
303 &ldc,
4cd4375efd5d ZP for sounding result imports: Use precise name of the depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3504
diff changeset
304 &depthReference,
4cd4375efd5d ZP for sounding result imports: Use precise name of the depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3504
diff changeset
305 )
3479
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
306 switch {
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
307 case err == sql.ErrNoRows:
4177
8b75ac5e243e Made 'staticcheck' happy with pgxutils package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4171
diff changeset
308 return nil, errors.New("cannot load LDC value")
3479
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
309 case err != nil:
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
310 return nil, err
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
311 }
3904
4b5c4007fc5d SR import: Fixed LDC treatment if depth reference system is ZPG.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3882
diff changeset
312
4b5c4007fc5d SR import: Fixed LDC treatment if depth reference system is ZPG.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3882
diff changeset
313 // LDC is cm. The data is in m.
4b5c4007fc5d SR import: Fixed LDC treatment if depth reference system is ZPG.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3882
diff changeset
314 ldc /= 100
3945
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
315 xform = chainTransforms(
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
316 xform,
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
317 func(v octree.Vertex) octree.Vertex {
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
318 return octree.Vertex{X: v.X, Y: v.Y, Z: v.Z + ldc}
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
319 })
3518
4cd4375efd5d ZP for sounding result imports: Use precise name of the depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3504
diff changeset
320 m.DepthReference = depthReference
3479
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
321 }
83b58f6356e7 Translate sounding results Z values import to LDC if ZPG was found as depth reference system.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3219
diff changeset
322
1328
d753ce6cf588 To make golint happier made context.Context to be the first argument in all calls.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1317
diff changeset
323 if err := m.Validate(ctx, conn); err != nil {
1392
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1384
diff changeset
324 return nil, common.ToError(err)
988
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
325 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
326
3748
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
327 var xyz octree.MultiPointZ
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
328
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
329 if z != nil { // Scanning ZIP file for *.xyz file.
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
330 var xyzf *zip.File
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
331 for _, ext := range []string{".xyz", ".txt"} {
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
332 feedback.Info("Looking for '*%s'", ext)
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
333 if xyzf = common.FindInZIP(z, ext); xyzf != nil {
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
334 break
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
335 }
1528
5874cedd7f91 Sounding result import: Accept *.txt files for XYZ data, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1502
diff changeset
336 }
3748
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
337 if xyzf == nil {
4177
8b75ac5e243e Made 'staticcheck' happy with pgxutils package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4171
diff changeset
338 return nil, errors.New("cannot find any *.xyz or *.txt file")
3748
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
339 }
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
340 xyz, err = loadXYZ(xyzf, feedback, xform)
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
341 } else { // TXT file mode
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
342 xyz, err = loadXYZFile(zpath, feedback, xform)
1528
5874cedd7f91 Sounding result import: Accept *.txt files for XYZ data, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1502
diff changeset
343 }
988
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
344 if err != nil {
1392
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1384
diff changeset
345 return nil, err
988
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
346 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
347
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
348 if len(xyz) == 0 {
1392
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1384
diff changeset
349 return nil, errors.New("XYZ does not contain any vertices")
988
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
350 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
351
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
352 // Is there a boundary shapefile in the ZIP archive?
3591
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
353 boundary, err := loadBoundary(z)
988
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
354 if err != nil {
1392
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1384
diff changeset
355 return nil, err
988
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
356 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
357
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
358 tx, err := conn.BeginTx(ctx, nil)
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
359 if err != nil {
1392
0e1d89241cda Imports: An Import (e.g. a sounding result import) can now write a 'summary' of a successful import. This is done if the import switches to to state 'pending'.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1384
diff changeset
360 return nil, err
988
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
361 }
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
362 defer tx.Rollback()
7dfd3db94e6d In preparation of persisting import jobs logging is done through an interface.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 984
diff changeset
363
4629
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
364 summary, err := sr.processScan(
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
365 ctx,
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
366 tx,
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
367 feedback,
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
368 importID,
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
369 m,
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
370 xyz,
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
371 boundary,
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
372 )
3587
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
373 if err != nil {
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
374 return nil, err
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
375 }
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
376
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
377 if err = tx.Commit(); err != nil {
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
378 feedback.Error(
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
379 "Storing sounding result failed after %v.", time.Since(start))
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
380 return nil, err
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
381 }
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
382
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
383 feedback.Info(
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
384 "Storing sounding result was successful after %v.", time.Since(start))
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
385
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
386 return summary, nil
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
387 }
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
388
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
389 func (sr *SoundingResult) processScan(
3587
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
390 ctx context.Context,
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
391 tx *sql.Tx,
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
392 feedback Feedback,
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
393 importID int64,
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
394 m *models.SoundingResultMeta,
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
395 xyz octree.MultiPointZ,
3591
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
396 boundary polygonSlice,
3587
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
397 ) (interface{}, error) {
3591
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
398
4629
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
399 if sr.singleBeam() {
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
400 feedback.Info("Processing as single beam scan.")
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
401 } else {
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
402 feedback.Info("Processing as multi beam scan.")
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
403 }
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
404
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
405 feedback.Info("Reproject XYZ data.")
3591
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
406
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
407 start := time.Now()
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
408
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
409 xyzWKB := xyz.AsWKB()
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
410 var reproj []byte
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
411 var epsg uint32
3591
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
412
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
413 if err := tx.QueryRowContext(
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
414 ctx,
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
415 reprojectPointsSingleBeamSQL,
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
416 xyzWKB,
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
417 m.EPSG,
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
418 ).Scan(&reproj, &epsg); err != nil {
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
419 return nil, err
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
420 }
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
421
3600
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
422 if err := xyz.FromWKB(reproj); err != nil {
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
423 return nil, err
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
424 }
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
425
3591
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
426 feedback.Info("Reprojecting points to EPSG %d took %v.",
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
427 epsg, time.Since(start))
3600
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
428 feedback.Info("Number of reprojected points: %d", len(xyz))
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
429 feedback.Info("Triangulate XYZ data.")
3600
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
430
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
431 start = time.Now()
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
432 tri, err := octree.Triangulate(xyz)
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
433 if err != nil {
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
434 return nil, err
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
435 }
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
436 feedback.Info("Triangulation took %v.", time.Since(start))
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
437 feedback.Info("Number triangles: %d.", len(tri.Triangles)/3)
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
438
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
439 var (
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
440 clippingPolygon octree.Polygon
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
441 clippingPolygonBuffered octree.Polygon
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
442 removed map[int32]struct{}
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
443 polygonArea float64
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
444 clippingPolygonWKB []byte
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
445 tin *octree.Tin
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
446 )
3646
810b28f59b8b Generate random points for second mesh.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3609
diff changeset
447
3600
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
448 if boundary == nil {
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
449 feedback.Info("No boundary given. Calulate from XYZ data.")
3733
ec86a7155377 Estimated too large triangles as triangles which have an edge which is at least 3.5 times as long as the standard dev of the longest egde per inner triangle.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3664
diff changeset
450 tooLongEdge := tri.EstimateTooLong()
3655
a6c671abbc35 Started with cleaning up the single beam import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3654
diff changeset
451 feedback.Info("Eliminate triangles with edges longer than %.2f meters.", tooLongEdge)
3646
810b28f59b8b Generate random points for second mesh.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3609
diff changeset
452
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
453 var polygon octree.LineStringZ
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
454 start = time.Now()
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
455 polygon, removed = tri.ConcaveHull(tooLongEdge)
3646
810b28f59b8b Generate random points for second mesh.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3609
diff changeset
456
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
457 polygonArea = polygon.Area()
3646
810b28f59b8b Generate random points for second mesh.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3609
diff changeset
458 if polygonArea < 0.0 { // counter clockwise
810b28f59b8b Generate random points for second mesh.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3609
diff changeset
459 polygonArea = -polygonArea
810b28f59b8b Generate random points for second mesh.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3609
diff changeset
460 polygon.Reverse()
810b28f59b8b Generate random points for second mesh.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3609
diff changeset
461 }
810b28f59b8b Generate random points for second mesh.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3609
diff changeset
462
3651
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
463 clippingPolygon.FromLineStringZ(polygon)
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
464
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
465 var buffered []byte
3651
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
466 if err := tx.QueryRowContext(
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
467 ctx,
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
468 repairBoundarySQL,
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
469 clippingPolygon.AsWKB(),
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
470 epsg,
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
471 ).Scan(&clippingPolygonWKB, &buffered); err != nil {
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
472 return nil, err
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
473 }
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
474
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
475 if err := clippingPolygon.FromWKB(clippingPolygonWKB); err != nil {
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
476 return nil, err
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
477 }
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
478 if err := clippingPolygonBuffered.FromWKB(buffered); err != nil {
3651
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
479 return nil, err
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
480 }
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
481
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
482 feedback.Info("Calculating took %v.", time.Since(start))
3600
6590733e2ebc SR import: If in single beam mode and no boundary is given eliminate triangles which have at least an edge with is longer than 98% of the overall edge lengths.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3596
diff changeset
483
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
484 } else { // has Boundary
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
485 feedback.Info("Using uploaded boundary polygon.")
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
486 var buffered []byte
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
487 if err = tx.QueryRowContext(
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
488 ctx,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
489 reprojectPointsBufferedSQL,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
490 boundary.asWKB(),
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
491 m.EPSG,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
492 epsg,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
493 ).Scan(&clippingPolygonWKB, &buffered, &polygonArea); err != nil {
3651
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
494 return nil, err
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
495 }
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
496 if err := clippingPolygon.FromWKB(clippingPolygonWKB); err != nil {
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
497 return nil, err
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
498 }
3651
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
499 if err := clippingPolygonBuffered.FromWKB(buffered); err != nil {
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
500 return nil, err
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
501 }
c368a9a20478 Tried to fix invalid new boundary polygon with ST_MakeValid() ... and failed.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3650
diff changeset
502
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
503 tin := tri.Tin()
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
504 tin.EPSG = epsg
3650
01ce3ba9b0d0 Fixed generating of random points.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3646
diff changeset
505
01ce3ba9b0d0 Fixed generating of random points.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3646
diff changeset
506 var str octree.STRTree
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
507 str.Build(tin)
3655
a6c671abbc35 Started with cleaning up the single beam import.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3654
diff changeset
508
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
509 removed = str.Clip(&clippingPolygon)
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
510 }
3650
01ce3ba9b0d0 Fixed generating of random points.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3646
diff changeset
511
4629
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
512 if sr.singleBeam() {
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
513
4751
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
514 origDensity := float64(len(xyz)) / polygonArea
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
515
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
516 feedback.Info("Boundary area: %.2fm²", polygonArea)
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
517 feedback.Info("Original point density: %.2f points/m²", origDensity)
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
518
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
519 if origDensity > multiBeamThreshold {
4776
0feddad5bb6d Fixed another formatting oversight.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4768
diff changeset
520 feedback.Warn("The density is greater than %.2f points/m².", multiBeamThreshold)
4751
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
521 feedback.Warn("It is assumed that the data is already interpolated.")
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
522 goto multibeam
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
523 }
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
524
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
525 // Build the first mesh to generate random points on.
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
526
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
527 feedback.Info("Build virtual DEM based on original XYZ data.")
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
528
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
529 start = time.Now()
3650
01ce3ba9b0d0 Fixed generating of random points.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3646
diff changeset
530
4654
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
531 tin := tri.Tin()
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
532 var virtual octree.STRTree
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
533 virtual.BuildWithout(tin, removed)
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
534
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
535 feedback.Info("Building took %v", time.Since(start))
3650
01ce3ba9b0d0 Fixed generating of random points.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3646
diff changeset
536
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
537 numPoints := int(math.Ceil(polygonArea * pointsPerSquareMeter))
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
538
4763
d786c37b02c0 Fixed a formatting mistake.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4762
diff changeset
539 feedback.Info("Generate %d random points for an average density of ~%.2f points/m².",
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
540 numPoints, pointsPerSquareMeter)
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
541
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
542 start = time.Now()
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
543
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
544 generated := make(octree.LineStringZ, 0, numPoints+clippingPolygon.NumVertices(0))
3650
01ce3ba9b0d0 Fixed generating of random points.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3646
diff changeset
545
4654
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
546 octree.GenerateRandomVertices(
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
547 numPoints,
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
548 tin.Min, tin.Max,
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
549 virtual.Value,
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
550 func(vertices []octree.Vertex) {
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
551 generated = append(generated, vertices...)
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
552 })
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
553
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
554 feedback.Info("Generating %d points took %v.", len(generated), time.Since(start))
3591
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
555
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
556 // Add the boundary to new point cloud.
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
557 dupes := map[[2]float64]struct{}{}
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
558 clippingPolygon.Vertices(0, func(x, y float64) {
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
559 key := [2]float64{x, y}
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
560 if _, found := dupes[key]; found {
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
561 return
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
562 }
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
563 dupes[key] = struct{}{}
4654
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
564 if z, ok := virtual.Value(x, y); ok {
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
565 generated = append(generated, octree.Vertex{X: x, Y: y, Z: z})
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
566 }
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
567 })
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
568
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
569 feedback.Info("Triangulate new point cloud.")
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
570 xyz = octree.MultiPointZ(generated)
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
571 start = time.Now()
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
572
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
573 tri, err = octree.Triangulate(xyz)
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
574 if err != nil {
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
575 return nil, err
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
576 }
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
577 feedback.Info("Second triangulation took %v.", time.Since(start))
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
578 feedback.Info("Number triangles: %d.", len(tri.Triangles)/3)
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
579 feedback.Info("Clipping triangles from new mesh.")
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
580 }
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
581
4751
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
582 multibeam:
dbf07d0c364e SR import: If the point density is greater than 0.2 values per meter² the interpolating step is omitted.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4734
diff changeset
583
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
584 start = time.Now()
3742
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
585 tin = tri.Tin()
879c297c47e9 SR import: Use home-brew concave hull algorithm in multi-beam scan case, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3733
diff changeset
586 tin.EPSG = epsg
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
587
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
588 var str octree.STRTree
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
589 str.Build(tin)
4660
6eab3ac0e849 Speak of meshes instead od octrees as they are not longer used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4654
diff changeset
590 feedback.Info("Building clipping index took %v", time.Since(start))
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
591
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
592 start = time.Now()
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
593
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
594 clippingPolygonBuffered.Indexify()
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
595 removed = str.Clip(&clippingPolygonBuffered)
4660
6eab3ac0e849 Speak of meshes instead od octrees as they are not longer used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4654
diff changeset
596 feedback.Info("Clipping took %v.", time.Since(start))
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
597 feedback.Info("Number of triangles to clip %d.", len(removed))
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
598
4646
89a72e0e2f9b Add a method to serialize an STRTree.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4640
diff changeset
599 start = time.Now()
4654
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
600 final := octree.STRTree{Entries: 16}
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
601 final.BuildWithout(tin, removed)
4646
89a72e0e2f9b Add a method to serialize an STRTree.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4640
diff changeset
602
4660
6eab3ac0e849 Speak of meshes instead od octrees as they are not longer used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4654
diff changeset
603 feedback.Info("Building final mesh took %v.", time.Since(start))
6eab3ac0e849 Speak of meshes instead od octrees as they are not longer used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4654
diff changeset
604 feedback.Info("Store mesh.")
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
605
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
606 start = time.Now()
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
607
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
608 var (
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
609 id int64
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
610 dummy uint32
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
611 lat, lon float64
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
612 )
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
613
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
614 var hull []byte
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
615
3664
58508f50d192 SR import: Made error messages symmetrical to multi beam if no matching bottleneck is found for single beam imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3659
diff changeset
616 err = tx.QueryRowContext(
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
617 ctx,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
618 insertHullSQL,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
619 m.Bottleneck,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
620 m.Date.Time,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
621 m.DepthReference,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
622 nil,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
623 clippingPolygonWKB,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
624 epsg,
4629
7acb03e8a737 SR import: Store in sounding_result table if import is single or multi beam.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4578
diff changeset
625 sr.surtype(),
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
626 ).Scan(
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
627 &id,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
628 &lat,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
629 &lon,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
630 &dummy,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
631 &hull,
3664
58508f50d192 SR import: Made error messages symmetrical to multi beam if no matching bottleneck is found for single beam imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3659
diff changeset
632 )
58508f50d192 SR import: Made error messages symmetrical to multi beam if no matching bottleneck is found for single beam imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3659
diff changeset
633
58508f50d192 SR import: Made error messages symmetrical to multi beam if no matching bottleneck is found for single beam imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3659
diff changeset
634 switch {
58508f50d192 SR import: Made error messages symmetrical to multi beam if no matching bottleneck is found for single beam imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3659
diff changeset
635 case err == sql.ErrNoRows:
58508f50d192 SR import: Made error messages symmetrical to multi beam if no matching bottleneck is found for single beam imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3659
diff changeset
636 return nil, fmt.Errorf(
4177
8b75ac5e243e Made 'staticcheck' happy with pgxutils package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4171
diff changeset
637 "no matching bottleneck of given name or time available: %v", err)
3664
58508f50d192 SR import: Made error messages symmetrical to multi beam if no matching bottleneck is found for single beam imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3659
diff changeset
638 case err != nil:
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
639 return nil, err
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
640 }
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
641
4654
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
642 index, err := final.Bytes()
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
643 if err != nil {
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
644 return nil, err
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
645 }
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
646
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
647 h := sha1.New()
4654
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
648 h.Write(index)
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
649 checksum := hex.EncodeToString(h.Sum(nil))
4660
6eab3ac0e849 Speak of meshes instead od octrees as they are not longer used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4654
diff changeset
650 _, err = tx.ExecContext(ctx, insertMeshSQL, id, checksum, index)
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
651 if err != nil {
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
652 return nil, err
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
653 }
4660
6eab3ac0e849 Speak of meshes instead od octrees as they are not longer used.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4654
diff changeset
654 feedback.Info("Storing mesh index took %s.", time.Since(start))
4654
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
655 err = generateIsos(ctx, tx, feedback, &final, id)
3658
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
656 if err != nil {
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
657 return nil, err
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
658 }
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
659
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
660 // Store for potential later removal.
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
661 if err = track(ctx, tx, importID, "waterway.sounding_results", id); err != nil {
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
662 return nil, err
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
663 }
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
664
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
665 summary := struct {
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
666 Bottleneck string `json:"bottleneck"`
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
667 Date models.Date `json:"date"`
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
668 Lat float64 `json:"lat"`
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
669 Lon float64 `json:"lon"`
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
670 }{
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
671 Bottleneck: m.Bottleneck,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
672 Date: m.Date,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
673 Lat: lat,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
674 Lon: lon,
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
675 }
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
676
1c3df921361d Handle th case that a boundary polygon is uploaded along side with the single beam scan.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3655
diff changeset
677 return &summary, nil
3587
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
678 }
a9d140c7db8d SR import: Prepare the separate code paths for single and multibeam scans.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3586
diff changeset
679
1498
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
680 // CleanUp removes the folder containing the ZIP file with the
5df748916fcf Finished API docs for sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1496
diff changeset
681 // the sounding result import.
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
682 func (sr *SoundingResult) CleanUp() error {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
683 return os.RemoveAll(sr.Dir)
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
684 }
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
685
1292
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1239
diff changeset
686 func (sr *SoundingResult) completeOverride() bool {
1384
8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1338
diff changeset
687 // sr.EPSG == nil -> WGS84
3586
b9adb0ea4c41 SR import: a complete parameter overwrite now includes single beam flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3585
diff changeset
688 return sr.Bottleneck != nil &&
b9adb0ea4c41 SR import: a complete parameter overwrite now includes single beam flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3585
diff changeset
689 sr.Date != nil &&
b9adb0ea4c41 SR import: a complete parameter overwrite now includes single beam flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3585
diff changeset
690 sr.DepthReference != nil &&
3946
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
691 sr.SingleBeam != nil &&
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
692 sr.NegateZ != nil
1292
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1239
diff changeset
693 }
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1239
diff changeset
694
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
695 func (sr *SoundingResult) loadMeta(f *zip.File) (*models.SoundingResultMeta, error) {
1292
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1239
diff changeset
696 if f == nil {
1384
8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1338
diff changeset
697 var epsg uint
8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1338
diff changeset
698 if sr.EPSG != nil {
8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1338
diff changeset
699 epsg = *sr.EPSG
8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1338
diff changeset
700 } else {
8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1338
diff changeset
701 epsg = models.WGS84
8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1338
diff changeset
702 }
1292
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1239
diff changeset
703 return &models.SoundingResultMeta{
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1239
diff changeset
704 Date: *sr.Date,
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1239
diff changeset
705 Bottleneck: *sr.Bottleneck,
1384
8d59457a5651 Sounding result import: Consider parameter override as complete even if there is no EPSG code. Use WGS84 in this case as a default.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1338
diff changeset
706 EPSG: epsg,
1292
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1239
diff changeset
707 DepthReference: *sr.DepthReference,
3946
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
708 SingleBeam: sr.singleBeam(),
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
709 NegateZ: sr.negateZ(),
1292
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1239
diff changeset
710 }, nil
5aeda02c51b9 Sounding result import: Don't fail if an import ZIP does not contain an meta.json but has a full set of overrides.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1239
diff changeset
711 }
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
712 r, err := f.Open()
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
713 if err != nil {
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
714 return nil, err
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
715 }
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
716 defer r.Close()
1224
bc4b642c8d04 Started with an upload sounding result to temp upload area.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1193
diff changeset
717 var m models.SoundingResultMeta
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
718 if err := m.Decode(r); err != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
719 return nil, err
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
720 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
721
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
722 // Apply overrides
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
723 if sr.Date != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
724 m.Date = *sr.Date
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
725 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
726 if sr.Bottleneck != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
727 m.Bottleneck = *sr.Bottleneck
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
728 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
729 if sr.EPSG != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
730 m.EPSG = *sr.EPSG
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
731 }
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
732 if sr.DepthReference != nil {
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
733 m.DepthReference = *sr.DepthReference
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
734 }
3585
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3518
diff changeset
735 if sr.SingleBeam != nil {
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3518
diff changeset
736 m.SingleBeam = *sr.SingleBeam
479da494bc09 SR import: access beam-type parameter supplied by client. TODO: Implement single beam code path.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3518
diff changeset
737 }
3946
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
738 if sr.NegateZ != nil {
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
739 m.NegateZ = *sr.NegateZ
d668742c8978 SR import: Made `negate-z` flag more symmetrical to `single-beam` flag.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3945
diff changeset
740 }
1239
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
741
d842d9d10872 Sounding result import: Added the feature to override bottleneck, EPSG, depth reference and date in meta.json by POST arguments.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1225
diff changeset
742 return &m, nil
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
743 }
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
744
3487
fdb0439850d5 Generalized transformation of vertices during sounding result import a bit.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3479
diff changeset
745 type vertexTransform func(octree.Vertex) octree.Vertex
fdb0439850d5 Generalized transformation of vertices during sounding result import a bit.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3479
diff changeset
746
3945
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
747 func identityTransform(v octree.Vertex) octree.Vertex { return v }
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
748
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
749 func negateZTransform(v octree.Vertex) octree.Vertex {
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
750 return octree.Vertex{X: v.X, Y: v.Y, Z: -v.Z}
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
751 }
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
752
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
753 func chainTransforms(a, b vertexTransform) vertexTransform {
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
754 return func(v octree.Vertex) octree.Vertex { return b(a(v)) }
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
755 }
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
756
3487
fdb0439850d5 Generalized transformation of vertices during sounding result import a bit.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3479
diff changeset
757 func loadXYZReader(r io.Reader, feedback Feedback, xform vertexTransform) (octree.MultiPointZ, error) {
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
758 mpz := make(octree.MultiPointZ, 0, 250000)
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
759 s := bufio.NewScanner(r)
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
760
4171
5a650cde0574 Move warning limiter to common package to flesh up the misc package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4055
diff changeset
761 warnLimiter := common.WarningLimiter{Log: feedback.Warn, MaxWarnings: 100}
3504
6e748f31777a AGM/SR import. Limit logging of warnings to 100.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3487
diff changeset
762 warn := warnLimiter.Warn
6e748f31777a AGM/SR import. Limit logging of warnings to 100.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3487
diff changeset
763 defer warnLimiter.Close()
2963
27ffd94afcb5 SR import: Limit number of warnings in parsing XYZ files to 100.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2873
diff changeset
764
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
765 for line := 1; s.Scan(); line++ {
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
766 text := s.Text()
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
767 var p octree.Vertex
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
768 // fmt.Sscanf(text, "%f,%f,%f") is 4 times slower.
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
769 idx := strings.IndexByte(text, ',')
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
770 if idx == -1 {
2963
27ffd94afcb5 SR import: Limit number of warnings in parsing XYZ files to 100.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2873
diff changeset
771 warn("format error in line %d", line)
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
772 continue
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
773 }
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
774 var err error
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
775 if p.X, err = strconv.ParseFloat(text[:idx], 64); err != nil {
2963
27ffd94afcb5 SR import: Limit number of warnings in parsing XYZ files to 100.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2873
diff changeset
776 warn("format error in line %d: %v", line, err)
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
777 continue
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
778 }
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
779 text = text[idx+1:]
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
780 if idx = strings.IndexByte(text, ','); idx == -1 {
1138
443fc80a315f Don't issue new lines at end of log messages when importing.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1137
diff changeset
781 feedback.Warn("format error in line %d", line)
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
782 continue
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
783 }
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
784 if p.Y, err = strconv.ParseFloat(text[:idx], 64); err != nil {
2963
27ffd94afcb5 SR import: Limit number of warnings in parsing XYZ files to 100.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2873
diff changeset
785 warn("format error in line %d: %v", line, err)
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
786 continue
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
787 }
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
788 text = text[idx+1:]
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
789 if p.Z, err = strconv.ParseFloat(text, 64); err != nil {
2963
27ffd94afcb5 SR import: Limit number of warnings in parsing XYZ files to 100.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2873
diff changeset
790 warn("format error in line %d: %v", line, err)
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
791 continue
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
792 }
3945
3bdbaf1b282a SR import: Negate the Z values in the XYZ files if the flag `negate-z` is set.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3904
diff changeset
793 p = xform(p)
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
794 mpz = append(mpz, p)
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
795 }
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
796
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
797 if err := s.Err(); err != nil {
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
798 return nil, err
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
799 }
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
800
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
801 return mpz, nil
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
802 }
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
803
3487
fdb0439850d5 Generalized transformation of vertices during sounding result import a bit.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3479
diff changeset
804 func loadXYZ(f *zip.File, feedback Feedback, xform vertexTransform) (octree.MultiPointZ, error) {
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
805 r, err := f.Open()
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
806 if err != nil {
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
807 return nil, err
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
808 }
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
809 defer r.Close()
3487
fdb0439850d5 Generalized transformation of vertices during sounding result import a bit.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3479
diff changeset
810 return loadXYZReader(r, feedback, xform)
959
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
811 }
6ab012d0f0c2 Started with writing an importer job for sounding results.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
812
3748
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
813 func loadXYZFile(f string, feedback Feedback, xform vertexTransform) (octree.MultiPointZ, error) {
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
814 r, err := os.Open(f)
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
815 if err != nil {
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
816 return nil, err
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
817 }
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
818 defer r.Close()
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
819 return loadXYZReader(r, feedback, xform)
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
820 }
4bb5dfa0b7e3 SR import: Accept TXT file for uploads.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3742
diff changeset
821
1786
09349ca27dd7 Imports: Removed duplicated code path to store WKB polygons into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1754
diff changeset
822 func loadBoundary(z *zip.ReadCloser) (polygonSlice, error) {
1225
4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1224
diff changeset
823 shpF := common.FindInZIP(z, ".shp")
960
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
824 if shpF == nil {
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
825 return nil, nil
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
826 }
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
827 prefix := strings.TrimSuffix(shpF.Name, path.Ext(shpF.Name))
1225
4d7c44f7044e Factored out som zip lookup code to be reusable in sounding result upload controller.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1224
diff changeset
828 dbfF := common.FindInZIP(z, prefix+".dbf")
960
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
829 if dbfF == nil {
4177
8b75ac5e243e Made 'staticcheck' happy with pgxutils package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4171
diff changeset
830 return nil, fmt.Errorf("no DBF file found for %s", shpF.Name)
960
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
831 }
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
832
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
833 shpR, err := shpF.Open()
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
834 if err != nil {
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
835 return nil, err
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
836 }
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
837
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
838 dbfR, err := dbfF.Open()
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
839 if err != nil {
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
840 shpR.Close()
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
841 return nil, err
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
842 }
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
843 sr := shp.SequentialReaderFromExt(shpR, dbfR)
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
844 defer sr.Close()
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
845
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
846 if !sr.Next() {
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
847 return nil, sr.Err()
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
848 }
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
849
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
850 _, s := sr.Shape()
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
851 if s == nil {
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
852 return nil, sr.Err()
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
853 }
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
854
972
17a03a84b0e8 Split out polygon code out of sounding result importer source file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 971
diff changeset
855 return shapeToPolygon(s)
960
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
856 }
e23ae2c83427 Load boundary polygon of sounding result from ZIP file.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 959
diff changeset
857
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
858 func generateIsos(
3591
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
859 ctx context.Context,
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
860 tx *sql.Tx,
3882
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
861 feedback Feedback,
4654
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
862 tree *octree.STRTree,
3591
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
863 id int64,
062dc9b54b86 SR import: Using context.Context aware SQL statements.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3587
diff changeset
864 ) error {
977
4a2ca0e20006 Fixed build error.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 976
diff changeset
865
3882
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
866 heights, err := octree.LoadClassBreaks(
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
867 ctx, tx,
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
868 "morphology_classbreaks",
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
869 )
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
870
4654
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
871 minZ, maxZ := tree.Min().Z, tree.Max().Z
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
872
3882
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
873 if err != nil {
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
874 feedback.Warn("Loading class breaks failed: %v", err)
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
875 feedback.Info("Using default class breaks")
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
876 heights = nil
4654
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
877 h := contourStepWidth * math.Ceil(minZ/contourStepWidth)
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
878 for ; h <= maxZ; h += contourStepWidth {
3882
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
879 heights = append(heights, h)
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
880 }
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
881 } else {
4654
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
882 heights = octree.ExtrapolateClassBreaks(heights, minZ, maxZ)
1184
064d44ccc6f2 Adjust contour lines heights to multiples of step width
Tom Gottfried <tom@intevation.de>
parents: 1168
diff changeset
883 }
064d44ccc6f2 Adjust contour lines heights to multiples of step width
Tom Gottfried <tom@intevation.de>
parents: 1168
diff changeset
884
3882
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
885 /*
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
886 for i, v := range heights {
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
887 fmt.Printf("%d %.2f\n", i, v)
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
888 }
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
889 log.Printf("%.2f - %.2f\n", tree.Min.Z, tree.Max.Z)
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
890 */
37d5c4441c70 Use custom morpho class breaks in sounding result contour generation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3762
diff changeset
891
4256
4ca884dfc470 Prevent heights duplicates in iso line calculations.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4214
diff changeset
892 heights = common.DedupFloat64s(heights)
4ca884dfc470 Prevent heights duplicates in iso line calculations.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4214
diff changeset
893
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
894 return generateIsoAreas(ctx, tx, feedback, tree, heights, id)
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
895 }
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
896
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
897 func generateIsoAreas(
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
898 ctx context.Context,
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
899 tx *sql.Tx,
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
900 feedback Feedback,
4654
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
901 tree *octree.STRTree,
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
902 heights []float64,
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
903 id int64,
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
904 ) error {
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
905 feedback.Info("Generate iso areas")
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
906 total := time.Now()
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
907 defer func() {
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
908 feedback.Info("Generating iso areas took %s.",
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
909 time.Since(total))
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
910 }()
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
911
4768
a2f16bbcc846 Morph differences: Directly raster A and subtract B as a raster.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4763
diff changeset
912 box := octree.Box2D{
a2f16bbcc846 Morph differences: Directly raster A and subtract B as a raster.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4763
diff changeset
913 X1: tree.Min().X,
a2f16bbcc846 Morph differences: Directly raster A and subtract B as a raster.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4763
diff changeset
914 Y1: tree.Min().Y,
a2f16bbcc846 Morph differences: Directly raster A and subtract B as a raster.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4763
diff changeset
915 X2: tree.Max().X,
a2f16bbcc846 Morph differences: Directly raster A and subtract B as a raster.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4763
diff changeset
916 Y2: tree.Max().Y,
a2f16bbcc846 Morph differences: Directly raster A and subtract B as a raster.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4763
diff changeset
917 }
a2f16bbcc846 Morph differences: Directly raster A and subtract B as a raster.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4763
diff changeset
918
a2f16bbcc846 Morph differences: Directly raster A and subtract B as a raster.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4763
diff changeset
919 raster := octree.NewRaster(box, isoCellSize)
a2f16bbcc846 Morph differences: Directly raster A and subtract B as a raster.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4763
diff changeset
920 raster.Rasterize(tree.Value)
a2f16bbcc846 Morph differences: Directly raster A and subtract B as a raster.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4763
diff changeset
921 areas := raster.Trace(heights)
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
922
4563
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
923 return storeAreas(
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
924 ctx, tx, feedback,
4654
3eda5a7215ab Generate STRTrees instaead of Octree during sounding result imports.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4646
diff changeset
925 areas, tree.EPSG(), heights, id)
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
926 }
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
927
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
928 func storeAreas(
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
929 ctx context.Context,
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
930 tx *sql.Tx,
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
931 feedback Feedback,
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
932 areas []wkb.MultiPolygonGeom,
4563
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
933 epsg uint32,
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
934 heights []float64,
4563
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
935 id int64,
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
936 ) error {
4564
6b107d4e6810 Add more debug output to trace problem why reprojected iso areas are not in expected bounds.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4563
diff changeset
937 feedback.Info("Store iso areas.")
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
938 total := time.Now()
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
939 defer func() {
4564
6b107d4e6810 Add more debug output to trace problem why reprojected iso areas are not in expected bounds.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4563
diff changeset
940 feedback.Info("Storing iso areas took %v.",
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
941 time.Since(total))
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
942 }()
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
943
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
944 stmt, err := tx.PrepareContext(ctx, insertIsoAreasSQL)
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
945 if err != nil {
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
946 return err
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
947 }
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
948 defer stmt.Close()
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
949
4637
0ed1489af8b6 Removed ST_MakeValid from inserting sounding results and differences ... in hope that they are not broken.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4629
diff changeset
950 var size int
0ed1489af8b6 Removed ST_MakeValid from inserting sounding results and differences ... in hope that they are not broken.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4629
diff changeset
951
4563
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
952 for i, a := range areas {
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
953 if len(a) == 0 {
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
954 continue
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
955 }
4637
0ed1489af8b6 Removed ST_MakeValid from inserting sounding results and differences ... in hope that they are not broken.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4629
diff changeset
956 wkb := a.AsWKB()
0ed1489af8b6 Removed ST_MakeValid from inserting sounding results and differences ... in hope that they are not broken.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4629
diff changeset
957 size += len(wkb)
4563
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
958 if _, err := stmt.ExecContext(
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
959 ctx,
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
960 id, heights[i], epsg,
4637
0ed1489af8b6 Removed ST_MakeValid from inserting sounding results and differences ... in hope that they are not broken.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4629
diff changeset
961 wkb,
4563
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
962 contourTolerance,
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
963 ); err != nil {
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
964 return err
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
965 }
a4042ab02e05 Store geometries in database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4562
diff changeset
966 }
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
967
4639
4380f5483c03 Log the size of the transferred WKB to database for differences, too.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4637
diff changeset
968 feedback.Info("Transferred WKB size: %.2fMB.",
4637
0ed1489af8b6 Removed ST_MakeValid from inserting sounding results and differences ... in hope that they are not broken.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4629
diff changeset
969 float64(size)/(1024*1024))
0ed1489af8b6 Removed ST_MakeValid from inserting sounding results and differences ... in hope that they are not broken.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4629
diff changeset
970
4562
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
971 return nil
5cc4042cf07c Started with integrating iso area generation into gemma server. WIP
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 4256
diff changeset
972 }