Mercurial > gemma

annotate cmd/octreediff/main.go @ 2483:620038ade708 octree-diff

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model. License: MIT Home: https://github.com/fogleman/delaunay
author Sascha L. Teichmann <sascha.teichmann@intevation.de>
date Fri, 01 Mar 2019 15:33:27 +0100
parents 242104c338ff
children 4fa92d468164
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2465
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
1 // This is Free Software under GNU Affero General Public License v >= 3.0
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2 // without warranty, see README.md and license for details.
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
3 //
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
4 // SPDX-License-Identifier: AGPL-3.0-or-later
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
5 // License-Filename: LICENSES/AGPL-3.0.txt
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
6 //
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
7 // Copyright (C) 2018 by via donau
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 // – Österreichische Wasserstraßen-Gesellschaft mbH
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 // Software engineering by Intevation GmbH
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 //
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 // Author(s):
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
13
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14 package main
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import (
2472
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
17 "bytes"
2465
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 "context"
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 "database/sql"
2472
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
20 "encoding/binary"
2468
27cc04bbda46 Bail out if the EPSG codes of the datasets don't match.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2467
diff changeset
21 "errors"
2465
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
22 "flag"
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
23 "fmt"
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
24 "log"
2472
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
25 "math"
2465
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26 "runtime"
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
27 "sync"
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
28 "time"
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
29
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
30 "gemma.intevation.de/gemma/pkg/common"
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
31 "gemma.intevation.de/gemma/pkg/octree"
2472
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
32 "gemma.intevation.de/gemma/pkg/wkb"
2465
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
33 )
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
34
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
35 var (
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
36 bottleneck = flag.String("bottleneck", "", "name of the bottleneck")
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
37 first = flag.String("first", "", "date of the first sounding result")
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
38 second = flag.String("second", "", "date of the second sounding result")
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
39 )
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
40
2479
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
41 const contourTolerance = 0.1
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
42
2465
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
43 const (
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
44 loadOctreeSQL = `
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
45 SELECT sr.octree_index
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
46 FROM waterway.sounding_results sr JOIN waterway.bottlenecks bn
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
47 ON sr.bottleneck_id = bn.id
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
48 WHERE bn.bottleneck_id = $1 AND sr.date_info = $2::date
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
49 AND sr.octree_index IS NOT NULL`
2473
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
50
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
51 triangulateSQL = `
2479
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
52 WITH joined AS (
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
53 SELECT
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
54 sr.area AS area,
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
55 sr.date_info AS date_info
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
56 FROM waterway.sounding_results sr JOIN
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
57 waterway.bottlenecks bn ON sr.bottleneck_id = bn.id
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
58 WHERE bn.bottleneck_id = $1
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
59 ),
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
60 inter AS (
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
61 SELECT
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
62 ST_Buffer(
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
63 ST_intersection(
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
64 (SELECT ST_Transform(area::geometry, $2::int) FROM joined WHERE date_info = $3::date),
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
65 (SELECT ST_Transform(area::geometry, $2::int) FROM joined WHERE date_info = $4::date)
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
66 ),
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
67 0.001) AS area
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
68 ),
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
69 triangles AS (
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
70 SELECT t.geom AS geom, ST_MakePolygon(ST_ExteriorRing(t.geom)) AS poly FROM (
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
71 SELECT (ST_Dump(
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
72 ST_DelaunayTriangles(ST_GeomFromWKB($5, $2::int), 0, 2))).geom
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
73 ) t
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
74 )
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
75 SELECT ST_AsBinary(ST_Collect(triangles.geom)) FROM triangles, inter
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
76 WHERE ST_Covers(inter.area, triangles.poly)
2473
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
77 `
2479
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
78
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
79 //INSERT INTO redis_diff_countour_lines (
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
80 //INSERT INTO diff_contour_lines (
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
81 //INSERT INTO diff_contour_lines_extern (
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
82 insertContourSQL = `
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
83 INSERT INTO diff_contour_lines (
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
84 height,
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
85 lines
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
86 )
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
87 SELECT
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
88 $1,
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
89 ST_Transform(
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
90 ST_Multi(
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
91 ST_CollectionExtract(
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
92 ST_SimplifyPreserveTopology(
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
93 ST_Multi(ST_Collectionextract(
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
94 ST_MakeValid(ST_GeomFromWKB($2, $3::integer)), 2)),
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
95 $4
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
96 ),
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
97 2
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
98 )
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
99 ),
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
100 4326
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
101 )`
2465
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
102 )
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
103
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
104 func check(err error) {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
105 if err != nil {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
106 log.Fatalf("error: %v\n", err)
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
107 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
108 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
109
2470
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
110 type point struct {
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
111 x float64
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
112 y float64
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
113 }
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
114
2472
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
115 type pointMap map[point]float64
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
116
2483
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
117 func (pm pointMap) triangulate() {
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
118 start := time.Now()
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
119 points := make([]octree.Vertex, len(pm))
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
120 var i int
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
121 for p, z := range pm {
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
122 points[i] = octree.Vertex{X: p.x, Y: p.y, Z: z}
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
123 i++
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
124 }
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
125 _, err := octree.Triangulate(points)
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
126 if err != nil {
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
127 log.Printf("triangulate error: %v\n", err)
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
128 }
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
129 log.Printf("in memory triangulation (%d points) took %s\n", i, time.Since(start))
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
130 }
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
131
2472
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
132 func (pm pointMap) asWKB() []byte {
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
133 size := 1 + 4 + 4 + len(pm)*(1+4+3*8)
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
134
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
135 buf := bytes.NewBuffer(make([]byte, 0, size))
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
136
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
137 binary.Write(buf, binary.LittleEndian, wkb.NDR)
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
138 binary.Write(buf, binary.LittleEndian, wkb.MultiPointZ)
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
139 binary.Write(buf, binary.LittleEndian, uint32(len(pm)))
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
140
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
141 perPoint := bytes.NewBuffer(make([]byte, 0, 1+4))
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
142 binary.Write(perPoint, binary.LittleEndian, wkb.NDR)
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
143 binary.Write(perPoint, binary.LittleEndian, wkb.PointZ)
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
144 hdr := perPoint.Bytes()
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
145
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
146 for p, z := range pm {
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
147 buf.Write(hdr)
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
148 binary.Write(buf, binary.LittleEndian, math.Float64bits(p.x))
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
149 binary.Write(buf, binary.LittleEndian, math.Float64bits(p.y))
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
150 binary.Write(buf, binary.LittleEndian, math.Float64bits(z))
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
151 }
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
152
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
153 return buf.Bytes()
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
154 }
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
155
2469
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
156 func sliceWork(
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
157 vs []octree.Vertex,
2472
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
158 dst pointMap,
2469
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
159 fn func([]octree.Vertex, func([]octree.Vertex) []octree.Vertex),
2470
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
160 ) {
2469
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
161 n := runtime.NumCPU()
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
162
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
163 wg := new(sync.WaitGroup)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
164
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
165 slices := make(chan []octree.Vertex)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
166 out := make(chan []octree.Vertex)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
167
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
168 pool := make(chan []octree.Vertex, n)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
169
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
170 const pageSize = 2048
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
171
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
172 turn := func(p []octree.Vertex) []octree.Vertex {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
173 if p != nil {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
174 out <- p
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
175 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
176 select {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
177 case p = <-pool:
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
178 default:
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
179 p = make([]octree.Vertex, 0, pageSize)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
180 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
181 return p
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
182 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
183
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
184 for i := 0; i < n; i++ {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
185 wg.Add(1)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
186 go func() {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
187 defer wg.Done()
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
188 for slice := range slices {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
189 fn(slice, turn)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
190 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
191 }()
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
192 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
193 done := make(chan struct{})
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
194 go func() {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
195 defer close(done)
2470
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
196 for s := range out {
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
197 for i := range s {
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
198 v := &s[i]
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
199 key := point{x: v.X, y: v.Y}
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
200 if z, found := dst[key]; found {
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
201 dst[key] = (z + v.Z) * 0.5
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
202 } else {
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
203 dst[key] = v.Z
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
204 }
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
205 }
2469
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
206 select {
2470
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
207 case pool <- s[:0:pageSize]:
2469
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
208 default:
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
209 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
210 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
211 }()
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
212
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
213 size := len(vs)/n + 1
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
214 for len(vs) > 0 {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
215 var l int
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
216 if len(vs) < size {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
217 l = len(vs)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
218 } else {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
219 l = size
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
220 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
221 slices <- vs[:l]
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
222 vs = vs[l:]
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
223 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
224 close(slices)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
225 wg.Wait()
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
226 close(out)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
227 <-done
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
228 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
229
2465
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
230 func process(bottleneck string, firstDate, secondDate time.Time) error {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
231 start := time.Now()
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
232 defer func() { log.Printf("processing took %v\n", time.Since(start)) }()
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
233
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
234 ctx := context.Background()
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
235
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
236 return run(func(db *sql.DB) error {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
237
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
238 conn, err := db.Conn(ctx)
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
239 if err != nil {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
240 return err
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
241 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
242 defer conn.Close()
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
243
2480
242104c338ff Wrap writing of iso-lines of differences into a transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2479
diff changeset
244 tx, err := conn.BeginTx(ctx, nil)
242104c338ff Wrap writing of iso-lines of differences into a transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2479
diff changeset
245 if err != nil {
242104c338ff Wrap writing of iso-lines of differences into a transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2479
diff changeset
246 return err
242104c338ff Wrap writing of iso-lines of differences into a transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2479
diff changeset
247 }
242104c338ff Wrap writing of iso-lines of differences into a transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2479
diff changeset
248
2465
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
249 type load struct {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
250 date time.Time
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
251 data []byte
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
252 err *error
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
253 dst **octree.Tree
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
254 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
255
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
256 out := make(chan *load)
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
257 wg := new(sync.WaitGroup)
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
258
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
259 n := runtime.NumCPU()
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
260 if n > 2 {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
261 n = 2
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
262 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
263
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
264 for i := 0; i < n; i++ {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
265 wg.Add(1)
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
266 go func() {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
267 defer wg.Done()
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
268 for l := range out {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
269 if *l.err == nil {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
270 *l.dst, *l.err = octree.Deserialize(l.data)
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
271 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
272 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
273 }()
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
274 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
275
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
276 var firstErr, secondErr error
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
277 var first, second *octree.Tree
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
278
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
279 for _, l := range []*load{
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
280 {date: firstDate, dst: &first, err: &firstErr},
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
281 {date: secondDate, dst: &second, err: &secondErr},
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
282 } {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
283 var data []byte
2480
242104c338ff Wrap writing of iso-lines of differences into a transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2479
diff changeset
284 if err := tx.QueryRowContext(
2465
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
285 ctx,
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
286 loadOctreeSQL,
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
287 bottleneck,
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
288 l.date,
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
289 ).Scan(&data); err != nil {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
290 *l.err = err
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
291 } else {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
292 l.data = data
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
293 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
294 out <- l
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
295 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
296 close(out)
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
297
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
298 wg.Wait()
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
299
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
300 if firstErr != nil || secondErr != nil {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
301 if firstErr != nil && secondErr != nil {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
302 return fmt.Errorf("%v, %v", firstErr, secondErr)
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
303 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
304 if firstErr != nil {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
305 return firstErr
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
306 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
307 return secondErr
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
308 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
309
2468
27cc04bbda46 Bail out if the EPSG codes of the datasets don't match.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2467
diff changeset
310 if first.EPSG != second.EPSG {
27cc04bbda46 Bail out if the EPSG codes of the datasets don't match.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2467
diff changeset
311 return errors.New("EPSG codes mismatch. Needs transformation slow pass.")
27cc04bbda46 Bail out if the EPSG codes of the datasets don't match.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2467
diff changeset
312 }
27cc04bbda46 Bail out if the EPSG codes of the datasets don't match.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2467
diff changeset
313
2470
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
314 now := time.Now()
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
315 log.Printf("loading took %v\n", now.Sub(start))
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
316 last := now
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
317
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
318 firstVs, secondVs := first.Vertices(), second.Vertices()
2465
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
319
2472
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
320 result := make(pointMap, len(firstVs)+len(secondVs))
2466
a1e751c08c56 Calculate difference on single core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
321
2470
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
322 sliceWork(
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
323 firstVs,
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
324 result,
2469
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
325 func(
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
326 slice []octree.Vertex,
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
327 turn func([]octree.Vertex) []octree.Vertex,
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
328 ) {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
329 p := turn(nil)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
330 for i := range slice {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
331 v := &slice[i]
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
332 if z, found := second.Value(v.X, v.Y); found {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
333 p = append(p, octree.Vertex{v.X, v.Y, v.Z - z})
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
334 if len(p) == cap(p) {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
335 p = turn(p)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
336 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
337 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
338 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
339 if len(p) > 0 {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
340 turn(p)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
341 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
342 })
2466
a1e751c08c56 Calculate difference on single core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2465
diff changeset
343
2470
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
344 sliceWork(
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
345 secondVs,
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
346 result,
2469
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
347 func(
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
348 slice []octree.Vertex,
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
349 turn func([]octree.Vertex) []octree.Vertex,
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
350 ) {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
351 p := turn(nil)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
352 for i := range slice {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
353 v := &slice[i]
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
354 if z, found := first.Value(v.X, v.Y); found {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
355 p = append(p, octree.Vertex{v.X, v.Y, z - v.Z})
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
356 if len(p) == cap(p) {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
357 p = turn(p)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
358 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
359 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
360 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
361 if len(p) > 0 {
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
362 turn(p)
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
363 }
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
364 })
1312953defab Do difference calculation multi-core.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2468
diff changeset
365
2470
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
366 now = time.Now()
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
367 log.Printf("setting in took %v\n", now.Sub(last))
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
368 last = now
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2469
diff changeset
369 log.Printf("num points: %d\n", len(result))
2465
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
370
2483
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
371 result.triangulate()
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2480
diff changeset
372
2472
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
373 data := result.asWKB()
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
374
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
375 now = time.Now()
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
376 log.Printf("turing into WKB took %v\n", now.Sub(last))
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
377 last = now
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
378
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
379 log.Printf("WKB size %.3fMB\n", float64(len(data))/(1024*1024))
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2470
diff changeset
380
2473
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
381 var tin octree.Tin
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
382
2480
242104c338ff Wrap writing of iso-lines of differences into a transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2479
diff changeset
383 if err := tx.QueryRowContext(
2473
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
384 ctx,
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
385 triangulateSQL,
2479
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
386 bottleneck,
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
387 first.EPSG,
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
388 firstDate,
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
389 secondDate,
2473
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
390 data,
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
391 ).Scan(&tin); err != nil {
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
392 return err
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
393 }
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
394
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
395 now = time.Now()
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
396 log.Printf("triangulation took %v\n", now.Sub(last))
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
397 last = now
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
398
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
399 builder := octree.NewBuilder(&tin)
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
400 builder.Build()
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
401
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
402 now = time.Now()
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
403 log.Printf("building octree took %v\n", now.Sub(last))
19beb7d17337 Triangulate and build octree index.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2472
diff changeset
404 last = now
2476
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
405
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
406 tree := builder.Tree()
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
407
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
408 log.Printf("min/max: %f %f\n", tree.Min.Z, tree.Max.Z)
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
409
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
410 var heights []float64
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
411
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
412 switch {
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
413 case tree.Min.Z >= 0: // All values positive.
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
414 for v := 0.0; v <= tree.Max.Z; v += 0.1 {
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
415 if v >= tree.Min.Z {
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
416 heights = append(heights, v)
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
417 }
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
418 }
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
419 case tree.Max.Z <= 0: // All values negative.
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
420 for v := 0.0; v >= tree.Min.Z; v -= 0.1 {
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
421 if v <= tree.Max.Z {
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
422 heights = append(heights, v)
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
423 }
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
424 }
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
425 default: // Positive and negative.
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
426 for v := 0.1; v <= tree.Max.Z; v += 0.1 {
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
427 heights = append(heights, v)
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
428 }
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
429 for i, j := 0, len(heights)-1; i < j; i, j = i+1, j-1 {
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
430 heights[i], heights[j] = heights[j], heights[i]
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
431 }
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
432 for v := 0.0; v >= tree.Min.Z; v -= 0.1 {
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
433 heights = append(heights, v)
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
434 }
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
435 }
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
436
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
437 var dataSize int
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
438
2480
242104c338ff Wrap writing of iso-lines of differences into a transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2479
diff changeset
439 stmt, err := tx.PrepareContext(ctx, insertContourSQL)
2479
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
440 if err != nil {
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
441 return err
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
442 }
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
443 defer stmt.Close()
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
444
2476
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
445 octree.DoContours(tree, heights, func(res *octree.ContourResult) {
2479
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
446 if err == nil && len(res.Lines) > 0 {
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
447 log.Printf("%f: lines: %d\n", res.Height, len(res.Lines))
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
448 wkb := res.Lines.AsWKB2D()
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
449 dataSize += len(wkb)
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
450 _, err = stmt.ExecContext(
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
451 ctx,
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
452 res.Height,
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
453 wkb,
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
454 first.EPSG,
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
455 contourTolerance,
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
456 )
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
457 }
2476
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
458 })
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
459
2479
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
460 if err != nil {
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
461 return err
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
462 }
c85b16db8a02 Calculate better triangulation and store it into database.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2476
diff changeset
463
2476
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
464 now = time.Now()
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
465 log.Printf("Number of iso lines: %d\n", len(heights))
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
466 log.Printf("Total WKB size: %.2fMB\n", float64(dataSize)/(1024*1024))
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
467 log.Printf("generating iso lines took %v\n", now.Sub(last))
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
468 last = now
efe332e985b9 Calculate the iso lines of the difference.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2473
diff changeset
469
2480
242104c338ff Wrap writing of iso-lines of differences into a transaction.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2479
diff changeset
470 return tx.Commit()
2465
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
471 })
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
472 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
473
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
474 func main() {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
475
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
476 flag.Parse()
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
477
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
478 firstDate, err := time.Parse(common.DateFormat, *first)
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
479 check(err)
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
480 secondDate, err := time.Parse(common.DateFormat, *second)
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
481 check(err)
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
482
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
483 if *bottleneck == "" {
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
484 log.Fatalln("Missing bottleneck name")
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
485 }
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
486
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
487 check(process(*bottleneck, firstDate, secondDate))
86c7a023400e Started experimental octree diff branch.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
488 }