gemma: cmd/octreediff/main.go annotate

annotate cmd/octreediff/main.go @ 2513:1534df518201 octree-diff

Called STT tree building.

author	Sascha L. Teichmann <sascha.teichmann@intevation.de>
date	Tue, 05 Mar 2019 15:00:42 +0100
parents	e0a7571ac13a
children	1ec4c5633eb6

rev	line source
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	1 // This is Free Software under GNU Affero General Public License v >= 3.0
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	2 // without warranty, see README.md and license for details.
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	3 //
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	4 // SPDX-License-Identifier: AGPL-3.0-or-later
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	5 // License-Filename: LICENSES/AGPL-3.0.txt
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	6 //
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	7 // Copyright (C) 2018 by via donau
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	8 // – Österreichische Wasserstraßen-Gesellschaft mbH
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	9 // Software engineering by Intevation GmbH
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	10 //
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	11 // Author(s):
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	13
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14 package main
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16 import (
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	17 "context"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	18 "database/sql"
2468 27cc04bbda46 Bail out if the EPSG codes of the datasets don't match. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2467 diff changeset	19 "errors"
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	20 "flag"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	21 "fmt"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	22 "log"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	23 "runtime"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	24 "sync"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	25 "time"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	26
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	27 "gemma.intevation.de/gemma/pkg/common"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	28 "gemma.intevation.de/gemma/pkg/octree"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	29 )
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	30
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	31 var (
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	32 bottleneck = flag.String("bottleneck", "", "name of the bottleneck")
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	33 first = flag.String("first", "", "date of the first sounding result")
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	34 second = flag.String("second", "", "date of the second sounding result")
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	35 )
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	36
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	37 const contourTolerance = 0.1
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	38
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	39 const (
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	40 loadOctreeSQL = `
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	41 SELECT sr.octree_index
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	42 FROM waterway.sounding_results sr JOIN waterway.bottlenecks bn
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	43 ON sr.bottleneck_id = bn.id
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	44 WHERE bn.bottleneck_id = $1 AND sr.date_info = $2::date
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	45 AND sr.octree_index IS NOT NULL`
2473 19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	46
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	47 insertContourSQL = `
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	48 INSERT INTO diff_contour_lines (
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	49 height,
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	50 lines
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	51 )
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	52 SELECT
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	53 $4,
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	54 ST_Transform(
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	55 ST_Multi(
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	56 ST_CollectionExtract(
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	57 ST_SimplifyPreserveTopology(
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	58 ST_Multi(ST_Collectionextract(
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	59 ST_MakeValid(ST_GeomFromWKB($1, $2::integer)), 2)),
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	60 $3
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	61 ),
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	62 2
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	63 )
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	64 ),
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	65 4326
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	66 )
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	67 `
2488 cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	68 clippingPolygonSQL = `
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	69 WITH joined AS (
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	70 SELECT
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	71 sr.area AS area,
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	72 sr.date_info AS date_info
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	73 FROM waterway.sounding_results sr JOIN
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	74 waterway.bottlenecks bn ON sr.bottleneck_id = bn.id
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	75 WHERE bn.bottleneck_id = $1
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	76 )
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	77 SELECT ST_AsBinary(
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	78 ST_intersection(
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	79 (SELECT ST_Transform(area::geometry, $2::int) FROM joined WHERE date_info = $3::date),
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	80 (SELECT ST_Transform(area::geometry, $2::int) FROM joined WHERE date_info = $4::date)
2504 e0a7571ac13a Eliminate bad triangles. Not really the right solution. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2496 diff changeset	81 )
e0a7571ac13a Eliminate bad triangles. Not really the right solution. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2496 diff changeset	82 ) AS area
2488 cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	83 `
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	84 insertContourSQLClipped = `
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	85 WITH joined AS (
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	86 SELECT
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	87 sr.area AS area,
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	88 sr.date_info AS date_info
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	89 FROM waterway.sounding_results sr JOIN
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	90 waterway.bottlenecks bn ON sr.bottleneck_id = bn.id
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	91 WHERE bn.bottleneck_id = $1
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	92 ),
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	93 inter AS (
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	94 SELECT
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	95 ST_Buffer(
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	96 ST_intersection(
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	97 (SELECT ST_Transform(area::geometry, $2::int) FROM joined WHERE date_info = $3::date),
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	98 (SELECT ST_Transform(area::geometry, $2::int) FROM joined WHERE date_info = $4::date)
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	99 ),
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	100 0.001) AS area
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	101 )
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	102 INSERT INTO diff_contour_lines (
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	103 height,
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	104 lines
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	105 )
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	106 SELECT
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	107 $5,
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	108 ST_Transform(
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	109 ST_Multi(
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	110 ST_intersection(
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	111 ST_CollectionExtract(
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	112 ST_SimplifyPreserveTopology(
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	113 ST_Multi(ST_Collectionextract(
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	114 ST_MakeValid(ST_GeomFromWKB($6, $2::integer)), 2)),
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	115 $7
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	116 ),
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	117 2
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	118 ),
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	119 area
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	120 )
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	121 ),
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	122 4326
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	123 )
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	124 FROM inter`
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	125 )
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	126
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	127 func check(err error) {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	128 if err != nil {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	129 log.Fatalf("error: %v\n", err)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	130 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	131 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	132
2470 5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	133 type point struct {
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	134 x float64
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	135 y float64
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	136 }
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	137
2472 db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2470 diff changeset	138 type pointMap map[point]float64
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2470 diff changeset	139
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	140 func (pm pointMap) triangulate() (*octree.Triangulation, error) {
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	141 vertices := make([]octree.Vertex, len(pm))
2483 620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2480 diff changeset	142 var i int
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2480 diff changeset	143 for p, z := range pm {
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	144 vertices[i] = octree.Vertex{X: p.x, Y: p.y, Z: z}
2483 620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2480 diff changeset	145 i++
620038ade708 Incorporated fogleman's fast Delaunay triangulation adjuted to our vertex model. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2480 diff changeset	146 }
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	147 return octree.Triangulate(vertices)
2472 db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2470 diff changeset	148 }
db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2470 diff changeset	149
2469 1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	150 func sliceWork(
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	151 vs []octree.Vertex,
2472 db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2470 diff changeset	152 dst pointMap,
2469 1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	153 fn func([]octree.Vertex, func([]octree.Vertex) []octree.Vertex),
2470 5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	154 ) {
2469 1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	155 n := runtime.NumCPU()
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	156
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	157 wg := new(sync.WaitGroup)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	158
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	159 slices := make(chan []octree.Vertex)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	160 out := make(chan []octree.Vertex)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	161
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	162 pool := make(chan []octree.Vertex, n)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	163
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	164 const pageSize = 2048
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	165
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	166 turn := func(p []octree.Vertex) []octree.Vertex {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	167 if p != nil {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	168 out <- p
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	169 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	170 select {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	171 case p = <-pool:
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	172 default:
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	173 p = make([]octree.Vertex, 0, pageSize)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	174 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	175 return p
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	176 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	177
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	178 for i := 0; i < n; i++ {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	179 wg.Add(1)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	180 go func() {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	181 defer wg.Done()
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	182 for slice := range slices {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	183 fn(slice, turn)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	184 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	185 }()
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	186 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	187 done := make(chan struct{})
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	188 go func() {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	189 defer close(done)
2470 5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	190 for s := range out {
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	191 for i := range s {
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	192 v := &s[i]
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	193 key := point{x: v.X, y: v.Y}
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	194 if z, found := dst[key]; found {
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	195 dst[key] = (z + v.Z) * 0.5
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	196 } else {
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	197 dst[key] = v.Z
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	198 }
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	199 }
2469 1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	200 select {
2470 5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	201 case pool <- s[:0:pageSize]:
2469 1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	202 default:
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	203 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	204 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	205 }()
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	206
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	207 size := len(vs)/n + 1
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	208 for len(vs) > 0 {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	209 var l int
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	210 if len(vs) < size {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	211 l = len(vs)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	212 } else {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	213 l = size
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	214 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	215 slices <- vs[:l]
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	216 vs = vs[l:]
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	217 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	218 close(slices)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	219 wg.Wait()
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	220 close(out)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	221 <-done
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	222 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	223
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	224 func process(bottleneck string, firstDate, secondDate time.Time) error {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	225 start := time.Now()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	226 defer func() { log.Printf("processing took %v\n", time.Since(start)) }()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	227
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	228 ctx := context.Background()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	229
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	230 return run(func(db *sql.DB) error {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	231
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	232 conn, err := db.Conn(ctx)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	233 if err != nil {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	234 return err
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	235 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	236 defer conn.Close()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	237
2480 242104c338ff Wrap writing of iso-lines of differences into a transaction. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2479 diff changeset	238 tx, err := conn.BeginTx(ctx, nil)
242104c338ff Wrap writing of iso-lines of differences into a transaction. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2479 diff changeset	239 if err != nil {
242104c338ff Wrap writing of iso-lines of differences into a transaction. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2479 diff changeset	240 return err
242104c338ff Wrap writing of iso-lines of differences into a transaction. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2479 diff changeset	241 }
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	242 defer tx.Rollback()
2480 242104c338ff Wrap writing of iso-lines of differences into a transaction. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2479 diff changeset	243
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	244 type load struct {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	245 date time.Time
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	246 data []byte
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	247 err *error
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	248 dst **octree.Tree
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	249 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	250
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	251 out := make(chan *load)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	252 wg := new(sync.WaitGroup)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	253
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	254 n := runtime.NumCPU()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	255 if n > 2 {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	256 n = 2
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	257 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	258
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	259 for i := 0; i < n; i++ {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	260 wg.Add(1)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	261 go func() {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	262 defer wg.Done()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	263 for l := range out {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	264 if *l.err == nil {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	265 l.dst, l.err = octree.Deserialize(l.data)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	266 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	267 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	268 }()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	269 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	270
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	271 var firstErr, secondErr error
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	272 var first, second *octree.Tree
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	273
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	274 for _, l := range []*load{
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	275 {date: firstDate, dst: &first, err: &firstErr},
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	276 {date: secondDate, dst: &second, err: &secondErr},
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	277 } {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	278 var data []byte
2480 242104c338ff Wrap writing of iso-lines of differences into a transaction. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2479 diff changeset	279 if err := tx.QueryRowContext(
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	280 ctx,
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	281 loadOctreeSQL,
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	282 bottleneck,
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	283 l.date,
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	284 ).Scan(&data); err != nil {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	285 *l.err = err
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	286 } else {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	287 l.data = data
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	288 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	289 out <- l
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	290 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	291 close(out)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	292
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	293 wg.Wait()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	294
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	295 if firstErr != nil \|\| secondErr != nil {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	296 if firstErr != nil && secondErr != nil {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	297 return fmt.Errorf("%v, %v", firstErr, secondErr)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	298 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	299 if firstErr != nil {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	300 return firstErr
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	301 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	302 return secondErr
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	303 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	304
2468 27cc04bbda46 Bail out if the EPSG codes of the datasets don't match. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2467 diff changeset	305 if first.EPSG != second.EPSG {
27cc04bbda46 Bail out if the EPSG codes of the datasets don't match. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2467 diff changeset	306 return errors.New("EPSG codes mismatch. Needs transformation slow pass.")
27cc04bbda46 Bail out if the EPSG codes of the datasets don't match. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2467 diff changeset	307 }
27cc04bbda46 Bail out if the EPSG codes of the datasets don't match. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2467 diff changeset	308
2470 5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	309 now := time.Now()
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	310 log.Printf("loading took %v\n", now.Sub(start))
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	311 last := now
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	312
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	313 firstVs, secondVs := first.Vertices(), second.Vertices()
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	314
2472 db0e4ab57977 Turn new point cloud into WKB form of MultiPointZ. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2470 diff changeset	315 result := make(pointMap, len(firstVs)+len(secondVs))
2466 a1e751c08c56 Calculate difference on single core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2465 diff changeset	316
2470 5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	317 sliceWork(
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	318 firstVs,
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	319 result,
2469 1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	320 func(
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	321 slice []octree.Vertex,
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	322 turn func([]octree.Vertex) []octree.Vertex,
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	323 ) {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	324 p := turn(nil)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	325 for i := range slice {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	326 v := &slice[i]
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	327 if z, found := second.Value(v.X, v.Y); found {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	328 p = append(p, octree.Vertex{v.X, v.Y, v.Z - z})
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	329 if len(p) == cap(p) {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	330 p = turn(p)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	331 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	332 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	333 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	334 if len(p) > 0 {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	335 turn(p)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	336 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	337 })
2466 a1e751c08c56 Calculate difference on single core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2465 diff changeset	338
2470 5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	339 sliceWork(
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	340 secondVs,
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	341 result,
2469 1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	342 func(
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	343 slice []octree.Vertex,
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	344 turn func([]octree.Vertex) []octree.Vertex,
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	345 ) {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	346 p := turn(nil)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	347 for i := range slice {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	348 v := &slice[i]
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	349 if z, found := first.Value(v.X, v.Y); found {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	350 p = append(p, octree.Vertex{v.X, v.Y, z - v.Z})
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	351 if len(p) == cap(p) {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	352 p = turn(p)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	353 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	354 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	355 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	356 if len(p) > 0 {
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	357 turn(p)
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	358 }
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	359 })
1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	360
2470 5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	361 now = time.Now()
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	362 log.Printf("setting in took %v\n", now.Sub(last))
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	363 last = now
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	364 log.Printf("num points: %d\n", len(result))
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	365
2488 cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	366 var clip []byte
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	367
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	368 if err := tx.QueryRowContext(
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	369 ctx, clippingPolygonSQL,
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	370 bottleneck,
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	371 first.EPSG,
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	372 firstDate, secondDate,
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	373 ).Scan(&clip); err != nil {
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	374 return err
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	375 }
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	376
2496 12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	377 var clippingPolygon octree.Polygon
12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	378 if err := clippingPolygon.FromWKB(clip); err != nil {
12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	379 return err
12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	380 }
12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	381 clippingPolygon.Indexify()
12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	382
2488 cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	383 now = time.Now()
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	384 log.Printf("loading clipping polygon took %v\n", now.Sub(last))
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	385 last = now
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	386
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	387 tri, err := result.triangulate()
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	388 if err != nil {
2473 19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	389 return err
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	390 }
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	391
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	392 now = time.Now()
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	393 log.Printf("triangulation took %v\n", now.Sub(last))
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	394 last = now
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	395
2513 1534df518201 Called STT tree building. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2504 diff changeset	396 var str octree.STRTree
1534df518201 Called STT tree building. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2504 diff changeset	397
1534df518201 Called STT tree building. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2504 diff changeset	398 str.Build(tri)
1534df518201 Called STT tree building. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2504 diff changeset	399
1534df518201 Called STT tree building. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2504 diff changeset	400 now = time.Now()
1534df518201 Called STT tree building. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2504 diff changeset	401 log.Printf("building STR tree took %v\n", now.Sub(last))
1534df518201 Called STT tree building. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2504 diff changeset	402 last = now
1534df518201 Called STT tree building. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2504 diff changeset	403
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	404 builder := octree.NewBuilder(tri.Tin())
2473 19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	405 builder.Build()
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	406
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	407 now = time.Now()
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	408 log.Printf("building octree took %v\n", now.Sub(last))
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	409 last = now
2476 efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	410
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	411 tree := builder.Tree()
2496 12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	412 tree.Clip(&clippingPolygon)
2504 e0a7571ac13a Eliminate bad triangles. Not really the right solution. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2496 diff changeset	413 //tree.Dump()
2496 12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	414
12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	415 now = time.Now()
12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	416 log.Printf("clipping octree took %v\n", now.Sub(last))
12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	417 last = now
2476 efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	418
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	419 log.Printf("min/max: %f %f\n", tree.Min.Z, tree.Max.Z)
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	420
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	421 var heights []float64
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	422
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	423 switch {
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	424 case tree.Min.Z >= 0: // All values positive.
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	425 for v := 0.0; v <= tree.Max.Z; v += 0.1 {
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	426 if v >= tree.Min.Z {
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	427 heights = append(heights, v)
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	428 }
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	429 }
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	430 case tree.Max.Z <= 0: // All values negative.
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	431 for v := 0.0; v >= tree.Min.Z; v -= 0.1 {
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	432 if v <= tree.Max.Z {
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	433 heights = append(heights, v)
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	434 }
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	435 }
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	436 default: // Positive and negative.
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	437 for v := 0.1; v <= tree.Max.Z; v += 0.1 {
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	438 heights = append(heights, v)
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	439 }
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	440 for i, j := 0, len(heights)-1; i < j; i, j = i+1, j-1 {
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	441 heights[i], heights[j] = heights[j], heights[i]
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	442 }
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	443 for v := 0.0; v >= tree.Min.Z; v -= 0.1 {
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	444 heights = append(heights, v)
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	445 }
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	446 }
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	447
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	448 log.Printf("num heights: %d\n", len(heights))
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	449
2476 efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	450 var dataSize int
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	451
2496 12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	452 stmt, err := tx.PrepareContext(ctx, insertContourSQL)
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	453 if err != nil {
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	454 return err
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	455 }
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	456 defer stmt.Close()
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	457
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	458 log.Println("do contours")
2476 efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	459 octree.DoContours(tree, heights, func(res *octree.ContourResult) {
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	460 if err == nil && len(res.Lines) > 0 {
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	461 log.Printf("%.2f: lines: %d\n", res.Height, len(res.Lines))
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	462 wkb := res.Lines.AsWKB2D()
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	463 dataSize += len(wkb)
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	464 _, err = stmt.ExecContext(
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	465 ctx,
2496 12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	466 wkb,
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	467 first.EPSG,
2496 12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	468 contourTolerance,
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	469 res.Height,
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	470 )
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	471 }
2476 efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	472 })
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	473
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	474 if err != nil {
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	475 return err
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	476 }
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	477
2476 efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	478 now = time.Now()
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	479 log.Printf("Number of iso lines: %d\n", len(heights))
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	480 log.Printf("Total WKB size: %.2fMB\n", float64(dataSize)/(1024*1024))
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	481 log.Printf("generating iso lines took %v\n", now.Sub(last))
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	482 last = now
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	483
2480 242104c338ff Wrap writing of iso-lines of differences into a transaction. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2479 diff changeset	484 return tx.Commit()
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	485 })
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	486 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	487
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	488 func main() {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	489
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	490 flag.Parse()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	491
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	492 firstDate, err := time.Parse(common.DateFormat, *first)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	493 check(err)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	494 secondDate, err := time.Parse(common.DateFormat, *second)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	495 check(err)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	496
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	497 if *bottleneck == "" {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	498 log.Fatalln("Missing bottleneck name")
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	499 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	500
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	501 check(process(*bottleneck, firstDate, secondDate))
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	502 }

Mercurial > gemma

annotate cmd/octreediff/main.go @ 2513:1534df518201 octree-diff