gemma: cmd/octreediff/main.go annotate

annotate cmd/octreediff/main.go @ 2583:5c6a7e69b02d

Morphological differences: Store calculated iso-lines in database.

author	Sascha L. Teichmann <sascha.teichmann@intevation.de>
date	Mon, 11 Mar 2019 16:42:38 +0100
parents	647a58ee9ae9
children

rev	line source
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	1 // This is Free Software under GNU Affero General Public License v >= 3.0
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	2 // without warranty, see README.md and license for details.
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	3 //
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	4 // SPDX-License-Identifier: AGPL-3.0-or-later
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	5 // License-Filename: LICENSES/AGPL-3.0.txt
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	6 //
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	7 // Copyright (C) 2018 by via donau
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	8 // – Österreichische Wasserstraßen-Gesellschaft mbH
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	9 // Software engineering by Intevation GmbH
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	10 //
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	11 // Author(s):
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	12 // * Sascha L. Teichmann <sascha.teichmann@intevation.de>
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	13
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14 package main
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16 import (
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	17 "context"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	18 "database/sql"
2468 27cc04bbda46 Bail out if the EPSG codes of the datasets don't match. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2467 diff changeset	19 "errors"
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	20 "flag"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	21 "fmt"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	22 "log"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	23 "runtime"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	24 "sync"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	25 "time"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	26
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	27 "gemma.intevation.de/gemma/pkg/common"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	28 "gemma.intevation.de/gemma/pkg/octree"
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	29 )
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	30
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	31 var (
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	32 bottleneck = flag.String("bottleneck", "", "name of the bottleneck")
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	33 first = flag.String("first", "", "date of the first sounding result")
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	34 second = flag.String("second", "", "date of the second sounding result")
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	35 )
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	36
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	37 const contourTolerance = 0.1
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	38
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	39 const (
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	40 loadOctreeSQL = `
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	41 SELECT sr.octree_index
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	42 FROM waterway.sounding_results sr JOIN waterway.bottlenecks bn
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	43 ON sr.bottleneck_id = bn.id
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	44 WHERE bn.bottleneck_id = $1 AND sr.date_info = $2::date
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	45 AND sr.octree_index IS NOT NULL`
2473 19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	46
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	47 insertContourSQL = `
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	48 INSERT INTO diff_contour_lines (
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	49 height,
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	50 lines
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	51 )
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	52 SELECT
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	53 $4,
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	54 ST_Transform(
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	55 ST_Multi(
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	56 ST_CollectionExtract(
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	57 ST_SimplifyPreserveTopology(
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	58 ST_Multi(ST_Collectionextract(
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	59 ST_MakeValid(ST_GeomFromWKB($1, $2::integer)), 2)),
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	60 $3
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	61 ),
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	62 2
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	63 )
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	64 ),
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	65 4326
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	66 )
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	67 `
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	68 )
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	69
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	70 func check(err error) {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	71 if err != nil {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	72 log.Fatalf("error: %v\n", err)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	73 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	74 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	75
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	76 func process(bottleneck string, firstDate, secondDate time.Time) error {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	77 start := time.Now()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	78 defer func() { log.Printf("processing took %v\n", time.Since(start)) }()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	79
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	80 ctx := context.Background()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	81
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	82 return run(func(db *sql.DB) error {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	83
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	84 conn, err := db.Conn(ctx)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	85 if err != nil {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	86 return err
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	87 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	88 defer conn.Close()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	89
2480 242104c338ff Wrap writing of iso-lines of differences into a transaction. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2479 diff changeset	90 tx, err := conn.BeginTx(ctx, nil)
242104c338ff Wrap writing of iso-lines of differences into a transaction. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2479 diff changeset	91 if err != nil {
242104c338ff Wrap writing of iso-lines of differences into a transaction. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2479 diff changeset	92 return err
242104c338ff Wrap writing of iso-lines of differences into a transaction. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2479 diff changeset	93 }
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	94 defer tx.Rollback()
2480 242104c338ff Wrap writing of iso-lines of differences into a transaction. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2479 diff changeset	95
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	96 type load struct {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	97 date time.Time
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	98 data []byte
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	99 err *error
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	100 dst **octree.Tree
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	101 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	102
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	103 out := make(chan *load)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	104 wg := new(sync.WaitGroup)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	105
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	106 n := runtime.NumCPU()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	107 if n > 2 {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	108 n = 2
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	109 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	110
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	111 for i := 0; i < n; i++ {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	112 wg.Add(1)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	113 go func() {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	114 defer wg.Done()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	115 for l := range out {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	116 if *l.err == nil {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	117 l.dst, l.err = octree.Deserialize(l.data)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	118 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	119 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	120 }()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	121 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	122
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	123 var firstErr, secondErr error
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	124 var first, second *octree.Tree
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	125
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	126 for _, l := range []*load{
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	127 {date: firstDate, dst: &first, err: &firstErr},
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	128 {date: secondDate, dst: &second, err: &secondErr},
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	129 } {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	130 var data []byte
2480 242104c338ff Wrap writing of iso-lines of differences into a transaction. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2479 diff changeset	131 if err := tx.QueryRowContext(
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	132 ctx,
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	133 loadOctreeSQL,
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	134 bottleneck,
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	135 l.date,
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	136 ).Scan(&data); err != nil {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	137 *l.err = err
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	138 } else {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	139 l.data = data
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	140 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	141 out <- l
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	142 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	143 close(out)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	144
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	145 wg.Wait()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	146
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	147 if firstErr != nil \|\| secondErr != nil {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	148 if firstErr != nil && secondErr != nil {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	149 return fmt.Errorf("%v, %v", firstErr, secondErr)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	150 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	151 if firstErr != nil {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	152 return firstErr
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	153 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	154 return secondErr
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	155 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	156
2468 27cc04bbda46 Bail out if the EPSG codes of the datasets don't match. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2467 diff changeset	157 if first.EPSG != second.EPSG {
27cc04bbda46 Bail out if the EPSG codes of the datasets don't match. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2467 diff changeset	158 return errors.New("EPSG codes mismatch. Needs transformation slow pass.")
27cc04bbda46 Bail out if the EPSG codes of the datasets don't match. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2467 diff changeset	159 }
27cc04bbda46 Bail out if the EPSG codes of the datasets don't match. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2467 diff changeset	160
2470 5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	161 now := time.Now()
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	162 log.Printf("loading took %v\n", now.Sub(start))
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	163 last := now
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	164
2572 7686c7c23506 Morphological differences: Moved some code into octree package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	165 result := first.Diff(second)
2469 1312953defab Do difference calculation multi-core. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2468 diff changeset	166
2470 5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	167 now = time.Now()
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	168 log.Printf("setting in took %v\n", now.Sub(last))
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	169 last = now
5bd3236f2b1f Integrate deduplication with setting in. Even if there is more contention in total it seems to be cheaper. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2469 diff changeset	170 log.Printf("num points: %d\n", len(result))
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	171
2574 2833ff156cb2 Morphological differences: Moved loading of clipping polygon into octree package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2572 diff changeset	172 clippingPolygon, err := octree.LoadClippingPolygon(
2833ff156cb2 Morphological differences: Moved loading of clipping polygon into octree package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2572 diff changeset	173 ctx,
2833ff156cb2 Morphological differences: Moved loading of clipping polygon into octree package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2572 diff changeset	174 conn,
2833ff156cb2 Morphological differences: Moved loading of clipping polygon into octree package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2572 diff changeset	175 first.EPSG,
2488 cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	176 bottleneck,
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	177 firstDate, secondDate,
2574 2833ff156cb2 Morphological differences: Moved loading of clipping polygon into octree package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2572 diff changeset	178 )
2833ff156cb2 Morphological differences: Moved loading of clipping polygon into octree package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2572 diff changeset	179 if err != nil {
2488 cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	180 return err
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	181 }
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	182
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	183 now = time.Now()
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	184 log.Printf("loading clipping polygon took %v\n", now.Sub(last))
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	185 last = now
cb55d7eaaa36 Started with the idea to clip an octree by an bounding polygon. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2484 diff changeset	186
2572 7686c7c23506 Morphological differences: Moved some code into octree package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2516 diff changeset	187 tri, err := result.Triangulate()
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	188 if err != nil {
2473 19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	189 return err
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	190 }
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	191
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	192 now = time.Now()
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	193 log.Printf("triangulation took %v\n", now.Sub(last))
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	194 last = now
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	195
2516 1ec4c5633eb6 Clip STR tree and not Octree. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2513 diff changeset	196 tin := tri.Tin()
1ec4c5633eb6 Clip STR tree and not Octree. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2513 diff changeset	197
2575 59e7a011d347 Morphological differences: Moved TIN clipping to octree package and hide the technical details there. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2574 diff changeset	198 removed := tin.Clip(clippingPolygon)
2513 1534df518201 Called STT tree building. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2504 diff changeset	199 now = time.Now()
2575 59e7a011d347 Morphological differences: Moved TIN clipping to octree package and hide the technical details there. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2574 diff changeset	200 log.Printf("clipping tin took %v\n", now.Sub(last))
2516 1ec4c5633eb6 Clip STR tree and not Octree. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2513 diff changeset	201 last = now
1ec4c5633eb6 Clip STR tree and not Octree. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2513 diff changeset	202
1ec4c5633eb6 Clip STR tree and not Octree. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2513 diff changeset	203 log.Printf("Number of triangles to clip: %d\n", len(removed))
1ec4c5633eb6 Clip STR tree and not Octree. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2513 diff changeset	204
1ec4c5633eb6 Clip STR tree and not Octree. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2513 diff changeset	205 builder := octree.NewBuilder(tin)
1ec4c5633eb6 Clip STR tree and not Octree. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2513 diff changeset	206 builder.Build(removed)
2473 19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	207
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	208 now = time.Now()
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	209 log.Printf("building octree took %v\n", now.Sub(last))
19beb7d17337 Triangulate and build octree index. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2472 diff changeset	210 last = now
2476 efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	211
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	212 tree := builder.Tree()
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	213
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	214 log.Printf("min/max: %f %f\n", tree.Min.Z, tree.Max.Z)
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	215
2576 647a58ee9ae9 Morphological differences: Centralized generation of height values for differences in octree package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2575 diff changeset	216 heights := octree.SampleDiffHeights(tree.Min.Z, tree.Max.Z, 0.1)
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	217 log.Printf("num heights: %d\n", len(heights))
4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	218
2476 efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	219 var dataSize int
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	220
2496 12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	221 stmt, err := tx.PrepareContext(ctx, insertContourSQL)
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	222 if err != nil {
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	223 return err
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	224 }
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	225 defer stmt.Close()
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	226
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	227 log.Println("do contours")
2476 efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	228 octree.DoContours(tree, heights, func(res *octree.ContourResult) {
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	229 if err == nil && len(res.Lines) > 0 {
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	230 log.Printf("%.2f: lines: %d\n", res.Height, len(res.Lines))
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	231 wkb := res.Lines.AsWKB2D()
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	232 dataSize += len(wkb)
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	233 _, err = stmt.ExecContext(
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	234 ctx,
2496 12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	235 wkb,
2484 4fa92d468164 Use fogleman's triangulation. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2483 diff changeset	236 first.EPSG,
2496 12ed6feefea5 Use octree clipping. Not working, yet! Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2488 diff changeset	237 contourTolerance,
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	238 res.Height,
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	239 )
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	240 }
2476 efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	241 })
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	242
2479 c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	243 if err != nil {
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	244 return err
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	245 }
c85b16db8a02 Calculate better triangulation and store it into database. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2476 diff changeset	246
2476 efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	247 now = time.Now()
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	248 log.Printf("Number of iso lines: %d\n", len(heights))
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	249 log.Printf("Total WKB size: %.2fMB\n", float64(dataSize)/(1024*1024))
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	250 log.Printf("generating iso lines took %v\n", now.Sub(last))
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	251 last = now
efe332e985b9 Calculate the iso lines of the difference. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2473 diff changeset	252
2480 242104c338ff Wrap writing of iso-lines of differences into a transaction. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2479 diff changeset	253 return tx.Commit()
2465 86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	254 })
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	255 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	256
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	257 func main() {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	258
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	259 flag.Parse()
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	260
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	261 firstDate, err := time.Parse(common.DateFormat, *first)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	262 check(err)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	263 secondDate, err := time.Parse(common.DateFormat, *second)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	264 check(err)
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	265
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	266 if *bottleneck == "" {
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	267 log.Fatalln("Missing bottleneck name")
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	268 }
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	269
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	270 check(process(*bottleneck, firstDate, secondDate))
86c7a023400e Started experimental octree diff branch. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	271 }

Mercurial > gemma

annotate cmd/octreediff/main.go @ 2583:5c6a7e69b02d