Mercurial > gemma
annotate cmd/soundingresults/main.go @ 612:d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
author | Sascha L. Teichmann <sascha.teichmann@intevation.de> |
---|---|
date | Mon, 10 Sep 2018 17:48:56 +0200 |
parents | effd22c0ae5a |
children | 99cd44f19e86 |
rev | line source |
---|---|
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
1 package main |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
2 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
3 import ( |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
4 "bufio" |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
5 "database/sql" |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
6 "encoding/hex" |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
7 "flag" |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
8 "fmt" |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
9 "io" |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
10 "log" |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
11 "os" |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
12 "path/filepath" |
603
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
13 "runtime" |
612
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
14 "sort" |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
15 "strings" |
603
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
16 "sync" |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
17 "time" |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
18 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
19 "github.com/jackc/pgx" |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
20 "github.com/jackc/pgx/stdlib" |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
21 ) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
22 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
23 var ( |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
24 dump = flag.Bool("dump", true, "dump SQL insert statements") |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
25 insecure = flag.Bool("insecure", false, "skip SSL verification") |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
26 dbhost = flag.String("dbhost", "localhost", "database host") |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
27 dbport = flag.Uint("dbport", 5432, "database port") |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
28 dbname = flag.String("dbname", "gemma", "database user") |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
29 dbuser = flag.String("dbuser", "scott", "database user") |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
30 dbpassword = flag.String("dbpw", "tiger", "database password") |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
31 dbssl = flag.String("dbssl", "prefer", "database SSL mode") |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
32 ) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
33 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
34 func run(fn func(*sql.DB) error) error { |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
35 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
36 // To ease SSL config ride a bit on parsing. |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
37 cc, err := pgx.ParseConnectionString("sslmode=" + *dbssl) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
38 if err != nil { |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
39 return err |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
40 } |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
41 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
42 // Do the rest manually to allow whitespace in user/password. |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
43 cc.Host = *dbhost |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
44 cc.Port = uint16(*dbport) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
45 cc.User = *dbuser |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
46 cc.Password = *dbpassword |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
47 cc.Database = *dbname |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
48 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
49 db := stdlib.OpenDB(cc) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
50 defer db.Close() |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
51 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
52 return fn(db) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
53 } |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
54 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
55 type meta struct { |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
56 date time.Time |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
57 name string |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
58 depthReference string |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
59 } |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
60 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
61 func substituteName(fname, name string) string { |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
62 dir := filepath.Dir(fname) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
63 info := filepath.Join(dir, "INFO.txt") |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
64 f, err := os.Open(info) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
65 if err != nil { |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
66 log.Printf("warn: %v\n", err) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
67 return name |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
68 } |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
69 defer f.Close() |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
70 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
71 s := bufio.NewScanner(f) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
72 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
73 for search := strings.ToLower(name); s.Scan(); { |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
74 line := strings.TrimSpace(s.Text()) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
75 if line == "" || strings.HasPrefix(line, "#") { |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
76 continue |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
77 } |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
78 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
79 if parts := strings.SplitN(line, "=", 2); len(parts) == 2 && |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
80 strings.TrimSpace(strings.ToLower(parts[0])) == search { |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
81 return strings.TrimSpace(parts[1]) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
82 } |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
83 } |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
84 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
85 if err := s.Err(); err != nil { |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
86 log.Printf("error: %v\n", err) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
87 } |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
88 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
89 return name |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
90 } |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
91 |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
92 func parseFilename(fname string) (*meta, error) { |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
93 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
94 base := filepath.Base(fname) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
95 |
602
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
96 compressed := strings.ToLower(filepath.Ext(base)) |
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
97 for _, ext := range []string{".gz", ".bz2"} { |
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
98 if ext == compressed { |
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
99 base = base[:len(base)-len(ext)] |
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
100 break |
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
101 } |
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
102 } |
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
103 |
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
104 // Cut .txt |
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
105 base = base[:len(base)-len(filepath.Ext(base))] |
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
106 |
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
107 if !strings.HasSuffix(strings.ToUpper(base), "_WGS84") { |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
108 return nil, fmt.Errorf("%s is not in WGS84", base) |
602
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
109 } |
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
110 |
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
111 base = base[:len(base)-len("_WGS84")] |
10f898bbe50f
Sounding results: Accept if XYZ files are compressed with GZIP or BZIP2
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
600
diff
changeset
|
112 |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
113 idx := strings.IndexRune(base, '_') |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
114 if idx == -1 { |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
115 return nil, fmt.Errorf("%s has no date", base) |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
116 } |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
117 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
118 datePart := base[:idx] |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
119 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
120 date, err := time.Parse("20060102", datePart) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
121 if err != nil { |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
122 return nil, fmt.Errorf("error %s: %v\n", datePart, err) |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
123 } |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
124 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
125 rest := base[idx+1:] |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
126 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
127 if idx = strings.LastIndex(rest, "_"); idx == -1 { |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
128 return nil, fmt.Errorf("%s has no depth reference", base) |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
129 } |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
130 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
131 depthReference := rest[idx+1:] |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
132 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
133 rest = rest[:idx] |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
134 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
135 if !strings.HasSuffix(strings.ToUpper(rest), "_MB") { |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
136 return nil, fmt.Errorf("%s is not in WGS84", base) |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
137 } |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
138 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
139 name := rest[:len(rest)-len("_MB")] |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
140 |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
141 name = substituteName(fname, name) |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
142 |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
143 return &meta{ |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
144 name: name, |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
145 depthReference: depthReference, |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
146 date: date, |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
147 }, nil |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
148 } |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
149 |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
150 type result struct { |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
151 m *meta |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
152 points points3d |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
153 wkb string |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
154 } |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
155 |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
156 func processor(fnames <-chan string, results chan<- result, wg *sync.WaitGroup) { |
603
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
157 defer wg.Done() |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
158 |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
159 for fname := range fnames { |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
160 log.Printf("Processing %s\n", fname) |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
161 m, err := parseFilename(fname) |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
162 if err != nil { |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
163 log.Printf("error: %v\n", err) |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
164 continue |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
165 } |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
166 _ = m |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
167 points, err := parseXYZ(fname) |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
168 if err != nil { |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
169 log.Printf("error: %v\n", err) |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
170 continue |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
171 } |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
172 log.Printf("Number of points: %d\n", len(points)) |
607
21f1f03a8a39
Sounding results: Added simple WKB encoder for points3d.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
606
diff
changeset
|
173 |
21f1f03a8a39
Sounding results: Added simple WKB encoder for points3d.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
606
diff
changeset
|
174 wkb := points.asWKB() |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
175 log.Printf("WKB size %.2f MB\n", float64(len(wkb))/(1024*1024)) |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
176 |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
177 results <- result{m, points, wkb} |
603
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
178 } |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
179 } |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
180 |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
181 func quote(s string) string { |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
182 return "'" + strings.Replace(s, "'", "'''", -1) + "'" |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
183 } |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
184 |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
185 func main() { |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
186 flag.Parse() |
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
187 |
603
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
188 var wg sync.WaitGroup |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
189 |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
190 fnames := make(chan string) |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
191 results := make(chan result) |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
192 done := make(chan struct{}) |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
193 |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
194 handler := func(result) {} |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
195 flush := func() {} |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
196 |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
197 if *dump { |
612
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
198 var results []result |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
199 handler = func(r result) { |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
200 if r.m.depthReference != "ADR" { |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
201 return |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
202 } |
612
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
203 results = append(results, r) |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
204 } |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
205 flush = func() { |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
206 sort.Slice(results, func(i, j int) bool { |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
207 if a, b := results[i].m.name, results[j].m.name; a != b { |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
208 return a < b |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
209 } |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
210 return results[i].m.date.Before(results[j].m.date) |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
211 }) |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
212 out := bufio.NewWriter(os.Stdout) |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
213 fmt.Fprintln(out, "BEGIN;") |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
214 for i := range results { |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
215 r := &results[i] |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
216 fmt.Fprintln(out, "INSERT INTO waterway.sounding_results (") |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
217 fmt.Fprintln(out, " bottleneck_id,") |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
218 fmt.Fprintln(out, " date_info,") |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
219 fmt.Fprintln(out, " depth_reference,") |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
220 fmt.Fprintln(out, " point_cloud") |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
221 fmt.Fprintln(out, ") VALUES (") |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
222 fmt.Fprintf(out, " %s,\n", quote(r.m.name)) |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
223 fmt.Fprintf(out, " '%s'::date,\n", r.m.date.Format("2006-01-02")) |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
224 fmt.Fprintf(out, " %s,\n", quote(r.m.depthReference)) |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
225 fmt.Fprintf(out, " '") |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
226 io.Copy(hex.NewEncoder(out), strings.NewReader(r.wkb)) |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
227 fmt.Fprintln(out, "'") |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
228 fmt.Fprintln(out, ");") |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
229 } |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
230 fmt.Fprintln(out, "COMMIT;") |
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
231 out.Flush() |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
232 } |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
233 } else { |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
234 // TODO: Implement database stuff. |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
235 } |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
236 |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
237 fin := make(chan struct{}) |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
238 |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
239 go func() { |
612
d127856e689d
Sounding results: Sort the data sets by name and date to make the output deterministic.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
611
diff
changeset
|
240 defer func() { flush(); close(fin) }() |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
241 for { |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
242 select { |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
243 case <-done: |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
244 return |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
245 case r := <-results: |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
246 handler(r) |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
247 } |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
248 } |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
249 }() |
603
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
250 |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
251 for i, n := 0, runtime.NumCPU(); i < n; i++ { |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
252 wg.Add(1) |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
253 go processor(fnames, results, &wg) |
603
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
254 } |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
255 |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
256 for _, fname := range flag.Args() { |
603
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
257 fnames <- fname |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
258 } |
603
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
259 |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
260 close(fnames) |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
261 |
3d33c53db1e3
Sounding results: Read point data from xyz files.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
602
diff
changeset
|
262 wg.Wait() |
611
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
263 |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
264 close(done) |
effd22c0ae5a
Sounding result: Write simple SQL insert dumper. Not deterministic, yet.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
607
diff
changeset
|
265 <-fin |
600
30eb8c823ff7
Started with importing the sounding results. WIP.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
266 } |