Mercurial > kallithea

annotate scripts/update-copyrights.py @ 6860:665dfa112f2c

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
py3: replace "file" with "open"
author Lars Kruse <devel@sumpfralle.de>
date Fri, 25 Aug 2017 14:30:57 +0200
parents 5dde581f703f
children 710b2eb8cc37
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6777
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
1 #!/usr/bin/env python2
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
2 # -*- coding: utf-8 -*-
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
3
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
4 """
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
5 Kallithea script for maintaining contributor lists from version control
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
6 history.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
7
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
8 This script and the data in it is a best effort attempt at reverse engineering
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
9 previous attributions and correlate that with version control history while
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
10 preserving all existing copyright statements and attribution. This script is
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
11 processing and summarizing information found elsewhere - it is not by itself
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
12 making any claims. Comments in the script are an attempt at reverse engineering
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
13 possible explanations - they are not showing any intent or confirming it is
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
14 correct.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
15
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
16 Three files are generated / modified by this script:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
17
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
18 kallithea/templates/about.html claims to show copyright holders, and the GPL
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
19 license requires such existing "legal notices" to be preserved. We also try to
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
20 keep it updated with copyright holders, but do not claim it is a correct list.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
21
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
22 CONTRIBUTORS has the purpose of giving credit where credit is due and list all
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
23 the contributor names in the source.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
24
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
25 kallithea/templates/base/base.html contains the copyright years in the page
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
26 footer.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
27
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
28 Both make a best effort of listing all copyright holders, but revision control
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
29 history might be a better and more definitive source.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
30
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
31 Contributors are sorted "fairly" by copyright year and amount of
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
32 contribution.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
33
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
34 New contributors are listed, without considering if the contribution contains
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
35 copyrightable work.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
36
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
37 When the copyright might belong to a different legal entity than the
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
38 contributor, the legal entity is given credit too.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
39 """
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
40
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
41
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
42 # Some committers are so wrong that it doesn't point at any contributor:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
43 total_ignore = set()
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
44 total_ignore.add('*** failed to import extension hggit: No module named hggit')
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
45 total_ignore.add('<>')
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
46
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
47 # Normalize some committer names where people have contributed under different
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
48 # names or email addresses:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
49 name_fixes = {}
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
50 name_fixes['Andrew Shadura'] = "Andrew Shadura <andrew@shadura.me>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
51 name_fixes['aparkar'] = "Aparkar <aparkar@icloud.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
52 name_fixes['Aras Pranckevicius'] = "Aras Pranckevičius <aras@unity3d.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
53 name_fixes['Augosto Hermann'] = "Augusto Herrmann <augusto.herrmann@planejamento.gov.br>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
54 name_fixes['"Bradley M. Kuhn" <bkuhn@ebb.org>'] = "Bradley M. Kuhn <bkuhn@sfconservancy.org>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
55 name_fixes['dmitri.kuznetsov'] = "Dmitri Kuznetsov"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
56 name_fixes['Dmitri Kuznetsov'] = "Dmitri Kuznetsov"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
57 name_fixes['domruf'] = "Dominik Ruf <dominikruf@gmail.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
58 name_fixes['Ingo von borstel'] = "Ingo von Borstel <kallithea@planetmaker.de>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
59 name_fixes['Jan Heylen'] = "Jan Heylen <heyleke@gmail.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
60 name_fixes['Jason F. Harris'] = "Jason Harris <jason@jasonfharris.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
61 name_fixes['Jelmer Vernooij'] = "Jelmer Vernooij <jelmer@samba.org>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
62 name_fixes['jfh <jason@jasonfharris.com>'] = "Jason Harris <jason@jasonfharris.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
63 name_fixes['Leonardo Carneiro<leonardo@unity3d.com>'] = "Leonardo Carneiro <leonardo@unity3d.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
64 name_fixes['leonardo'] = "Leonardo Carneiro <leonardo@unity3d.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
65 name_fixes['Leonardo <leo@unity3d.com>'] = "Leonardo Carneiro <leonardo@unity3d.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
66 name_fixes['Les Peabody'] = "Les Peabody <lpeabody@gmail.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
67 name_fixes['"Lorenzo M. Catucci" <lorenzo@sancho.ccd.uniroma2.it>'] = "Lorenzo M. Catucci <lorenzo@sancho.ccd.uniroma2.it>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
68 name_fixes['Lukasz Balcerzak'] = "Łukasz Balcerzak <lukaszbalcerzak@gmail.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
69 name_fixes['mao <mao@lins.fju.edu.tw>'] = "Ching-Chen Mao <mao@lins.fju.edu.tw>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
70 name_fixes['marcink'] = "Marcin Kuźmiński <marcin@python-works.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
71 name_fixes['Marcin Kuzminski'] = "Marcin Kuźmiński <marcin@python-works.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
72 name_fixes['nansenat16@null.tw'] = "nansenat16 <nansenat16@null.tw>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
73 name_fixes['Peter Vitt'] = "Peter Vitt <petervitt@web.de>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
74 name_fixes['philip.j@hostdime.com'] = "Philip Jameson <philip.j@hostdime.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
75 name_fixes['Søren Løvborg'] = "Søren Løvborg <sorenl@unity3d.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
76 name_fixes['Thomas De Schampheleire'] = "Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
77 name_fixes['Weblate'] = "<>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
78 name_fixes['xpol'] = "xpol <xpolife@gmail.com>"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
79
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
80
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
81 # Some committer email address domains that indicate that another entity might
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
82 # hold some copyright too:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
83 domain_extra = {}
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
84 domain_extra['unity3d.com'] = "Unity Technologies"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
85 domain_extra['rhodecode.com'] = "RhodeCode GmbH"
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
86
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
87 # Repository history show some old contributions that traditionally hasn't been
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
88 # listed in about.html - preserve that:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
89 no_about = set(total_ignore)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
90 # The following contributors were traditionally not listed in about.html and it
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
91 # seems unclear if the copyright is personal or belongs to a company.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
92 no_about.add(('Thayne Harbaugh <thayne@fusionio.com>', '2011'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
93 no_about.add(('Dies Koper <diesk@fast.au.fujitsu.com>', '2012'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
94 no_about.add(('Erwin Kroon <e.kroon@smartmetersolutions.nl>', '2012'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
95 no_about.add(('Vincent Caron <vcaron@bearstech.com>', '2012'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
96 # These contributors' contributions might be too small to be copyrightable:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
97 no_about.add(('philip.j@hostdime.com', '2012'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
98 no_about.add(('Stefan Engel <mail@engel-stefan.de>', '2012'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
99 no_about.add(('Ton Plomp <tcplomp@gmail.com>', '2013'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
100 # Was reworked and contributed later and shadowed by other contributions:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
101 no_about.add(('Sean Farley <sean.michael.farley@gmail.com>', '2013'))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
102
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
103 # Preserve contributors listed in about.html but not appearing in repository
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
104 # history:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
105 other_about = [
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
106 ("2011", "Aparkar <aparkar@icloud.com>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
107 ("2010", "RhodeCode GmbH"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
108 ("2011", "RhodeCode GmbH"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
109 ("2012", "RhodeCode GmbH"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
110 ("2013", "RhodeCode GmbH"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
111 ]
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
112
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
113 # Preserve contributors listed in CONTRIBUTORS but not appearing in repository
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
114 # history:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
115 other_contributors = [
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
116 ("", "Andrew Kesterson <andrew@aklabs.net>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
117 ("", "cejones"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
118 ("", "David A. Sjøen <david.sjoen@westcon.no>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
119 ("", "James Rhodes <jrhodes@redpointsoftware.com.au>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
120 ("", "Jonas Oberschweiber <jonas.oberschweiber@d-velop.de>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
121 ("", "larikale"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
122 ("", "RhodeCode GmbH"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
123 ("", "Sebastian Kreutzberger <sebastian@rhodecode.com>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
124 ("", "Steve Romanow <slestak989@gmail.com>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
125 ("", "SteveCohen"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
126 ("", "Thomas <thomas@rhodecode.com>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
127 ("", "Thomas Waldmann <tw-public@gmx.de>"),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
128 ]
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
129
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
130
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
131 import os
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
132 import re
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
133 from collections import defaultdict
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
134
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
135
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
136 def sortkey(x):
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
137 """Return key for sorting contributors "fairly":
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
138 * latest contribution
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
139 * first contribution
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
140 * number of contribution years
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
141 * name (with some unicode normalization)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
142 The entries must be 2-tuples of a list of string years and the unicode name"""
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
143 return (x[0] and -int(x[0][-1]),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
144 x[0] and int(x[0][0]),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
145 -len(x[0]),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
146 x[1].decode('utf8').lower().replace(u'\xe9', u'e').replace(u'\u0142', u'l')
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
147 )
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
148
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
149
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
150 def nice_years(l, dash='-', join=' '):
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
151 """Convert a list of years into brief range like '1900-1901, 1921'."""
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
152 if not l:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
153 return ''
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
154 start = end = int(l[0])
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
155 ranges = []
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
156 for year in l[1:] + [0]:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
157 year = int(year)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
158 if year == end + 1:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
159 end = year
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
160 continue
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
161 if start == end:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
162 ranges.append('%s' % start)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
163 else:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
164 ranges.append('%s%s%s' % (start, dash, end))
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
165 start = end = year
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
166 assert start == 0 and end == 0, (start, end)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
167 return join.join(ranges)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
168
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
169
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
170 def insert_entries(
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
171 filename,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
172 all_entries,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
173 no_entries,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
174 domain_extra,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
175 split_re,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
176 normalize_name,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
177 format_f):
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
178 """Update file with contributor information.
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
179 all_entries: list of tuples with year and name
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
180 no_entries: set of names or name and year tuples to ignore
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
181 domain_extra: map domain name to extra credit name
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
182 split_re: regexp matching the part of file to rewrite
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
183 normalize_name: function to normalize names for grouping and display
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
184 format_f: function formatting year list and name to a string
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
185 """
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
186 name_years = defaultdict(set)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
187
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
188 for year, name in all_entries:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
189 if name in no_entries or (name, year) in no_entries:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
190 continue
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
191 domain = name.split('@', 1)[-1].rstrip('>')
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
192 if domain in domain_extra:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
193 name_years[domain_extra[domain]].add(year)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
194 name_years[normalize_name(name)].add(year)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
195
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
196 l = [(list(sorted(year for year in years if year)), name)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
197 for name, years in name_years.items()]
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
198 l.sort(key=sortkey)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
199
6860
665dfa112f2c py3: replace "file" with "open"
Lars Kruse <devel@sumpfralle.de>
parents: 6777
diff changeset
200 with open(filename) as f:
6777
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
201 pre, post = re.split(split_re, f.read())
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
202
6860
665dfa112f2c py3: replace "file" with "open"
Lars Kruse <devel@sumpfralle.de>
parents: 6777
diff changeset
203 with open(filename, 'w') as f:
6777
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
204 f.write(pre +
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
205 ''.join(format_f(years, name) for years, name in l) +
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
206 post)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
207
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
208
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
209 def main():
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
210 repo_entries = [
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
211 (year, name_fixes.get(name) or name_fixes.get(name.rsplit('<', 1)[0].strip()) or name)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
212 for year, name in
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
213 (line.strip().split(' ', 1)
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
214 for line in os.popen("""hg log -r '::.' -T '{date(date,"%Y")} {author}\n'""").readlines())
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
215 ]
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
216
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
217 insert_entries(
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
218 filename='kallithea/templates/about.html',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
219 all_entries=repo_entries + other_about,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
220 no_entries=no_about,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
221 domain_extra=domain_extra,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
222 split_re=r'(?: <li>Copyright &copy; [^\n]*</li>\n)*',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
223 normalize_name=lambda name: name.split('<', 1)[0].strip(),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
224 format_f=lambda years, name: ' <li>Copyright &copy; %s, %s</li>\n' % (nice_years(years, '&ndash;', ', '), name),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
225 )
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
226
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
227 insert_entries(
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
228 filename='CONTRIBUTORS',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
229 all_entries=repo_entries + other_contributors,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
230 no_entries=total_ignore,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
231 domain_extra=domain_extra,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
232 split_re=r'(?: [^\n]*\n)*',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
233 normalize_name=lambda name: name,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
234 format_f=lambda years, name: (' %s%s%s\n' % (name, ' ' if years else '', nice_years(years))),
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
235 )
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
236
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
237 insert_entries(
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
238 filename='kallithea/templates/base/base.html',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
239 all_entries=repo_entries,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
240 no_entries=total_ignore,
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
241 domain_extra={},
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
242 split_re=r'(?<=&copy;) .* (?=by various authors)',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
243 normalize_name=lambda name: '',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
244 format_f=lambda years, name: ' ' + nice_years(years, '&ndash;', ', ') + ' ',
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
245 )
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
246
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
247
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
248 if __name__ == '__main__':
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
249 main()
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
250
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
251
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
252 # To list new contributors since last tagging:
5dde581f703f scripts: add update-copyrights.py
Mads Kiilerich <mads@kiilerich.com>
parents:
diff changeset
253 # { hg log -r '::tagged()' -T ' {author}\n {author}\n'; hg log -r '::.' -T ' {author}\n' | sort | uniq; } | sort | uniq -u